Similar funtionality for Structured Output metadata?

[rwjdk]

Hi, and thank you for a cool repo 👍

Have you considered also supporting dynamic metadata for structured output objects (That is actually there I mostly need the dynamic nature to avoid multiple Structured Output object for various scenarios)?

[lsiddiquee]

Hi @rwjdk thanks for reaching out. Can you maybe elaborate and provide a few examples.

[rwjdk]

@lsiddiquee : Take, as an example, the StructuredOutput example in this repo as a simple sample:

https://github.com/rwjdk/AiSamples

More specifically, this file: https://github.com/rwjdk/AiSamples/blob/main/src/StructuredOutputExample/Models/MovieResult.cs

In it, there is a Description Attribute on the Top10Movies property

[Description("Order them by IMDB Score")]
public required Movie[] Top10Movies { get; set; }

If I had two use-cases where I need these movies ordered by score, and one where I wanted it by year of release. Because the Description needs to be a constant, I would need 2 different classes for the two different use-cases

I know the sample is a bit contrived, as this ordering could be done in code after the fact, but here is a bit more advanced scenario for context:

I use AI and structured output to make AI Reviews on Pull Requests... In some repo I have C# code, but in others I have md files (Docs site)... For this, I can't use the same Review class for structured output as the descriptions on the properties need to be different (one needs to explain to look for code style and bug, while the other needs to look for spelling errors and md structure errors).

Hope it makes sense... If not, or not fitting this, then feel free to close this issue 👍

[lsiddiquee]

Coming back to this, was trying to think of potential solution. The issue here is that the plugins are SK specific and this is OpenAI functionality.

There are two ways to achieve this.

• We can either emit a new type with different attributes
• Or preferably better to hook into json schema generation and update the comments accordingly

        var agent = new ChatCompletionAgent
        {
            Kernel = kernel,
            Instructions = "You are a Movie Expert",
            Arguments = new KernelArguments(new AzureOpenAIPromptExecutionSettings
            {
                ResponseFormat = GenerateAndPatchJsonSchema(
                    typeof(MovieResult),
                    new Dictionary<string, string>
                    {
                        { "Top10Movies", "Order them by IMDB Score" },
                        { "Top10Movies.Title", "The title of the movie" }
                    })
            }),
        };

Every entry in the dictionary could be the path in the json to override the descriptions.

[lsiddiquee]

@rwjdk thoughts?

[rwjdk]

@isiddiquee: I think the best adoption would happen if it was not a new type, so hook into JSON sounds like the best option to me.

A params of a new type called OverriddenDescription(path, newDescription), might make it easier to understand over a dictionary

[lsiddiquee]

I agree. Will keep the issue as is for now. Hoping that can find some free time in the comin month to add this capability.

[rwjdk]

@lsiddiquee : You might also await whats happening to Semantic Kernel. As I understand it SK is now in maintenance mode while the SK Team + The Autogen Team are tasked with building a new AI Framework