Tool grouping prototype

[MackinnonBuck]

Summary

This PR demonstrates a prototype middleware that allows grouping related tools. Groups are hidden from the inner chat client by default, but a well-known "expand" function can be called to expand one group at a time. By reducing the number of tools available to the model at a time, its ability to select the correct tool improves.

Following is an overview of the changes in this PR:

• Add ToolGroupingChatClient, which provides tool grouping and expansion functionality.
• Introduce AIToolGroup to model stable, named batches of tools.
• Add integration tests validating the feasibility of approach.

Usage example

using Microsoft.Extensions.AI;

IChatClient client = baseClient
    .AsBuilder()
    .UseToolGrouping(options =>
    {
        options.Groups.Add(new AIToolGroup(
            name: "TravelUtilities",
            description: "Plan trips end-to-end.",
            tools: [itineraryTool, lodgingTool]));

        options.Groups.Add(new AIToolGroup(
            name: "FinanceUtilities",
            description: "Budgeting helpers.",
            tools: [budgetTool]));
    })
    .UseToolReduction(/* ... */) // Optionally further reduce the set of available tools
    .UseFunctionInvocation()
    .Build();

var response = await client.GetResponseAsync("Plan a weekend getaway to Lisbon and include a budget tip.");
// ...

Implementation notes

This middleware contains a lot of the same complexities present in FunctionInvokingChatClient: cloning options, honoring ConversationId, aggregating Usage across inner calls, and replaying tool-result messages into the history. The expansion loop is deliberately limited to one active group per top-level request to prevent degradation of tool selection as the number of expanded groups increases.

Microsoft Reviewers: Open in CodeFlow

[ericstj]

This is very clever, I had a couple small questions on behavior.

[ericstj]

So we only group tools known at initialization? What about those added per call to GetResponse? I could imagine some sort of callback to assign a group. Or we could let folks annotate a tool with a group and store that in AdditionalProperties.

[ericstj]

Another thought is auto-grouping based on similarity of descriptions / embedding.

[MackinnonBuck]

That's definitely something to consider. We could have a way to annotate a tool with a "category", similar to how tool descriptions are defined. Then chat clients like ToolGroupingChatClient can utilize the category to perform grouping on a per-response basis. This keeps the tool configuration completely in ChatOptions rather than having it be split between ChatOptions and other middleware.

Or maybe we could change the AIToolGroup type to extend AITool. That way you could put tool groups directly in the ChatOptions. You could even make AIToolGroup unsealed and have a virtual GetToolsAsync() method that can be overridden if you want to dynamically expose different sets of tools based on some other logic (this could be used to implement the "get more tools" function you mentioned in your other comment). I'll think about this a bit more.

[ericstj]

Interesting, so we create a signature that the model can call, but it does nothing. We just observe that in the results when handling them from the inner client and use it to modify the tools we then send back to the client.

When an inner client gets a function invocation like this, does it know to halt communcation with the model and yield the result?

[MackinnonBuck]

Yep - FuncitonInvokingChatClient has some logic to detect if an AITool is not an AIFunction. Such tools are considered "non-invokable" and terminate the function calling loop to allow the tool to be handled by the caller:

extensions/src/Libraries/Microsoft.Extensions.AI/ChatCompletion/FunctionInvokingChatClient.cs

Lines 828 to 832 in d5cb8af

	if (tool is not AIFunction)
	{
	// The tool was found but it's not invocable. Regardless of TerminateOnUnknownCallRequests,
	// we need to break out of the loop so that callers can handle all the call requests.
	return true;

[ericstj]

So we always return all Tools in a group, even if those tools were not part of the options for this request? I can see how that might make sense - we have to choose a behavior - are the groupings additive to call, or does the call filter the groupings. Given groupings are new and the call behavior exists, I had assumed the call would filter the groupings (but I wouldn't trust my perspective here 😄).

[MackinnonBuck]

Yeah, we could definitely treat the group as a "filter" more than an extra set of available tools if we wanted. Maybe this is another reason to define groups directly on ChatOptions rather than configuring them on the middleware.