Ungroup tools based on embedding rankings

[vritant24]

Add embeddings-based tool ranking and caching for virtual tool grouper to predict which tools are most relevant to user queries, improving the tool selection experience.

The feature is currently behind an experiment flag (chat.virtualTools.embeddingRanking)

[vritant24]

cc @connor4312

[Copilot]

Pull Request Overview

This PR adds embeddings-based tool ranking and caching to the virtual tool grouper to predict which tools are most relevant to user queries, improving the tool selection experience. The feature is currently behind an experiment flag (chat.advanced.virtualTools.embeddingRanking) and includes infrastructure for computing tool embeddings and expanding virtual tool groups based on similarity rankings.

Key changes include:

• Implementing an embeddings-based prediction system for virtual tools
• Adding caching for tool embeddings with both pre-computed and runtime computation
• Integrating the ranking system into the tool grouping workflow

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/platform/embeddings/common/embeddingsIndex.ts	Adds new Tools remote cache type for storing tool embeddings
src/platform/configuration/common/configurationService.ts	Adds experiment flag for virtual tool embedding ranking feature
src/extension/tools/test/node/virtualTools/virtualToolGrouping.spec.ts	Updates test calls to include query parameter in grouping methods
src/extension/tools/test/node/virtualTools/virtualToolGrouper.spec.ts	Updates test calls and adds comprehensive tests for the new embedding-based expansion feature
src/extension/tools/common/virtualTools/virtualToolTypes.ts	Updates interface methods to accept query string parameter for context
src/extension/tools/common/virtualTools/virtualToolGrouper.ts	Core implementation of embedding-based tool prediction and virtual tool expansion logic
src/extension/tools/common/virtualTools/toolGrouping.ts	Updates method signatures to pass through query string for embedding computation
src/extension/tools/common/virtualTools/toolEmbeddingsCache.ts	New file implementing tool embeddings computation and caching infrastructure
src/extension/tools/common/test/toolEmbeddingsCache.spec.ts	Comprehensive unit tests for the tool embeddings computation system
src/extension/test/node/services.ts	Registers embeddings computer service for testing
src/extension/prompt/node/defaultIntentRequestHandler.ts	Updates method calls to pass query text for embedding-based tool ranking
src/extension/intents/node/toolCallingLoop.ts	Updates abstract method signature to include query parameter
src/extension/intents/node/agentIntent.ts	Updates method call to include query parameter

[connor4312]

I get an error when we try to compare here

https://github.com/vritant24/vscode-copilot-chat/blob/cbf04b59ee7dfb9e94b5d32d4b534ad780c8c2e1/src/platform/embeddings/common/embeddingsComputer.ts#L162

embedding is simply an array of the embedding values, not the

export interface Embedding {
	readonly type: EmbeddingType;
	readonly value: EmbeddingVector;
}

as expected

[connor4312]

How about we refactor this so _reExpandToolsToHitBudget is generic and takes a ranker or comparator argument. By default it would be something like this._reExpandToolsToHitBudget(root, g => g.tools.length) but for embeddings it would be like this._reExpandToolsToHitBudget(root, g => this._getGroupPredictedRelevancy(g, predictedTools))