When an agent has access to hundreds or thousands of tools, listing all of them in the context window is token-wasteful and degrades model performance. Dynamic tool retrieval — selecting the relevant subset of tools at runtime using embeddings or a lightweight classifier — is how production agentic platforms scale.

Research has shown that model performance on tool selection degrades measurably when the number of available tools in the context exceeds roughly 20–30. Tool descriptions consume tokens, introduce noise, and increase the probability that the model selects a plausible-sounding but wrong tool. Dynamic tool retrieval solves this by treating tool selection as a retrieval problem before it becomes a reasoning problem. The architecture mirrors RAG: each tool's name and description is embedded and stored in a vector index. When the agent needs to select a tool for a subtask, the subtask description is embedded and used to retrieve the top-k most semantically similar tools. Only these k tools are passed to the model for final selection — typically k=5 to 15. This keeps the tool selection prompt lean regardless of registry size. More sophisticated approaches use a lightweight classifier to categorize the agent's current intent and retrieve tools from the appropriate category, reducing false-positive retrievals from semantically adjacent but functionally wrong tools. Tool registry hygiene matters: tool descriptions must be precise and distinctive — vague or overlapping descriptions cause systematic misretrieval. Version management is a production concern: when a tool's interface changes, all downstream agents using that tool need coordinated updates, motivating registry-level versioning and deprecation signaling. MCP's server-based tool exposure model is pushing toward standardized tool metadata that enables cross-organizational tool discovery and retrieval.

Dynamic Tool Retrieval: Scaling Agent Tool Registries Beyond 100 Tools