We need to build a system where agents can dynamically discover and use new tools. What's the best framework for dynamic tool discovery?

When building systems where agents dynamically discover and use new tools (avoiding context window bloat and scaling past hundreds of tools), there isn't just one "framework," but rather a modern architectural pattern combined with specific orchestration tools.

The gold standard pattern for this is Progressive Disclosure / Dynamic Tool Selection (the "Discovery First, Injection Second" approach pioneered by Anthropic's Tool Search and advanced patterns in LangGraph).

Top Frameworks & Approaches for Dynamic Tool Discovery

- **Why it fits:** Unlike static agent runners, LangGraph allows you to manipulate state and runtime context dynamically. You can build a custom `ToolNode` or use runtime configuration/state injections to filter, search, and bind a subset of tools to the model per-step based on the user's immediate intent.
- **Best for:** Complex, production-grade agent loops where you need programmatic control over what the LLM can "see" at any given turn.[](https://google.com/goto?url=CAESTgHrOzAVjSfwzr8ETjjVPGGQk62iTZJ4-WohKT3ro92gKnqD5t1cNjBLUzgA-nZgd_LF8Jz1FZ_OfHvLF7OOoPrOelLYQyP3wAORE3F6HA) [[1]](https://google.com/goto?url=CAESTgHrOzAVjSfwzr8ETjjVPGGQk62iTZJ4-WohKT3ro92gKnqD5t1cNjBLUzgA-nZgd_LF8Jz1FZ_OfHvLF7OOoPrOelLYQyP3wAORE3F6HA)

- **Why it fits:** MCP standardizes how applications provide context and tools to LLMs. Instead of hardcoding tools into your agent framework, MCP allows agents to connect to external "servers" that dynamically expose or register tools at runtime. Combined with a gateway or search primitive, an agent can query an MCP registry to discover new capabilities on the fly.
- **Best for:** Decoupled, microservice-style tool ecosystems where tools live outside your core application codebase.[](https://google.com/goto?url=CAESfgHrOzAVDzPqaNrE53FtBJHhc2ZKSGF0Ux0N5LIW_3114u3sb3iGh14nr1bL1gxeQEPhTZ0WoXMmmO1-IKt4mPSagw8F6jwQD1FcHb81y-d_OmW0LQPauOm09ZVg7ZMTk_CG0OV8--oDgCgpAVAnHBM5-b2xKt0xNYD3LJJpLg) [[1]](https://google.com/goto?url=CAESfgHrOzAVDzPqaNrE53FtBJHhc2ZKSGF0Ux0N5LIW_3114u3sb3iGh14nr1bL1gxeQEPhTZ0WoXMmmO1-IKt4mPSagw8F6jwQD1FcHb81y-d_OmW0LQPauOm09ZVg7ZMTk_CG0OV8--oDgCgpAVAnHBM5-b2xKt0xNYD3LJJpLg)

- **Why it fits:** If you have hundreds or thousands of tools, putting all their descriptions in a prompt breaks context limits. Instead, you store tool descriptions and schemas in a vector database. When a user sends a prompt, you run a fast semantic search against the tool registry, retrieve the top K cap K𝐾 relevant tool definitions, and bind *only* those to the LLM for that execution step.
- **Best for:** Massive scale tool libraries where keyword or LLM-based tool searching is too slow or expensive.

The Blueprint: How to Implement Dynamic Tool Discovery

Regardless of the framework you choose, the architecture typically follows a three-step Discovery Loop:

To help narrow down the best choice for your stack, could you share:

• Where do your tools live? (e.g., local Python functions, external APIs, or third-party microservices?)
• Roughly how many tools do you expect the agents to choose from?

If the goal is agents that can discover tools at runtime rather than having every tool hard-coded into the agent, I’d use Model Context Protocol (MCP) as the foundation, with a semantic/hybrid tool-discovery layer on top.

My recommendation

MCP + tool registry + semantic retrieval + policy layer

MCP is currently the strongest choice for the protocol layer because it standardizes how agents discover and invoke external tools. The latest MCP specification supports tools/list, pagination, caching, deterministic tool ordering, and server discovery.

The important distinction is:

Don't dynamically load every tool into the model context. Dynamically discover only the tools relevant to the current task. AWS's current guidance explicitly identifies three patterns—static registration, loading all tools dynamically, and runtime search—and recommends filtering/semantic search as tool catalogs grow.

A good architecture looks like:

                    ┌─────────────────────┐
                    │      Agent/LLM      │
                    └──────────┬──────────┘
                               │
                    "I need a tool for X"
                               │
                               ▼
                    ┌─────────────────────┐
                    │  Tool Discovery     │
                    │                     │
                    │ BM25 + embeddings   │
                    │ metadata filtering  │
                    │ permissions/policy  │
                    └──────────┬──────────┘
                               │
                       top 3–10 tools
                               │
                               ▼
                    ┌─────────────────────┐
                    │    MCP Gateway      │
                    └──────────┬──────────┘
                               │
             ┌─────────────────┼─────────────────┐
             ▼                 ▼                 ▼
        MCP Server A       MCP Server B      MCP Server C
        Salesforce         GitHub            Internal APIs

Why not just MCP dynamic discovery?

If you have 10–20 tools, simply calling tools/list and giving the results to the model is perfectly reasonable.

But at hundreds or thousands of tools, it becomes problematic. Tool schemas consume context and make tool selection harder. Recent research on semantic MCP discovery reports substantial reductions in tool-context consumption by retrieving only a handful of relevant tools.

So I'd implement two levels:

1. Server discovery

• Which MCP servers/capability domains are relevant?
• e.g. CRM, finance, GitHub, databases.

2. Tool discovery

• Which specific tools within those servers solve the current subtask?
• Retrieve perhaps the top 5–10.

Then expose the selected tools to the agent.

Framework choices

The OpenAI Agents SDK now explicitly incorporates MCP alongside other agent primitives, so it can serve as the orchestration layer while MCP handles external tool integration.

One design I'd strongly recommend

Don't make the LLM responsible for unrestricted discovery.

Give it something conceptually like:

search_tools(query, constraints)
        ↓
candidate tools
        ↓
policy / authorization filter
        ↓
ranked tools
        ↓
load schema for selected tools
        ↓
agent calls tool

For ranking, I'd start with:

score =
    0.45 * semantic_similarity
  + 0.25 * keyword/BM25_score
  + 0.15 * capability/domain_match
  + 0.10 * historical_success
  + 0.05 * latency/cost_score

And attach metadata to every tool:

{
  "name": "create_invoice",
  "description": "Create an invoice for an existing customer",
  "domain": "billing",
  "risk": "financial_write",
  "required_scopes": ["invoices.write"],
  "cost": "low",
  "latency_ms": 250
}

That gives you dynamic discovery without dynamic authorization—the latter should remain deterministic and enforced outside the model.

Bottom line

If I were starting this system today, I'd choose:

MCP for the tool protocol → MCP Registry/internal catalog for discovery → hybrid BM25 + vector retrieval for tool selection → policy engine for authorization → Agents SDK/LangGraph for orchestration.

MCP is particularly attractive now because the July 2026 specification moved to a stateless core and made tool-list results cacheable, which makes large-scale MCP deployments considerably easier to operate.

If you're building this for hundreds/thousands of tools, I would not implement plain tools/list → dump everything into context; I'd build the tool-search layer as a first-class component.

If dynamic tool discovery is the core requirement, I’d make Model Context Protocol (MCP) the tool layer rather than choosing an agent framework that invents its own discovery mechanism.

Recommended architecture

MCP + an agent orchestration framework

• MCP — standardized tool discovery, schemas, invocation, and capability negotiation.
• Agent framework — handles planning, state, retries, delegation, and workflows.
• Tool registry/catalog — optional layer for finding which MCP servers are relevant when you have hundreds or thousands of tools.
• Policy layer — controls which discovered tools the agent is actually allowed to use.

MCP explicitly supports tools/list, pagination, tool schemas, and notifications when the tool set changes. The newer 2026 MCP specification also adds server/discover for discovering server capabilities before interacting with it.

The important distinction

There are really two discovery problems:

1. "What tools does this server provide?" → MCP handles this extremely well.
2. "Which of 10,000 available tools should I even expose to this agent?" → MCP alone isn't enough. Build a semantic tool registry/search layer on top.

For example:

                    ┌─────────────────────┐
                    │      Agent          │
                    │  planner / executor │
                    └──────────┬──────────┘
                               │
                         tool search
                               │
                    ┌──────────▼──────────┐
                    │   Tool Registry     │
                    │                     │
                    │ embeddings +        │
                    │ metadata + ACLs     │
                    └──────────┬──────────┘
                               │
                     relevant MCP servers
                               │
              ┌────────────────┼────────────────┐
              ▼                ▼                ▼
         ┌─────────┐      ┌─────────┐      ┌─────────┐
         │ MCP     │      │ MCP     │      │ MCP     │
         │ Server A│      │ Server B│      │ Server C│
         └─────────┘      └─────────┘      └─────────┘

This prevents the classic problem of dumping thousands of tool definitions into the model's context.

Which framework?

If you're using OpenAI models, OpenAI Agents SDK + MCP is particularly attractive because the SDK already supports MCP servers, hosted MCP tools, tool filtering, and dynamic tool filtering. It also supports deferred loading/tool search for hosted MCP servers, which is directly relevant to your dynamic-discovery requirement.

If you need complex stateful workflows, branching, human approval, and durable execution, I'd instead consider LangGraph as the orchestration layer while still using MCP for tools.

My recommendation

For a new system I'd choose:

MCP → tool interoperability OpenAI Agents SDK or LangGraph → agent orchestration Vector/metadata registry → large-scale tool discovery Policy engine → authorization

And I would not make the agent blindly discover and execute arbitrary newly registered tools. Tool descriptions and annotations should be treated as untrusted unless the server is trusted; MCP itself recommends approval/visibility mechanisms for sensitive operations.

The key design pattern is therefore "discover → rank → authorize → load → execute", rather than simply "give the agent all available tools."

If you're building this at 10s vs. 1,000s vs. 100,000s of tools, the optimal discovery architecture changes substantially.

Building a system where agents dynamically discover and use tools requires solving a major bottleneck: context window bloat . If you pass hundreds of tool definitions (JSON schemas) to an LLM upfront, accuracy plummets, latency increases, and costs spike.

The industry has largely converged on an architecture of "Discovery First, Injection Second" , supported by a few leading frameworks and protocols.

The Leading Standard: Model Context Protocol (MCP) + Tool Search

The emerging gold standard for dynamic tool discovery is Model Context Protocol (MCP) (originally pioneered by Anthropic) combined with native Tool Search or deferred tool patterns.

• How it works: Instead of hardcoding tools into your agent code, external capabilities live on modular MCP servers . The agent registers a massive catalog of tools as deferred or searchable, exposing only a lightweight "tool search primitive" in the primary context.
• The Discovery Loop: When a user asks a complex question, the LLM first queries the search primitive ("Find me a tool for managing Cloudflare workers" or "Find a database query tool"). The registry returns just the relevant 3–5 tool schemas, which are then injected into the active context for execution.
• Why it's the best approach: It scales cleanly to hundreds or thousands of enterprise tools without overwhelming the model's context or degrading reasoning performance.

Framework Options for Implementation

1. LangChain / LangGraph (Best for Custom, Fine-Grained Control)

• Approach: LangChain supports Dynamic Tool Selection where you can filter pre-registered tools or register/un-register tools at runtime based on user permissions, conversation state, or external registries. It also natively binds with provider-specific MCP toolsets.
• Best if: You need deep control over state management (using LangGraph) and want to swap between OpenAI, Anthropic, or open-source models while managing a dynamic tool registry.

2. LlamaIndex (Best for Data-Heavy & Retrieval-Based Discovery)

• Approach: LlamaIndex treats tool discovery similarly to RAG (Retrieval-Augmented Generation). You can store tool definitions, descriptions, and schemas inside a vector database. When a query comes in, the agent runs a semantic search over the tool vector store to retrieve and bind the most relevant tools on the fly.
• Best if: You have a massive, sprawling library of APIs or microservices and want semantic similarity matching to decide which tool an agent should use.

3. CrewAI / AutoGen (Best for Multi-Agent Task Delegation)

• Approach: These multi-agent frameworks handle discovery by abstracting tools to specific agent personas or specialized "sub-agents." Rather than one agent searching through 50 tools, a master router agent dynamically delegates sub-tasks to specialized worker agents that only carry the exact toolset required for that sub-task.
• Best if: Your dynamic discovery problem is better solved by organizational hierarchy (routing tasks to specialized agents) rather than flat function-calling catalogs.

Recommendation Summary

• If you are building a scale-out enterprise architecture with decoupled microservices, use MCP (Model Context Protocol) as your underlying transport layer paired with a deferred tool search mechanism.
• If you want to orchestrate the logic , implement it using LangGraph for reliable stateful routing or LlamaIndex if your discovery relies heavily on vector-based semantic retrieval of APIs.

If you can share how many tools you expect to manage, whether they are internal APIs or third-party integrations , and your preferred LLM provider , I can recommend a more tailored architecture.

The best industry standard and architecture for building a dynamic tool discovery system is the Model Context Protocol (MCP) , coupled with either an agent registry gateway or Anthropic's Advanced Tool Use framework . Rather than using a monolithic agent framework, the modern standard is to separate the tool providers from the agent orchestration using an open, protocol-based architecture.

An evaluation of the best approaches for dynamic tool discovery highlights the key options, architectural patterns, and execution strategies available.

The Leading Frameworks & Standards

1. Model Context Protocol (MCP) — The Standard Architecture

Developed as an open-source standard, MCP completely decouples tools (servers) from agents (clients).

• How Discovery Works: Tools register themselves dynamically with an MCP Registry or Gateway. The registry exposes a semantic metadata search. When an agent needs a capability it does not natively possess, it queries the registry using natural language.
• Why it’s the best: It provides a native notifications/tools/list_changed hook. This allows tool servers to broadcast real-time additions, removals, or permission state changes (e.g., hiding a tool if an API key expires) directly to the agent runtime.

2. Anthropic’s Advanced Tool Use — The Best Front-End/LLM Native Framework

If you are heavily leveraging Claude or similar state-of-the-art models, Anthropic's Tool Search Tool pattern is the premier framework for context-aware dynamic discovery.

• How Discovery Works: Instead of flooding the LLM's context window with thousands of tool definitions, you only give the agent a single primitive: search_available_tools . The agent calls this search tool, finds the top 3–5 exact tools it needs, and uses a second primitive (load_tool ) to dynamically inject the full JSON schema into its active context window on the fly.
• Why it's the best: It stops context window bloat, reduces token costs, and handles scale seamlessly.

3. Pydantic AI & LangGraph — The Best Code-First Orchestration Engines

If you need a programming framework to wire these components together, Pydantic AI and LangGraph are the top choices.

• Pydantic AI natively integrates with MCP, leveraging typed python graphs to validate runtime schemas securely.
• LangGraph allows you to manage state explicitly. You can create a specialized "Router/Selector Node" that reads a user prompt, hits a vector DB of your tool registry, and dynamically injects the chosen tool definitions into the agent state before routing to the LLM node.

The Recommended Architectural Pattern

To build a production-grade system, combine these technologies into a Gateway Architecture:

[ New Tool / MCP Server ] ──(Auto-Register)──► [ Tool Registry / Vector DB ]
                                                       ▲
                                                (Semantic Search)
                                                       │
[ User Prompt ] ─────────────────────────────► [ Agent Orchestrator ]

1. The Registry: Store all your tool schemas in an external database (like a vector database or an MCP registry gateway) rather than in your application code. Index them by semantic description.
2. The Search Step: When a user issues a request, a lightweight "Selector" loop or the LLM itself utilizes a search utility to find relevant tools matching the intent.
3. The Just-In-Time Injection: Only inject the specific schemas of the discovered tools into the model’s active prompt context.
4. Execution Sandbox: Ensure the dynamically loaded code or API calls run in a containerized sandbox with minimum permissions to mitigate prompt injection risks.

Direct Comparison of Approaches

To help narrow down your engineering design, let me know:

• Approximately how many tools do you expect the system to scale to?
• What programming language is your primary agent orchestration layer built on?
• Are your tools internal APIs, third-party services, or local scripts?

For a system where agents can dynamically discover and use new tools, the best architecture is usually not a single agent framework. The strongest pattern today is:

Model Context Protocol (MCP) for tool discovery + an agent orchestration framework for reasoning/workflows. MCP is emerging as the interoperability layer that lets agents discover tools exposed by external servers instead of hard-coding every integration.

A practical stack:

My default recommendation

For a greenfield production system:

MCP + LangGraph + semantic tool retrieval

Architecture:

                 User Request
                      |
                      v
              Agent Orchestrator
                 (LangGraph)
                      |
          +-----------+-----------+
          |                       |
          v                       v
   Tool Discovery Service     Memory/State
          |
          v
   MCP Tool Registry
          |
   +------+------+------+
   |             |      |
 MCP Server   MCP    MCP
  (CRM)     (Search) (Code)

Instead of:

Agent context:
- tool1
- tool2
- tool3
...
- tool500

do:

User asks:
"Find unpaid invoices"

        |
        v

Tool retriever:
returns:
- accounting.search_invoices
- accounting.get_customer_balance

        |
        v

Agent invokes MCP tool

This avoids context bloat and improves tool selection. Research on semantic MCP tool discovery specifically targets this problem: exposing hundreds of tools directly to an LLM increases token cost and can reduce accuracy, while retrieval-based selection can narrow the active tool set.

Framework choices

1. LangGraph — best general choice

• Good when agents need:
  • durable state
  • complex workflows
  • human approval steps
  • retries/checkpoints
  • explicit control flow

It has a strong ecosystem around tool calling and MCP integrations.

2. Microsoft Agent Framework / Semantic Kernel — best enterprise choice

• Strong if you are:
  • .NET-heavy
  • Azure-based
  • building plugin-style enterprise agents

Semantic Kernel’s plugin model maps naturally to dynamically discovered capabilities, and MCP support makes it suitable for external tool ecosystems.

3. OpenAI Agents SDK — best for simpler agent systems

• Good when:
  • you have fewer tools
  • you want minimal orchestration code
  • workflows are relatively straightforward

4. CrewAI — good for role-based multi-agent teams

• Better when the problem is "multiple specialists collaborating" rather than "one agent dynamically acquiring capabilities."

Avoid this pattern

A common early design mistake:

Every tool → every agent → every prompt

It fails when you reach hundreds of tools because:

• context windows fill up
• tool descriptions compete with task instructions
• the model chooses similar-but-wrong tools
• security boundaries become unclear

For a serious platform, I would build:

1. MCP servers as the integration boundary
2. Tool registry
   • name
   • description
   • permissions
   • cost
   • latency
   • embeddings
   • version
3. Tool retrieval layer
   • semantic search
   • capability filtering
   • user/agent authorization
4. Agent runtime
   • LangGraph or Microsoft Agent Framework
5. Execution sandbox
   • rate limits
   • audit logs
   • approval gates

So the short answer:

Best framework combination: MCP for dynamic discovery + LangGraph for orchestration + a semantic tool registry for scaling beyond dozens of tools.

If you are building a platform intended to support thousands of tools across many agents, the tool registry/retrieval layer is actually the more important design decision than the agent framework itself.

Building a system where agents dynamically discover and use new tools requires decoupling tool registration and discovery from agent compilation/prompt injection . Cramming hundreds of tool schemas into an LLM's prompt causes context rot, token waste, and degraded decision-making.

The industry has largely converged on a protocol-plus-registry architecture rather than relying on any single monolithic framework.

The Modern Standard Stack for Dynamic Tool Discovery

• The Protocol Layer (Model Context Protocol / MCP): Originally pioneered by Anthropic and widely adopted across the ecosystem, MCP serves as the universal "USB-C for AI". Instead of hardcoding or writing custom glue code for new tools, external services and internal microservices expose themselves as MCP Servers . Agents query the server at runtime via standard protocols (Stdio, SSE, or Streamable HTTP) to dynamically list available tools and schemas.
• The Discovery & Search Layer (Tool Search / Vector Indexing): Instead of exposing every discovered tool to the LLM immediately, use a Tool Search Pattern . The LLM initially sees only a lightweight core tool (like a search_tools primitive). When faced with a task, the agent queries the tool catalog using natural language/vector embeddings, fetches only the 3 to 5 relevant tool definitions, and injects them into the active context on demand.
• The Orchestration Layer (LangGraph or CrewAI): Once a tool is discovered and its schema is pulled into context, you hand it off to an execution framework.
  • LangGraph is ideal if you want fine-grained, low-level control over state, custom runtime context injection, and conditional tool-fetching loops.
  • CrewAI provides great out-of-the-box native DSL integrations (mcps fields and adapters) if your system relies on multi-agent collaborative teams that need to aggregate tools from various remote servers.

Recommended Implementation Blueprint

1. Host a Central Registry/Gateway: Maintain an internal MCP-compliant registry or API gateway that tracks all enterprise tools, microservices, and user permissions.
2. Apply Discovery-First Logic: Index your tool metadata in a semantic/vector store. Do not pass full tool arrays to the model upfront.
3. Filter by Identity & Context: Ensure the discovery mechanism evaluates runtime context (user permissions, active workspace, security policies) before returning tool signatures to the agent.
4. Bind and Execute: Pass the dynamically retrieved subset of tools to your orchestrator's execution node for the current turn.

If you'd like to dive deeper into the implementation, let me know:

• Are your tools local Python functions, remote REST/OpenAPI endpoints , or third-party services?
• Do you prefer an orchestration framework like LangGraph or CrewAI?

I can provide a code snippet tailored to your architecture.

If the core requirement is “agents should discover tools at runtime rather than having every tool hard-coded into the agent”, I’d use MCP (Model Context Protocol) as the tool-discovery layer, with your agent framework on top of it.

Recommended architecture

Agent runtime → MCP client → Tool registry / MCP servers → actual APIs/services

Why MCP:

• Standardized discovery: MCP exposes tools with names, descriptions, and JSON schemas, and clients discover them through tools/list. Servers can also signal when their tool catalog changes.
• Decouples agents from tools: You can add a new MCP server/tool without modifying the agent's core implementation.
• Works across frameworks: MCP is model/framework agnostic; LangGraph, OpenAI Agents SDK, and other agent runtimes can consume MCP tools.
• Supports remote discovery: Streamable HTTP is the current preferred approach for remote MCP servers; stdio is useful for local processes.
• Dynamic filtering: You can expose different tools based on agent, user, tenant, permissions, or runtime context. OpenAI's Agents SDK, for example, supports callable tool filters.

The important distinction

I'd separate tool discovery from tool execution:

                    ┌──────────────────┐
                    │   Agent / LLM    │
                    └────────┬─────────┘
                             │
                     "What can I use?"
                             │
                    ┌────────▼─────────┐
                    │ Tool Discovery   │
                    │ / MCP Client     │
                    └────────┬─────────┘
                             │
              ┌──────────────┼──────────────┐
              ▼              ▼              ▼
        ┌──────────┐   ┌──────────┐   ┌──────────┐
        │ CRM MCP  │   │ Git MCP  │   │ DB MCP   │
        │ Server   │   │ Server   │   │ Server   │
        └────┬─────┘   └────┬─────┘   └────┬─────┘
             ▼              ▼              ▼
           CRM            Git            Database

The agent shouldn't need to know that search_customer happens to live in your CRM service. It discovers the capability and its schema, then invokes it.

For a large tool ecosystem

Don't dump hundreds or thousands of tool schemas into the model's context. That's where deferred tool loading / tool search becomes important.

The current OpenAI Agents SDK supports deferred loading for tools and hosted MCP servers: the model can search for and load only the relevant tool definitions when needed.

So I'd use:

MCP = protocol/discovery layer
Tool registry = governance/catalog layer
Tool search = semantic selection layer
Agent framework = orchestration/execution layer

For example:

                    Tool Registry
                         │
          ┌──────────────┼──────────────┐
          │              │              │
       metadata        auth          embeddings
          │              │              │
          └──────────────┼──────────────┘
                         │
                   Tool Search
                         │
                  "Find tools for
                   refunding orders"
                         │
              ┌──────────▼──────────┐
              │ Relevant MCP tools │
              └──────────┬──────────┘
                         │
                    Agent chooses
                         │
                         ▼
                  MCP tool call

What I'd choose

If you're starting today:

One particularly useful feature for your use case is that the latest MCP specification has been moving toward a more scalable, stateless protocol core, with cache hints and deterministic tool-list ordering—useful characteristics when you have many agents and many dynamically changing tool servers.

Bottom line: I wouldn't build a proprietary dynamic-tool-discovery protocol unless you have an unusual requirement. MCP + a centralized tool catalog + semantic/deferred tool search is the architecture I'd recommend. It gives you dynamic discovery without coupling your agents to the underlying services.

Building a system for dynamic tool discovery requires solving two primary bottlenecks: context window bloat (cramming hundreds of tool schemas into an LLM prompt degrades reasoning) and governance/security (controlling who or what can access a tool at runtime).

The industry has largely converged on a protocol-plus-registry pattern rather than monolithic code frameworks.

The Leading Standard: Model Context Protocol (MCP)

The gold standard for dynamic tool discovery is Model Context Protocol (MCP) , originally pioneered by Anthropic and rapidly adopted across the broader ecosystem (including Azure AI Agent Service and Spring AI).

• How it works: Instead of hardcoding tools into your agent code, your agent connects to external MCP Servers at runtime. The server acts as a dynamic capability repository exposing structured metadata, schemas, and endpoints.
• Why it's the best approach: It decouples discovery (what tools exist) from invocation (how to execute them). New enterprise microservices or third-party APIs can launch an MCP server, and any compliant agent can discover and use those tools instantly without a rebuild.

Key Architectural Patterns for Implementation

When implementing tool discovery on top of MCP or a custom registry, use one of these three strategies depending on your scale:

1. The "Tool Search" (Deferred Loading) Pattern
   • Mechanism: You register your entire tool catalog with the model API, but mark the majority as "deferred". The LLM initially sees only a generic search_tools primitive. When a user asks a complex question, the model queries the search tool first, retrieves the specific schemas for the 2–3 relevant functions, and injects only those into its active context window.
   • Best for: Large tool libraries (dozens to hundreds of tools) where you want to eliminate token waste and context confusion.
2. Vectorized Semantic Tool Indexing (RAG for Tools)
   • Mechanism: Tool descriptions, inputs, and outputs are embedded and stored in a vector index. The agent is given a single retrieval tool. It translates the user's intent into a natural language semantic query, searches the tool bank, and pulls the relevant code/schema dynamically.
   • Best for: Massive tool ecosystems (hundreds to thousands of tools) where keyword matching or explicit categories fail.
3. Agent-as-a-Tool / Federated Directories
   • Mechanism: Emerging specifications like the Agentic Resource Discovery (ARD) standard allow distributed agents and resource registries to publish their skills across organizational bounds. A master orchestrator agent calls a directory service to find which specialized sub-agent or capability block handles a domain, then dispatches the task.
   • Best for: Enterprise multi-agent environments where different business units independently manage and update micro-tools.

Frameworks to Build With

• LangGraph: Excellent for implementing runtime context customization and graph-based orchestration where tools can be filtered and injected into the model's state dynamically per turn.
• Semantic Kernel / Microsoft Agent Framework: Strong enterprise backing for native MCP integration, Azure AI Foundry, and identity-aware tool filtering.
• Spring AI: Provides out-of-the-box ToolSearchToolCallingAdvisor implementations for deferred tool indexing and runtime expansion.

To help narrow down the exact architecture you need, tell me:

• Roughly how many tools will your system need to support (dozens, hundreds, or thousands)?
• Are these tools internal microservices or third-party user-facing integrations requiring OAuth?

For a system where agents can dynamically discover and use new tools, the strongest current architecture is usually:

Model Context Protocol (MCP) + an agent orchestration framework (LangGraph, Semantic Kernel, etc.) + a tool registry/discovery layer.

A useful way to think about the stack:

                 Agent
                   |
          Agent Framework
       (LangGraph / Semantic Kernel)
                   |
          Tool Discovery Layer
       (registry + semantic search)
                   |
              MCP Servers
                   |
        APIs / Databases / Services

1. MCP: the tool interface layer (recommended foundation)

Model Context Protocol is currently the best fit for the "discover tools I don't know about yet" problem.

Why:

• Standard tool schemas (name, description, inputs, outputs)
• Tools can live outside the agent process
• Agents can connect to new tool providers without custom integrations
• Works across different agent frameworks

Example:

Agent:
  "I need to analyze a PDF"

Discovery:
  Search available capabilities

Registry:
  Finds:
    pdf.extract_text
    pdf.summarize
    pdf.convert

Agent:
  Loads MCP tool definition

Agent:
  Calls tool

---

2. Agent framework choice

Best general production choice: LangGraph + MCP

LangChain / LangGraph gives you:

• Stateful workflows
• Human approval steps
• Multi-agent orchestration
• Runtime tool injection

LangChain supports MCP servers through adapters and can dynamically add tools at runtime.

Good for:

• Enterprise agents
• Complex workflows
• Long-running tasks

---

Best Microsoft ecosystem choice: Semantic Kernel

Good if you are building around:

• Azure
• .NET
• enterprise plugins
• governed tool access

---

Best lightweight approach

Build your own thin layer:

• MCP client
• Tool registry
• Vector search over tool descriptions
• Permission manager
• Execution sandbox

This often ends up being the architecture behind large internal agent platforms.

---

3. Don't expose every tool to the model

A common mistake is:

Agent gets 10,000 tools
        ↓
LLM chooses one

This causes:

• context bloat
• worse tool selection
• higher cost

Instead:

User request
      ↓
Semantic tool retriever
      ↓
Top 5 relevant tools
      ↓
Agent chooses
      ↓
Execute

Recent research around MCP tool retrieval explores exactly this pattern: embedding tool descriptions and retrieving only relevant capabilities rather than loading the whole catalog.

---

4. Reference architecture I would build

For a serious dynamic discovery system:

Tool registry

• Stores:
  • tool name
  • description
  • JSON schema
  • auth requirements
  • permissions
  • owner
  • version

Discovery service

• Keyword search
• Vector similarity search
• Capability taxonomy

MCP gateway

• Normalizes tool access
• Handles authentication
• Rate limits
• Audits calls

Agent runtime

• LangGraph or Semantic Kernel
• Dynamically loads tools
• Tracks tool usage

Governance

• "This agent may call finance tools but not delete records"
• Human approval for risky actions

---

My recommendation by scenario

The key point: MCP solves tool interoperability; it does not fully solve discovery. For truly dynamic agents, add a discovery layer that sits above MCP and retrieves only the tools relevant to the current task.