
Gemini API Tool Retrieval: Load Tool Schemas Only on Demand
Gemini API Tool Retrieval lets agents keep hundreds of tools on standby and load schemas only when needed, cutting input tokens at no separate charge.
Gemini API Tool Retrieval is Google's answer to a problem every agent builder eventually hits: the more tools you connect, the more tokens you burn describing them. Announced on November 19 and available in preview through the Interactions API, the feature lets an agent keep its full tool library on standby and pull in a tool's parameter schema only when the task calls for it.
- What it does: defers tool parameter schemas so they stay out of the prompt until the model asks for them.
- How to enable it: add a tool_search tool and set defer_loading to true on the tools you want held back.
- Cost: no separate charge; loaded schemas are billed as normal input tokens.
- Model support: Gemini 3.6 and later, including all Gemini 4 models.
How Does Gemini API Tool Retrieval Work?
Normally, an agent with 50 functions sends all 50 full schemas on every request. That inflates input token counts, adds latency and, as Google's post notes, lowers tool-selection accuracy as the list of candidates grows.
With Tool Retrieval, the model still sees the name and description of every deferred tool, so it knows what exists. When it decides it needs one, it requests that tool by name, the API loads the full schema, and the model makes the function call, all inside the same request. Each step shows up in the response, which helps when you are debugging why a tool was or was not picked.
There is no index to build or host in the default mode because Gemini runs the search on the server. Google suggests keeping your most-used tools loaded up front and deferring the long tail.
Does It Work With MCP Servers and Caching?
Yes to both. Setting defer_loading on a remote MCP server tool means every tool that server exposes is loaded on demand, so a large server can be connected without listing its tools individually. If you are wiring up third-party servers, our MCP server security checklist is worth a read first.
The feature is also designed to cooperate with prefix caching. Deferred schemas stay out of the start of the prompt, and anything the model loads is appended later in the conversation, so the cached prefix stays stable from turn to turn. That pairs neatly with the techniques in our prompt caching explainer.
Teams with their own tool registry, embedding index or per-user permission filter can run the search client-side instead. That adds one round trip per search, but it lets an app expose new tools mid-conversation, for example right after a user connects an integration.
Will Tool Retrieval Save You Money?
Usually, though Google is careful not to promise a number. Turning on tool search adds a small fixed block of input tokens, and each schema the model loads is billed for the rest of the conversation. You come out ahead when the schemas you never load outweigh the ones you do, which the company calls the usual case for large libraries. Its advice is to measure against your own traffic.
For developers building AI agents with sprawling toolsets, this is a practical quality-of-life upgrade: smaller prompts, steadier caching and a cleaner path to connecting big MCP servers. It is a preview on the v1beta API, so expect the surface to shift before general availability.
Sources: Google Blog — November 19, 2026; Gemini API documentation — November 2026.
More AI Stories

OpenAI Chip Design Results: Up to 14% PPA Gains on AMD GPU
OpenAI says its models delivered up to 14% better power, performance and area on AMD's flagship GPU design, and AMD has sent some changes to the fab.

Qwen 4 Max Explained: 2.4T Parameters, $1.50 API Pricing
Alibaba's Qwen 4 Max pairs 2.4 trillion parameters with a 1M-token context at $1.50 per million input tokens. Here is what developers get today.

Gemini 4 Neon: Carbon-Class Smarts at $3 Per Million Tokens
Gemini 4 Neon scores 61 on the Artificial Analysis index at $3 per million input tokens and cuts cost per task 32% versus Carbon. See who should switch.
