vooy books taxis, finds restaurants, manages calendars, and sends email. A useful agent should get better as it learns to do more. So we kept adding tools. When the catalog approached 200, the opposite happened.
The agent could do more, but its answers slowed down and tool selection got worse. We had reached the strange point where adding a feature could make the product less capable.
More tools started producing worse answers
A model needs to see a tool's name, description, and input schema before it can call it. With a few dozen tools, putting the entire catalog into context worked fine. The setup was simple, and we always knew which capabilities the model could see.
At roughly 200 tools, that stopped working. A user could send one sentence such as "find somewhere good to eat nearby," and the model would first read the entire catalog. Tool definitions alone took about 150k input tokens per turn. Similar names and schemas pushed selection accuracy down to 0.83. As conversation history grew, some requests failed before the agent could do anything useful.
Users saw two symptoms: "Why is this so slow?" and "Why did it choose that tool?" The internal diagnosis was simpler. Showing the full tool catalog on every turn did not scale.
vooy is not alone here. Anthropic described the same problem when it introduced the Claude API Tool Search Tool. Instead of preloading everything, the model discovers tools when it needs them.
We agreed with that direction, but vooy routes across several models and handles Korean and Japanese requests alongside English. Rather than tie the runtime to one model's native feature, we rebuilt the idea around our own product constraints.
Before
Ship the full tool catalog every turn
Tool defs ~150k tokens
~200 names · descriptions · schemas
Conversation history
User utterance
After
Show only the few tools that matter
Stable surface + retrieved tools ~19k
Conversation history
Retrieved detailed schemas
User utterance
"Why not load only the relevant tools?"
That was our first attempt. We classified the request, selected a few tools, and put only those in the turn's tools array. Context size dropped immediately.
Then a different cost appeared. LLM prompt caches work best when the beginning of a prompt stays the same. The tools array usually sits near that beginning. Changing it on every turn also invalidated the cached conversation history that followed it.
For a short request, this still looked like a win. Long sessions told a different story. Cache hits fell, net cost climbed, and users who moved from calendar to restaurants to email in one conversation paid the largest penalty. That happens to be a normal vooy conversation.
This failure set the direction for the final design.
Dynamic tool loading and prompt caching naturally fight each other. Keeping both means designing not only what changes, but where change is allowed to happen.
We separated finding a tool from handing it to the model
We needed two pieces: a searcher that finds the right tool now, and a delivery layer that gives it to the model without breaking the cache.
User utterance
“Book me a taxi to Incheon Airport”
Tool search
Combine lexical and semantic matching to pick the tools needed now.
LexicalSemanticTop candidatesDelivery layer
Keep the stable surface; append only detailed schemas.
Stable surfaceSchema appendModel
See a small set, then select and execute
Fixed every turn
- Always-on core tools
- Search / execute meta tools
Added only when needed
- Detailed schemas for relevant tools
- Reused within the session
Search for how users ask, not just what tools are called
We treated tool discovery as an information retrieval problem. The user's message is the query. Tool names and descriptions are the documents. BM25 catches exact clues such as "KakaoMap," multilingual embeddings catch intent such as "somewhere nearby for dinner," and RRF combines the two rankings.
The corpus mattered more than a fancier search model. Tool descriptions usually explain what a tool does. Users do not speak that way. They say "is it going to rain today?" or "find parking near here."
We enriched each tool with realistic Korean, English, and Japanese requests. After that change, recall at five, the share of queries whose correct tool appeared in the top five results, rose from 0.33 to 0.964.
Keep the tool surface stable and append schema later
Putting search results straight into the tools array would recreate the cache problem. We separated the stable tool surface visible on every turn from the detailed schemas revealed after search.
The stable surface contains a small set of core capabilities plus the meta-tools used to search and execute everything else. It does not change during a session. When the agent discovers a tool, we append its detailed schema to the conversation side of the prompt instead of rewriting the front. The cacheable prefix remains intact, and the model can read and call the tool it just found.
A discovered tool stays available for the rest of the session and is cleaned up when context is compacted. Users only see that the agent picked the right capability. Internally, discovery and schema delivery move through different parts of the prompt.
150k input tokens became 19k
The results were clear in a synthetic benchmark using Sonnet.
Full load vs search-based
Input tokens / turn
Tool-selection accuracy
Tokens
−87%
Context headroom
Accuracy
+0.15
0.83 → 0.98
Long-session cost
~−40%
With cache intact
Tool-selection accuracy rose from 0.83 to 0.98. Input tokens fell from roughly 150k to 19k per turn, an 87% reduction, and net cost in long sessions dropped by about 40% while prompt caching stayed intact.
We also ran Anthropic's native tool search on the same evaluation set. Our system led on both overall accuracy and the Japanese slice. We believe the difference came from building the corpus around the way multilingual users actually ask for things.
Production replays were less uniform. Models with strong function calling improved the most. On some models, the result was closer to "cheaper and faster at roughly the same accuracy." We do not treat tool search as magic that makes every model smarter. We treat it as a way to protect accuracy while recovering context headroom.
Search success and execution success are different
Two early taxi requests exposed two different gaps. In the first, a tool with missing user setup disappeared from the search corpus entirely. The model did not know the taxi tool existed, so it searched 17 times.
In the second, search found the correct tool, but production made it callable on the next user turn instead of the next model step. The model had the answer and still searched 21 times because it could not use it.
The causes were different. The result for the user was the same: nothing happened.
After those incidents, we stopped measuring discovery in isolation. A tool now has to meet three conditions:
- the agent can discover it even when setup is incomplete
- the agent can call it on the next model step of the same turn
- the input schema is clear enough that the model does not have to guess
The last condition mattered more than we expected. When every tool was preloaded, runtime cleanup could hide weak schemas. Dynamic discovery turned those weaknesses into immediate failures. We tightened tool-authoring rules and automated checks alongside the search system. Tool search became a sensor for catalog quality too.
Adding a tool no longer has to penalize every turn
If vooy is going to handle more of daily life, its catalog of apps and connectors has to keep growing. In the old design, every new tool made every conversation slightly heavier. Even users who never touched that feature paid for it on every turn.
With search-based loading, growing the catalog from 100 tools to 1,000 does not make the model read all 1,000. It still sees a small active set. Adding a capability changes from "one more option that can confuse the model" to "one more thing the agent can find when it is relevant."
Deep multi-turn conversations, domain switching, and session-level optimization still leave work to do. One thing has changed already: giving an agent more tools no longer has to mean making it less smart.
Instead of making the model read 200 tools at once, we find and deliver the right one when it is needed. As vooy learns to do more, the amount the model must know on every turn can actually get smaller.
vooy
Try an agent that stays light even with 200 tools
Book taxis, find restaurants, and move across calendar and email with vooy. Be first in line.