Tool calling
How a model that can only write text gets real work done: it writes a request, and your software acts on it.
Read first:Next-token prediction
Step 1 of 6· Illustrative order-lookup tool, data and timings; message shapes follow Anthropic's Messages API
A language model can only produce text. On its own it cannot check a live order, query a database or send an email. Tool calling bridges that gap. The app tells the model which tools exist, each with a name, a description and the arguments it takes. When a tool would help, the model replies with a short structured request, such as get_order_status for order 4812, instead of an answer. The app, not the model, decides whether to run it, runs the real code and adds the result to the conversation. The model reads the result and answers, or asks for another tool. The model proposes; your software acts.
Why it matters for your product
Tool calling is how a model reaches live data and takes actions: order lookups, database queries, bookings. Because your code runs every call, your code is where safety lives: validate arguments, check permissions, ask a person before anything that writes or spends, and treat results as untrusted text. Each round trip adds latency and resends the tool definitions, so long chains get slow and costly. Write tool descriptions with care: they are all the model knows about when to use each tool.
For engineersShow the maths
model requests = sequential tool rounds + 1
Each time the model must wait for a result, the app sends the whole conversation back for another model request. Calls made together in one turn share a round.
Worked example: Look up an order, then look up its courier with the tracking number from the first result: 2 rounds, so 3 model requests. Asking for two orders at once in parallel is 1 round, so 2 requests.
input tokens per request ≈ tool-use system prompt + tool definitions + conversation so far
Tool definitions and the growing transcript are sent again on every request. Providers also add a hidden system prompt that enables tool use.
Worked example: Anthropic lists 286 tokens of tool-use system prompt for Claude Opus 5 with tool_choice auto. Add 20 tools at an illustrative 150 tokens each: 3,286 tokens per request before any conversation, about 10,000 over three requests (prompt caching makes the repeats cheaper).
Models still choose the wrong tool, invent arguments or skip a call they needed; the Gorilla study found models prone to hallucinating API calls, so every call must be validated. Results come back as text the model reads, so a web page or email a tool returns can carry instructions that try to steer it, known as indirect prompt injection.
The words you will hear
Tool definition
Primary sources
- 01Tool use with ClaudeAnthropic · 2026
- 02Handle tool callsAnthropic · 2026
- 03Toolformer: Language Models Can Teach Themselves to Use ToolsSchick et al. · 2023
- 04Parallel tool useAnthropic · 2026
- 05Gorilla: Large Language Model Connected with Massive APIsPatil et al. · 2023
- 06Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt InjectionGreshake et al. · 2023
Connected ideas
Agents and protocols
Model Context Protocol (MCP)
One standard plug between AI apps and the tools and data they use, and exactly what travels over it.
ExploreAgents and protocols
The agent loop
How a model works through a task: think, use a tool, read the result, and go round again.
ExploreInterpretability and safety
Prompt injection
When text the model reads on your behalf starts giving it orders.
ExploreRetrieval and search
Retrieval-augmented generation (RAG)
How a model answers from your documents, and shows you exactly where each claim came from.
ExploreWant this working on your data?
We design and build the systems these ideas power: retrieval, agents, voice and the models behind them. Start with a free discovery call.