How tool calling works | Constellation Gate AI

The short answer

Tool calling is how a model asks your application to do something: it returns a structured request to run a named operation, your code runs it, and the result goes back into the conversation. The model never executes anything itself. It chooses what to ask for, based on nothing but the tool descriptions you supplied.

That division of labour is the entire mechanism, and most confusion about agents dissolves once it’s clear.

The round trip

A tool call is not one request. It is at least two, with your code in the middle.

You send the request with a list of tools. Each tool carries a name, a description, and the shape of its inputs.
The model answers with a call instead of an answer. The reply comes back flagged as a request to run something rather than as an answer, naming the tool it wants and the values to pass it, as Anthropic sets out.
Your application runs it. OpenAI states the obligation plainly: when the model calls a function, you must execute it and return the result.
You send everything back. The original messages, the model’s request, and the result, as a new request.
Repeat while the model keeps asking. The loop ends when the model returns an ordinary answer instead of another call.

Fig. 1

Execution happens on your side of the line, which is where permission lives.

Anthropic’s own summary of why it works this way: “The model can’t run your code, so every tool call is a round trip: the model asks, you execute, you report back, the model continues.”

Who does what

	The model	Your application
Decides which tool to use	Yes, from the descriptions	No
Decides the arguments	Yes	No, though you validate them
Runs the operation	No	Yes
Decides whether it’s allowed	No	Yes. This is where permission lives
Sees the implementation	No, only the schema and the result	Yes
Ends the loop	Signals it	Enforces it

The two rows worth dwelling on are the middle ones. Every action taken in an agent’s name is an action your code chose to perform, which means the security boundary is not inside the model. It’s in the function that decides whether to honour the request.

Where tools run

Not every tool executes on your side. Anthropic documents three arrangements, and the difference decides what you’re responsible for.

Tools you defineYou write the schema, your code executes, you return the result. The common case, and the one where the model “never sees your implementation; it only sees the schema you provided and the result you returned”.

Vendor-schema toolsThe provider publishes a standard schema for common operations like shell commands or file edits, and your application still does the execution.

Server toolsSearch, code execution and similar run on the provider’s infrastructure. You get results without handling execution, and the provider runs its own internal loop with an iteration cap, signalling a paused turn if it hits the ceiling before finishing.

The last one is easy to miss on a bill and in an audit, because work happened several times over inside a single request you made.

What the model knows about a tool

Very little. The consequences follow from that.

The model chooses a tool from its name and description. Anthropic puts it plainly: the model “determines when to call a tool based on the user’s request and the tool’s description”. OpenAI’s best practices follow from that, telling developers to describe explicitly what the function is for, what each parameter means, and to use the system prompt to say when not to use it.

Which means a tool description is not documentation. It is instructions the model will act on, sitting in the same channel as everything else in the request. The MCP specification draws the security conclusion for tools arriving from elsewhere: “descriptions of tool behavior such as annotations should be considered untrusted, unless obtained from a trusted server.”

Tool definitions also cost. OpenAI states that callable function definitions count against the model’s context limit and are billed as input tokens, on every request, whether the model uses them or not.

Where it goes wrong

Too many tools. OpenAI recommends keeping the initially available set small for accuracy, suggesting fewer than twenty at the start of a turn. A sprawling tool surface costs tokens and makes wrong selections more likely.
Vague descriptions. The model picks from the text you wrote. Two tools with overlapping descriptions produce a coin flip.
Assuming one call per turn. A response can contain zero, one, or several calls, and OpenAI’s guidance is to assume there are several. Code that handles only the first quietly drops work.
Trusting the result. Whatever a tool returns goes straight into the model’s context and shapes what it does next. If the tool fetched a web page or read a document, that content came from outside your control, which is the injection problem.
No ceiling. The loop is driven by your code, so an unbounded loop is your bug rather than the model’s.

Common questions

Does the model run my code?+

No. It returns a structured request naming a tool and its arguments; your application decides whether to execute it. That is worth being precise about, because it means every action an agent takes went through code you control.

What is the difference between tool calling and function calling?+

Nothing meaningful. OpenAI’s documentation treats them as the same thing, and the industry uses both names for the same mechanism.

How does the model know which tool to pick?+

From the name and description you supplied, weighed against what the user asked for. There is no discovery beyond that, which is why description quality is the main lever on tool-use accuracy.

Do tools slow things down?+

Yes, unavoidably. Each call is an extra round trip plus the time the operation itself takes, and the conversation grows each time, so later steps are also larger requests.

Is this how agents work?+

It is the mechanism underneath them. An agent is this loop, run repeatedly, with the model deciding each next step until a stopping condition ends it.

Do I have to write a tool for every system I want to connect?+

Not necessarily. That integration burden is what MCP exists to reduce, by standardising how tools are exposed so the same server works with any client that speaks the protocol.

Keep learning

Agents and tools What an AI agent is Agents and tools What MCP is Requests and tokens What happens when an app calls an AI model