What a request to an AI model costs | Constellation Gate AI

The short answer

A request is billed on the tokens it uses, at two different rates: one for what you send, and a higher one for what the model writes back. Output runs five to six times the price of input on current models, the whole conversation is re-sent and re-billed on every turn, and repeated content can be charged at a tenth of the normal rate if it’s cached. Those four facts explain most of a bill.

Everything else is line items, and there are more of them than most people expect.

Input and output are priced separately

The single most useful thing to know about model pricing is that it is not one number per model. It is at least two.

Anthropic prices Claude Opus 5 at $5 per million input tokens against $25 per million output. Haiku 4.5 runs $1 against $5. OpenAI is steeper still, listing its current family at a six-fold gap: $5.00 input against $30.00 output at the top, $0.20 against $1.20 at the bottom.

The ratio holds across tiers, which makes it a structural fact rather than a quirk of one price list. Reading is cheap. Writing is not.

Fig. 1Anthropic and OpenAI price lists, Aug 2026

Published rates per million tokens. Output runs five to six times input at the top of a family and at the bottom of it.

The practical consequence is that a short prompt producing a long answer can cost more than a long prompt producing a short one. Summarising a 20,000-token document into a paragraph is an inexpensive request. Asking a model to write 20,000 tokens from a one-line brief is not. If you are trying to guess where money is going before you measure it, look at reply length first.

The rates are also not permanent. Anthropic’s own table currently lists a single model twice, at $2/$10 through 31 August 2026 and $3/$15 after it, because an introductory rate is expiring. Prices move in both directions, and a cost model built on today’s number needs a date attached.

Why a conversation costs more as it goes

The bill doesn’t grow with the length of your question. It grows with the length of the exchange, because the model has no memory of its own and the whole conversation goes back on every turn.

Turn one sends the system instructions and one message, and is billed for both.
Turn two sends the system instructions, the first message, the first reply, and the new message.
Turn ten sends everything that came before it, and pays input rates on all of it, again.

Fig. 2Counted from the request shape, not measured

One block is one message. Turn four sends eight of them and pays input rates on all but the last.

Letting the provider hold the conversation changes none of this. OpenAI states it flatly: even when using its stored-conversation parameter, all previous input tokens in the chain are billed as input tokens. Server-side state saves you the trouble of resending. It does not save you the tokens.

Anything that puts content into the conversation therefore has a tail. Anthropic notes that web search results are counted as input tokens both in the search iterations during a turn and in subsequent conversation turns, which is the same effect in a less obvious costume: a document pulled in at turn three is still being paid for at turn twelve.

Caching and batching

Two discounts are large enough to change the shape of a bill, and both are documented rather than negotiated.

Caching

It re-prices repeated content. Anthropic publishes it as multipliers on the base input rate: a cache read costs 0.1x, a five-minute cache write costs 1.25x, and a one-hour write costs 2x. It also publishes the break-even. Caching pays off after a single read on the five-minute duration, or after two on the one-hour. OpenAI’s numbers land in the same place, cached input at one tenth of the uncached rate and writes at 1.25x on its current family, where earlier models had no write fee at all.

The catch is structural. Cache hits need an exact matching prefix, so the reusable material has to sit at the front of the request and stay identical, with the variable content at the end. That includes images and tool definitions. And there is a floor: OpenAI caches automatically above 1,024 tokens, and Google’s per-model minimums for context caching are 2,048 or 4,096 tokens. Below the floor, there is nothing to save.

Batching

It re-prices patience. Anthropic, OpenAI and Google all discount asynchronous work by 50% on both input and output. OpenAI’s version carries a 24-hour completion window and a separate pool of higher rate limits, and it can fail: a batch that doesn’t finish inside the window expires, returning errors rather than answers. That is the real cost of the discount, and it’s why batch suits overnight classification and not a chat interface.

The two stack. Anthropic states that caching multipliers combine with the batch discount, which makes a cached, batched request dramatically cheaper than a naive one for the same work.

The line items that aren’t tokens

A bill has more rows than input and output, and the extras are where estimates go wrong.

ThinkingReasoning the model does before answering is billed as output, at output rates. Anthropic reports it in a dedicated usage field so you can see how much of a bill was internal reasoning, and notes that it counts against the turn’s output limit. Google labels its price column “output price (including thinking tokens)”.

ToolsBuilt-in search is charged as a call and again as tokens. Anthropic prices web search at $10 per 1,000 searches plus the tokens the results add. OpenAI bills search content tokens fed in alongside your prompt, and for two named models charges a fixed block of 8,000 input tokens per call.

TimeSome things are metered by the clock rather than the token. Anthropic prices agent session runtime at $0.08 per session-hour and container time beyond an allowance at $0.05 per hour. OpenAI bills eligible container sessions by the minute with a five-minute minimum.

LengthWhether a long request costs more per token is a vendor policy, not a rule. OpenAI publishes a separate long-context tier at twice the short-context input rate. Anthropic charges one rate across its full window, stating that a 900k-token request is billed at the same per-token rate as a 9k-token one.

Where and how fastAnthropic applies a 1.1x multiplier across every token category when you pin inference to US-only, and prices its fast mode from a separate, higher table that is not available with the batch discount.

None of these are hidden. They are simply on different pages from the headline rate, which is how a project comes in at three times its estimate while every individual number was correct.

Why two teams with identical usage get different bills

Same volume of work, same model, and a gap of several times. The differences that produce it are all upstream of the price list.

Cache hit rate is the big one, because it moves a large share of input tokens to a tenth of their price. It is engineered, not granted: OpenAI documents a routing key to improve hit rates, a rough ceiling of 15 requests per minute per key before hits start being missed, and a default cache lifetime of 30 minutes. A team that puts its stable instructions first and its variable content last pays a different bill from a team that interleaves them.

Where the work runs. Classification and extraction on a small model cost a fraction of the same work on a frontier one, and the same job does not need the same model every time.

Reply length. Output is the expensive rate, so an unbounded maximum and a verbose instruction cost real money at volume.

Retries and loops. A failed request that gets retried is billed twice. An agent that takes eleven steps to do a two-step job pays for eleven, and every step carries the whole conversation so far.

The good news is that all of this is measurable rather than mysterious. Providers report token usage per request, including cache reads and writes and thinking tokens, which is what makes per-request cost attribution possible at all. Anthropic publishes a worked example of a single session where moving 40,000 input tokens onto cache reads takes the total from $0.705 to $0.525, and the arithmetic is entirely reproducible from the published rates.

Common questions

How much does one request cost?+

Multiply input tokens by the input rate, add output tokens times the output rate, then add any per-call tool charges and metered runtime. For a short exchange on a small model, fractions of a cent. For a long agent session on a frontier model with search, dollars. The variance between those two is the entire subject.

Why am I paying for my own prompt?+

Because the model has to read it before it can write anything. That includes the system instructions, the conversation so far, tool definitions, and any documents or images you attached, on every turn.

Is a bigger context window more expensive?+

Only if the vendor prices it that way. OpenAI charges a higher rate above a context threshold; Anthropic charges the same rate across its full window on current models. What is always true is that filling a bigger window means sending more tokens, and those are billed whatever the rate.

Does caching happen automatically?+

Sometimes. OpenAI and Google apply it by default above a minimum prompt length, while Anthropic’s is explicitly marked. In all three cases you only benefit if the repeated content is identical and at the front, so the automatic part gets you less than the deliberate part.

Are reasoning tokens billed even though I never see them?+

Yes, at the output rate, and they count against the output limit for the turn. Providers report the count separately so you can see what the reasoning cost.

What is the cheapest change I can make?+

Cap the maximum output length, then look at your cache hit rate. Those two require no product changes and no model migration. The full set of levers goes further, in rough order of what they cost you to adopt.

Keep learning

Cost How to reduce what you spend on AI models Models How to choose a model for a task Requests and tokens What happens when an app calls an AI model