Models · Decision guide

How to choose a model for a task

For operatorsFor developers 8 min read · Updated Jul 2026

On this page
  1. Start with the job
  2. What differs between models
  3. Which model to start with
  4. Decide it with twenty examples
  5. When models retire
  6. Common questions
  7. Where Gate fits

The short answer

Choosing a model is a decision about one task, not a standing commitment to one vendor. Work out what the job demands, accuracy first and then cost and speed, and pick the cheapest model that still clears the bar. The only evidence worth acting on is a set of your own test cases run against the models you are considering.

A leaderboard tells you which model was better at somebody else’s problem.

break-even at 85.8% accuracy, so the target became 90% Set the target from unit economics Build 20 examples your own cases Run candidates cheapest first Cheapest that clears per task, not per vendor nothing clears it: move up a tier a leaderboard is somebody else's problem, measured on somebody else's cases
Set the target from unit economics 85.8% to 90% Build 20 examples your own cases Run candidates cheapest first Cheapest that clears per task, not per vendor move up a tier a leaderboard is somebody else's problem
Fig. 1OpenAI model-selection guidance, Aug 2026
The method, in four steps and one loop. The accuracy target comes out of what the task is worth, not out of a leaderboard.

Start with the job

Every model vendor publishes the same advice, and it is the opposite of how most teams shop. OpenAI states the ordering plainly: optimise for accuracy until you hit your accuracy target, then aim to hold that accuracy with the cheapest, fastest model possible.

Which raises the question the ordering depends on: what is your accuracy target? It is not a number you can borrow. It comes out of what the task is worth. OpenAI works its own example through the arithmetic: a correct classification saves the cost of a human review, a wrong one costs several times that, and the two figures together set the accuracy at which the whole thing breaks even. In their case that was 85.8%, so the target became 90%.

Do that arithmetic before you look at a single model, and most of the choosing is already done. Tasks where a wrong answer is cheap and easily caught can run on a small model. Tasks where a wrong answer reaches a customer cannot.

What differs between models

Price gets the attention, and it is one line in a list of things vendors publish per model.

PriceSeparate rates for input and output. Output usually costs more. The arithmetic is in what a request costs.
RoomHow much the model can hold at once, and separately, how long a single reply is allowed to run. Both are published per model, and they are not the same number.
SpeedVendors rate models by latency class, and speed is sometimes sold on its own: Anthropic offers a fast mode delivering up to 2.5x higher output speed at premium pricing.
What it acceptsText always. Images, PDFs, audio and video vary by model. Google publishes the accepted input types per model alongside the token limits.
What it can doFunction calling, structured output, code execution, search grounding, computer use. These are per-model support flags, not a family-wide guarantee.
ThinkingSome models reason before answering. It costs: Google labels the relevant pricing row “output price (including thinking tokens)”.
What it knowsTraining stops on a date. Anthropic publishes two of them, a training data cutoff and an earlier reliable knowledge cutoff, defined as the date through which a model’s knowledge is most extensive and reliable.
What you can getRate limits differ between tiers of the same family, so a cheaper model can also be the one that lets you run more of them at once.

Which model to start with

Two vendors give opposite starting advice, and both are right about different work.

Start with the smallestStart with the strongest
The moveBuild on the fastest, cheapest model and upgrade only where it visibly failsBuild on the frontier model, then swap in a smaller one and check whether accuracy holds
Whose adviceAnthropic calls this efficiency-firstOpenAI recommends using the most capable model to reach the target first
FitsPrototyping, tight latency budgets, high volume, straightforward tasksComplex reasoning, work where accuracy outweighs cost, high-autonomy agents
The riskYou conclude the task is impossible when the model was simply too smallYou ship at four times the necessary price and never find out

Anthropic names the same split from the other side: capability-first, meaning implement with the strong model, applies to scientific work, nuanced understanding, and advanced coding. The deciding question is whether you already know the task is achievable. If you do, start cheap. If you don’t, start strong, because a failure on a small model tells you nothing about whether the job can be done at all.

Some models accept an effort setting that trades intelligence against latency and cost inside a single model, and Anthropic’s own guidance is that tuning effort is often a better lever than switching models. Try it before you migrate anything.

Decide it with twenty examples

Every vendor’s guidance converges here. Anthropic puts it bluntly: having a good evaluation set is the most important step in the process.

  1. Write the success criterion so it can be checked. Not “good performance” but the specific thing you want, with a number attached and a test set to measure it on.
  2. Collect real cases, including the awkward ones. Anthropic’s guidance is to mirror your real-world task distribution and to factor in edge cases, which is where models differ from each other.
  3. Favour volume over polish. More cases with automated grading beats a handful of carefully hand-graded ones. Automated and slightly noisy wins because you can re-run it.
  4. Run the candidates on identical prompts. Same inputs, same instructions, same grader. A prompt tuned for one model flatters that model.
  5. Compare against your break-even accuracy, not against each other. The question is which models clear the bar, and then which of those is cheapest.
  6. Keep it. The set you built to choose a model is the set that tells you whether next quarter’s model is an upgrade, and whether a prompt change quietly broke something.

Twenty to fifty real cases is usually enough to separate candidates. That is an afternoon, and it replaces an argument about benchmarks with a number about your own work.

When models retire

The model you choose has an end date, and how much warning you get varies by vendor. That rarely makes it into the decision.

OpenAI commits to at least six months for generally available models and at least three for specialised variants, but preview models “may be retired with much shorter notice, such as 2 weeks”, and its own docs advise against preview models for business-critical work unless you can migrate quickly. Anthropic commits to at least 60 days for publicly released models, and is explicit about what retirement means: requests to a retired model fail. Google publishes shutdown dates but describes them as the earliest possible dates on which a model might be retired, and its own table shows preview models released and shut down inside four months.

Three practical consequences. Model names belong in configuration, not scattered through application code, because you will change them on someone else’s schedule. Version aliases are not the escape hatch they look like, since Anthropic warns that even its dateless model IDs are pinned snapshots rather than evergreen pointers. And where you buy matters: Anthropic notes that partner platforms such as Amazon Bedrock and Google Cloud set their own retirement schedules, so the same model can have different end dates depending on the route you take to it.

When a retirement notice does land, the awkward question is which of your systems still calls the old model. That is answerable if request logs record the model name per call, and a research project if they don’t.

Common questions

Should I just use the most capable model available?+
It is a reasonable place to start and a poor place to stop. OpenAI’s own recommendation is to reach your accuracy target on the strongest model and then test whether a smaller one holds that accuracy at lower cost and latency. In its worked classification example the smaller fine-tuned model matched the frontier model’s accuracy at a fraction of the cost.
Do benchmark scores matter at all?+
They narrow the field and they don’t settle it. A published score measures a fixed task set, and your task is not in it. Every vendor’s guidance points the same way: build a small evaluation set from your own cases and test the shortlist against that.
How many models should we be using?+
As many as the work differs. A cheap model for extraction and classification, a strong one for multi-step reasoning and code, is a common and defensible split. The cost of running several only stays low if adding one is configuration rather than an integration project, which is what an AI gateway is for.
What is a knowledge cutoff, and when does it bite?+
It is the date the model’s training data ends. Ask about anything after it and the model has no knowledge to draw on, though it may answer anyway. Anthropic publishes two dates per model, the broader training cutoff and an earlier date through which knowledge is most reliable. For anything current, supply the facts in the request rather than trusting recall.
Can we switch models later without rewriting everything?+
Largely, if you set it up that way. Requests to different models look similar enough that the model name can live in configuration, and the practical work is re-running your eval set and adjusting prompts. What makes switching expensive is a model name hard-coded in a dozen places.
How often should we revisit the choice?+
When a retirement notice arrives, when your eval set shows a new model clearing the bar more cheaply, or when the task itself changes. Rechecking every release costs more time than it saves. Rechecking never is how you end up paying frontier prices for classification.

Keep learning

Cost What a request to an AI model costs Models Open-weight and closed models Models Why models get things wrong
Back to top