Skip to content

Knowledge Base

Choosing AI models with OpenRouter

OpenRouter is a single connection to hundreds of AI models from many providers, with one key and one bill.

Why it matters to product people: when switching models takes minutes instead of a sprint, model choice stops being a one-time engineering decision and becomes an ongoing product decision about quality, cost, speed, reliability and privacy.

After this page you can explain where OpenRouter sits in a product, compare models without writing code, and make a model choice you can defend. The map has two parts: where OpenRouter sits (top) and the workflow I use to choose a model (bottom). Click any box for details, or walk through every part in order.

Terms used on this page

Model
The trained AI system that reads a prompt and writes a reply.
Model maker
The company that trained a model, such as Anthropic, OpenAI or Google.
Provider
A company running a model on its servers and selling access to it. It may or may not be the maker.
Token
A small chunk of text, roughly three quarters of a word. Models are priced per million tokens in and out.
API key
A secret string your code sends with each request to prove who is paying.
Credits
Prepaid US dollars in your OpenRouter account. Each request's cost is deducted.
Fallback
A backup that takes over when the first choice fails: another provider, or another model.
BYOK (bring your own key)
Using your own provider account through OpenRouter, so that provider bills you directly.
Zero data retention (ZDR)
A provider promise to store none of your prompts or replies, for any length of time.
Test set
A fixed list of real or realistic inputs you run through every candidate model to compare them fairly.
Eval (evaluation)
A structured test of an AI step: a fixed test set with expected answers, scored against bars set before you run it. The workflow on this page is a model-choice eval.
Temperature
A setting that controls how much a model varies its wording. On OpenRouter it runs from 0 to 2, and the default is 1. Rules of thumb:
  • 0: nearly the same answer every time. For tasks with one right answer, such as sorting.
  • 0.3 to 0.7: natural wording that stays factual. For replies and summaries.
  • Around 1: more variety. For brainstorming and creative copy.
  • Above 1: rarely useful; output gets erratic.
Some models ignore this setting.
Key Your product OpenRouter Model providers People and workflow steps Setup and alternative routes

Where OpenRouter sits

Requests flow down
Your product
sends requests
Every request, from code or chat, passes through
Records every request:
Done once, by you, on openrouter.ai:

My workflow

Read top to bottom

This is the eval workflow I use to choose a model for each AI step in a feature.

Where an example helps, the steps use a feature that sorts an input and then writes a reply: the sorting step and the writing step.

See this workflow applied to a real feature, with the prompts, test set and my results:

When models launch or prices change, I rerun steps 7 to 9 with the same test set.

See this workflow applied to a real feature, with the prompts, test set and my results:

Rules of thumb

  • Match the model to the task, not the brand. A sorting step and a writing step rarely need the same model.
  • One example at a time, the way the product will run. A batch pasted into one message tests something the product never does.
  • Test each step with a fixed input from the step before. Otherwise one step's mistakes show up as the next step's score.
  • Keep test inputs fixed. Same cases, same prompt, every model. Otherwise you are comparing luck.
  • Test before assuming premium wins. Cheap models often match frontier models on narrow tasks.
  • Revisit the choice on a schedule. Models and prices change monthly; a test set makes rechecking cheap.

Is OpenRouter worth it?

It tends to fit when

  • You are still exploring which models suit a feature.
  • One product uses several models.
  • Volume is small to mid, so a fee on credit purchases is minor.
  • You want provider and model fallbacks without building them.

Something else may fit better when

  • You have settled on one model for good.
  • Volume is high enough that the fee matters.
  • You need a contractual uptime guarantee or enterprise governance below the Enterprise plan.
Going direct to providers. No platform fee, and each provider's own terms. You manage a key and bill per provider and build fallbacks yourself.
LiteLLM. An open-source library and self-hosted gateway that calls 100+ model APIs in one format, with cost tracking and an admin UI. Your team runs it.
Portkey. An AI gateway with routing, fallbacks, logs and guardrails. Its gateway is open source, with a managed version and private-cloud hosting for enterprises.
Vercel AI Gateway. A hosted gateway that charges providers' list prices with no token markup, including on BYOK traffic, using prepaid credits. It does not require hosting on Vercel.

Listed neutrally, not ranked. Each was checked against its own docs on 17 October 2026.