Model tiering

As model pricing diversified through 2025 and 2026, the price spread between the cheapest and most expensive models in a family reached roughly 25x on input tokens alone. Running every task through a frontier model stopped being defensible, and model tiering emerged as the standard architectural response. The idea is simple: classify incoming tasks by complexity, latency requirements, and domain sensitivity, then route each task to the model tier that clears it at acceptable quality for the lowest cost.

In practice, most teams operate three tiers: a fast, inexpensive model for high-volume tasks like classification, summarization, data extraction, and formatting; a mid-tier model for everyday drafting, code completion, and question answering; and a frontier model reserved for tasks that require deep reasoning, complex code generation, or top-tier output. OpenAI made this architecture explicit by naming their tiers (Luna for cheap, Terra for mid, Sol for frontier), and other labs followed with similarly structured lineups.

Model tiering is closely related to model routing, which is the software layer that actually makes the routing decision per request. Manual tiering (teams choose which tier to call in code) is simple but brittle as models evolve. Automated routers evaluate each prompt and assign it dynamically, which adds latency but adapts as the model landscape shifts. The business impact compounds quickly: teams that get tiering right report cutting inference costs by 60 to 90% without measurable quality loss on the tasks routed to cheaper tiers. The biggest mistake is treating tiering as a one-time configuration rather than a system that needs ongoing evaluation as model capabilities change and prices drop.