You probably don’t need the frontier model for that

A client asked us recently why their AI feature — a system that tags incoming support tickets by category and urgency — was costing more per month than the support team's help-desk software. The answer was almost funny once we found it: every single ticket, including the ones that said things like "how do I reset my password," was being routed through the same large frontier model as the genuinely gnarly billing disputes. The model was massively overqualified for 90% of what it was doing, and the bill reflected exactly that.

This is an extremely common pattern, and it's not really about the models being expensive in the abstract — token prices have actually fallen a lot. It's that most teams default every call to whichever model is best, because that decision is easy and correctness feels safer, and then never revisit it once the feature ships. Meanwhile a small, task-specific model run on that same brief-tagging job costs a fraction as much — in the range of ten to thirty times cheaper for equivalent accuracy on a narrow task — and sits unused, because nobody built the routing logic that would let simple requests go there.

The accuracy gap that used to justify defaulting to the biggest model has mostly closed for narrow, well-defined tasks. A small model fine-tuned or well-prompted for "classify this ticket" is now, in most benchmarks we've seen and run ourselves, within a couple of points of a frontier model on that specific job — while responding faster and costing a small fraction as much. The frontier model still earns its cost on the genuinely open-ended 10 to 20% of requests: the ambiguous case, the multi-step reasoning task, the thing that doesn't fit a template. Routing correctly means most of your traffic never needs to touch it.

There are three common ways teams actually build the router, and they trade off complexity against accuracy in a fairly predictable way. The simplest is rule-based: a ticket containing certain keywords or matching a known template goes to the small model, everything else goes to the frontier model. It's cheap to build and easy to reason about, but it's blunt — it misses nuance a keyword list was never going to catch. A step up is an embedding-similarity check against a set of known-simple examples: if the incoming request is close enough in vector space to things the small model has handled well before, route it there. The most capable, and most expensive to run, is using a small, fast model as the router itself — not to answer the request, just to classify its difficulty in a single cheap call before deciding where the real request goes. That third approach is usually where the ten-to-thirty-times cost advantage of the small model actually gets realized at scale, because the routing decision itself stays cheap even as accuracy improves.

Put real numbers on it: a system handling 10,000 support-ticket classifications a day, if every one goes through a frontier model, runs somewhere in the neighborhood of $50 to $60 a day at typical current pricing. Route 80% of that volume — the tickets a well-tuned small model handles at near-identical accuracy — to a task-specific small model, and the same volume drops to roughly $10 to $15 a day, with the frontier model still handling the genuinely hard 20%. That's not a rounding-error optimization. Over a year, it's the difference between an AI feature that's a minor line item and one a finance team starts asking pointed questions about — for identical output quality, provided the routing decision itself is actually reliable.

Routing isn't free to build, and there's a real threshold below which it isn't worth it. If a feature is doing a few hundred calls a day rather than tens of thousands, the engineering time spent building and maintaining a router will often cost more than it saves — better to just use whichever model gets the job done and revisit the decision once volume actually justifies it. The same is true early in a feature's life, before you have enough real traffic to know what "simple" and "complex" actually look like for your specific use case; building a router on guessed categories is a good way to route confidently and wrongly. Routing earns its complexity at scale and with real usage data behind the difficulty classification, not as a default architecture decision made on day one.

Building that router is real engineering work, not a config toggle, which is probably why so many teams skip it. It means classifying incoming requests by actual difficulty before deciding where they go, not by which team happens to own that part of the product. It means monitoring the small model's error rate on the traffic it's handling and escalating automatically when confidence drops, rather than assuming the split you set up on day one stays correct forever. And it means being honest that "simple" and "complex" aren't fixed categories — a ticket-tagging task that's simple today gets harder the day your product adds a new category the small model was never tuned on.

The counterintuitive part is that this isn't really a cost-cutting story, even though it usually cuts cost by more than half. It's a reliability story. A small model doing one narrow job is easier to evaluate, easier to notice when it starts drifting, and easier to reason about when something goes wrong than a general-purpose model being asked to be good at everything at once. Routing by task, not by habit, tends to make a system both cheaper and easier to trust — which is a better trade than most cost-optimization exercises manage.

Model routing is also not a one-time decision, and treating it that way is its own quiet failure mode. Model capability, and pricing, both shift every few months — a model that was the clear frontier choice a year ago might now be squarely in "small model" territory in terms of relative capability and cost, while still being the model a routing config from a year ago is sending traffic to out of habit. The teams getting real, compounding value out of routing revisit the split on a quarterly cadence at minimum: re-running the small model against a held-out set of recent real requests, checking whether its error rate has drifted as the product's actual traffic mix has changed, and re-evaluating whether newer, cheaper models can now handle a larger share of what used to require the frontier tier.