Ask an engineering team what their AI workload costs and they will quote a price per million tokens. That number is real, but it is one point on a surprisingly wide surface. The same tokens, from the same model, producing the same output, are sold at meaningfully different prices depending on how, when, and where the request is executed.
We spent the last few months mapping that surface across the major providers. Three mechanisms account for most of the spread.
Batch is a contractual 50%
Every major provider now offers a batch tier at roughly half the interactive price, with a completion window measured in hours instead of seconds. This is not a promotion; it is published, contractual pricing. Any workload step that does not sit between a user tapping a key and a spinner disappearing — evaluation, enrichment, classification backfills, overnight summarization — is eligible.
In the workloads we profile, it is common for a third or more of traffic to be latency-tolerant once someone actually asks the question. That traffic is usually running at the interactive price anyway, because the integration was written once and never revisited.
Priority tiers work the other direction
The same providers sell faster, more predictable variants of the same model at roughly twice the standard price. Teams reach for priority tiers under latency pressure, and often leave everything on them long after the pressure has passed. Between the batch floor and the priority ceiling, identical tokens span about a 4x price range before you have changed a single model name.
Flexible and off-peak processing tiers sit in between: same model, discounted price, softer availability guarantees, sometimes only during defined windows. They are drop-in for traffic that can tolerate occasional queuing.
The spread across providers is larger than the spread within one
For open-weight models the same checkpoint is served by many providers, and the differences are not subtle. At a single price point we measured close to a 6x spread in serving speed for the same model across hosts — and price at a single speed varies almost as widely. Model quality is identical by construction; the execution is what differs.
None of this requires new research or new models. It requires knowing, per request, how urgent the request actually is, and having the machinery to route it accordingly — with evidence that quality held. That is an execution-policy decision, and it is exactly the kind of decision that should be continuously optimized rather than frozen into the first integration someone shipped.