The reason most teams run production AI on an expensive default is not ignorance — everyone suspects a cheaper model would probably be fine. The reason is risk. “Probably fine” is not a standard anyone wants to defend after a quality incident, and a hand-run eval on a synthetic dataset does not move that conversation much.
Valite's answer is to make the evidence come from your own traffic, and to make it arrive before any production request is touched.
Capture, then fingerprint
The gateway relays traffic byte-for-byte and logs every call off the hot path: model, tokens, latency, cost, full request and response. From that log, Valite fingerprints calls into callsites — the distinct prompts your code makes. A support agent might have a triage callsite, a drafting callsite, and a summarization callsite, each with different difficulty and different headroom.
That distinction matters because savings are not uniform. A model that fails your hardest callsite is often flawless on your easiest ones. Optimizing per callsite is where most of the recoverable margin lives.
Replay against your own baseline
For each callsite, Valite assembles a golden set of representative production requests and replays it against candidate policies — different models, providers, tiers, parameters — entirely offline. Every candidate response is graded against your captured production baseline by a blinded, calibrated pairwise judge. The question is never “is this good in the abstract,” it is “is this at least as good as what you ship today, on your requests.”
Candidates that miss the quality floor are discarded, whatever they cost. Candidates that hold it are ranked by economics, and the result is a per-callsite plan with the evidence attached: replay counts, judged worse-rates, and the projected savings.
Promotion is guarded, not brave
A proven plan still does not touch traffic by itself. Activation runs through shadow mode and canary percentages, with error-rate and quality tripwires comparing the new policy against baseline continuously. A failed threshold triggers automatic rollback to the exact prior configuration.
In our internal benchmarks across real open-source agent workloads, replay-proven routing cut per-run cost between 17% and 97% — the spread itself is the point. Nobody can tell you your number from a pricing page. The pipeline exists so that your number arrives as a proof, not a guess.