Cost Optimization

Most LLM spend hides in a few callsites: structured extraction, classification, and short transformations running on a frontier model because that's what the team started with. These are exactly the callsites where smaller models tend to hold quality — and where replay can prove it cheaply, because outputs are short and compliance is checkable.

The profile job makes this visible: cost per run, decomposed by callsite, from your real traffic. Triage then sorts callsites into candidates worth testing, risky candidates, and anchors to leave alone.

Run the pipeline per workflow, in order:

1. profile    know your cost per run and its breakdown
2. triage     shortlist movable callsites; protect anchors
3. bar        freeze the quality bar from baseline runs
4. calibrate  verify the judge is unbiased on your traffic
5. replay     test candidates within a fixed budget
6. plan       accept only routes with evidence attached
7. canary     apply at 10–25% of runs, hashed per run
8. validate   score routed runs against the frozen bar
9. active     promote — the tripwire keeps watching

Because every call logs both requested_model and routed_model, realized savings are computed from actual traffic, not projections. The plan's projection gives you the expected number (savings_pct); validation gives you the realized cost per routed run; the report ties them together with the quality outcome.

Pricing comes from live provider catalogs (cached six hours). Models without published pricing are flagged in the profile's pricing notes rather than silently costed at zero.

A healthy plan often routes only some callsites. Common reasons a candidate is rejected — all recorded in the plan notes: compliance below threshold on schema-strict output, quality proxy under the floor, latency ratio above your cap, or the callsite being a quality anchor. An empty plan is a valid, useful result: it means the evidence says your current models are earning their cost.