When an AI request is slow, expensive, or surprising, “the model answered badly” is not a useful incident report. A useful trace explains what the organization allowed, what the router considered, which provider path ran, and how much the request consumed.
The evidence chain behind one request
decision pathWhat to inspect first
Start with the request class and active policy revision. Then check the candidate set: a model may be absent because of region, capability, provider health, budget, or an explicit deny rule. Only after that should you inspect provider latency and output tokens.
- Policy decision and revision
- Candidate models and scores
- Fallback reason and provider health
- Input/output/cache tokens
- Latency, cost, and final status
$ policate trace req_84f2 --jsonUse traces to improve the model mix
A trace is not just an audit artifact. Aggregate traces by task class to find expensive defaults, repeated cache misses, fallback spikes, and quality-sensitive routes. Then update a preset or policy, run a canary, and compare the new distribution against the old one.
In Gateway mode, the control plane remains authoritative. In Direct mode, the binary can still preserve a local receipt and route explanation, but the organization must accept that central redaction, budgets, and server-side audit authority are reduced.