Pattern · last reviewed 2026-11-01

Your cost meter is a copy of the price list

Context & problem

Spend control for an LLM feature usually starts as a small function: multiply input tokens by an input rate, output tokens by an output rate, add them up, store the sum per user, refuse past a limit. It is correct on the day you write it. After that it is a statement about a document you do not own.

Three things make that statement go false without anyone touching your code. Vendors change a rate. They add a token category that is billed differently from the ones you model. They introduce a model id your table has never seen. In each case the function keeps running, keeps returning plausible numbers, and keeps feeding a cap that now means something slightly different from what you wrote on the sticky note above it.

You can see the same dependency from the other side. Whatever sits downstream of the meter inherits its errors: the per-user cap, the figure you show a member after a turn, the price table you publish. If those three read from one rate table, one stale entry is wrong in three places at once, and none of them has any way to notice.

This page is about the check that notices. It is deliberately narrow. How a team finds out that a vendor changed a number, and how wide to search once it has, is a weekly practice covered in the vendor change triage workflow; that page owns the story of one cache-rate change in detail and this one only uses it as a data point. How a ceiling is enforced, and how reserving an estimate before a call differs from recording the actual afterwards, is in enforceable spend ceilings. What remains once both are in place is the question of whether the number the ceiling reads is right, and how you would find out.

Forces

• You need a number at call time, and sometimes before it. A cap that waits for the provider's report is a cap that reads yesterday, and reserving an estimate ahead of a call needs a rate table by definition. That is why the copy exists at all. Where the response carries its own cost you can record that instead, which takes the table off the enforcement path but not off your list of things to audit.
• The provider's number is authoritative and late. It may arrive on the response, which is fast, or in a usage report that lags and gets revised, which is slow. Either way it answers a different question from the one the cap asks.
• Errors are not symmetric. An overcount costs goodwill. An undercount costs money. A check that treats both as "mismatch" will page for the cheap one as loudly as for the expensive one, and will be muted within a month.
• Units and scope differ between the two numbers. Credits against dollars, cents against dollars, list price against after-discount, your traffic against everything on the account. A comparison that ignores these reports drift on day one and teaches everyone to ignore it.
• Some gaps are latent. A model that no route calls yet can sit mispriced in the table for months. The gap is real and the cost is zero, until a one-line routing change makes it live.

The pattern

Four decisions, in the order they pay back.

• Write the direction next to the gap. For every place the meter is known to disagree with the vendor, record whether it overstates or understates, and what happens to the cap in each case. Do this before you build anything else, because it tells you which gaps are urgent. The table below is the shape of it.
• Keep the provider's figure next to yours, twice. Per call, when the response or gateway carries a cost: store your computed cost and theirs and keep the ratio. This catches a stale rate or a missing bucket within hours, but it only gives you a total per call. Per period, from the billing or usage report: this is slower and revised after the fact, but it breaks cost down by token type and sees traffic that never touched your meter, such as a script with its own key. The two checks catch different failures; running only one leaves the other class invisible.
• Enforce at call time, audit on the slow report. Do not move the cap onto a periodic usage report to fix a drift problem. It arrives after the spend it would have to prevent, and a hot path that waits on a usage report is a new outage. Recording the cost a response reports about itself is a legitimate option, and the as-built section shows one use of it. It still leaves you without an estimate before the call and without any figure for calls that return nothing. The reconciliation exists to correct the table, not to replace the meter.
• Make the alert asymmetric and normalise before comparing. A ratio of your cost to theirs below one means you undercount: page someone. Above one means you overcount: open a ticket. Convert both sides to the same currency unit, the same price basis and the same scope first, or the ratio measures your conversion mistakes rather than your table.

How a drift errs

Each row is a way the meter can disagree with the price list. The examples are from this platform's own meter; the last column says whether the gap was ever live.

Of the five cases in the table above (the price-rise row has no instance), two were live, one in each direction. The cache-read overstatement failed safe. The cache-write undercount ran the expensive way, and we inferred it from documentation rather than measured it. One case was latent and is now closed, and two are still latent: each costs nothing until a routing change or a vendor update makes it live. That is the useful finding. The meter flagged neither live drift: a vendor note found one and a code reading found the other. Several more wait on a future decision, and nothing would tell us when one of them switched on.

Reference implementation notes

Neither check is large. Most of the work is deciding what to compare, and the two sources have traps in the comparison itself.

Per call: the cost on the response

OpenRouter returns usage on every response without an opt-in. Its usage-accounting page lists usage.cost as the total amount charged to your account and labels it cost in credits, alongside prompt_tokens_details.cached_tokens and prompt_tokens_details.cache_write_tokens on the Chat Completions shape. The older request flag for enabling this is documented as deprecated with no effect. The write counter appears only for models that have explicit caching with a write price. There is also a cost_details.upstream_inference_cost field, which the page says, when you fetch usage by generation ID, is only populated for bring-your-own-key requests (0 or null otherwise).

The comparison is a few lines. The part that matters is what it refuses to compare:

// one row per call; ratio below 1 means the meter undercounts
// both amounts in the same unit: convert credits or cents first
function driftOf(aRoute, aModel, aLocalUsd, aProviderUsd) {
  if (!(aProviderUsd > 0) || !Number.isFinite(aProviderUsd)) return null;  // unreported or free: nothing to compare
  if (!Number.isFinite(aLocalUsd)) return null;
  return { route: aRoute, model: aModel, ratio: aLocalUsd / aProviderUsd };
}

The guard is written as "is it greater than zero" for the reason given in the spend-ceiling page: an undefined or NaN value fails every comparison, so only a real positive number passes, and a ratio with a missing denominator should not be a number.

Two things limit what this tells you. A total per call cannot say which token category is off, only that something is, so it is a tripwire rather than a diagnosis. And the page does not say whether cost already reflects cache discounts, so do not assume your own arithmetic and the response should agree to the cent. Measure the baseline ratio on your own traffic before you set a band, and set the band from that.

Per period: the usage and cost report

When your organisation holds the key that the provider bills, the provider's usage report is the other reference. Anthropic's Usage and Cost Analytics API is in beta. Every request carries the header anthropic-beta: analytics-usage-cost-2026-08-24 together with an anthropic-version header, and the endpoints are GET /v1/organizations/analytics/usage and GET /v1/organizations/analytics/cost. For a Claude Console organisation it needs an Admin API key or another admin credential, not an ordinary inference key.

The useful part for this pattern is the grouping. The cost endpoint accepts group_by[]=token_type and group_by[]=cost_type, so you can see cost per token category per model. That is the view that tells you a per-call ratio of 0.9 is a cache-read rate and not an output rate. Its usage response also splits cache creation into five-minute and one-hour counts, a distinction a single combined counter cannot carry.

The traps are all in the comparison:

• Units. The cost endpoint returns, in the page's words, decimal strings in cents with six decimal places; the page's own example reads "41280.000000" as $412.80 USD. A meter in dollars compared with that string is off by a factor of one hundred, which is the kind of mismatch that gets the check disabled. OpenRouter's figure is in credits.
• Price basis. Each row has amount, defined as the cost after discounts and before any credits are applied, and list_amount, defined as the cost at list prices. A meter built from list prices belongs next to list_amount. Comparing it to the discounted figure produces a permanent gap that is a contract, not a bug.
• Coverage. The report counts requests served under a priority-tier commitment as zero token cost. If any of your traffic runs under one, the vendor's total will sit below your meter for a reason that has nothing to do with your table.
• Lag and revision. The report is a moving target, so a reconciliation job has to know what it can trust. Each response names a data_refreshed_at cut-off, and anything newer than it has not fully landed yet. Do not judge the newest hours of a window; the vendor side is incomplete there, so your meter looks like it overcounts when it does not. Compare days that end before a stamp you have already stored, and treat a month as open until roughly a week after it closes, since the page says totals can still be revised until then.

Reconcile against whoever bills you

The Anthropic report describes the Anthropic organisation that holds the key. If your Claude calls go through an aggregator or a gateway that holds its own account, Anthropic's report is that intermediary's usage, not yours, and the figure you owe is on the intermediary's side. The choice of reference is decided by the invoice, not by which vendor made the model. Check which party sends the bill before you pick a source.

Trade-offs

• You now run two ledgers. The second one is read-only and off the hot path, but it needs storage, a job and an owner. Without the owner it becomes a table nobody has opened since the week it was built.
• The reference is itself a claim. The provider's figure can be revised, can exclude things you pay for, and can disagree with an invoice that includes credits and commitments. Reconciliation moves your doubt to a better-informed number; it does not remove it.
• The check can be loudly wrong in the safe direction. An overcount is cheap, and a design that pages on it burns attention that the undercount needs. That is why the alert is asymmetric. It also means a fail-safe drift can sit unfixed for days, by design.
• Per-call ratios are noisy for small calls. Rounding on a fraction of a cent swamps the signal, so you aggregate by route and model over a window before you judge. That delays the tripwire, which is the opposite of what a per-call check promised.
• A baseline has to exist before a threshold does. Choose the band from your own measured ratio on normal traffic. Any number written here instead would be a guess presented as a rule.

When not to use this

When the bill cannot exceed a fixed ceiling you do not set. A flat-rate plan, a prepaid balance that stops serving at zero, or a provider-enforced hard cap makes your meter a display feature. An undercount there costs you a wrong number on a screen, not money, and a second ledger is not worth running for it.

When the spend is too small for the check to cost less than the drift. A prototype that spends a few dollars a month has no use for per-route ratios. Read the vendor's dashboard monthly and move on.

When there is no number to reconcile against. If your gateway returns no cost on the response and you hold no key that can read a usage report, the only available reference is the invoice. A monthly manual comparison of invoice to meter is still worth doing; it is just not this pattern's automated form.

When the problem is a runaway, not a rate. A meter that is perfectly accurate does nothing for an agent stuck in a loop, because it reports each call correctly while the calls keep coming. That is a bounded loop problem, and reconciling the numbers afterwards only confirms the damage.

And never as a replacement for the cap. If the reconciliation is the only thing between a drift and the bill, it needs to run as fast as the spend. Otherwise it is a report on what already happened.

As-built: what this platform runs

The meter is costOf in src/lib/cost.ts: a PRICING table keyed by model id, a default cache-read multiplier of 0.1 that individual entries override, a flat 1.25 multiplier on cache-creation tokens that the file documents as the five-minute rate, and a clamp that turns any non-finite or negative token count into zero. An id missing from the table falls back to Sonnet 5's rates. The coach adds each turn's cost to a CostLog and src/coach/CoachAgent.ts passes costLog.total() to recordSpend in src/lib/spendCap.ts. So the per-user cap is only as correct as the table and the token counts mapUsage feeds it. The same table also feeds the Claude rows of the public LLM pricing tracker through src/lib/pricing/seed.ts, which means the cap, the figure a member sees and the page you can read are one copy, not three.

Two drifts were live, one in each direction. The first is the Sonnet 5.5 cache-read rate, in the first row of the table above: an overstatement, so the safe direction, caught on 2026-10-19 and now pinned by a test. The triage workflow tells how it was found and how widely it was restated. Our weekly sweep first recorded it on 19 October, twelve days after the vendor note. Nothing in the meter would have shown it.

That is the honest state of reconciliation here: we do not run it. mapUsage in src/lib/llm.ts reads the response's prompt tokens, completion tokens, cached prompt tokens and cache-write tokens, and it does not read usage.cost. Until the November 2026 fix it skipped the cache-write count as well. That is the second live drift, in the last row of the table. OpenRouter's usage-accounting example places written tokens inside the prompt count, and we mark every Anthropic call for caching. On that reading, each written token was priced at the plain input rate instead of the 1.25x write rate: an under-priced bucket, so the meter under-recorded. If written tokens sit outside the prompt count instead, the old code missed them entirely, which is the same direction and a larger gap. We found it by reading the code against the provider's documentation, not from a recorded response, and the meter now maps the write count to the write rate. On the other reading the fix still under-records each written token at its input rate, which only a recorded response would settle. Nothing on the model-call path in src/ reads usage.cost, as of a search on 2026-11-01. The one reader in the repository is the image-generation script, scripts/graphics/generate-background.mjs, which adds up OpenRouter's per-image usage.cost against its own monthly cap. It enforces on the provider's figure and keeps no local rate table, so there is no copy to drift. That works there because each image's cost returns on its own response before the next request is made, the cap is checked once before a batch with up to one batch of overshoot accepted, and a missing figure stops the script instead of counting as zero. Our Claude traffic reaches the model through a gateway and OpenRouter, so the per-call figure to compare against is OpenRouter's, and Anthropic's organisation report would show that intermediary's usage, not ours. We make no claim that any spend rule is configured on our gateway: that is dashboard state, and nothing in this repository can read it. The spend ceilings we can evidence are the ones in src/lib/spendCap.ts and the daily counters in src/lib/rateLimit.ts.

Two live gaps are not in the table above, and both undercount. When the injection judge's call aborts or errors it may still have been billed upstream, but no usage reaches us, and the code comment calls this a known accounting gap that it chose not to patch with an estimate. When the coach's output scanner blocks a reply mid-stream, the stream ends before usage is delivered, so nothing is recorded for the call that was cut off. A per-call ratio cannot see either, because no response comes back to compare. Only the per-period report can.

The latent rows in the table above are from comments in src/lib/cost.ts. The Haiku 5.5 tier is guarded only by a note to add a tier before routing a role to that model. The injection-judge ratio has no guard of ours: its comment records that a vendor flag change, with no code change on our side, would make the gap live. A note is a reminder to a person. A per-call ratio is the version that does not depend on someone remembering.

Changelog

• 2026-11-01 — Initial publication. OpenRouter usage accounting and Anthropic's Usage and Cost Analytics API read 2026-11-01. As-built claims verified against the tree the same day.