Core concepts

Cost and quotas

Every model call writes a row. Every turn reads those rows before it is allowed to make one. That loop is the whole cost-control story, and it is closed.

The ledger

One append-only row per model call, written off the request path through a background channel — so a slow database never sits between a user and their answer, and a shutdown drains what is queued rather than losing it.

Column	Notes
`organization_id`, `location_id`, `user_id`	Attribution
`surface`	Which path spent it — chat, agent, embeddings
`model`	The model actually used, not the one configured
`prompt_tokens`, `completion_tokens`, `total_tokens`	As the provider reported them
`cached_input_tokens`	Cache reads. On a warm turn, almost all of the input
`cache_write_tokens`	Cache writes. Billed above ordinary input on some models
`reasoning_tokens`	Telemetry only — never billed twice
`estimated_cost_usd`	Computed at write time from the price table
`request_id`, `created_at`	Correlation

Reasoning tokens deserve the callout: the provider already counts them inside completion_tokens. They are stored separately so output ceilings can be tuned against real data, and adding them to the bill would charge the same tokens twice.

Pricing

Rates are per million tokens, matched by longest prefix, so a dated snapshot like gpt-4o-2024-11-20 resolves to its family. Provider prefixes (azure/, openai:) are stripped before matching.

With a provider token breakdown, the input side is split three ways and each part priced separately. Without one, every input token is charged at the dearest input-side rate for that model.

The practical consequence: a turn whose provider response omitted the token breakdown is recorded as more expensive than it really was. Early spend figures on a new model can read high for that reason alone.

The breaker

Before any model call, four independent ceilings are checked against the ledger. A cap of 0 means unlimited, which is the default for all four — the breaker ships dark until you configure it.

{
  "AI": {
    "Budget": {
      "OrgDailyCapUsd": 5,
      "OrgMonthlyCapUsd": 60,
      "GlobalDailyCapUsd": 200,
      "GlobalMonthlyCapUsd": 3000
    }
  }
}

Cap	Trips on	Code
`OrgDailyCapUsd`	One tenant's spend today	`ORG_DAILY_BUDGET_EXCEEDED`
`OrgMonthlyCapUsd`	One tenant's spend this month	`ORG_MONTHLY_BUDGET_EXCEEDED`
`GlobalDailyCapUsd`	Everyone's spend today	`GLOBAL_DAILY_BUDGET_EXCEEDED`
`GlobalMonthlyCapUsd`	Everyone's spend this month	`GLOBAL_MONTHLY_BUDGET_EXCEEDED`

Day and month boundaries are UTC. Sums are memoized briefly — 15 seconds for a tenant, 60 for the platform — so the guard costs a couple of indexed sums per window rather than per request.

The daily caps are the ones that matter. A monthly-only cap does not notice a runaway loop until the month is most of the way gone; a daily cap trips within the cache window. The platform-wide sums deliberately ignore tenant scoping and return only a boolean — no tenant learns anything about another from them.

Caps that belong to a plan

The four ceilings above are one number for every tenant, which is the wrong shape when the point of a plan is that it is capped while the others are not. PlanCaps adds ceilings keyed by plan code:

{
  "AI": {
    "Budget": {
      "PlanCaps": {
        "free": { "DailyCapUsd": 0.50, "MonthlyCapUsd": 5.00 }
      }
    }
  }
}

A plan cap and an org cap both apply and the tighter one decides. 0 is unlimited here too, and unlimited never loosens a real ceiling — a plan cap of 0.50 still trips even when OrgDailyCapUsd is unset.

A tenant that trips a plan cap gets the same ORG_DAILY_BUDGET_EXCEEDED or ORG_MONTHLY_BUDGET_EXCEEDED code as any other tenant-scoped refusal. Nothing in the response says which ceiling it was; that distinction is yours, not the caller's.

What a refused turn looks like

Not an HTTP error. On /v1/chat it is a normal 200 block document:

{
  "content_type": "application/vnd.tether.blocks.v1+json",
  "title": "AI budget reached",
  "blocks": [
    { "type": "text", "id": "budget_exceeded_1", "variant": "body",
      "markdown": "The AI assistant has reached its usage budget for now. It will be available again when the budget window resets — your data and all other features keep working." }
  ]
}

On the streaming path it is an error frame carrying the code. Either way, no model call was made and nothing was billed. Telemetry records result_path: "budget_exceeded" with the specific cap in fallback_mode.

Observability thresholds

Three optional thresholds log without ever refusing a request:

Setting	Emits
`OrgMonthlyTargetUsd`	Information — a tenant is above the planning target
`OrgMonthlyWarnUsd`	Warning
`OrgMonthlyReviewUsd`	Warning — worth a human looking

They share the enforcing check's cache key, so turning them on adds no queries. Use them to find out whether a hard cap is warranted before imposing one on a customer.

Request quotas

Distinct from dollars: a per-plan request count, checked before the spend breaker. A tenant can hit either first.

Where a turn's cost surfaces

Three places, at different grains:

The ledger — one row per model call. The billing-grade record.
Telemetry — one row per request, carrying prompt_tokens, completion_tokens and estimated_cost_usd summed across the turn's calls. The debugging-grade record.
The breaker — reads the ledger, spends nothing itself.

See Cost accounting for the queries.