Operations

Telemetry

An assistant that behaves oddly is not debuggable from logs. ai_request_telemetry records the decisions a turn made, so "why did it not use my tool?" has an answer.

Where it lives

One row per request in ai_request_telemetry, written best-effort off the request path. Loss-tolerant by design: analytics must never take a turn down. Indexed on (organization_id, created_at) and on result_path.

Enabled with AI:Telemetry:Enabled, which defaults to on.

The fields

Identity

Field	Notes
`request_id`	Correlates with `ai_usage_record` rows for the same turn
`session_id`	The conversation thread
`organization_id`, `location_id`	Tenant and scope
`endpoint`	Which API surface
`capability_mode`	`standard` or `deep`, after clamping

Routing — why the turn did what it did

Field	Notes
`intent`, `route`, `route_kind`	The deterministic classification
`matched_alias`	The alias key that matched, when one did
`positive_signals`, `negative_signals`	Which rules fired, and which downgraded a route
`routing_confidence`	Margin between the top and runner-up route, 0–1
`used_ambiguity_fallback`	Low confidence triggered clarification or a tie-breaker
`used_prior_session_context`	An elliptical follow-up reused the previous turn's route
`deterministic_bypass`	The turn was answered without a model call
`selected_surface`, `surface_from_header`	Client surface, and whether it was explicit

Tools and grounding

Field	Notes
`tool_count`	Tools actually called
`selected_tool_count`, `selected_tool_domains`	Tools offered to the model, and their domains
`context_sections`, `context_cache_hit`	Pre-turn context assembly
`retrieved_chunk_count`, `retrieved_chunk_ids`	Retrieval, capped at 12 ids
`source_types_used`	Which grounding sources contributed

Latency, in milliseconds

scope_ms, context_ms, history_ms, openai_ms, validation_ms, total_ms.

Outcome

Field	Notes
`schema_valid`	Whether the model's output validated first time
`result_path`	`ok`, `out_of_scope`, `budget_exceeded`, `error`, `canceled`
`fallback_mode`	How it degraded — including the specific budget cap that tripped
`model`	`none` on a turn that never called a model
`prompt_tokens`, `completion_tokens`, `estimated_cost_usd`	Summed across the turn's calls

The four questions it answers

"Why did it not use my tool?"

SELECT route, intent, selected_tool_count, selected_tool_domains, tool_count,
       routing_confidence, negative_signals
FROM ai_request_telemetry
WHERE organization_id = 42 AND created_at >= now() - interval '1 hour'
ORDER BY created_at DESC
LIMIT 20;

selected_tool_count = 0 means the router decided the turn did not need tools — a known weakness for domains outside the engine's origin. selected_tool_count > 0 with tool_count = 0 means the tools were offered and the model chose not to call them, which is a description problem, not a routing one.

"Why is it slow?"

SELECT date_trunc('hour', created_at) AS hour,
       percentile_cont(0.5)  WITHIN GROUP (ORDER BY total_ms)  AS p50,
       percentile_cont(0.95) WITHIN GROUP (ORDER BY total_ms)  AS p95,
       percentile_cont(0.95) WITHIN GROUP (ORDER BY openai_ms) AS p95_model,
       avg(context_ms) AS ctx, avg(history_ms) AS hist
FROM ai_request_telemetry
WHERE created_at >= now() - interval '1 day'
GROUP BY 1 ORDER BY 1;

total_ms far above openai_ms puts the time in context assembly, history, or your tool endpoint.

"Is the output contract holding?"

SELECT date_trunc('day', created_at) AS day,
       count(*) FILTER (WHERE NOT schema_valid)::numeric / count(*) AS invalid_rate,
       count(*) FILTER (WHERE fallback_mode IS NOT NULL) AS fallbacks
FROM ai_request_telemetry
WHERE created_at >= now() - interval '14 days'
GROUP BY 1 ORDER BY 1;

A rising invalid_rate after a model change is the earliest signal you get that answers are degrading, and it arrives before anyone complains.

"What is failing?"

SELECT result_path, fallback_mode, count(*)
FROM ai_request_telemetry
WHERE created_at >= now() - interval '1 day'
GROUP BY 1, 2 ORDER BY 3 DESC;

`result_path`

Value	Meaning
`ok`	Normal, including turns that never called a model
`out_of_scope`	Policy refused before any spend
`budget_exceeded`	The breaker refused. `fallback_mode` names the cap
`error`	The turn failed. `fallback_mode` is `rate_limited`, `provider_validation`, or similar
`canceled`	The client disconnected mid-turn

Worth alerting on

Signal	Why
`result_path = 'budget_exceeded'` appears at all	A customer is being refused right now
`invalid_rate` up week over week	Output quality is drifting
`selected_tool_count = 0` on a rising share of turns	The router is declining to offer tools
`p95(total_ms)` up while `p95(openai_ms)` is flat	Something other than the model got slower
`result_path = 'error'` with `fallback_mode = 'rate_limited'`	You are being throttled upstream

What it does not contain

No prompts, and no answers. Telemetry records decisions and measurements, not content. Conversations live in ai_chat_message; tool results are never persisted at all.

No tool-level detail. There is no per-tool latency or failure breakdown — tool_count and selected_tool_domains are the granularity. Your own endpoint's logs are the place to see which tool was slow.