# wai-metrics

The text exposition format (version 0.0.4) that a WAI service answers `GET /metrics` with, in a
crate of its own with no dependencies. The services beside `wai-rs` (`wai-audit`, `wai-ledger`,
`wai-fleet`, `wai-key-proxy`, `wai-quantum-service`) and the room service, ingest gateway and relay
of `joule-code` depend on it by path.

- **`Exposition`** writes counters and gauges family by family: each family's `# HELP` and
  `# TYPE` lines, then its samples. Names are checked against the format's grammars, a family is
  written once, label values are escaped (`\`, `"` and a line feed), help text likewise (`\` and a
  line feed), and values are written as the format reads them (`+Inf`, `-Inf`, `NaN`).
- **Every label is fixed in the code.** A label's name and value are `&'static str`: a value that
  exists only at run time (a room's id, a principal, an account, a client's address) cannot become
  a label. The one value written at run time is the `code` of `Requests`, a request's status, which
  is a number. A service's series are the ones its code names, and their number is bounded by it.
- **The scrape is the operator's.** `through_a_front` tells a request that came through a front by
  the fields a front sets (`Forwarded`, `Via`, `X-Forwarded-For`, `X-Forwarded-Host`,
  `X-Forwarded-Proto`, `X-Real-IP`, `CF-Connecting-IP`). A service answers such a request by any
  method as it does a path it does not have, so `/metrics` is read at the service's own address. A
  front that sets none of these fields is configured not to pass the path on.
- **`Counts`** counts by labels fixed in the code (an outcome, a role, a protocol), each series
  written from the start at 0 when the service names it so.
- **`Requests`** counts a service's requests by route, method and status. The route is the pattern
  a path matched (`/v1/channels/{name}/report`), never the path; a method outside HTTP's own is
  `other`.
- **`read`** reads the format for a service's tests. It reads every text the format's grammar
  allows: blanks and tabs between tokens and around a line, comments, and every spelling of a
  value (`Inf`, `Infinity` and `NaN` in any case). It refuses a text that breaks one of the rules
  it holds, with the line and the reason: names, quoting and escapes, one `# HELP` and one `# TYPE`
  a family, each before the first sample, a family's lines together, no series twice, the text
  ended by a line feed. It also refuses a counter that is negative or NaN, and a family named as a
  counter is without its `_total`, which are a service's rules rather than the grammar's.
  `read::identifier_like` tells a label value that looks like a key, a digest, an address or an
  account, which a service's tests refuse in its exposition. `read::declared_routes` reads the
  routes a router's source declares and `read::path_of` makes a path of each, so that a service's
  tests ask every route it has and a route left out of its labels is seen.

```rust
let requests = wai_metrics::Requests::default();
requests.count("/v1/health", wai_metrics::method("GET"), 200);

let mut e = wai_metrics::Exposition::new();
e.gauge("wai_example_sessions", "Sessions open now.", 2u64);
e.family("wai_example_refused_total", wai_metrics::Kind::Counter, "Requests refused, by why.")
    .sample(&[("why", "full")], 1u64);
requests.write(&mut e, "wai_example_requests_total", "Requests answered.");
let body = e.finish(); // answered as wai_metrics::CONTENT_TYPE
```

## What each service exposes

Every service's counters start from 0 when it starts. Every family below is in every answer, with
no sample where nothing was counted yet. Each service answers `HEAD /metrics` as it does `GET`,
without the text.

| service | families |
|---|---|
| `wai-room` | `wai_room_requests_total`, `wai_room_grants_total{role}`, `wai_room_present`, `wai_room_publishers`, `wai_room_audience`, `wai_room_viewer_grants`, `wai_room_rooms_total`, `wai_room_relay_connected` |
| `wai-ingest` | `wai_ingest_requests_total`, `wai_ingest_sessions{protocol}`, `wai_ingest_sessions_opened_total{protocol}`, `wai_ingest_refused_total`, `wai_ingest_records_total`, `wai_ingest_energy_unmetered` |
| `wai-audit` | `wai_audit_requests_total`, `wai_audit_sessions_total{outcome}`, `wai_audit_cmcd_batches_total{outcome}`, `wai_audit_reports_total{outcome}`, `wai_audit_channels` |
| `wai-ledger` | `wai_ledger_requests_total`, `wai_ledger_tariffs`, `wai_ledger_sources`, `wai_ledger_invoices`, `wai_ledger_verifications` |
| `wai-fleet` | `wai_fleet_requests_total`, `wai_fleet_enrolments`, `wai_fleet_records_held`, `wai_fleet_reports`, `wai_fleet_records_total{outcome}`, `wai_fleet_verifications` |
| `wai-key-proxy` | `wai_key_proxy_requests_total`, `wai_key_proxy_receipts_total{decision,delivery}`, `wai_key_proxy_receipts_held`, `wai_key_proxy_forwarding`, `wai_key_proxy_proofs_in_flight`, `wai_key_proxy_upstream_total{class}` |
| `wai-relay` (on its control listener) | `wai_relay_sessions`, `wai_relay_sessions_opened_total`, `wai_relay_tracks`, `wai_relay_subscriptions`, `wai_relay_queued`, `wai_relay_objects_in_total`, `wai_relay_fetches_served_total{from}`, `wai_relay_door_refused_total`, `wai_relay_door_groups_dropped_total`, `wai_relay_door_running_total{outcome}`, `wai_relay_door_admitted_on_run_total`, `wai_relay_door_runs_contradicted_total`, `wai_relay_catalog_receipts_refused_total{why}`, `wai_relay_delivery_gap_claims_total{outcome}`, `wai_relay_requests_refused_total{why}`, `wai_relay_age_withheld_total`, `wai_relay_subscriptions_too_far_behind_total`, `wai_relay_objects_dropped_too_far_behind_total`, `wai_relay_probes_total{direction}`, `wai_relay_probes_refused_total`, `wai_relay_probes_ended_for_shape_total`, `wai_relay_probe_bytes_total{direction}`, `wai_relay_upstream_total{outcome}`, `wai_relay_ratings_held`, `wai_relay_going_away`, `wai_relay_energy_unmetered`, `wai_relay_control_requests_total{op}`, `wai_relay_control_refused_total{why}` |
| `wai-quantum-service` | `wai_quantum_requests_total`, `wai_quantum_jobs{status}`, `wai_quantum_memory_held_bytes`, `wai_quantum_restored_jobs{outcome}`, `wai_quantum_lent_turns_total{pool,event}`, `wai_quantum_energy_unmetered` |

Each `*_requests_total` is labelled `route`, `method` and `code`. The relay answers on its control
listener (the operator's surface, loopback or a Unix socket of its own user), not on its media
ports, and so has no `*_requests_total`. Each service's README says what
its families count.

## Checks

```sh
cargo test --locked
```