Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions 5

wai: media jobs in the receipt ledger, and the int_motion receipts written down (stacked on #542) #544

Merged
dcharlot merged 20 commits from wai/routed-jobs-ledger into main 2026-10-30 05:29:26 -04:00
Conversation 0 Commits 20 Files changed 52 +12593 -212
52 changed files with 12593 additions and 212 deletions
Download patch file Download diff file Expand all files Collapse all files

42
.github/workflows/wai-conformance.yml vendored
View file

@ -216,6 +216,10 @@ on:
# fleet_verify.py reads that table, and the registry's wire identifiers test reads its
# sections. Its corpus is generated from the metering corpus, under the paths above.
- 'wai/extensions/fleet.md'
# tests/route_record.rs and the module's tests require this extension's §8 to list the
# codes the reference reports, route_record_verify.py reads that table, and the registry's
# wire identifiers test reads its sections.
- 'wai/extensions/route-record.md'
# tests/native_sink.rs requires this extension's §8 to list exactly the reasons its
# corpus reaches, and native_sink_verify.py reads that table.
- 'wai/extensions/native-sink.md'
@ -1168,7 +1172,7 @@ jobs:
name: independent verifiers · claims, JWP, key discovery and integer payloads (Python)
runs-on: linux-arm64
env:
WAI_FEATURES: key_discovery,http,video_manifest,stage_measure,live_events,live_room,capture_take,live_compose,live_record,timed_text,access_services,live_metering,energy_steering,live_moderation,dred_exact,live_ads,live_provenance,synthesis,synthesis_mark,neural_int,exact_upscale,robustness,ai_usage,contribution,settlement,browser_sink,receipt_claims,takedown,influence,direction,co_performance,age_assurance,bearer_receipts,exploration,co_watch,telemetry_aggregate,exact_tonemap,splat_exact,live_watermark,derive_edge,sink_energy,key_release,stitched_presentation,prior_wire,scene_map,avatar_face,receipt_ledger,stream_audit,fleet,native_sink,platform_encode,remote_desktop,stream_termination,live_world,format_probe,live_messaging,conform_result,revenue_share,opportunity_report,channel_lineup
WAI_FEATURES: key_discovery,http,video_manifest,stage_measure,live_events,live_room,capture_take,live_compose,live_record,timed_text,access_services,live_metering,energy_steering,live_moderation,dred_exact,live_ads,live_provenance,synthesis,synthesis_mark,neural_int,exact_upscale,robustness,ai_usage,contribution,settlement,browser_sink,receipt_claims,takedown,influence,direction,co_performance,age_assurance,bearer_receipts,exploration,co_watch,telemetry_aggregate,exact_tonemap,splat_exact,live_watermark,derive_edge,sink_energy,key_release,stitched_presentation,prior_wire,scene_map,avatar_face,receipt_ledger,stream_audit,fleet,native_sink,platform_encode,remote_desktop,stream_termination,live_world,format_probe,live_messaging,conform_result,revenue_share,opportunity_report,channel_lineup,route_record
WAI_PQ_FEATURES: synthesis,pq_signatures
WAI_RELEASE_FEATURES: dred_exact,synthesis_mark,robustness,avatar_face
steps:
@ -1265,7 +1269,7 @@ jobs:
--test stitched_presentation --test prior_wire --test scene_map --test receipt_ledger \
--test stream_audit --test fleet --test native_sink --test platform_encode --test remote_desktop \
--test stream_termination --test live_world --test format_probe --test live_messaging \
--test conform_result --test revenue_share --test opportunity_report --test channel_lineup
--test conform_result --test revenue_share --test opportunity_report --test channel_lineup --test route_record
cargo test --release --no-run --no-default-features --features "$WAI_RELEASE_FEATURES" \
--test dred_exact --test dred_exact_measure --test synthesis_mark --test robustness \
--test avatar_face
@ -2000,18 +2004,20 @@ jobs:
python3 -I live-conformance/settlement_verify.py live-conformance/settlement
# wai.delivery.ledger (extensions/receipt-ledger.md): the reference module's unit tests and
# tests/receipt_ledger.rs (vectors.json regenerates byte for byte from its fixed seeds and
# from the sources it takes from the metering, settlement, ads, presentation and stitched
# corpora and the packs of pack-conformance, and no other file is there; wai-rs gives every vector its
# from the sources it takes from the metering, settlement, ads, presentation, stitched and
# routes corpora and the packs of pack-conformance, and no other file is there; wai-rs gives every vector its
# stated outcome; every reason of §10 and every reused code refuses a vector, and so does
# each source kind's own extension; §10 lists exactly the reference's codes); then the
# corpus's second implementation, receipt_ledger_verify.py, written from the extension's
# text and sharing no code with wai-rs, which verifies each source with that corpus's own
# second implementation (live_metering_verify.py, settlement_verify.py, live_ads_verify.py
# with ad_presentation_verify.py, stitched_presentation_verify.py, pack_verify.py) and
# with ad_presentation_verify.py, stitched_presentation_verify.py, pack_verify.py, and for a
# media job route_record_verify.py and _int_motion_receipt.py) and
# reports a stitched stream's statement as that verifier gives it, checks signatures with
# WAI's strict Ed25519 rule, recomputes every line, citation, energy row and report,
# requires every mutated copy of the corpus to fail, and drops each of its own checks in turn and requires
# the corpus to catch the weakened verifier; standard library only, run isolated (-I).
# the corpus to catch the weakened verifier; run isolated (-I), with the `cryptography` package
# installed above for the OER/2 reference verifier that checks a job's hop receipts.
- name: receipt ledger · wai-rs and receipt_ledger_verify.py
working-directory: wai
shell: bash
@ -2109,6 +2115,28 @@ jobs:
(cd wai-rs && cargo test --no-default-features --features "$WAI_FEATURES" --lib fleet::)
(cd wai-rs && cargo test --no-default-features --features "$WAI_FEATURES" --test fleet)
python3 -I live-conformance/fleet_verify.py live-conformance/fleet
# wai.route.record (extensions/route-record.md): the reference module's unit tests (a
# document read strictly and held to the subset, §2's encodings, an estimate past the subset
# unknown, a null ranked last, §8 listing exactly the reference's codes) and
# tests/route_record.rs (vectors.json regenerates byte for byte from its seeds and no other
# file is there; wai-rs gives every vector its stated outcome; every reason of §8 and every
# reused code refuses a vector; the accepted vectors are §10's, every limit removing a
# candidate; a record is bound to each of its documents; `wai route verify` prints the
# outcome and exits by it); then the corpus's second implementation, route_record_verify.py,
# written from the extension's text and sharing no code with wai-rs, which checks the
# router's signature with WAI's strict Ed25519 rule and the hop receipts with the OER/2
# reference verifier (the `cryptography` package, installed above), recomputes every
# candidate, removal, rank, order, decision and attempt, requires every mutated copy of the
# corpus to fail, and drops each of its own checks in turn and requires the corpus to catch
# the weakened verifier; run isolated (-I).
- name: route record · wai-rs and route_record_verify.py
working-directory: wai
shell: bash
run: |
set -euo pipefail
(cd wai-rs && cargo test --no-default-features --features "$WAI_FEATURES" --lib route_record::)
(cd wai-rs && cargo test --no-default-features --features "$WAI_FEATURES" --test route_record)
python3 -I live-conformance/route_record_verify.py live-conformance/routes
# The native sink (extensions/native-sink.md): the reference module's unit tests (dates,
# durations and attribute lists read as a player reads them, origins serialized as RFC 6454
# does, names that state a content hash and names that do not, the acknowledgements
@ -2744,7 +2772,7 @@ jobs:
synthesis,pq_signatures sink_energy key_release stitched_presentation prior_wire \
prior_wire,neural_int scene_map avatar_face receipt_ledger stream_audit fleet native_sink \
platform_encode remote_desktop stream_termination live_world format_probe live_messaging \
conform_result revenue_share opportunity_report channel_lineup; do
conform_result revenue_share opportunity_report channel_lineup route_record; do
echo "::group::$f"
cargo check --no-default-features --features "$f" --tests
echo "::endgroup::"

26
.github/workflows/wai-pack.yml vendored
View file

@ -144,7 +144,7 @@ jobs:
--test=floor_only 2
--test=ladder 3
--test=pack_alloc 1
--test=packs 8
--test=packs 9
--test=review 9
--test=refusals 6
--test=repack 3
@ -158,6 +158,30 @@ jobs:
working-directory: wai/wai-pack
run: cargo run --release --locked --bin wai-pack -- conformance check ../pack-conformance
# The catalogue's package.build and package.verify (wai-rs/src/bin/wai_cli/package.rs)
# run this packer as itself: a dry run of `wai package build` writes nothing; a real one
# writes the golden package of video-base-01 byte for byte; `wai package verify` accepts
# it and exits 1 on a copy whose receipt was edited.
- name: The catalogue's package doors run this packer
shell: bash
run: |
set -euo pipefail
(cd wai/wai-pack && cargo build --release --locked --bin wai-pack)
(cd wai/wai-rs && cargo build --locked --no-default-features --bin wai)
P="$PWD/wai/wai-pack/target/release/wai-pack"; W="$PWD/wai/wai-rs/target/debug/wai"
G=wai/pack-conformance/packs/video-base-01; D=$(mktemp -d)
cp -r "$G/policy.toml" "$G/floor" "$G/key.hex" "$D/"
"$P" fixture gen video-base-01 -o "$D/video-base-01.y4m"
"$W" package build --dry-run --packer "$P" --policy "$D/policy.toml" --out "$D/out" --key "$D/key.hex"
test ! -e "$D/out"
"$W" package build --packer "$P" --policy "$D/policy.toml" --out "$D/out" --key "$D/key.hex"
diff -r "$D/out" "$G/package"
"$W" package verify --packer "$P" "$D/out"
cp -r "$D/out" "$D/edited"
sed -i.bak 's/"total_entries":[0-9]*/"total_entries":1/' "$D/edited/package.receipt.json" && rm "$D/edited/package.receipt.json.bak"
set +e; "$W" package verify --packer "$P" "$D/edited"; code=$?; set -e
[ "$code" = 1 ] || { echo "::error::an edited package's verify exited $code, not 1"; exit 1; }
# The steps a GPU runs (packaging §11.6), on the one leg whose host has a GPU. Each
# test prints the adapter it ran on; the step requires those lines, so a run that
# found no adapter, or filtered the tests away, fails.

88
wai/CHANGELOG.md
View file

@ -6,6 +6,94 @@ wire format may change and such changes are recorded here.
## Unreleased
### Media jobs in the receipt ledger, and the receipts of `wai.video.int_motion` written down
- **The receipt ledger prices media jobs** ([extensions/receipt-ledger.md](extensions/receipt-ledger.md) §4.5): a
tariff and its invoices of **version 2** (`WLT1`, `WLI1` with version byte 2) carry job prices, each a capability, a
unit (`jobs`, or `subpixels`: the `work_pixel_ops` of the job's output receipt), a divisor and a price, and bill a
**job source**: a route record's verification document (`wai.route.record`) with its hops' OER/2 receipts and
WAI's decode receipt of its output with its codestream. A job is billed at its serving capability's prices from its
billed hop alone, so a failed hop, a hop past the declared deadline and a job no hop served bill nothing; its energy
is the serving hop's figure in the new scope `jobs`, by class, the unmetered marker counted apart. New record kinds
(14 an OER/2 receipt, 15 an `int_motion` decode receipt, 16 a route record), a scope kind for a router's key, a
line source kind 5, and two reasons, `output_invalid` and `job_figure_invalid`; one hop and one output belong to one
job (`record_duplicate`), and two records of one router for one job are `record_overlap`. A tariff and an invoice
of version 1 are read, verified and billed exactly as before, and every existing vector keeps its outcome but one:
the control that refused an invoice of version 2 as `unknown_version` now gives version 3.
- **One registry of record kinds** (receipt-ledger §5.2): the table now holds every kind a WAI text names a record
by, revenue-share §7.4's run (10) and invoice (11) and opportunity-report §7.3's cue event (12) and room policy (13)
among them, each pointing to the text that defines it, unchanged; those texts say their kinds are registered there.
Kinds **128 to 255 are reserved for private and provisional use**, so an implementation's own records never collide
with a kind WAI assigns later: a WAI record never states one, and an invoice that lists one is refused
`private_kind` (in either version; such an invoice was refused `invoice_invalid` before). An invoice of version 2
lists kinds 1 to 9 and 14 to 16, never another text's.
- **The `int_motion` receipts** ([extensions/integer-payloads.md](extensions/integer-payloads.md) §15): the decode
receipt and its JSON form, which only the code defined, are written down unchanged, and an **encode receipt**
(`wai.video.int_motion/encode-receipt`) is added: the source frames' hashes, the encoder's parameters, the SSE the
payload decodes with, the work and the encode's energy and class; `VideoEncodeReceipt` seals it over the reference
encoder's payload and re-derives it from the payload and the source. `live-conformance/_int_motion_receipt.py`
reads and checks a decode receipt from the text, for the ledger's second implementation.
- **`oer::micro_joules`**: an OER/2 figure as whole microjoules, exactly, a half to the even one.
- **The route corpus** binds each hop's receipt to its job and gives the jobs the ledger bills distinct ids; two
accepted vectors are added (a hop that served unmetered, and one whose figure states no class).
- **Breaking:** `api:rust:wai::receipt_ledger::Tariff::version`: a tariff states its version. A struct literal adds
`version: receipt_ledger::VERSION` and `job: vec![]`, and reads and writes as before.
- **Breaking:** `api:rust:wai::receipt_ledger::Tariff::job`: the job prices of a tariff of version 2; empty in version 1.
- **Breaking:** `api:rust:wai::receipt_ledger::Invoice::version`: an invoice states its version, its tariff's.
- **Breaking:** `api:rust:wai::receipt_ledger::Invoice::unmetered`: five counts, scope 5 (`jobs`) last; a version-1
invoice writes and reads the first four, its fifth 0.
- **Breaking:** `api:rust:wai::receipt_ledger::Computed::unmetered`: the same five.
- **Breaking:** `api:rust:wai::receipt_ledger::Facts::Job`: the facts of a job source; a match over the sources'
facts adds the variant.
- **Breaking:** `api:rust:wai::receipt_ledger::Reason::OutputInvalid`: a job's output receipt that does not read or
does not re-derive from its codestream (§5.1); a match over the reasons adds it.
- **Breaking:** `api:rust:wai::receipt_ledger::Reason::JobFigureInvalid`: a job's serving hop whose figure no row can
state (§5.1); a match over the reasons adds it.
- **Breaking:** `api:rust:wai::receipt_ledger::Reason::PrivateKind`: an invoice listing a record of a kind reserved
for private use (§5.2); a match over the reasons adds it.
- **Breaking:** `api:rust:wai::receipt_ledger::Reason::ALL`: the codes of §10, the three above among them.
- **Breaking:** `record:corpus:wai/live-conformance/ledger/vectors.json`: the control `invoice_version` gives an
invoice of version 3, since version 2 is now defined, and is refused `unknown_version` as before; every other
vector keeps its outcome, and the version-2 vectors are added.
### Routed media jobs: the route record, OER/2 minor version 1, and the operations a job names
- **`wai.route.record`** ([extensions/route-record.md](extensions/route-record.md)): what a router considered for
one media job and why it chose. The job's request (an operation of the catalogue, a capability and a fallback, its
quantities, hard limits and soft preferences), the price book and a named, versioned routing policy are documents
pinned by SHA-256 over their RFC 8785 forms; from them alone come the candidates, every limit that removes one (a
value the price book does not know counted as the most restrictive), the ranking, the order and the decision, which
the router signs with every attempt, its hop's OER/2 job receipt and whether it was billed. The attempts follow the
order, only the last serves, and a failed hop or one that served past the declared deadline is not billed. Feature
`route_record` (`route`, `seal`, `verify`); the corpus `live-conformance/routes/`, written by
`wai_route_record_vectors`, each control refused at the reason and place it declares; its second implementation
`live-conformance/route_record_verify.py`, which verifies the hop receipts with the OER/2 reference verifier. The
document kinds `wai-route-request`, `wai-route-pricebook`, `wai-route-policy` and `wai-route-record` are wire
identifiers.
- **OER/2 minor version 1** ([extensions/oer-receipts.md](extensions/oer-receipts.md) §3.1): a receipt whose `oer` is
`"2.1"` may state its figure as the unmetered marker, `energy.joules_total` `null`, which a verifier reports as
unmetered and never as a figure, or beside its acquisition class, `energy.class`, which must support exactly the
receipt's grade. Three reasons (`unmetered_invalid`, `class_invalid`, `class_grade_mismatch`), a third canonical
fixture, and `oer-conformance/job_vectors.json` from the hand-declared `generate_job_vectors.py`. A receipt of minor
version 0 reads as before: every existing vector keeps its reasons. A `job` receipt names its operation under
`oer.job.*`, and a route record holds each hop's receipt to that name. The differential fuzzer mutates job receipts
of minor version 1 too.
- **The catalogue**: `route.verify` (`wai route verify <document>`, a build with `route_record`); `package.build` and
`package.verify` (`wai package build|verify`), which run the reference packager `wai-pack` as itself (found with
`--packer`, `$WAI_PACK_BIN`, beside `wai` or on `PATH`), so a job can name its packaging by an operation of the
catalogue. `wai/catalogue/ops.json` is regenerated.
- **`VideoReceipt::from_json`**: the inverse of `to_json`, so a holder of a `wai.video.int_motion` receipt's JSON runs
`verify` and `verify_reconstruction_wiv1` with WAI's own code. `to_json` writes the signer id as an escaped JSON
string, which changes nothing for an id that needs no escape.
- **`wai-pack`'s meter slot is `Send`**: `PackOpts::meter` is `Option<Box<dyn Meter + Send>>`, so a host that packs on
a worker thread hands the packer its meter and the package receipt states the figure.
- **Breaking:** `api:rust:wai::energy_meter::CounterMeter::new`: its read and clock closures are `Send`, so the meter
can cross threads. A caller whose closure holds an `Rc` or a `RefCell` moves its state into the closure or holds it
in an `Arc<Mutex<_>>`.
- **Breaking:** `api:rust:wai::energy_meter::CounterMeter::calibrated_instrument`: the same, for a calibrated
instrument's read and clock.
- **Breaking:** `api:rust:wai::energy_meter::ModelMeter::new`: the same, for the model meter's clock.
### The opportunity report and the statement check: what a channel's records prove, beside what a platform reported
- **`wai.delivery.opportunity`** ([`extensions/opportunity-report.md`](extensions/opportunity-report.md)), a new

5
wai/REGISTRY.md
View file

@ -281,6 +281,7 @@ and a role are fixed once registered.
| `wai.quantum.circuit` | payload | registered | binary | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.quantum.circuit2` | payload | registered | binary | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.quantum.job` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |
| `wai.route.record` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |
| `wai.semantic.clip` | payload | registered | image | standalone | royalty-free | behavioral | — | §5 Neural capabilities |
| `wai.semantic.perco2` | payload | registered | image | standalone | non-commercial | behavioral | — | §5 Neural capabilities |
| `wai.sink.energy` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |
@ -554,6 +555,10 @@ and a role are fixed once registered.
| `wai-robustness-claim` | document-kind | defined | robustness-evidence §8 |
| `wai-robustness-corpus` | document-kind | defined | robustness-evidence §3 |
| `wai-robustness-harness:1` | scheme | defined | robustness-evidence §6 |
| `wai-route-policy` | document-kind | defined | route-record §3.3 |
| `wai-route-pricebook` | document-kind | defined | route-record §3.2 |
| `wai-route-record` | document-kind | defined | route-record §5.1 |
| `wai-route-request` | document-kind | defined | route-record §3.1 |
| `wai-scene/1` | scheme | defined | scene-carriage §2.3 |
| `wai-sequence` | document-kind | defined | direction §6 |
| `wai-session` | document-kind | defined | contribution §3.1 |

17
wai/SPEC.md
View file

@ -1577,6 +1577,23 @@ summary contradicts, a break no window places, two rooms' breaks at once, and a
does not recompute. No parser of any platform's own format:
[`extensions/opportunity-report.md`](extensions/opportunity-report.md).
And for **a routed media job**, what a router considered and why it chose —
`wai.route.record`: a job names an operation of WAI's catalogue, a capability and a fallback,
the quantities it expects, hard limits (regions and data residency, no training on its inputs,
zero retention, generally available endpoints only, consent on file, a price per unit and a
spend ceiling) and soft preferences (a time to first byte, a quality floor, a sort by price,
speed, energy or quality); a price book states what each endpoint offers and costs, and a
named, versioned routing policy orders the survivors and bounds the attempts, the three pinned
by SHA-256 over their RFC 8785 forms. From those three documents alone come the candidates,
every limit that removes one (a value the price book does not know counted as the most
restrictive), the ranking, the order and the first key on which the first two survivors
differ; the router signs a route record of them and of every attempt, each with its hop's
OER/2 job receipt and whether it was billed. The attempts follow the order and only the last
may serve; a failed hop, and a hop that served after the deadline the job declared, is not
billed. A verifier recomputes the record from the three documents, holds each attempt to its
receipt, and refuses a record whose choice they do not give. No transport and no payment:
[`extensions/route-record.md`](extensions/route-record.md).
And for **auditing a stream another service delivers** — `wai.delivery.audit`: a content owner's
players, with nothing asked of the service that packages and delivers the stream over HTTP. For each
playback session a player makes a fresh key and signs with it settlement acknowledgements of every

264
wai/catalogue/ops.json
View file

@ -1277,6 +1277,208 @@
"sdk": "wai.encoder.verify"
}
},
{
"id": "package.build",
"summary": "A package packed by the reference packager wai-pack from a policy, a source per component and a supplied floor, with its signed package receipt",
"cli": "wai package build",
"aliases": [],
"usage": [
"wai package build --policy <policy.toml> --out <dir> --key <seed.hex> [--source <id>=<file>]... [--floor <dir>]",
" [--signer-id <id>] [--meter none|model:<mW>] [--packer <wai-pack>] [wai-pack pack's other flags]",
" the reference packager's pack (extensions/packaging.md): the policy's components packed into <dir>, new or empty,",
" with the package receipt (package.receipt.json); the packer is --packer, $WAI_PACK_BIN, wai-pack beside wai,",
" or wai-pack on PATH"
],
"placement": "local",
"effect": "writes-project",
"dry_run": true,
"confirm": "when-replacing-an-unnamed-file",
"route": null,
"doors": {
"cli": true,
"mcp": false,
"http": false
},
"toolset": "core",
"stability": "preview",
"input": {
"type": "object",
"properties": {
"policy": {
"type": "string",
"format": "path",
"description": "The packaging policy (extensions/packaging.md §3).",
"x-cli": {
"flag": "--policy"
}
},
"out": {
"type": "string",
"format": "path",
"description": "A new or empty directory, written with the package.",
"x-cli": {
"flag": "--out"
}
},
"key": {
"type": "string",
"format": "path",
"description": "The packager's signing key: a 32-byte seed, in hexadecimal.",
"x-cli": {
"flag": "--key"
}
},
"source": {
"type": "array",
"items": {
"type": "string"
},
"description": "A component's source, <id>=<file>; each component's own by default.",
"x-cli": {
"flag": "--source"
}
},
"floor": {
"type": "string",
"format": "path",
"description": "A supplied floor's directory (floor/ beside the policy by default).",
"x-cli": {
"flag": "--floor"
}
},
"signer_id": {
"type": "string",
"description": "The receipt's signer id (the key's did:key by default).",
"x-cli": {
"flag": "--signer-id"
}
},
"meter": {
"type": "string",
"description": "none, or model:<milliwatts>: the meter of the receipt's figure.",
"x-cli": {
"flag": "--meter"
}
},
"packer": {
"type": "string",
"format": "path",
"description": "The packer's binary (wai-pack).",
"x-cli": {
"flag": "--packer"
}
}
},
"required": [
"policy",
"out",
"key"
],
"additionalProperties": false
},
"output": {
"type": "object",
"properties": {
"text": {
"type": "string"
}
},
"required": [
"text"
]
},
"stream": null,
"since": "0.3.0",
"deprecated": null,
"receipt": true,
"needs": null,
"names": {
"cli": "wai package build",
"mcp": null,
"c": "wai_package_build",
"rust": "wai::package::build",
"sdk": "wai.package.build"
}
},
{
"id": "package.verify",
"summary": "A package verified by the reference packager wai-pack: every reason it is refused, or its counts and pack digest",
"cli": "wai package verify",
"aliases": [],
"usage": [
"wai package verify <dir> [--parent <dir>] [--packer <wai-pack>] the reference packager's verify",
" (extensions/packaging.md §9): the package's receipt, stage indexes, objects and floor; with --parent, a re-pack against",
" the package its receipt names; exit 1 when refused"
],
"placement": "local",
"effect": "reads",
"dry_run": false,
"confirm": "never",
"route": null,
"doors": {
"cli": true,
"mcp": false,
"http": false
},
"toolset": "core",
"stability": "preview",
"input": {
"type": "object",
"properties": {
"dir": {
"type": "string",
"format": "path",
"description": "The package's directory.",
"x-cli": {
"positional": true
}
},
"parent": {
"type": "string",
"format": "path",
"description": "The package a re-pack's receipt names.",
"x-cli": {
"flag": "--parent"
}
},
"packer": {
"type": "string",
"format": "path",
"description": "The packer's binary (wai-pack).",
"x-cli": {
"flag": "--packer"
}
}
},
"required": [
"dir"
],
"additionalProperties": false
},
"output": {
"type": "object",
"properties": {
"text": {
"type": "string"
}
},
"required": [
"text"
]
},
"stream": null,
"since": "0.3.0",
"deprecated": null,
"receipt": false,
"needs": null,
"names": {
"cli": "wai package verify",
"mcp": null,
"c": "wai_package_verify",
"rust": "wai::package::verify",
"sdk": "wai.package.verify"
}
},
{
"id": "registry.list",
"summary": "Every registered capability with its fidelity, licence class, replicate class and role, and whether this build decodes it",
@ -3301,6 +3503,68 @@
"sdk": "wai.report.export"
}
},
{
"id": "route.verify",
"summary": "A route record checked offline against its request, price book and routing policy, and against its hops' OER/2 receipts where given",
"cli": "wai route verify",
"aliases": [],
"usage": [
"wai route verify <document.json> a verification document (extensions/route-record.md §7.1): the record's outcome",
" (§7.3) as JSON, recomputed from the request, the price book and the policy; exit 1 when it is refused"
],
"placement": "local",
"effect": "reads",
"dry_run": false,
"confirm": "never",
"route": null,
"doors": {
"cli": true,
"mcp": false,
"http": false
},
"toolset": "core",
"stability": "preview",
"input": {
"type": "object",
"properties": {
"document": {
"type": "string",
"format": "path",
"description": "The verification document: the request, price book, policy and record, and the hops' receipts.",
"x-cli": {
"positional": true
}
}
},
"required": [
"document"
],
"additionalProperties": false
},
"output": {
"type": "object",
"required": [
"outcome"
],
"description": "The outcome of extensions/route-record.md §7.3: accept with the record's digest, the served and billed hops, the deadline, the order, the decision, the removals and the unmetered hops; or refuse with the reason and where"
},
"stream": null,
"since": "0.3.0",
"deprecated": null,
"receipt": false,
"needs": {
"all": [
"route_record"
]
},
"names": {
"cli": "wai route verify",
"mcp": null,
"c": "wai_route_verify",
"rust": "wai::route::verify",
"sdk": "wai.route.verify"
}
},
{
"id": "channel.import",
"summary": "A linear channel's feeds (an XMLTV schedule, Media RSS catalogues or the JSON form, and a Media Playlist's SCTE-35 date ranges) imported into its signed receipt, its signed lineup and its breaks' cues",

117
wai/extensions/integer-payloads.md
View file

@ -749,3 +749,120 @@ leave `i64` on. Some of these checks are new refusals of input no committed corp
contains, and some now come before the streams where an earlier decoder made them when
the network ran ([integer-operators](integer-operators.md) §15.3, SPEC §8 *Planned
breaks*).
## 15. Receipts of `wai.video.int_motion`
Two signed receipts describe a `WIV1` payload of `wai.video.int_motion` (keyframe 0, §3).
The **decode receipt** (§15.1) is a sink's statement of what the payload reconstructs to;
the **encode receipt** (§15.3) is an encoder's statement of the payload it wrote, the
source it wrote it from, its parameters and its error. Each binds a measured energy
figure and its acquisition class ([energy-measurement](energy-measurement.md) §2) beside
the figures anyone can recompute, and keeps the two apart. The reference is
`wai-rs`'s `video_receipt` (`VideoReceipt`, `VideoEncodeReceipt`; feature
`video_receipt`).
In this section, integers in a signing payload are big-endian and those hashed into a
frame or clip commitment little-endian, as the reference writes them; BLAKE3 is the
hash of every commitment.
### 15.1 The decode receipt
| field | what it is |
|---|---|
| `codestream_hash` | BLAKE3 over the payload's bytes |
| `h`, `w` | the payload's geometry |
| `frames` | for each decoded frame `i`, from 0: BLAKE3 over `wai:video-rframe` (ASCII), 0x01, `i` (u64) and the frame's RGB8 bytes |
| `clip_hash` | BLAKE3 over `wai:video-clip`, 0x01, `h` (u32), `w` (u32), the number of frames (u64), then each frame's `i` (u64) and hash |
| `merkle_root` | the [jwp-receipts](jwp-receipts.md) §2.2 Merkle root over the frame hashes, in order |
| `work_pixel_ops` | frames × `h` × `w` × 3, saturating at 2⁶⁴ − 1: the decoded subpixels |
| `joules_micro`, the class | the energy the sink spent reconstructing, in microjoules, and its acquisition class; `0` and no class is the unmetered marker |
| `parent_receipt_hash` | the receipt hash of the receipt before it, or none |
| `signer_pubkey`, `signer_id`, `sig` | the signer's Ed25519 key, its id, and the signature |
The **signing payload** is `wai:video-receipt` and 0x01, then `codestream_hash`,
`clip_hash`, `h` (u32), `w` (u32), `merkle_root`, the number of frames (u64),
`work_pixel_ops` (u64), `joules_micro` (u64), the class, the parent (0x00, or 0x01 and
the hash), each frame's `i` (u64) and hash, `signer_pubkey`, and `signer_id`'s length
(u32) and UTF-8 bytes. The class is one tag byte and its evidence: 0 the unmetered
marker; 1 `HwShunt` (legacy), 2 `ModelBased`, 3 `Estimator`, with none; 4
`OnChipCounter`, then `relative_ppm` (u32), `window_us` (u64) and the filtering state
(u8: 0 `Unknown`, 1 `Off`, 2 `On`); 5 `CalibratedInstrument`, then `relative_ppm` (u32),
`window_us` (u64), and the certificate reference's length (u32) and UTF-8 bytes. `sig`
is Ed25519 over the payload, verified strictly ([SPEC](../SPEC.md) §7 *Signatures*). The
**receipt hash** is BLAKE3 over `wai:video-receipt-id`, 0x01, the payload and `sig`.
A receipt is **consistent** when `merkle_root`, `clip_hash` and `work_pixel_ops` are
what its frames give, its frames are numbered 0 upwards, its figure and class cohere (0
with no class, or a figure above 0 with a class whose evidence energy-measurement §2.3
requires and whose declared `relative_ppm` × `joules_micro` reaches the resolution floor
of 288676), and its signature verifies. It is **re-derived** from a payload when the
payload's BLAKE3 is its `codestream_hash`, the payload decodes (§3), and the decode has
its geometry and, frame by frame, its frame hashes; and then it is consistent. The figure
is attested by the signature and by nothing else: another sink measures another figure
for the same decode.
### 15.2 The JSON form
A decode receipt's JSON form is one object of these members, in this order:
`profile` (`"wai.video.int_motion/receipt"`), `codestream_hash`, `clip_hash`, `h`, `w`,
`frames` (each frame's hash), `merkle_root`, `work_pixel_ops`, `joules_micro`,
`uj_per_kilopixel_op`, `energy_provenance`, the class's evidence (for `OnChipCounter`,
`energy_uncertainty` `{"relative_ppm", "window_us"}` and `energy_counter_filtering`; for
`CalibratedInstrument`, `energy_uncertainty` and `energy_calibration_ref`; for the
others, none), `parent_receipt_hash`, `signer_pubkey`, `signer_id`, `sig` and
`receipt_hash`. Hashes, keys and signatures are lowercase hexadecimal; `h`, `w` and the
figures are integers in plain decimal; `energy_provenance` is the class's label, or
`null` when unmetered, and `parent_receipt_hash` `null` when there is none.
`uj_per_kilopixel_op` is derived: `null` when the receipt is unmetered or its work is 0,
and otherwise `joules_micro` × 1000 / `work_pixel_ops`, computed in IEEE 754 binary64
(the product, then the quotient) and written with six digits after the point, correctly
rounded.
A reader reads the text strictly (RFC 8259, no member named twice), requires exactly the
members the receipt's class writes, each of its form, reads the class and its evidence
as energy-measurement §2.7 reads them, and refuses a `uj_per_kilopixel_op` that is not,
spelled as above, the receipt's own, and a `receipt_hash` that is not its receipt hash.
The members' order and the whitespace between them are not held. A receipt read so is
then consistent or not, and re-derived from its payload or not, as §15.1 says.
### 15.3 The encode receipt
An encode receipt (`wai.video.int_motion/encode-receipt`) states the payload an encoder
wrote and the source it wrote it from:
| field | what it is |
|---|---|
| `codestream_hash` | BLAKE3 over the payload's bytes |
| `h`, `w` | the source's geometry |
| `source_frames` | for each source frame `i`, from 0: BLAKE3 over `wai:video-sframe` (ASCII), 0x01, `i` (u64) and the frame's RGB8 bytes |
| `source_hash` | BLAKE3 over `wai:video-source`, 0x01, `h` (u32), `w` (u32), the number of frames (u64), then each frame's `i` (u64) and hash |
| `block`, `search`, `mv_precision`, `q` | the encoder's parameters: the payload's `block`, `mv_precision` and `q` (§3), and the motion search's radius, which the payload does not carry |
| `sse` | Σ (source − decode)² over every sample of every frame (u64): the error the payload decodes with |
| `work_pixel_ops` | frames × `h` × `w` × 3, saturating at 2⁶⁴ − 1: the encoded subpixels |
| `joules_micro`, the class | the energy the encoder spent, in microjoules, and its acquisition class; `0` and no class is the unmetered marker |
| `parent_receipt_hash`, `signer_pubkey`, `signer_id`, `sig` | as §15.1's |
Its **signing payload** is `wai:video-encode-receipt` and 0x01, then `codestream_hash`,
`source_hash`, `h` (u32), `w` (u32), the number of frames (u64), `block` (u32), `search`
(u32), `mv_precision` (u8), `q` (u32), `sse` (u64), `work_pixel_ops` (u64),
`joules_micro` (u64), the class (§15.1's encoding), the parent, each source frame's `i`
(u64) and hash, `signer_pubkey`, and `signer_id`'s length (u32) and bytes. Its **receipt
hash** is BLAKE3 over `wai:video-encode-receipt-id`, 0x01, the payload and `sig`. It is
a profile defined after the hardware class was refined (energy-measurement §2.6, *New
profiles*): its class is never the legacy `HwShunt`.
An encode receipt is **consistent** as §15.1 says of a decode receipt, with `source_hash`
for `clip_hash` and no Merkle root, and the new-profile rule for its class. It is
**re-derived** from a payload and a source when the payload's BLAKE3 is its
`codestream_hash`; the payload reads as `WIV1` keyframe 0 (§3) of its `h`, `w` and
number of frames, with its `block`, `mv_precision` and `q`; each source frame hashes to
its own; the payload decodes; and the decode against the source gives its `sse`; and
then it is consistent. Re-deriving it proves the payload decodes to frames within `sse`
of the stated source. It does not prove which encoder wrote it, or that `search` was the
radius used.
Its JSON form is one object of `profile` (`"wai.video.int_motion/encode-receipt"`),
`codestream_hash`, `source_hash`, `h`, `w`, `source_frames`, `block`, `search`,
`mv_precision`, `q`, `sse`, `work_pixel_ops`, `joules_micro`, `energy_provenance`, the
class's evidence (as §15.2), `parent_receipt_hash`, `signer_pubkey`, `signer_id`, `sig`
and `receipt_hash`, read as §15.2 reads a decode receipt's.

119
wai/extensions/oer-receipts.md
View file

@ -1,6 +1,8 @@
# WAI Extension: OER Receipts (outcome and energy)
> Status: Draft. Ingests **OER/2 draft 0.1**, with the departures in §9. An OER
> Status: Draft. Ingests **OER/2 draft 0.1**, with the departures in §9 and the
> members minor version 1 adds (§3.1): the unmetered marker and the acquisition class
> of a receipt's figure. An OER
> receipt records what one unit of work was, what it cost, **how that cost is
> graded**, and what it rests on — canonicalised, content-addressed, hash-chained
> and Ed25519-signed, so a third party can check every figure without trusting the
@ -16,8 +18,9 @@
acquired* — the acquisition class. It does not say how much weight the figure can
bear. An OER receipt adds that: every figure carries an **evidential grade**, and
the receipt as a whole is canonical, content-addressed and chained, so the grade
cannot be quietly upgraded after the fact. An OER/2 receipt carries the grade and
no acquisition class; the two are different axes (§2).
cannot be quietly upgraded after the fact. A receipt of minor version 0 carries the
grade and no acquisition class; one of minor version 1 may carry both (§3.1). The two
are different axes (§2).
A receipt describes one unit of work of a declared `kind`, one of the strings
`decision`, `task`, `reconstruction`, `calibration`, `job`, `compile`, `decode`,
@ -44,9 +47,10 @@ A figure's grade is its **evidential standing**; its acquisition class
([`energy-measurement.md`](energy-measurement.md) §2: `CalibratedInstrument`,
`OnChipCounter`, `ModelBased`, `Estimator`, and the legacy `HwShunt`) is **how it
was obtained**. They are different axes and a receipt **SHOULD** carry both where
it carries an acquisition class at all. An OER/2 receipt has no member for the
class, and cannot gain one without becoming non-canonical (§3). Its descriptive
members name none. `provenance.device` is text; the corpus writes `hardware`,
it carries an acquisition class at all. A receipt of minor version 0 has no member
for the class, and cannot gain one without becoming non-canonical (§3); one of minor
version 1 may state it as `energy.class` (§3.1). Their descriptive members name
none. `provenance.device` is text; the corpus writes `hardware`,
`emulation` and `simulation`. `energy.method` is text too; the corpus writes
`hardware_meter` and `model_estimated`. In particular, neither `hardware` nor
`hardware_meter` is `HwShunt` or either refined hardware class.
@ -81,16 +85,17 @@ The two axes correspond in one direction:
`OnChipCounter` or `CalibratedInstrument`. Mapped into a vocabulary that has
only `HwShunt` / `ModelBased` / `Estimator`, both refined classes become
`HwShunt` (§2 there); the grade does not change, so a `HwShunt` figure
produced that way supports `measured` too. An OER receipt records no class
at all, so a figure graded `measured` in one says nothing of which hardware
class, if any, is behind it.
produced that way supports `measured` too. A receipt that records no class
says, by a figure graded `measured`, nothing of which hardware class, if any,
is behind it.
- The correspondence does not run the other way: `measured` also covers a
datasheet or a published figure, which no acquisition class names, and
`simulated` has no acquisition-class counterpart at all.
A producer **MUST NOT** grade `measured` a figure whose acquisition class is
`Estimator`. A receipt carries no acquisition-class member (§3), so no reason in
§7 checks this; it binds the producer.
`Estimator`. A receipt of minor version 0 carries no acquisition-class member (§3),
so no reason in §7 checks this there, and it binds the producer; in a receipt of minor
version 1 that states its class, `class_grade_mismatch` checks it (§3.1).
`provenance.weakest_input` names the lowest-graded input the figure rests on, so a
`measured` total built on a `projected` constant is visible as such.
@ -103,10 +108,11 @@ holding that receipt then checks that it verifies under this document, is about
same object (its `subject.output` is the object's jwp-receipts §2.1 content hash, as
the claim writes it), and states the same grade and, to the microjoule, the same
figure, read as §4 reads a quantity. That is where the producer rule above becomes
checkable, against the class the claim's signer states: a verifier that checks an
accepted `energy` claim against the OER receipt it names holds a receipt graded no
higher than that class supports. It changes nothing about the OER receipt itself, which
still carries no class.
checkable for a receipt that states no class, against the class the claim's signer
states: a verifier that checks an accepted `energy` claim against the OER receipt it
names holds a receipt graded no higher than that class supports. It changes nothing
about the OER receipt itself. A receipt that states the unmetered marker (§3.1) states
no figure, so no claim's figure is that receipt's.
## 3. Receipt members
@ -142,9 +148,15 @@ its measurement boundary cannot be compared with any other.
**Capability.** `capability` names what the receipt is about, in the receipt's own
namespace: a decision receipt names its decision class under `oer.decision.*`
(`oer.decision.ev_charging_authorization`), and bundles and anchors use `oer.bundle.*`
and `oer.anchor.*`. These are not WAI payload capabilities, which name formats a sink
reconstructs ([`SPEC.md`](../SPEC.md) §5), and a receipt **MUST NOT** name them in
the `wai.` namespace.
and `oer.anchor.*`. A `job` receipt names the operation its unit of work ran under `oer.job.*`:
`oer.job.<operation>`, where `<operation>` is the id of an operation of WAI's
operation catalogue (`wai/catalogue/ops.json`, `noun.verb`), so a hop that encoded
names `oer.job.media.encode` and one that packaged `oer.job.package.build`. A record
that names a job's operation, such as a route record
([route-record](route-record.md) §6), holds each hop's receipt to that name. These are
not WAI payload capabilities, which name formats a sink reconstructs
([`SPEC.md`](../SPEC.md) §5), and a receipt **MUST NOT** name them in the `wai.`
namespace.
**Status, escalation and abstention.** `work.status` is `1` for a unit of work that
completed and `-1` for one that was wasted; a bundle's `efficiency` counts them as
@ -167,6 +179,55 @@ first-class result. An abstention is completed work (`status` `1`), so
`efficiency.joules_per_completed` counts it; a figure per *settled* decision
**MUST NOT**.
### 3.1 Minor version 1: the unmetered marker and the acquisition class
A receipt whose `oer` is exactly `"2.1"` is of **minor version 1**; any other `"2.x"`
is read as minor version 0, under §3's members alone. A receipt of minor version 1 may
carry two things a receipt of minor version 0 cannot, and nothing else:
| path | members, in canonical order |
|---|---|
| `energy` | `joules_total`, `boundary`, `breakdown`, `method`, `temperature_k`, `class` |
| `energy.class` | `label`, `relative_ppm`, `window_us`, `filtering`, `calibration_ref` |
The other paths keep §3's members. Under minor version 0 an `energy.class` is a member
not listed for its path (`canonical_failure`) and a `null` `joules_total` is not a
quantity (`malformed`), as before this revision, so no receipt of minor version 0 reads
differently. A verifier that knows only minor version 0 refuses a receipt that uses
either, rather than misreading it.
**The unmetered marker.** An `energy.joules_total` of `null` states that nobody measured
the unit of work's energy. It is not a figure of zero joules, and a verifier **MUST**
report the receipt as unmetered, never as a figure (energy-measurement §2). A receipt
that states it **MUST** write every `energy.breakdown` term as `null`, **MUST** be
graded `projected` (no figure stands behind it to be measured or derived), **MUST NOT**
carry `energy.class`, and **MUST NOT** be a `bundle`, whose figure is its stages' sum
(§6); a receipt that breaks any of these is `unmetered_invalid`. Its `energy.boundary`
is still stated: it names what would have been measured. Nothing sums to the marker, so
the sum check (§6) does not run for it, and a ledger identity over a chain that holds
one does not hold (§6, `ledger_mismatch`).
**The acquisition class.** `energy.class` states how `joules_total` was acquired, as
[energy-measurement](energy-measurement.md) §2 names the classes. `label` is one of
`CalibratedInstrument`, `OnChipCounter`, `ModelBased` and `Estimator`; the legacy
`HwShunt` is not admitted (energy-measurement §2.6, *New profiles*). Each label carries
exactly its evidence and no other member:
| `label` | members |
|---|---|
| `OnChipCounter` | `label`, `relative_ppm` (an integer `1` to `2³² − 1`: the declared standard relative uncertainty in parts per million, energy-measurement §2.3), `window_us` (an integer `1` to `2⁵³ − 1`: the integration window in microseconds), `filtering` (`"Unknown"`, `"Off"` or `"On"`: whether the counter filtered its output, energy-measurement §2.4) |
| `CalibratedInstrument` | `label`, `relative_ppm`, `window_us`, `calibration_ref` (a non-empty string: the calibration certificate's identifier) |
| `ModelBased`, `Estimator` | `label` |
A class that is not an object of these members, each of its form, is `class_invalid`.
A valid class **MUST** support exactly the receipt's grade, under §2's correspondence
(`CalibratedInstrument` and `OnChipCounter` `measured`, `ModelBased` `derived`,
`Estimator` `projected`); a receipt graded otherwise is `class_grade_mismatch`, checked
when the grade is one of §2's (an unknown grade is `grade_unknown` alone). So the
producer rule of §2 is a reason here: a counter reading graded `derived`, an estimate
graded `measured` and a simulated figure with any class are each refused. A receipt of
minor version 1 may omit the class, and then reads as one of minor version 0 does.
## 4. Canonical form
The canonical body is the receipt **without** `chain` and `receipt_id`, with every
@ -234,6 +295,11 @@ vector can tell them apart. The fixture's stages carry a two-byte and a four-byt
character, with their members out of order, against literal bytes and their
`sha-256` id (`sha256:c1caa0ce…f2638b`).
A third fixture pins minor version 1's members (§3.1): a body whose `energy.class` and
whose class's own members are out of order, against literal bytes and their `sha-256`
id (`sha256:b473c94d…388b94`); and the same input under `"2.0"`, which the canonical
form refuses.
## 5. Identity, chain and signature
- `receipt_id` = `H(canonical body)`.
@ -285,7 +351,9 @@ figure it names, so a negative figure is judged exactly as a positive one.
- **Ledger.** The receipts' `joules_total` plus the ledger's `unallocated_joules` and
`carried_forward_joules` (`0` when absent) **MUST** equal its `meter_joules`
within `1e-9 × max(1, |meter|)`. A ledger figure that is not a quantity (§4)
makes the identity fail.
makes the identity fail, and so does a receipt that states the unmetered marker
(§3.1): a meter's reading cannot be accounted for by a receipt that states no
figure.
## 7. Reasons
@ -340,6 +408,14 @@ one is a named reason rather than `malformed`:
| `signature_invalid` | the signature does not verify, or the published key fails §5's strict decoding |
| `record_mismatch` | a cited record does not rehash to `record_hash`, or cannot be rehashed (§6) |
| `ledger_mismatch` | the ledger identity does not hold |
| `unmetered_invalid` | a receipt of minor version 1 states the unmetered marker beside a breakdown term that is not `null`, a grade other than `projected`, a class, or as a bundle (§3.1) |
| `class_invalid` | a receipt of minor version 1 states an `energy.class` that is not §3.1's: an unknown or legacy label, or evidence missing, extra or of the wrong form |
| `class_grade_mismatch` | a valid `energy.class` does not support the receipt's grade, one of §2's (§3.1) |
Beside its reasons, a verifier reports each receipt that states the unmetered marker
as **unmetered**, by its `receipt_id`, whatever else it reports of it. A vector that
declares `unmetered`, a list of receipt ids in the order of its receipts, passes only
when the verifier reports exactly those receipts unmetered.
Reasons for receipts that share a `receipt_id` **accumulate**; a repeated id
**MUST NOT** replace, and so hide, the reasons of an earlier receipt.
@ -355,6 +431,7 @@ Reasons for receipts that share a `receipt_id` **accumulate**; a repeated id
| `vectors.json` | 9 | the reference generator: a simulated corridor day, two valid vectors and seven tampered twins |
| `firmware_vectors.json` | 1 | the portable C kernel (`settle/`): 400 receipts, independent SHA-256 and Ed25519 |
| `edge_vectors.json` | 99 | `generate_edge_vectors.py`: one case for each reason the upstream nine never exercise, each number shape and decimal-string defect §4 forbids and each limit from the accepting side, each bound and tolerance edge, each step of the order of the reasons reported alone, each member judged on its value, each claim an abstention cannot make, each key a signature cannot rest on, and the extremes a document may hold |
| `job_vectors.json` | 38 | `generate_job_vectors.py`: `job` receipts of minor version 1 (§3.1) under `oer.job.media.encode`: metered hops of each class with the grade it supports, the unmetered marker on its own, in a failover chain and on a failed hop, each rule of the marker and of the class broken once, each class held to a grade it does not support, a ledger over a chain that holds the marker, and the same members under minor version 0 and a minor version this text does not define; every vector declares the receipts it states unmetered, and every expectation is written by hand |
| `unreadable/` | 11 documents | one for each way §4 refuses a document whole; every verifier must refuse all of them |
The upstream vectors cover eight reason names; the edge vectors pin the rest,
@ -368,7 +445,8 @@ base; for each of the four, a verdict that flips if it is computed to 34 digits
keys that are small-order or non-canonical encodings, each under a signature that a
lenient verifier accepts. Their expectations are written by hand from this document
— none is copied from a verifier's output, which would only prove the code agrees
with itself.
with itself. The job vectors pin minor version 1's three reasons and its unmetered
report the same way.
Two rules bind the producer and no reason in §7 can enforce them: the capability
namespace, and receipting an unearned decision as an abstention (§3). A verifier
@ -379,6 +457,7 @@ abstention claims none.
wai_oer_vectors verify oer-conformance # wai-rs, --features oer: every file and unreadable/
python3 oer-conformance/oer2_verify.py <file> # reference verifier, per file
python3 oer-conformance/oer2_verify.py --unreadable oer-conformance/unreadable
python3 oer-conformance/generate_job_vectors.py # regenerates job_vectors.json byte for byte
python3 oer-conformance/fuzz_differential.py # the two verifiers against each other
make -C oer-conformance/settle && oer-conformance/settle/settle_host out.json 400 # the C kernel
```

3
wai/extensions/opportunity-report.md
View file

@ -277,7 +277,8 @@ are the policy's (kind 13, the policy's digest, live-ads §3.2), and its run (ki
for a scheduled break, its run (kind 10). `cited` is their number, each record counted once, and
`cites` is SHA-256 over the bytes `wai:opportunity-cites` (ASCII) and 0x01, followed by each cited
record's `kind` (u8) and identity (32), in ascending order of (`kind`, identity). Kind 10 is
revenue-share §7.4's; 12 and 13 are this text's.
revenue-share §7.4's; 12 and 13 are this text's. Every kind a citation digest names, these
among them, is registered in [receipt-ledger](receipt-ledger.md) §5.2.
### 7.4 Digest and signature

155
wai/extensions/receipt-ledger.md
View file

@ -21,8 +21,10 @@
> invoice, the ledger states two **reports**: the ad decisions reconciled against the players'
> presentation claims (for a stitched stream, stitched-presentation §7.5's own statement), and
> the delivery of each party under the one tariff with the bytes two parties were acknowledged
> for twice. No new transport, no payment and no tax: an invoice is
> the recomputable figures a payment rests on. The reference implementation is the
> for twice. A tariff of version 2 also prices **media jobs** (§4.5): a job is billed from
> its route record ([route-record](route-record.md)), its hops' OER/2 receipts and WAI's
> receipt of its output, in the units those receipts state. No new transport, no payment
> and no tax: an invoice is the recomputable figures a payment rests on. The reference implementation is the
> `receipt_ledger` module of `wai-rs` (feature `receipt_ledger`) and the service
> `wai/wai-ledger`; the corpus is [`live-conformance/ledger/`](../live-conformance/ledger/),
> and its second implementation is `live-conformance/receipt_ledger_verify.py`.
@ -94,7 +96,8 @@ consistent with what the records say:
party and class, so the two can differ by less than one millionth of a minor unit per payer
line, and the delivery report (§8.2) states both.
3. **Units are never converted.** A room is billed in its own units, a delivery in bytes, a
package in packages, entries and stored bytes, an ad in impressions. No line turns one unit
package in packages, entries and stored bytes, an ad in impressions, a job in jobs or in
the subpixels its output's receipt states (§4.5). No line turns one unit
into another, and no line is priced from joules: the energy of an invoice is stated beside
its lines (§6.6), per class, and is not an amount.
@ -109,7 +112,7 @@ records recomputes all three.
| field | size | meaning |
|---|---|---|
| `magic` | 4 | `WLT1` (ASCII) |
| `version` | 1 | 1 in this revision |
| `version` | 1 | 1, or 2 for a tariff that prices jobs (§4.5) |
| `seller` | 32 | the seller's key, which signs the tariff and every invoice under it |
| `buyer` | 32 | the buyer's key, which accepts the tariff by signing it |
| `start`, `end` | 8 each | u64: the period the tariff is good for |
@ -118,6 +121,7 @@ records recomputes all three.
| `delivery` | 32 | four u64: millionths of a minor unit per 10^9 acknowledged bytes of each object class, in class order (settlement §7.1) |
| `package` | 24 | three u64: millionths of a minor unit per package receipt, per entry, and per 10^9 stored bytes |
| `ads` | 16 | two u64: millionths of a minor unit per presented impression, and per billable impression |
| `job` | 2 + Σ (18 + *c*) | version 2 only, absent in version 1: a u16 count *j*, then each job price (§4.5): `cap_len` (u8), `capability` (`cap_len` bytes), `unit` (u8), `divisor` (u64) and `price` (u64) |
| `scope` | 2 + 33 × *n* | a u16 count *n*, then each entry: `kind` (u8) and `id` (32), in strictly ascending order of (`kind`, `id`) |
| `seller_sig` | 64 | §4.2 |
| `buyer_sig` | 64 | §4.2 |
@ -136,14 +140,16 @@ tariff the buyer did not sign bills nobody.
A reader reads exactly the bytes it is given as one tariff, in the order of §4.1, and refuses at
the first of these that holds: fewer than 4 bytes, `truncated`; a magic other than `WLT1`,
`malformed`; no fifth byte, `truncated`; a version other than 1, `unknown_version`; the bytes end
before a field the layout requires, `truncated`; bytes after `buyer_sig`, `malformed`. It then
refuses as `tariff_invalid` a tariff that breaks any of these:
`malformed`; no fifth byte, `truncated`; a version other than 1 and 2, `unknown_version`; the
bytes end before a field the layout requires, `truncated`; bytes after `buyer_sig`, `malformed`.
It then refuses as `tariff_invalid` a tariff that breaks any of these:
- `start` is below `end`; `currency` is three capitals `A`–`Z`;
- `seller` and `buyer` are not zero, and not the same key;
- `scope` has at least one entry, in strictly ascending order of (`kind`, `id`), each `kind` 1 to
4 and each `id` not zero.
4 (1 to 5 in version 2) and each `id` not zero;
- in version 2, `job` has at most 255 prices, each as §4.5 requires, in strictly ascending
order of (`capability`'s bytes, `unit`).
Then the seller's signature, then the buyer's: `bad_key`, `bad_signature`.
@ -157,10 +163,28 @@ A scope entry names whose records the tariff bills:
| 2 | a party's key | a settlement statement's `party` is the `id` |
| 3 | a packager's key | a package receipt's `signer_pubkey` is the `id` |
| 4 | an ad decider's or a stitcher's key | every receipt of an ads source has `signer_pubkey` the `id` (live-ads §3.4); a stitched source's records are accepted only when their `signer` is such an `id`: the verifier's **signers** (stitched-presentation §7.1) are the scope's entries of kind 4 |
| 5 | a router's key (version 2) | a job source's route record has `signer` the `id` (route-record §5.1) |
The scope is what the buyer agreed to pay for: a record signed by a party the tariff does not name
is refused `outside_scope` (§7.2 step 5), whoever submits it.
### 4.5 Job prices (version 2)
A tariff of version 2 prices the media jobs a router routed and a hop served. A **job price**
is a `capability` (route-record §2: 1 to 128 bytes, lowercase ASCII letters, digits, `_` and
`.`, beginning `wai.`), a `unit`, a `divisor` of at least 1 and a `price`: millionths of a
minor unit per `divisor` units of the unit, for a job whose serving hop served that capability.
The units are the records' own figures, never converted:
| `unit` | name | a job's quantity |
|---|---|---|
| 1 | `jobs` | 1 |
| 2 | `subpixels` | the sum of its output receipts' `work_pixel_ops` (integer-payloads §15.1): the subpixels its output decodes to |
A job the tariff has no price for, by its serving capability, is billed nothing, and a buyer
pays for a capability only at a price it signed. A package job stays a package source (§5.1),
billed by §4.1's package prices.
## 5. Sources
### 5.1 Kinds and their documents
@ -175,6 +199,7 @@ extension's verifier is given. Bytes are lowercase hexadecimal strings.
| `package` | `receipt` (the text of `package.receipt.json`), `indexes` (the index files in the receipt's component order) | packaging §9, a receipt of version 2 against its indexes |
| `ads` | `descriptor`, `grants`, `key_log`, `chains`, `policy`, `pt0_ms`, `receipts` (form-2 receipts in their JSON carriage), `pods`, `claims`, `creatives`, `issuers` | live-ads §9, with ad-presentation §9 under a policy that requires claims |
| `stitched` | `records` (pod-input records), `claims` (stitched claims), `manifests`, `responses` (VAST and VMAP documents), `receipts` (form-1 receipts in their JSON carriage), `creatives`, `issuers` | stitched-presentation §7, its signers the tariff's (§4.4) |
| `job` (version 2) | `request`, `pricebook`, `policy`, `record`, `receipts`, `keys` (a route record's verification document, route-record §7.1, its `receipts` and `keys` not `null`) and `outputs`, a list of objects of exactly `receipt` (the text of a `wai.video.int_motion` decode receipt's JSON form, integer-payloads §15.2) and `codestream` (the `WIV1` payload it describes) | route-record §7, accepted with its hop receipts checked; then each output receipt read by integer-payloads §15.2 and re-derived from its `codestream` (§15.1 there), or `output_invalid` at `outputs` and its index |
A source is accepted when its extension's verifier accepts it; for `ads`, when the outcome is
**accept** (live-ads §9.4), a **gap** being `source_gap`. A `stitched` source is always
@ -182,11 +207,24 @@ accepted, since its verification refuses nothing as a whole (stitched-presentati
record it does not accept, of a signer outside the scope among them, bills nothing, and a claim
it does not find presented counts nothing. A source that is not such an object,
whose `kind` is none of these, or whose members are not exactly its kind's, each of its form, is
refused `malformed`.
refused `malformed`. Under a tariff of version 1, a `job` source is not a kind, and is
`malformed`.
A job source's **billed hop** is the attempt of its route record that is billed (route-record
§5.2), if any; its **serving capability** is that attempt's candidate's `capability`, and its
**figure** is the figure of that attempt's OER/2 receipt: the unmetered marker
(oer-receipts §3.1), or `energy.joules_total` in microjoules, rounded to the nearest and a
half to the even one, with the receipt's `energy.class`. A billed hop's receipt whose figure
has no class, or rounds below 1 µJ or past what a u64 holds, cannot be stated in a row
(§6.6): the source is refused `job_figure_invalid`, at `receipts` and the receipt's index.
### 5.2 What a source contributes
Each source contributes **records**, each named by a kind and an identity:
Each source contributes **records**, each named by a kind and an identity. This table is the
**one registry of record kinds**: every WAI text that names a record by a kind (in a citation
digest, or in a list of records) takes the kind from here, and a new kind is assigned here, the
next free one. Kinds 10 to 13 are defined by the texts that cite them and are no ledger source's;
the others are the ledger's own:
| record kind | record | identity | contributed by |
|---|---|---|---|
@ -199,11 +237,27 @@ Each source contributes **records**, each named by a kind and an identity:
| 7 | a presentation claim, `WPC1` | its digest (ad-presentation §5.3) | an `ads` source: each claim that reads |
| 8 | a pod-input record, `WSI1` | its digest (stitched-presentation §4.4) | a `stitched` source: each record it accepts |
| 9 | a stitched claim, `WSC1` | its digest (stitched-presentation §5.3) | a `stitched` source: each claim that reads |
| 10 | a run of a channel's program-as-run | its identity ([revenue-share](revenue-share.md) §7.4) | no source: cited by revenue-share §7.4 and opportunity-report §7.3 |
| 11 | an invoice, `WLI1` | its digest (§6.7, as [revenue-share](revenue-share.md) §7.4 cites it) | no source: cited by revenue-share §7.4 |
| 12 | a cue event | its event digest ([opportunity-report](opportunity-report.md) §7.3, live-events §3.2) | no source: cited by opportunity-report §7.3 |
| 13 | a room policy | its digest ([opportunity-report](opportunity-report.md) §7.3, live-ads §3.2) | no source: cited by opportunity-report §7.3 |
| 14 | an OER/2 receipt | the 32 bytes its `receipt_id` names (oer-receipts §5) | a `job` source (version 2): the receipt of each attempt of its record |
| 15 | a `wai.video.int_motion` decode receipt | its receipt hash (integer-payloads §15.1) | a `job` source (version 2): each output receipt |
| 16 | a route record | its digest (route-record §5.3) | a `job` source (version 2): its record |
Kinds 17 to 127 are unassigned. Kinds **128 to 255 are reserved for private and provisional
use**: an implementation's own records, such as a hosted service's provisional records carried
beside WAI's until WAI defines them, may name records by them, and no kind WAI assigns later will
ever be one of them. A WAI record never states one, and a WAI verifier refuses one wherever a WAI
record states a kind, `private_kind`: in this extension, an invoice's `records` (§6.8). No citation
digest WAI defines covers one.
A source's **head** is its receipt, statement or package receipt; an `ads` source's heads are its
receipts, and a `stitched` source's its accepted records. A head of one source that another source of the invoice also contributes is refused
receipts, a `stitched` source's its accepted records, and a `job` source's its route record. A head of one source that another source of the invoice also contributes is refused
`record_duplicate`; a per-hop record, which a room receipt and a statement can both name, is
contributed once.
contributed once. An OER/2 receipt or an output receipt a `job` source contributes that an earlier
`job` source of the invoice also contributes is refused `record_duplicate` too: one hop serves one
job, and one output is one job's, so neither is billed for two.
### 5.3 Placement in time
@ -214,11 +268,13 @@ A source is **in** a period when:
- `delivery`: the statement's period lies within it;
- `ads`: the `t` of the break each receipt names, placed by the room's `t0`, is within it;
- `package` and `stitched`: always: a package receipt and a pod-input record state no time, and
the chain (§6.9) bills each once.
the chain (§6.9) bills each once;
- `job`: its route record's `accepted` is within it.
Two room receipts of one room whose sessions overlap, two statements of one party whose periods
overlap, and two receipts of the invoice's ads sources naming one break and one session, bill one
thing twice: the later is refused `record_overlap`.
overlap, two receipts of the invoice's ads sources naming one break and one session, and two
route records of one router naming one `job`, bill one thing twice: the later is refused
`record_overlap`.
## 6. The invoice
@ -227,7 +283,7 @@ thing twice: the later is refused `record_overlap`.
| field | size | meaning |
|---|---|---|
| `magic` | 4 | `WLI1` (ASCII) |
| `version` | 1 | 1 in this revision |
| `version` | 1 | its tariff's version, 1 or 2 |
| `seller`, `buyer` | 32 each | the tariff's |
| `tariff` | 32 | the tariff's digest |
| `prev` | 32 | the digest of the invoice before it in its chain (§6.9); zero for the first |
@ -236,7 +292,7 @@ thing twice: the later is refused `record_overlap`.
| `records` | 4 + 33 × *r* | a u32 count, then each record (§5.2): `kind` (u8) and `id` (32), in strictly ascending order of (`kind`, `id`) |
| `lines` | 2 + 94 × *l* | a u16 count, then each line (§6.3), in strictly ascending order of (`source_kind`, `source`, `unit`) |
| `energy` | 1 + 18 × *e* | a u8 count, then each row (§6.6): `scope` (u8), `tag` (u8), `figures` (u32), `joules` (u64, µJ), `max_ppm` (u32), in strictly ascending order of (`scope`, `tag`) |
| `unmetered` | 16 | four u32: the unmetered figures of scopes 1 to 4 (§6.6) |
| `unmetered` | 16 or 20 | four u32: the unmetered figures of scopes 1 to 4 (§6.6); in version 2, five, scopes 1 to 5 |
| `amount` | 8 | u64: the sum of the lines' amounts |
| `sig` | 64 | §6.7 |
@ -251,8 +307,8 @@ A line is 94 bytes:
| field | size | meaning |
|---|---|---|
| `source_kind` | 1 | 1 room, 2 delivery, 3 package, 4 ads |
| `source` | 32 | the room id, the delivering party, the packager or the ad decider |
| `source_kind` | 1 | 1 room, 2 delivery, 3 package, 4 ads, 5 job (version 2) |
| `source` | 32 | the room id, the delivering party, the packager, the ad decider or the router |
| `unit` | 1 | the unit (below) |
| `quantity` | 8 | u64: the quantity of the unit the records state |
| `price` | 8 | u64: the tariff's price for the unit |
@ -276,6 +332,7 @@ each:
| 3 | 3 | `stored_bytes` | their `stored_bytes` | `package` 2 | 10^9 |
| 4 | 1 | `presented_impressions` | under a policy that requires claims, the impressions that count (ad-presentation §9.2); of a stitched source, the presentations of its accepted records' ads (stitched-presentation §7.4), one per session | `ads` 0 | 1 |
| 4 | 2 | `billable_impressions` | under a policy that does not, the billable impressions (quartile at least 1) | `ads` 1 | 1 |
| 5 | *k* | the *k*-th job price's unit | over the router's job sources with a billed hop whose serving capability is that price's `capability`: one per job (`jobs`), or their output receipts' `work_pixel_ops` (`subpixels`) | the *k*-th job price's `price` | its `divisor` |
There is one line for each source and unit whose quantity is above 0, a price of 0 included, and
none for a quantity of 0. A delivery line's bytes are those its party's downstream parties signed
@ -301,7 +358,9 @@ Each line cites the records it came from:
- a package line: the package receipts of its packager whose figure of the unit is above 0;
- an ad line: the receipts of its decider with at least one impression counted under the unit, and,
for `presented_impressions`, each counted impression's own claim; of a stitched source, its
stitcher's accepted records, and each presented own claim of their ads.
stitcher's accepted records, and each presented own claim of their ads;
- a job line: for each job it counts, its route record and its billed hop's OER/2 receipt, and,
for `subpixels`, each of its output receipts whose `work_pixel_ops` is above 0.
`cited` is their number, and `cites` is SHA-256 over the 17 bytes `wai:ledger-cites` (ASCII) and
0x01, followed by each cited record's `kind` and identity, in ascending order of (`kind`,
@ -318,6 +377,7 @@ Each record that states an energy figure states it once, in one **scope**:
| 2 `delivery_hops` | the per-hop records named by the statements and by no room receipt of the invoice |
| 3 `packaging` | the package receipts' `joules_micro` and class |
| 4 `ad_decisions` | the ad receipts' `joules_micro` and class; a pod-input record states no figure |
| 5 `jobs` (version 2) | each job source's billed hop's figure (§5.1): the hop that served the job, in the class its OER/2 receipt states; a job with no billed hop states none |
For each scope, `unmetered` counts its figures that are the unmetered marker, which are in no row
and **MUST NOT** be presented as zero joules. For each scope and each tag among its metered
@ -343,13 +403,15 @@ covers, verified strictly (SPEC §7 *Signatures*).
A reader reads exactly the bytes it is given as one invoice, as §4.3 reads a tariff with `WLI1`
for `WLT1` (`truncated`, `malformed`, `unknown_version`), and refuses as `classes_summed` an
invoice with a row of tag 0. It refuses as `invoice_invalid` one that breaks any of these:
invoice with a row of tag 0. It refuses as `private_kind` an invoice with a record of a kind 128 to 255 (§5.2), and as
`invoice_invalid` one that breaks any of these:
- `start` is below `end`; `currency` is three capitals `A`–`Z`;
- each list is in the strictly ascending order §6.1 gives it; each record's `kind` is 1 to 9;
- each line's `source_kind` is 1 to 4, its `unit` a unit of its kind (§6.3), its `quantity` and
`cited` above 0;
- each row's `scope` is 1 to 4, its `tag` 2 to 5 and its `figures` above 0.
- each list is in the strictly ascending order §6.1 gives it; each record's `kind` is 1 to 9 (in
version 2, also 14 to 16: kinds 10 to 13 are no ledger source's, §5.2);
- each line's `source_kind` is 1 to 4 (1 to 5 in version 2), its `unit` a unit of its kind (§6.3;
for kind 5, at least 1), its `quantity` and `cited` above 0;
- each row's `scope` is 1 to 4 (1 to 5 in version 2), its `tag` 2 to 5 and its `figures` above 0.
Then the signature (§6.7): `bad_key`, `bad_signature`.
@ -377,9 +439,9 @@ It runs these steps and stops at the first refusal, which it reports with the re
1. **The tariff** (§4.3), at `tariff`, or `tariff` and `seller` or `buyer` for a signature:
`truncated`, `malformed`, `unknown_version`, `tariff_invalid`, `bad_key`, `bad_signature`.
2. **The invoice** (§6.8), at `invoice`: `truncated`, `malformed`, `unknown_version`,
`classes_summed`, `invoice_invalid`, `bad_key`, `bad_signature`.
3. **The binding**, at `invoice`: its `tariff` is the tariff's digest, and its `seller`, `buyer`
and `currency` the tariff's: otherwise `tariff_mismatch`; its period lies within the tariff's:
`classes_summed`, `private_kind`, `invoice_invalid`, `bad_key`, `bad_signature`.
3. **The binding**, at `invoice`: its `tariff` is the tariff's digest, and its `version`,
`seller`, `buyer` and `currency` the tariff's: otherwise `tariff_mismatch`; its period lies within the tariff's:
otherwise `outside_tariff`.
4. **The previous**, at `previous`, when `prev` is not zero: an invoice whose digest is `prev` is
given (`record_missing`); it reads and its signature verifies (§6.8, with those reasons); its
@ -409,9 +471,10 @@ A figure that cannot be held in its field is a difference.
- **refuse**: the reason and where.
- **accept**: the invoice's digest; the tariff's digest; the currency and the period; the number of
records; each line with its source kind, source, unit name, quantity, price, amount and its cited
records; each energy row by scope and class label with its figures, joules and `max_ppm`; each
scope's unmetered figures and coverage; the amount; and the reports of §8.
records; each line with its source kind, source, unit name (and, for a job line, the job
price's `capability`), quantity, price, amount and its cited records; each energy row by scope and class label with its figures, joules and `max_ppm`; each
scope's unmetered figures and coverage (scopes 1 to 4, and in version 2 scope 5); the amount; and
the reports of §8.
## 8. Reports
@ -501,13 +564,17 @@ The codes this extension adds, each with where it is raised:
| `quantity_not_recorded` | §7.2 step 7 |
| `citation_mismatch` | §6.5, §7.2 step 7 |
| `invoice_mismatch` | §7.2 step 7 |
| `private_kind` | §5.2, §6.8, §7.2 step 2 |
| `output_invalid` | §5.1, §7.2 step 5 |
| `job_figure_invalid` | §5.1, §7.2 step 5 |
A verifier of this extension also raises `truncated`, `malformed` and `unknown_version` for a
tariff or an invoice that does not read, and `malformed` for a source that does not read (§5.1); `bad_key` and `bad_signature` for their signatures;
`record_missing` (live-metering §9) for a previous invoice not given; `classes_summed`
(live-metering §9) for a row over classes; and, at a source, every code its extension's verifier
raises: live-metering §9 and the codes it lists, settlement §11 and the codes it lists, packaging
§12, and live-ads §10 with ad-presentation §11. A `stitched` source raises none: the codes of
§12, live-ads §10 with ad-presentation §11, and, at a `job` source, route-record §8 and the
codes it reuses. A `stitched` source raises none: the codes of
stitched-presentation §9 are the verdicts of its records and claims, stated in its report (§8.1).
## 11. Conformance
@ -516,8 +583,10 @@ stitched-presentation §9 are the verdicts of its records and claims, stated in
`vectors.json`, written by `wai_receipt_ledger_vectors gen` from fixed seeds and from sources taken
from the corpora of the extensions it reads (`live-conformance/metering/`,
`live-conformance/settlement/`, `live-conformance/ads/`, `live-conformance/presentation/`,
`live-conformance/stitched/` and `pack-conformance/packs/`), with delivery statements of two
parties signed for it. The sources are
`live-conformance/stitched/`, `live-conformance/routes/` and `pack-conformance/packs/`), with
delivery statements of two parties signed for it and, for each media job, a sink's
`wai.video.int_motion` decode receipt of its output over a payload of
`wai-rs/tests/vectors/int_video/`. The sources are
held once, by name, and each vector is a verification document (§7.1) naming its sources, with the
outcome §7.3 gives it:
@ -527,10 +596,19 @@ outcome §7.3 gives it:
object from both parties among the overlaps; the next invoice of its chain, billing a relay's
session whose energy is unmetered, and a second package; and an invoice whose room hops are of
three classes, one row each; and an invoice of streams stitched by others, one with a form-1
receipt whose own figures the report states beside the presented ones.
receipt whose own figures the report states beside the presented ones; and, under a tariff of
version 2, a month of media jobs one router routed, beside a package: an `int_motion` job billed
per job and per subpixel of its output's receipt, a failover billed at the fallback's price, a
job served past its deadline and one every allowed hop failed, which bill nothing, and a job
whose serving hop was unmetered, counted unmetered in the jobs' scope.
- **Refused**: each reason of §10 at least once, each vector a control of one defect; a source of
each kind outside the scope, a room receipt and a statement under other prices, and a source of
each timed kind outside the period; and a refusal of each source kind by its own extension.
each timed kind outside the period; a refusal of each source kind by its own extension; and, for
version 2, job prices out of order, a router in a version-1 tariff's scope, a job source under a
version-1 tariff, a version-1 invoice under a version-2 tariff, a job given twice, routed twice,
and claiming another job's output, and a job line and a jobs row that do not recompute; and an
invoice of each version listing a record of a kind reserved for private use, and one listing
another text's kind.
`wai_receipt_ledger_vectors verify` and the test `tests/receipt_ledger.rs` check that the file
regenerates byte for byte, that `wai-rs` gives every vector its outcome, that every reason of §10
@ -539,8 +617,9 @@ refuses a vector, and that §10 lists exactly the reference's codes.
it shares no code with `wai-rs`, verifies signatures with WAI's strict rule written from RFC 8032
(`claim-conformance/_ed25519.py`), verifies each source with that corpus's own second
implementation (`live_metering_verify.py`, `settlement_verify.py`, `live_ads_verify.py` with
`ad_presentation_verify.py`, `stitched_presentation_verify.py`, and
`pack-conformance/pack_verify.py`), recomputes every line,
`ad_presentation_verify.py`, `stitched_presentation_verify.py`, `route_record_verify.py`, and
`pack-conformance/pack_verify.py`) and each output receipt by `_int_motion_receipt.py`, written
from integer-payloads §15, recomputes every line,
citation, row and report itself, gives every vector the same outcome, and requires every mutated
copy of the corpus to fail.

3
wai/extensions/revenue-share.md
View file

@ -390,7 +390,8 @@ instant, and the invoice that billed it (kind 11, its digest). `cited` is their
counted once, and `cites` is SHA-256 over the bytes `wai:share-cites` (ASCII) and 0x01, followed by
each cited record's `kind` (u8) and identity (32), in ascending order of (`kind`, identity). A line
that cites no record has no source (`line_unsourced`, §8.2 step 7). An owner shows one line to an
auditor with the records it cites, the contract and the runs.
auditor with the records it cites, the contract and the runs. Kinds 10 and 11, like every kind a
citation digest names, are registered in [receipt-ledger](receipt-ledger.md) §5.2.
### 7.5 Digest and signature

499
wai/extensions/route-record.md Normal file
View file

@ -0,0 +1,499 @@
# WAI Extension: Route Record
> Status: Draft. Registers the record `wai.route.record`: what a router considered for one
> media job and why it chose. A job names an operation of WAI's catalogue, a capability and
> a fallback ([SPEC](../SPEC.md) §4), the quantities it expects, hard limits that remove
> candidates and soft preferences that reorder the rest. A router reads its candidates from
> a **price book**, removes those a hard limit removes, ranks the survivors under a named
> and versioned **routing policy**, tries them in that order, and signs a **route record**:
> every candidate with the figures the price book stated for it and the limits that removed
> it, the order, the decision and why, and every attempt with its hop's OER/2 receipt and
> whether it was billed. A verifier given the record, the request, the price book and the
> policy recomputes the candidates, every removal, the ranking, the order and the decision,
> holds the attempts to the order and the billing rules, and, given the hops' receipts,
> holds each attempt to its receipt; it refuses a record those inputs do not give. No
> transport, no payment and no account: a route record is the recomputable account of a
> routing decision. The reference implementation is the `route_record` module of `wai-rs`
> (feature `route_record`); the corpus is
> [`live-conformance/routes/`](../live-conformance/routes/), and its second implementation
> is `live-conformance/route_record_verify.py`.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
>
> Texts read for this revision: the WAI texts this one cites; RFC 8259; RFC 8785; RFC 8032;
> ISO 4217 (the alphabetic currency code); FIPS 180-4.
## 1. Scope
In scope: the three documents a routing decision rests on, the request, the price book and
the policy (§3); how a router computes its candidates, their removals, their ranking and
its decision from them (§4); the record, its attempts and its signature (§5); how an
attempt is held to its hop's OER/2 receipt (§6); verification (§7); and the reference
implementation and corpus (§10).
Out of scope: a transport; how a router learns its prices, which the price book states and
this extension takes as stated; accounts, credit and payment; the job's output and its own
receipts, which their extensions define; and the energy of a hop, which its OER/2 receipt
states ([oer-receipts](oer-receipts.md)). An invoice that bills a job from its route record
is the receipt ledger's ([receipt-ledger](receipt-ledger.md) §5).
Relationship to the core spec: `wai.route.record` is a **record** ([SPEC](../SPEC.md) §5): a
signed record a WAI tool emits and verifies. It names no payload format and is not
dispatchable.
## 2. Terms and encodings
- **Router**: an Ed25519 key (RFC 8032) that signs route records. **Signer**: the record's
`signer`.
- **Time**: microseconds since 1970-01-01T00:00:00Z, an integer.
- **Amount**: an integer of millionths of a currency's minor unit (ISO 4217), as the
receipt ledger's amounts are (receipt-ledger §2). A **price** is an amount per `divisor`
units of a unit.
- **Document**: a JSON text (RFC 8259) read strictly: no member name repeated within an
object, no string holding an unpaired surrogate, arrays and objects nested at most 64
deep (the document counting as one), and every number an integer from 0 to 2⁵³ − 1
written in plain decimal (no sign, fraction or exponent). A text that is not such a
document is `malformed`. Every value of a document is therefore in the subset RFC 8785
canonicalises without a choice (the subset `wai::jcs` writes), and its **canonical form**
is RFC 8785's.
- **Digest**: SHA-256 (FIPS 180-4) over a document's canonical form, written as 64
lowercase hexadecimal digits.
- **Token**: a string of 1 to 128 bytes, each an ASCII letter or digit or one of `.`, `_`,
`-`, `:` and `/`. Endpoints, providers, regions, residencies and a policy's name are
tokens. Tokens are ordered by their bytes.
- **Unit**: a string of 1 to 64 bytes, a lowercase ASCII letter and then lowercase letters,
digits and `_` (`subpixels`, `seconds`, `packages`): the name of what a quantity counts.
- **Capability**: a string of 1 to 128 bytes of lowercase ASCII letters, digits, `_` and
`.`, beginning `wai.`: a capability of WAI's registry (SPEC §5), which a router **SHOULD**
name only when it is registered.
- **Operation**: the id of an operation of WAI's catalogue (`wai/catalogue/ops.json`): two
words joined by `.`, each a lowercase ASCII letter and then lowercase letters and digits
(`media.encode`).
- **Key** and **signature**: 64 and 128 lowercase hexadecimal digits.
- **Class**: an acquisition class of [energy-measurement](energy-measurement.md) §2, by its
label: `CalibratedInstrument`, `OnChipCounter`, `ModelBased` or `Estimator`.
- ⌊x⌋ is x rounded down; products are computed in integers wide enough not to overflow.
## 3. The documents
Each document is an object of exactly the members its table lists, each of its form; a
document that breaks a rule of this section is refused with its own reason
(`request_invalid`, `pricebook_invalid`, `policy_invalid`).
### 3.1 The request
What the job asked for, as the router received it.
| member | form |
|---|---|
| `kind` | `"wai-route-request"` |
| `job` | a token: the job's id |
| `operation` | an operation |
| `capability` | a capability |
| `fallback` | a capability other than `capability`, or `null` |
| `quantities` | 1 to 16 objects of exactly `unit` (a unit) and `quantity` (an integer at least 1), in strictly ascending order of `unit`: what the job is expected to need, unit by unit, before anything ran |
| `limits` | an object of exactly the members below |
| `prefer` | an object of exactly `sort` (`null` or one of `price`, `ttfb`, `energy`, `quality`), `ttfb_ms` (`null` or an integer: a time to first byte, in milliseconds) and `quality` (`null` or an integer 0 to 1000: a quality floor) |
| `deadline_us` | `null` or an integer at least 1: the time after acceptance by which the job's output must be served |
`limits`, the hard limits:
| member | form |
|---|---|
| `regions` | `null`, or 1 to 64 tokens in strictly ascending order: the regions an endpoint may run in |
| `residency` | `null`, or 1 to 64 tokens in strictly ascending order: the places the job's data may rest |
| `no_training` | a boolean: no endpoint that trains on its inputs |
| `zero_retention` | a boolean: no endpoint that keeps the job's data |
| `ga_only` | a boolean: only endpoints generally available, not previews |
| `consent_on_file` | a boolean: whether the job's owner holds consent for what an endpoint that needs it does (a cloned voice) |
| `max_price` | 0 to 16 objects of exactly `unit`, `price` and `divisor` (an integer at least 1), in strictly ascending order of `unit`: the most the job pays per `divisor` units of the unit |
| `spend_ceiling` | `null`, or an object of exactly `currency` (three capitals `A`–`Z`) and `amount`: the most the job may cost |
### 3.2 The price book
What each endpoint offers and costs, as the router read it.
| member | form |
|---|---|
| `kind` | `"wai-route-pricebook"` |
| `currency` | three capitals `A`–`Z` |
| `issued` | a time: when the prices were stated |
| `entries` | 1 to 1024 objects of exactly the members below, in strictly ascending order of `endpoint` |
An entry:
| member | form |
|---|---|
| `endpoint` | a token: the endpoint's id |
| `provider` | a token: who runs it |
| `capability` | a capability: what it produces |
| `region`, `residency` | a token, or `null` when not known |
| `trains_on_inputs` | a boolean, or `null` when not known |
| `retention_days` | an integer, or `null` when not known: how long it keeps a job's data (0: not at all) |
| `ga` | a boolean, or `null` when not known: generally available |
| `needs_consent` | a boolean, or `null` when not known: whether what it does needs the consent of the person it imitates |
| `unit` | a unit: what it is priced by |
| `divisor` | an integer at least 1 |
| `price` | an integer: its price per `divisor` units |
| `ttfb_ms` | an integer, or `null` when not known: its time to first byte, in milliseconds |
| `energy_uj` | an integer, or `null` when not stated: the microjoules it states per `divisor` units |
| `energy_class` | the class of that figure, or `null` exactly when `energy_uj` is `null` |
| `quality` | an integer 0 to 1000, or `null` when not known |
A price book states each figure as its endpoint's provider gave it; a value that is not known
is `null`, and §4.3 reads every `null` limit as the most restrictive value.
### 3.3 The policy
How survivors are ranked and how many are tried.
| member | form |
|---|---|
| `kind` | `"wai-route-policy"` |
| `name` | a token |
| `version` | an integer at least 1 |
| `keys` | the four ranking keys `price`, `ttfb`, `energy` and `quality`, each once, in the policy's order |
| `max_attempts` | an integer 1 to 16: the most survivors tried |
| `exclude` | 0 to 64 provider tokens in strictly ascending order: providers the policy never routes to |
A policy is named and versioned so that a deployment pins it: the record states its name,
version and digest, and a policy whose rules change takes a new version.
## 4. Routing
A router computes the following from the request, the price book and the policy, and a
verifier computes the same. Nothing in it depends on the order of a document's members, on
the time, or on anything the three documents do not state.
### 4.1 Candidates
The **candidates** are the price book's entries whose `capability` is the request's
`capability`, which are **direct** (`via_fallback` false), and, when the request names a
`fallback`, those whose `capability` is the fallback (`via_fallback` true), in the price
book's order. An entry of any other capability is not a candidate, and the record does not
list it.
### 4.2 Estimates
For a candidate whose `unit` the request states a quantity `q` for:
- its **cost** is ⌊`q` × `price` / `divisor`⌋;
- its **energy** is ⌊`q` × `energy_uj` / `divisor`⌋ when `energy_uj` is stated.
An estimate is `null` when the request states no quantity in the candidate's unit, when
`energy_uj` is `null` (for the energy), and when it exceeds 2⁵³ − 1, which no document can
state. Units are never converted: a candidate priced in `seconds` is estimated from the
request's `seconds` and from nothing else. An estimate is the router's, from the request's
own quantities; what a job costs is what its invoice bills from its receipts.
### 4.3 Hard limits
Each candidate is held to every limit in this order, and its **removals** list each limit
that holds, in this order, each with the candidate's value the limit read (`null` when the
price book does not know it):
| limit | removes a candidate when | value |
|---|---|---|
| `policy` | its `provider` is in the policy's `exclude` | `provider` |
| `unit` | the request states no quantity in its `unit` | `unit` |
| `region` | the request's `regions` is not `null`, and its `region` is `null` or not in it | `region` |
| `residency` | the request's `residency` is not `null`, and its `residency` is `null` or not in it | `residency` |
| `training` | `no_training` is true and its `trains_on_inputs` is not `false` | `trains_on_inputs` |
| `retention` | `zero_retention` is true and its `retention_days` is not 0 | `retention_days` |
| `ga` | `ga_only` is true and its `ga` is not `true` | `ga` |
| `consent` | `consent_on_file` is false and its `needs_consent` is not `false` | `needs_consent` |
| `unit_price` | `max_price` has an entry for its `unit`, and `price` × that entry's `divisor` exceeds that entry's `price` × `divisor` | `price` |
| `spend` | `spend_ceiling` is not `null`, and its `currency` is not the price book's, or the candidate's cost is `null` or exceeds its `amount` | the cost |
So a value the price book does not know counts as the most restrictive one: an endpoint that
does not say it never trains on its inputs is removed by `no_training`, one with no stated
retention by `zero_retention`, and a cost that cannot be estimated, or a ceiling in another
currency, by `spend`. A candidate with no removal is a **survivor**.
### 4.4 Ranking
The survivors are ordered by a list of keys, the **key list**:
1. `targets`, when the request's `prefer.ttfb_ms` or `prefer.quality` is not `null`: a
survivor **meets** the targets when, for each target stated, its `ttfb_ms` is stated and
at most `prefer.ttfb_ms`, and its `quality` is stated and at least `prefer.quality`;
2. the request's `prefer.sort`, when it is not `null`;
3. the policy's `keys`, in its order, without `prefer.sort`;
4. `endpoint`.
Two survivors compare by the first key of the list on which they differ, and the one that
comes first under that key ranks first:
| key | first |
|---|---|
| `targets` | the one that meets the targets |
| `price` | the smaller cost |
| `ttfb` | the smaller `ttfb_ms` |
| `energy` | the smaller energy |
| `quality` | the larger `quality` |
| `endpoint` | the smaller `endpoint`, by its bytes |
A `null` figure comes after every stated one, and two `null`s do not differ. Endpoints are
distinct, so the order is total. The **order** is the survivors' endpoints in it, and each
survivor's **rank** is its position there, from 1. The energy compared is each figure as its
provider states it, whatever its class: the record states every class beside its figure, so
a reader sees what was compared.
### 4.5 The decision
`decided_by` is `none` when there is no survivor, `only_survivor` when there is one, and
otherwise the first key of the key list on which the first two survivors of the order
differ: the reason the first was tried before the second. `runner_up` is the second
survivor's endpoint, or `null` when there are fewer than two.
The number of survivors the router may try is the smaller of the survivors' number and the
policy's `max_attempts`, the **allowed** attempts.
## 5. The record
### 5.1 Members
A route record is a document of exactly these members:
| member | form |
|---|---|
| `kind` | `"wai-route-record"` |
| `version` | `1` in this revision |
| `job`, `operation`, `capability`, `fallback`, `quantities` | the request's |
| `request` | the request's digest |
| `policy` | an object of exactly `name`, `version` and `digest`: the policy's |
| `pricebook` | an object of exactly `digest`, `currency` and `issued`: the price book's |
| `candidates` | the candidates (§4.1), in their order, each an object of exactly the members below |
| `order` | the order (§4.4): the survivors' endpoints |
| `decision` | an object of exactly `decided_by` and `runner_up` (§4.5) |
| `accepted` | a time: when the router accepted the job |
| `deadline` | `null` when the request's `deadline_us` is `null`; otherwise `accepted` + `deadline_us` |
| `attempts` | 0 to 16 objects of exactly the members of §5.2, in the order they were made |
| `signer` | the router's key |
| `sig` | §5.3 |
A candidate:
| member | form |
|---|---|
| `endpoint`, `provider`, `capability`, `region`, `residency`, `trains_on_inputs`, `retention_days`, `ga`, `needs_consent`, `unit`, `divisor`, `price`, `ttfb_ms`, `energy_uj`, `energy_class`, `quality` | the price book's entry's, as it stated them |
| `via_fallback` | a boolean (§4.1) |
| `cost`, `energy` | its estimates (§4.2) |
| `removed` | its removals (§4.3): objects of exactly `limit` and `value`, in §4.3's order; empty for a survivor |
| `rank` | its rank (§4.4), or `null` for a candidate a limit removed |
### 5.2 Attempts
An attempt is one hop the router sent the job to:
| member | form |
|---|---|
| `hop` | its position among the attempts, from 1 |
| `endpoint` | the endpoint it was sent to |
| `outcome` | `served` (the hop returned the job's output) or `failed` |
| `error` | `null` when served; when failed, one of `timeout`, `server_error`, `rate_limited`, `refused` and `unavailable` |
| `start`, `end` | times: when it was sent and when it ended |
| `billed` | a boolean: whether the job is billed for it |
| `not_billed` | `null` when billed; otherwise `failed` or `deadline_missed` |
| `receipt` | the `receipt_id` of the hop's OER/2 receipt (oer-receipts §5): `sha256:` or `blake3:` and 64 lowercase hexadecimal digits |
The attempts **MUST** follow the order: the attempt at position *i* names the *i*-th survivor
of the order, and there are at most the allowed attempts (§4.5). So no attempt names a
removed candidate, and no survivor is tried before one ranked above it. Only the last attempt
may be `served`; a job whose last attempt failed was tried on every allowed survivor before
the router gave up. The times are in order: `accepted` is at most the first attempt's
`start`, each attempt's `start` is at most its `end`, and each attempt starts no earlier than
the one before it ended.
Billing:
- a failed attempt is not billed, `not_billed` `failed`: a hop that returned no output costs
the job nothing, whatever the provider charged the router;
- a served attempt that ended after the `deadline` is not billed, `not_billed`
`deadline_missed`: a job that missed the deadline it declared is not billed;
- every other served attempt is billed.
So at most one attempt of a record is billed, and a reader finds the hop that served, whether
it was billed and why not, and the receipt of every hop, failed ones too.
### 5.3 Signature and digest
The record's `sig` is Ed25519 by `signer` over the 17 bytes `wai:route-record` (ASCII) and
0x01, followed by the canonical form of the record without `sig`. It is verified strictly, as
[SPEC](../SPEC.md) §7 *Signatures* requires: a key that fails step 1 or 2 is `bad_key`, and a
signature that fails a later step is `bad_signature`. The record's **digest**, which names it
(receipt-ledger §5.2), is the digest of the whole record, `sig` included.
## 6. Hop receipts
Every hop is receipted under [oer-receipts](oer-receipts.md): a `job` receipt whose
`capability` is `oer.job.` followed by the record's `operation` (oer-receipts §3), whose
`work.status` is `1` for a hop that served and `-1` for one that failed, and whose figure is
the hop's energy, or the unmetered marker where nobody measured it (oer-receipts §3.1). When
a verifier is given the receipts and the keys they are published under (oer-receipts §5), it
holds each attempt to its receipt:
- a receipt whose `receipt_id` is the attempt's `receipt` is given: otherwise
`receipt_missing`;
- it verifies under oer-receipts on its own, with no reason (§7 there): otherwise
`receipt_invalid`;
- it is a `job` receipt of the record's operation whose `work.status` is the attempt's
outcome's: otherwise `receipt_mismatch`.
A receipt states the hop's energy with its grade, and, in minor version 1, its class or the
unmetered marker; the record states no figure of its own. A reader of the hops' energy reads
it from their receipts, and an invoice bills a job from them (receipt-ledger §5).
## 7. Verification
### 7.1 Inputs
A verifier is given a **verification document**: a JSON object of exactly `request`,
`pricebook`, `policy` and `record`, each a string holding that document's text, `receipts`,
`null` or a list of strings each holding an OER/2 receipt's text, and `keys`, `null` or an
object of a receipt's `chain.signer` names to keys, as oer-receipts §5 publishes them. The
texts are carried as strings so that each is read as written. A verification document that is
not such an object, or whose `receipts` is a list while its `keys` is `null`, is `malformed`.
### 7.2 Steps
A verifier runs these steps and stops at the first refusal, which it reports with its reason
and where (`at`, a list: the member, then indexes or members within it):
1. **The request** (§3.1), at `request`: `malformed`, `request_invalid`.
2. **The price book** (§3.2), at `pricebook`: `malformed`, `pricebook_invalid`.
3. **The policy** (§3.3), at `policy`: `malformed`, `policy_invalid`.
4. **The record** (§5), at `record`: a text that is not a document, or whose `kind` is not
`wai-route-record`, is `malformed`; a `version` that is an integer other than `1` is
`unknown_version`, at `record`, `version`; one that is not of §5.1's and §5.2's members,
each of the form its table gives it (a restated member of a candidate, the form §3.2 gives
it), is `record_invalid` (an attempt whose `hop` is not its position, or whose `error` is
not as its `outcome` requires, among them; the relations between a candidate's members,
such as an energy figure and its class, are step 6's); then its signature (§5.3):
`bad_key`, `bad_signature`, at `record`, `sig`.
5. **The binding**, at `record` and the member: its `job`, `operation`, `capability`,
`fallback`, `quantities`, `request` and `deadline` are not the request's
(`request_mismatch`); its `policy` is not the policy's name, version and digest
(`policy_mismatch`); its `pricebook` is not the price book's digest, currency and `issued`
(`pricebook_mismatch`).
6. **The candidates**, at `record`, `candidates` and the index, recomputed by §4, each in
turn and for each these three in this order: a candidate missing, added or out of order,
or with a member other than `removed` and `rank` not what §4 gives it
(`candidate_mismatch`); its `removed` not its removals (`removal_mismatch`); its `rank` not
its rank (`rank_mismatch`). A list of the wrong length is a `candidate_mismatch` at the
first index past the shorter. Then, at `record`, `order`, the order (`rank_mismatch`); and
at `record`, `decision`, the decision (`decision_mismatch`).
7. **The attempts**, at `record`, `attempts` and the index, each in turn, by §5.2: an attempt
past the allowed number (`attempts_exceeded`); one that names an endpoint that is not a
survivor (`attempt_removed`); one that names a survivor not at its position in the order
(`attempt_out_of_order`); a served attempt that is not the last (`served_not_last`); times
out of order (`times_invalid`); `billed` and `not_billed` not what §5.2 gives
(`billing_invalid`). Then, at `record`, `attempts`: a job whose last attempt failed, or
that made none while a survivor was allowed, with fewer attempts than allowed
(`attempts_incomplete`).
8. **The hop receipts** (§6), when `receipts` is not `null`: first each receipt's text, at
`receipts` and its index, which a receipt that is not a document under oer-receipts §4
fails (`receipt_invalid`); then each attempt in turn, at `record`, `attempts`, its index and
`receipt`: `receipt_missing`, `receipt_invalid`, `receipt_mismatch`, the first receipt of the
list whose `receipt_id` is the attempt's being its receipt.
### 7.3 Outcome
- **refuse**: the reason and where.
- **accept**: the record's digest; the job and its operation; the served endpoint and the hop
that served it, or `null`s for a job no hop served; the billed hop, or `null`; the deadline,
`none` when none was declared and otherwise `met` or `missed` by the served hop, or `none`
too when no hop served; the order; the decision; each removed candidate's endpoint with the
limits that removed it; whether the hop receipts were `checked` or `not_checked`; and, when
checked, the hops whose receipts state the unmetered marker (oer-receipts §3.1), which a
reader **MUST NOT** present as hops of zero joules.
## 8. Reasons
The codes this extension adds, each with where it is raised:
| reason | raised at |
|---|---|
| `request_invalid` | §3.1, §7.2 step 1 |
| `pricebook_invalid` | §3.2, §7.2 step 2 |
| `policy_invalid` | §3.3, §7.2 step 3 |
| `record_invalid` | §5.1, §5.2, §7.2 step 4 |
| `request_mismatch` | §7.2 step 5 |
| `policy_mismatch` | §7.2 step 5 |
| `pricebook_mismatch` | §7.2 step 5 |
| `candidate_mismatch` | §4.1, §4.2, §7.2 step 6 |
| `removal_mismatch` | §4.3, §7.2 step 6 |
| `rank_mismatch` | §4.4, §7.2 step 6 |
| `decision_mismatch` | §4.5, §7.2 step 6 |
| `attempts_exceeded` | §5.2, §7.2 step 7 |
| `attempt_removed` | §5.2, §7.2 step 7 |
| `attempt_out_of_order` | §5.2, §7.2 step 7 |
| `served_not_last` | §5.2, §7.2 step 7 |
| `times_invalid` | §5.2, §7.2 step 7 |
| `billing_invalid` | §5.2, §7.2 step 7 |
| `attempts_incomplete` | §5.2, §7.2 step 7 |
| `receipt_missing` | §6, §7.2 step 8 |
| `receipt_invalid` | §6, §7.2 step 8 |
| `receipt_mismatch` | §6, §7.2 step 8 |
A verifier of this extension also raises `malformed` for a text that is not a document (§2)
or a verification document that is not §7.1's, `unknown_version` for a record of another
version, and `bad_key` and `bad_signature` for its signature (SPEC §7).
## 9. Operation
The catalogue's operation `route.verify` (`wai route verify <document>`) verifies a
verification document (§7.1) and prints its outcome (§7.3), offline; it exits 0 when the
record is accepted and 1 when it is refused.
## 10. Conformance
[`live-conformance/routes/`](../live-conformance/routes/) holds the corpus, in one file,
`vectors.json`, written by `wai_route_record_vectors gen` from fixed seeds: a router's key and
a hop's, and the OER/2 receipts of minor version 1 the hops sign. Each vector is a
verification document with the outcome §7.3 gives it:
- **Accepted**: a job served by its first survivor, billed; a failover after a server error,
the failed hop unmetered and not billed; a served hop past the deadline, not billed; a job
every allowed survivor failed, with the policy's `max_attempts` below the survivors' number;
a job no candidate survives; a ranking decided by the targets, by the request's sort, by a
key of the policy and by the endpoint; an unknown energy and time to first byte ranked last;
every limit of §4.3 removing a candidate, a `null` value among them, and a candidate removed
by several; a ceiling in another currency; the fallback's candidates; and a record verified
without its receipts.
- **Refused**: each reason of §8 at least once, each vector a control of one defect, its
record signed again over the defect where the defect is not in the signature; and each code
§8 reuses.
`wai_route_record_vectors verify` and the test `tests/route_record.rs` check that the file
regenerates byte for byte, that `wai-rs` gives every vector its outcome, that every reason of
§8 refuses a vector, and that §8 lists exactly the reference's codes.
`live-conformance/route_record_verify.py` is the second implementation, written from this
text: it shares no code with `wai-rs`, verifies the router's signature with WAI's strict rule
written from RFC 8032 (`claim-conformance/_ed25519.py`), verifies the hop receipts with the
OER/2 reference verifier (`oer-conformance/oer2_verify.py`), recomputes every candidate,
removal, rank, order, decision and attempt itself, gives every vector the same outcome,
requires every mutated copy of the corpus to fail, and requires the corpus to catch the
verifier with each of its checks dropped.
## 11. Security and privacy considerations
- **What the record proves.** That the router, holding these three documents, chose by their
rules, and tried and billed as it says. It does not prove that the price book states what
the providers charge, or that an endpoint's stated retention or training is true: the price
book is the router's statement, pinned by digest, and a buyer who holds the published price
book of that digest compares it. A record whose candidates differ from its price book's
entries is refused (`candidate_mismatch`), so a router cannot list a price it did not
state.
- **Removed means never tried.** An attempt that names a removed candidate is refused
(`attempt_removed`): a job whose limits removed an endpoint was not sent there, by this
router's own signed account. Whether a hop's provider kept the data it was sent is outside
any record; the limit that removed a provider that says it keeps data is not.
- **Failed hops are not billed.** A record that bills a failed attempt, or a served one past
the declared deadline, is refused (`billing_invalid`); the receipt ledger bills a job from
the billed attempt alone.
- **Unknown is restrictive.** Every `null` a limit reads removes the candidate (§4.3), so a
price book that omits what a buyer cares about removes the endpoint rather than admitting
it.
- **What a record reveals.** Its job id, its operation and its quantities, every endpoint and
provider that could have served the job with their prices, and when each hop ran. It
travels between the router, the job's owner and an auditor they choose.

225
wai/live-conformance/_int_motion_receipt.py Normal file
View file

@ -0,0 +1,225 @@
"""The `wai.video.int_motion` decode receipt (extensions/integer-payloads.md §15.1, §15.2),
read and checked as the text states it.
Written from the extension's text; it shares no code with wai-rs. BLAKE3 is
../claim-conformance/_blake3.py, Ed25519 WAI's strict rule ../claim-conformance/_ed25519.py, and
the `WIV1` decode the integer-payload corpus's second implementation,
../int-payload-conformance/int_payload_verify.py. Standard library only.
read(text) -> the receipt, or raise Bad(why) (§15.2)
consistent(r) -> None, or why it is not (§15.1)
rederived(r, p) -> None, or why the payload does not give it (§15.1)
"""
import json
import os
import struct
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(HERE, "..", "claim-conformance"))
sys.path.insert(0, os.path.join(HERE, "..", "int-payload-conformance"))
import _ed25519 as ED # noqa: E402
from _blake3 import blake3 # noqa: E402
import int_payload_verify as IPV # noqa: E402
PROFILE = "wai.video.int_motion/receipt"
LEAF, NODE = b"moq-jwp:merkle-leaf\x01", b"moq-jwp:merkle-node\x01" # jwp-receipts §2.2
U32, U64 = 2 ** 32 - 1, 2 ** 64 - 1
RESOLUTION_FLOOR = 288676 # §15.1
TAGS = {"HwShunt": 1, "ModelBased": 2, "Estimator": 3, "OnChipCounter": 4, "CalibratedInstrument": 5}
FILTERING = {"Unknown": 0, "Off": 1, "On": 2}
# The Unicode White_Space characters: a calibration reference of these alone is empty.
WHITE_SPACE = set("\t\n\x0b\x0c\r \x85\xa0            "
"

  　")
MEMBERS = ["profile", "codestream_hash", "clip_hash", "h", "w", "frames", "merkle_root", "work_pixel_ops", "joules_micro",
"uj_per_kilopixel_op", "energy_provenance", "parent_receipt_hash", "signer_pubkey", "signer_id", "sig", "receipt_hash"]
EVIDENCE = {"OnChipCounter": ["energy_uncertainty", "energy_counter_filtering"],
"CalibratedInstrument": ["energy_uncertainty", "energy_calibration_ref"]}
class Bad(Exception):
pass
class Float:
"""A JSON number with a fraction or an exponent, kept as it is spelled."""
def __init__(self, text):
self.text = text
def _pairs(pairs):
d = {}
for k, v in pairs:
if k in d:
raise Bad("a member named twice")
d[k] = v
return d
def _hex(v, n):
if not isinstance(v, str) or len(v) != 2 * n or any(c not in "0123456789abcdef" for c in v):
raise Bad("not %d bytes of lowercase hexadecimal" % n)
return bytes.fromhex(v)
def _uint(v, top):
if isinstance(v, bool) or not isinstance(v, int) or not 0 <= v <= top:
raise Bad("not an integer in plain decimal")
return v
def _class(o):
"""energy-measurement §2.7's reading of the class and its evidence: (label, ppm, window,
filtering, calibration_ref), or None when unmetered."""
label = o.get("energy_provenance")
evidence = [k for k in ("energy_uncertainty", "energy_counter_filtering", "energy_calibration_ref") if o.get(k) is not None]
if label is None:
if evidence:
raise Bad("evidence with no class")
return None
if not isinstance(label, str) or label not in TAGS:
raise Bad("a label outside the vocabulary")
if label not in EVIDENCE:
if evidence:
raise Bad("evidence beside a class that carries none")
return (label, None, None, None, None)
u = o.get("energy_uncertainty")
if not isinstance(u, dict):
raise Bad("a refined class without its uncertainty")
ppm, win = u.get("relative_ppm"), u.get("window_us")
if isinstance(ppm, bool) or not isinstance(ppm, int) or ppm < 0 or isinstance(win, bool) or not isinstance(win, int) or win < 0:
raise Bad("an uncertainty that is not two integers")
if ppm > U32 or ppm == 0 or win == 0 or win > U64:
raise Bad("an uncertainty of zero, of no window, or out of range")
if label == "OnChipCounter":
if o.get("energy_calibration_ref") is not None:
raise Bad("a certificate beside a counter")
f = o.get("energy_counter_filtering")
f = "Unknown" if f is None else f
if not isinstance(f, str) or f not in FILTERING:
raise Bad("a filtering state outside the three")
return (label, ppm, win, f, None)
if o.get("energy_counter_filtering") is not None:
raise Bad("a filtering state beside an instrument")
ref = o.get("energy_calibration_ref")
if not isinstance(ref, str) or all(c in WHITE_SPACE for c in ref):
raise Bad("an instrument with no certificate")
return (label, ppm, win, None, ref)
def read(text):
"""§15.2: the receipt a text is the JSON form of."""
try:
o = json.loads(text, object_pairs_hook=_pairs, parse_float=Float, parse_constant=lambda s: Float(s))
except (ValueError, RecursionError) as e:
raise Bad("not JSON read strictly: %s" % e)
if not isinstance(o, dict) or o.get("profile") != PROFILE:
raise Bad("profile")
cls = _class(o)
want = MEMBERS + (EVIDENCE.get(cls[0], []) if cls else [])
if set(o) != set(want) or len(o) != len(want):
raise Bad("members")
if not isinstance(o["frames"], list) or not isinstance(o["signer_id"], str):
raise Bad("frames or signer id")
r = {
"codestream_hash": _hex(o["codestream_hash"], 32), "clip_hash": _hex(o["clip_hash"], 32),
"h": _uint(o["h"], U32), "w": _uint(o["w"], U32), "frames": [_hex(f, 32) for f in o["frames"]],
"merkle_root": _hex(o["merkle_root"], 32), "work": _uint(o["work_pixel_ops"], U64), "joules": _uint(o["joules_micro"], U64),
"class": cls, "parent": None if o["parent_receipt_hash"] is None else _hex(o["parent_receipt_hash"], 32),
"signer": _hex(o["signer_pubkey"], 32), "signer_id": o["signer_id"], "sig": _hex(o["sig"], 64),
}
rate = o["uj_per_kilopixel_op"]
want_rate = None
if cls is not None and r["joules"] > 0 and r["work"] > 0:
want_rate = "%.6f" % (float(r["joules"]) * 1000.0 / float(r["work"]))
got_rate = rate.text if isinstance(rate, Float) else (None if rate is None else json.dumps(rate))
if got_rate != want_rate or (rate is not None and not isinstance(rate, Float)):
raise Bad("uj_per_kilopixel_op is not the receipt's")
if _hex(o["receipt_hash"], 32) != receipt_hash(r):
raise Bad("receipt_hash is not the receipt's")
return r
def class_bytes(cls):
"""§15.1: the tag byte and its evidence."""
if cls is None:
return b"\x00"
label, ppm, win, filt, ref = cls
out = bytes([TAGS[label]])
if label == "OnChipCounter":
out += struct.pack(">IQ", ppm, win) + bytes([FILTERING[filt]])
elif label == "CalibratedInstrument":
ref = ref.encode("utf-8")
out += struct.pack(">IQI", ppm, win, len(ref)) + ref
return out
def payload(r):
"""§15.1's signing payload."""
o = b"wai:video-receipt\x01" + r["codestream_hash"] + r["clip_hash"] + struct.pack(">II", r["h"], r["w"])
o += r["merkle_root"] + struct.pack(">QQQ", len(r["frames"]), r["work"], r["joules"]) + class_bytes(r["class"])
o += b"\x00" if r["parent"] is None else b"\x01" + r["parent"]
for i, f in enumerate(r["frames"]):
o += struct.pack(">Q", i) + f
sid = r["signer_id"].encode("utf-8")
return o + r["signer"] + struct.pack(">I", len(sid)) + sid
def receipt_hash(r):
return blake3(b"wai:video-receipt-id\x01" + payload(r) + r["sig"])
def merkle_root(leaves):
"""jwp-receipts §2.2: the final leaf duplicated on an odd level; the empty root is zero."""
if not leaves:
return bytes(32)
level = [blake3(LEAF + h) for h in leaves]
while len(level) > 1:
if len(level) % 2:
level.append(level[-1])
level = [blake3(NODE + level[i] + level[i + 1]) for i in range(0, len(level), 2)]
return level[0]
def frame_hash(i, rgb):
return blake3(b"wai:video-rframe\x01" + struct.pack("<Q", i) + bytes(rgb))
def clip_hash(h, w, frames):
return blake3(b"wai:video-clip\x01" + struct.pack("<IIQ", h, w, len(frames))
+ b"".join(struct.pack("<Q", i) + f for i, f in enumerate(frames)))
def consistent(r):
"""§15.1: None when the receipt is consistent, otherwise why not."""
if merkle_root(r["frames"]) != r["merkle_root"]:
return "merkle root"
if clip_hash(r["h"], r["w"], r["frames"]) != r["clip_hash"]:
return "clip hash"
if min(len(r["frames"]) * r["h"] * r["w"] * 3, U64) != r["work"]:
return "work"
cls, j = r["class"], r["joules"]
if (cls is None) != (j == 0):
return "figure and class"
if cls is not None and cls[1] is not None and j * cls[1] < RESOLUTION_FLOOR:
return "below the resolution floor"
if ED.verify(r["signer"], payload(r), r["sig"]):
return "signature"
return None
def rederived(r, p):
"""§15.1: None when the payload re-derives the receipt, otherwise why not."""
if blake3(p) != r["codestream_hash"]:
return "codestream hash"
try:
v = IPV.parse_wiv1(p, 0)
frames = IPV.decode_wiv1_k0(v)
except Exception as e: # a refused payload re-derives nothing
return "the payload does not decode: %s" % e
if (v["h"], v["w"], len(frames)) != (r["h"], r["w"], len(r["frames"])):
return "geometry"
if any(frame_hash(i, f) != r["frames"][i] for i, f in enumerate(frames)):
return "a frame"
return consistent(r)

16
wai/live-conformance/ledger/README.md
View file

@ -36,7 +36,14 @@ The accepted vectors:
of [`stitched/`](../stitched/) (`hls_stitched`; `form1_prefetch`, whose form-1 receipt states
its own quartiles and spend beside the presented ones; and `squeezeback`, a shared break whose
sessions' own claims are not all presented), under a tariff whose ad scope is the stitchers
that corpus accepts; its report is stitched-presentation §7.5's statement, unchanged.
that corpus accepts; its report is stitched-presentation §7.5's statement, unchanged;
- `jobs_routed_and_served` (version 2, §4.5): a month of media jobs routed by the route corpus's
router, beside a package, under a tariff of version 2 that prices `wai.video.int_motion` per job
and per subpixel and `wai.video.av1` per job: the served `int_motion` jobs are billed from their
billed hop and their output's decode receipt (`work_pixel_ops`); the failover is billed at the
fallback's price; the job served past its deadline and the one every allowed hop failed bill
nothing; the jobs' energy is the serving hops' figures, a counter row and a model row, and the
job whose serving hop was unmetered is counted unmetered (coverage partial).
Every other vector is refused, a control of one defect: each reason of §10, the reused codes, and a
refusal of each source kind by its own extension.
@ -50,7 +57,12 @@ receipts from [`metering/`](../metering/) (`lesson`, `mixed_classes`, `receipt_b
[`presentation/`](../presentation/) (`video_rule`, `ceiling_exceeded_by_presented`), streams
stitched by others from [`stitched/`](../stitched/) (`hls_stitched`, `form1_prefetch`,
`squeezeback`, with the
manifests, responses and receipts they name, by their bytes), and the
manifests, responses and receipts they name, by their bytes), media jobs from
[`routes/`](../routes/) (`served_first_try`, `failover_after_a_server_error`,
`served_after_the_deadline`, `served_by_an_unmetered_hop`, `every_allowed_survivor_failed`,
`served_by_a_hop_whose_figure_states_no_class`, `a_failed_hop_billed`,
`an_unknown_energy_ranks_last`), each `int_motion` job's output a decode receipt sealed here over
`wai-rs/tests/vectors/int_video/int_video_bitstream.wiv1`, and the
packages `video-base-01` and `video-staged-01` of
[`pack-conformance/packs/`](../../pack-conformance/packs/). When one of those corpora is
regenerated, so is this one. Nothing here was fetched from outside the repository.

741
wai/live-conformance/ledger/vectors.json
View file

File diff suppressed because one or more lines are too long

220
wai/live-conformance/receipt_ledger_verify.py
View file

@ -10,7 +10,9 @@ settlement statement by settlement_verify.py (settlement §8), a package receipt
../pack-conformance/pack_verify.py (packaging §9), an ads room by live_ads_verify.py with
ad_presentation_verify.py (live-ads §9, ad-presentation §9), and a stream stitched by others by
stitched_presentation_verify.py (stitched-presentation §7), whose statement the ledger reports as
it is. Everything else is here:
it is; and, under a tariff of version 2, a media job by route_record_verify.py (route-record §7,
its hop receipts by the OER/2 reference verifier) and its output receipts by
_int_motion_receipt.py (integer-payloads §15). Everything else is here:
- the tariff (§4): reading, rules, both signatures, its scope;
- the invoice (§6): reading, rules, signature, chain;
@ -29,11 +31,13 @@ requires the corpus to catch the weakened verifier.
python3 -I receipt_ledger_verify.py --document <verification document>
Exit status 0 only if every vector verifies, every mutated copy fails and every weakened verifier
fails. Python 3.8 or later; standard library only.
fails. Python 3.8 or later; the standard library, and the `cryptography` package that the OER/2
reference verifier checks a job's hop receipts with.
"""
import copy
import hashlib
import io
import json
import os
import re
@ -51,6 +55,10 @@ import live_ads_verify as LAV # noqa: E402
import ad_presentation_verify as APV # noqa: E402
import pack_verify as PV # noqa: E402
import stitched_presentation_verify as STV # noqa: E402
import route_record_verify as RRV # noqa: E402 (it puts ../oer-conformance on the path)
import oer2_verify as OER # noqa: E402
import _int_motion_receipt as IMR # noqa: E402
from decimal import Decimal, ROUND_HALF_EVEN, localcontext, MAX_PREC, MAX_EMAX, MIN_EMIN # noqa: E402
EXTENSION = os.path.join(HERE, "..", "extensions", "receipt-ledger.md")
@ -61,11 +69,15 @@ INVOICE_DOMAIN = b"wai:ledger-invoice\x01" # §6.7
CITES_DOMAIN = b"wai:ledger-cites\x01" # §6.5
ZERO = bytes(32)
U64 = 2 ** 64 - 1
ROOM, DELIVERY, PACKAGE, ADS = 1, 2, 3, 4 # §6.3 source kinds
ROOM, DELIVERY, PACKAGE, ADS, JOB = 1, 2, 3, 4, 5 # §6.3 source kinds (5: version 2)
SESSION, HOP, STATEMENT, ACK, PACK, AD_RECEIPT, CLAIM, POD_INPUT, STITCHED_CLAIM = 1, 2, 3, 4, 5, 6, 7, 8, 9 # §5.2
KIND_NAMES = {ROOM: "room", DELIVERY: "delivery", PACKAGE: "package", ADS: "ads"}
OER_RECEIPT, VIDEO_RECEIPT, ROUTE_RECORD = 14, 15, 16 # §5.2, version 2 (10 to 13 are other texts')
PRIVATE = 128 # §5.2: 128 to 255, private and provisional use
KIND_NAMES = {ROOM: "room", DELIVERY: "delivery", PACKAGE: "package", ADS: "ads", JOB: "job"}
STITCHED = "stitched" # §5.1: a source kind billed as ads
SCOPES = {1: "room_hops", 2: "delivery_hops", 3: "packaging", 4: "ad_decisions"} # §6.6
SCOPES = {1: "room_hops", 2: "delivery_hops", 3: "packaging", 4: "ad_decisions", 5: "jobs"} # §6.6
JOB_UNITS = ["jobs", "subpixels"] # §4.5
CAPABILITY = re.compile(r"\Awai\.[a-z0-9_.]{0,124}\Z")
LABELS = {2: "ModelBased", 3: "Estimator", 4: "OnChipCounter", 5: "CalibratedInstrument"}
TAGS = {v: k for k, v in LABELS.items()}
CLASS_NAMES = ["live", "fetch", "fill", "ancillary"]
@ -118,7 +130,7 @@ def head(b, magic):
return "malformed"
if len(b) < 5:
return "truncated"
if b[4] != 1:
if b[4] not in (1, 2):
return "unknown_version"
return None
@ -148,6 +160,12 @@ def read_tariff(b):
t["delivery"] = [r.u(8) for _ in range(4)]
t["package"] = [r.u(8) for _ in range(3)]
t["ads"] = [r.u(8) for _ in range(2)]
t["version"] = b[4]
t["job"] = []
if t["version"] == 2: # §4.5
for _ in range(r.u(2)):
cap = r.take(r.u(1))
t["job"].append((cap, r.u(1), r.u(8), r.u(8)))
t["scope"] = [(r.u(1), r.take(32)) for _ in range(r.u(2))]
body_end = r.p
seller_sig, buyer_sig = r.take(64), r.take(64)
@ -155,9 +173,17 @@ def read_tariff(b):
raise Refuse("truncated", ["tariff"])
if r.p != len(b):
raise Refuse("malformed", ["tariff"])
def job_price(j):
try:
cap = j[0].decode("utf-8")
except UnicodeDecodeError:
return False
return CAPABILITY.match(cap) is not None and 1 <= j[1] <= len(JOB_UNITS) and j[2] >= 1
top = 5 if t["version"] == 2 else 4
rules = (t["start"] < t["end"] and capitals(t["currency"]) and t["seller"] != ZERO and t["buyer"] != ZERO
and t["seller"] != t["buyer"] and t["scope"] and ascending(t["scope"])
and all(1 <= k <= 4 and i != ZERO for k, i in t["scope"]))
and all(1 <= k <= top and i != ZERO for k, i in t["scope"])
and len(t["job"]) <= 255 and ascending([(j[0], j[1]) for j in t["job"]]) and all(job_price(j) for j in t["job"]))
if not rules and "tariff-rules" not in SKIP:
raise Refuse("tariff_invalid", ["tariff"])
body = b[:body_end]
@ -172,9 +198,15 @@ def read_tariff(b):
def price(t, kind, unit):
if kind == JOB:
return t["job"][unit - 1][3]
return {ROOM: t["room"], DELIVERY: t["delivery"], PACKAGE: t["package"], ADS: t["ads"]}[kind][unit - 1]
def divisor(t, kind, unit):
return t["job"][unit - 1][2] if kind == JOB else UNITS[kind][unit - 1][1]
# ---------------------------------------------------------------- §6 the invoice
def read_invoice(b, at):
@ -191,7 +223,9 @@ def read_invoice(b, at):
i["lines"].append({"kind": r.u(1), "source": r.take(32), "unit": r.u(1), "quantity": r.u(8), "price": r.u(8),
"amount": r.u(8), "cited": r.u(4), "cites": r.take(32)})
i["energy"] = [(r.u(1), r.u(1), r.u(4), r.u(8), r.u(4)) for _ in range(r.u(1))]
i["unmetered"] = [r.u(4) for _ in range(4)]
i["version"] = b[4]
v2 = i["version"] == 2
i["unmetered"] = [r.u(4) for _ in range(5 if v2 else 4)] + ([] if v2 else [0])
i["amount"] = r.u(8)
sig = r.take(64)
except ValueError:
@ -200,13 +234,18 @@ def read_invoice(b, at):
raise Refuse("malformed", at)
if any(e[1] == 0 for e in i["energy"]) and "row-of-no-class" not in SKIP:
raise Refuse("classes_summed", at)
if any(k >= PRIVATE for k, _ in i["records"]) and "private-kind" not in SKIP:
raise Refuse("private_kind", at)
keys = [(x["kind"], x["source"], x["unit"]) for x in i["lines"]]
def unit_ok(x):
if x["kind"] == JOB:
return v2 and x["unit"] >= 1
return x["kind"] in UNITS and 1 <= x["unit"] <= len(UNITS[x["kind"]])
rules = (i["start"] < i["end"] and capitals(i["currency"]) and ascending(i["records"])
and all(1 <= k <= 9 for k, _ in i["records"]) and ascending(keys)
and all(x["kind"] in UNITS and 1 <= x["unit"] <= len(UNITS[x["kind"]]) and x["quantity"] > 0 and x["cited"] > 0
for x in i["lines"])
and all(1 <= k <= 9 or (v2 and OER_RECEIPT <= k <= ROUTE_RECORD) for k, _ in i["records"]) and ascending(keys)
and all(unit_ok(x) and x["quantity"] > 0 and x["cited"] > 0 for x in i["lines"])
and ascending([(e[0], e[1]) for e in i["energy"]])
and all(1 <= e[0] <= 4 and 2 <= e[1] <= 5 and e[2] > 0 for e in i["energy"]))
and all(1 <= e[0] <= (5 if v2 else 4) and 2 <= e[1] <= 5 and e[2] > 0 for e in i["energy"]))
if not rules and "invoice-rules" not in SKIP:
raise Refuse("invoice_invalid", at)
bad = verify_sig(i["seller"], INVOICE_DOMAIN, b[:-64], sig)
@ -367,16 +406,86 @@ def stitched_source(s, signers):
# A source's verification is a pure function of the source, and the corpus verifies the same
# sources under every mutated copy and every weakened verifier (none of whose checks is a
# source's own): each is verified once.
def oer_identity(rid):
"""§5.2: the 32 bytes an OER/2 receipt id names."""
return bytes.fromhex(rid.split(":", 1)[1])
def micro(v):
"""§5.1: a figure in joules as whole microjoules, the nearest, a half to the even one; None
when it is not a non-negative quantity (oer-receipts §4)."""
try:
d = OER.dec(v)
except TypeError:
return None
with localcontext() as c:
c.prec, c.Emax, c.Emin = MAX_PREC, MAX_EMAX, MIN_EMIN
n = (d * 1000000).to_integral_value(rounding=ROUND_HALF_EVEN)
return int(n) if n >= 0 else None
def job_figure(r):
"""§5.1: the billed hop's figure as §6.6 counts it, or None when no row can state it."""
if OER.is_unmetered(r):
return (None, 0, 0)
e = r.get("energy") if isinstance(r.get("energy"), dict) else {}
c = e.get("class")
if not OER.class_valid(c):
return None
j = micro(e.get("joules_total"))
if j is None or j < 1 or j > U64:
return None
tag = TAGS[c["label"]]
return (tag, j, c["relative_ppm"] if tag in (4, 5) else 0)
def job_source(s):
if set(s) != {"kind", "request", "pricebook", "policy", "record", "receipts", "keys", "outputs"} \
or s["receipts"] is None or s["keys"] is None or not isinstance(s["outputs"], list):
raise Refuse("malformed", [])
doc = {k: s[k] for k in ("request", "pricebook", "policy", "record", "receipts", "keys")}
got = RRV.verify(doc)
if got["outcome"] != "accept":
raise Refuse(got["reason"], got["at"])
rec = json.loads(s["record"])
texts = [OER.load(io.StringIO(t)) for t in s["receipts"]]
billed, fig = None, None
for a in rec["attempts"]:
if a["billed"] is True:
cap = next(c["capability"] for c in rec["candidates"] if c["endpoint"] == a["endpoint"])
j = next(i for i, t in enumerate(texts) if isinstance(t, dict) and t.get("receipt_id") == a["receipt"])
fig = job_figure(texts[j]) if "job-figure" not in SKIP else (None, 0, 0)
if fig is None:
raise Refuse("job_figure_invalid", ["receipts", j])
billed = (cap.encode("utf-8"), oer_identity(a["receipt"]))
break
outputs = []
for i, x in enumerate(s["outputs"]):
if not isinstance(x, dict) or set(x) != {"receipt", "codestream"} or not isinstance(x["receipt"], str) \
or not isinstance(x["codestream"], str) or not re.fullmatch(r"(?:[0-9a-f]{2})*", x["codestream"]):
raise Refuse("malformed", [])
try:
r = IMR.read(x["receipt"])
except IMR.Bad:
raise Refuse("output_invalid", ["outputs", i])
if "output" not in SKIP and IMR.rederived(r, bytes.fromhex(x["codestream"])) is not None:
raise Refuse("output_invalid", ["outputs", i])
outputs.append((IMR.receipt_hash(r), r["work"]))
return {"kind": JOB, "digest": bytes.fromhex(RRV.digest(rec)), "router": bytes.fromhex(rec["signer"]), "job": rec["job"],
"accepted": rec["accepted"], "hops": [oer_identity(a["receipt"]) for a in rec["attempts"]], "billed": billed,
"figure": fig, "outputs": outputs}
_SOURCES = {}
def verify_source(s, signers):
"""§5.1: the source verified by its own extension, a stitched source with `signers`, or
Refuse with that extension's place."""
key = json.dumps([s, sorted(k.hex() for k in signers)], sort_keys=True)
def verify_source(s, signers, version=1):
"""§5.1: the source verified by its own extension, a stitched source with `signers`, a job
source under a tariff of version 2, or Refuse with that extension's place."""
key = json.dumps([s, sorted(k.hex() for k in signers), version, sorted(SKIP & {"job-figure", "output"})], sort_keys=True)
if key not in _SOURCES:
try:
_SOURCES[key] = read_source(s, signers)
_SOURCES[key] = read_source(s, signers, version)
except Refuse as e:
_SOURCES[key] = e
got = _SOURCES[key]
@ -385,11 +494,13 @@ def verify_source(s, signers):
return got
def read_source(s, signers):
def read_source(s, signers, version):
if not isinstance(s, dict):
raise Refuse("malformed", [])
if s.get("kind") == "stitched":
return stitched_source(s, signers)
if s.get("kind") == "job" and version == 2:
return job_source(s)
readers = {"room": room_source, "delivery": delivery_source, "package": package_source, "ads": ads_source}
if s.get("kind") not in readers:
raise Refuse("malformed", [])
@ -406,6 +517,8 @@ def heads(f):
return [(PACK, f["digest"])]
if f["kind"] == STITCHED:
return [(POD_INPUT, r["digest"]) for r in f["records"]]
if f["kind"] == JOB:
return [(ROUTE_RECORD, f["digest"])]
return [(AD_RECEIPT, r["identity"]) for r in f["receipts"]]
@ -420,6 +533,8 @@ def contributed(f):
out |= {(CLAIM, c) for c in f["claims"]}
elif f["kind"] == STITCHED:
out |= {(STITCHED_CLAIM, c) for c in f["claims"]}
elif f["kind"] == JOB:
out |= {(OER_RECEIPT, h) for h in f["hops"]} | {(VIDEO_RECEIPT, h) for h, _ in f["outputs"]}
return out
@ -451,6 +566,11 @@ def check_source(t, start, end, f):
return "outside_scope"
elif f["kind"] == STITCHED:
return None # accepted only under the tariff's signers (§4.4); it states no time (§5.3)
elif f["kind"] == JOB:
if not in_scope(t, 5, f["router"]):
return "outside_scope"
if "period" not in SKIP and not start <= f["accepted"] < end:
return "outside_period"
else:
if not all(in_scope(t, 4, r["decider"]) for r in f["receipts"]):
return "outside_scope"
@ -475,17 +595,24 @@ def bills_again(f, g):
if f["kind"] == ADS:
return f["room"] == g["room"] and any((x["cue"], x["session"]) == (y["cue"], y["session"])
for x in f["receipts"] for y in g["receipts"])
if f["kind"] == JOB:
return f["router"] == g["router"] and f["job"] == g["job"]
return False
def check_across(facts, previous):
"""§7.2 step 6: (the index, the reason) of the first source refused, or None."""
billed = set(previous["records"]) if previous else set()
seen = set()
seen, jobs = set(), set()
for k, f in enumerate(facts):
hs = heads(f)
if "duplicate" not in SKIP and (any(h in seen for h in hs) or len(set(hs)) != len(hs)):
return k, "record_duplicate"
if f["kind"] == JOB: # §5.2: one hop, one output, one job
own = contributed(f) - set(hs)
if "job-duplicate" not in SKIP and own & jobs:
return k, "record_duplicate"
jobs |= own
if "overlap" not in SKIP and any(bills_again(f, g) for g in facts[:k]):
return k, "record_overlap"
if "rebilled" not in SKIP and contributed(f) & billed:
@ -541,6 +668,20 @@ def compute(t, facts):
for r in f["records"]:
cites = [(POD_INPUT, r["digest"])] + [(STITCHED_CLAIM, c) for _, cs in r["ads"] for c in cs]
add(ADS, r["signer"], 1, sum(p for p, _ in r["ads"]), cites)
elif f["kind"] == JOB: # §6.3, §6.5
if f["billed"] is None:
continue
cap, hop = f["billed"]
for k, (pcap, unit, _, _) in enumerate(t["job"]):
if pcap != cap:
continue
cites = [(ROUTE_RECORD, f["digest"]), (OER_RECEIPT, hop)]
if unit == 1:
n = 1
else:
cites += [(VIDEO_RECEIPT, h) for h, w in f["outputs"] if w > 0]
n = sum(w for _, w in f["outputs"])
add(JOB, f["router"], k + 1, n, cites)
else:
unit = 1 if f["claims_required"] else 2
for r in f["receipts"]:
@ -551,7 +692,7 @@ def compute(t, facts):
for (kind, source, unit) in sorted(q):
n, cited = q[(kind, source, unit)]
p = price(t, kind, unit)
a = n * p // UNITS[kind][unit - 1][1]
a = n * p // divisor(t, kind, unit)
if n > U64 or a > U64:
return None
total += a
@ -575,6 +716,8 @@ def compute(t, facts):
figs.append((3, f["figure"]))
elif f["kind"] == ADS:
figs += [(4, r["figure"]) for r in f["receipts"]]
elif f["kind"] == JOB and f["figure"] is not None:
figs.append((5, f["figure"]))
rows, unmetered, sums = rows_of(figs)
if rows is None or total > U64:
return None
@ -582,7 +725,7 @@ def compute(t, facts):
def rows_of(figs):
unmetered = [0, 0, 0, 0]
unmetered = [0, 0, 0, 0, 0]
acc = {}
for s, (tag, joules, ppm) in figs:
if tag is None:
@ -741,7 +884,7 @@ def verify(doc):
raise Refuse("malformed", ["source"])
t = read_tariff(tb) # step 1
inv = read_invoice(ib, ["invoice"]) # step 2
if "binding" not in SKIP and (inv["tariff"] != t["digest"] or inv["seller"] != t["seller"]
if "binding" not in SKIP and (inv["tariff"] != t["digest"] or inv["seller"] != t["seller"] or inv["version"] != t["version"]
or inv["buyer"] != t["buyer"] or inv["currency"] != t["currency"]): # step 3
raise Refuse("tariff_mismatch", ["invoice"])
if "outside-tariff" not in SKIP and not (t["start"] <= inv["start"] and inv["end"] <= t["end"]):
@ -759,7 +902,7 @@ def verify(doc):
facts = []
for k, s in enumerate(sources): # step 5
try:
f = verify_source(s, [i for k, i in t["scope"] if k == 4])
f = verify_source(s, [i for k, i in t["scope"] if k == 4], t["version"])
except Refuse as e:
raise Refuse(e.reason, ["source", k] + list(e.at))
bad = check_source(t, inv["start"], inv["end"], f)
@ -777,12 +920,23 @@ def verify(doc):
raise Refuse(bad, ["invoice"])
except Refuse as e:
return {"outcome": "refuse", "reason": e.reason, "at": e.at}
return accepted(inv, facts, c)
return accepted(inv, facts, c, t)
def accepted(inv, facts, c):
def line_json(x, t):
"""§7.3: a line, a job line naming its job price's unit and capability."""
out = {"source_kind": KIND_NAMES[x["kind"]], "source": x["source"].hex(),
"unit": JOB_UNITS[t["job"][x["unit"] - 1][1] - 1] if x["kind"] == JOB else UNITS[x["kind"]][x["unit"] - 1][0],
"quantity": x["quantity"], "price": x["price"], "amount": x["amount"],
"cites": [[k, i.hex()] for k, i in x["cited_records"]]}
if x["kind"] == JOB:
out["capability"] = t["job"][x["unit"] - 1][0].decode("utf-8")
return out
def accepted(inv, facts, c, t):
unmetered, coverage = {}, {}
for s in range(1, 5):
for s in range(1, 6 if inv["version"] == 2 else 5):
n = inv["unmetered"][s - 1]
unmetered[SCOPES[s]] = n
coverage[SCOPES[s]] = "partial" if n > 0 else "whole" if any(r[0] == s for r in inv["energy"]) else "none"
@ -794,9 +948,7 @@ def accepted(inv, facts, c):
"start": inv["start"],
"end": inv["end"],
"records": len(inv["records"]),
"lines": [{"source_kind": KIND_NAMES[x["kind"]], "source": x["source"].hex(), "unit": UNITS[x["kind"]][x["unit"] - 1][0],
"quantity": x["quantity"], "price": x["price"], "amount": x["amount"],
"cites": [[k, i.hex()] for k, i in x["cited_records"]]} for x in c["lines"]],
"lines": [line_json(x, t) for x in c["lines"]],
"energy": [{"scope": SCOPES[r[0]], "class": LABELS[r[1]], "figures": r[2], "joules": r[3], "max_ppm": r[4]}
for r in inv["energy"]],
"unmetered": unmetered,
@ -870,6 +1022,14 @@ def mutations(doc):
m = copy.deepcopy(doc)
m["vectors"][0]["outcome"]["reports"]["ads"][0]["not_counted"] = {}
out.append(("a_day: its ad report's not-counted figures dropped", m))
if "job_int_motion" in doc["sources"]:
m = copy.deepcopy(doc)
m["sources"]["job_int_motion"]["outputs"] = []
out.append(("jobs: the int_motion job's output receipt dropped", m))
m = copy.deepcopy(doc)
k = next(i for i, v in enumerate(doc["vectors"]) if v["name"] == "jobs_routed_and_served")
m["vectors"][k]["outcome"]["coverage"]["jobs"] = "whole"
out.append(("jobs_routed_and_served: its jobs' coverage stated whole", m))
for k, v in enumerate(doc["vectors"]):
if v["outcome"]["outcome"] == "refuse":
m = copy.deepcopy(doc)
@ -881,7 +1041,7 @@ def mutations(doc):
CHECKS = ["signatures", "buyer-signature", "tariff-rules", "invoice-rules", "row-of-no-class", "binding", "outside-tariff",
"previous-given", "previous-overlap", "scope", "conflict", "period", "duplicate", "overlap", "rebilled", "quantity",
"summed", "citations", "mismatch"]
"summed", "citations", "mismatch", "job-figure", "output", "job-duplicate", "private-kind"]
def main(corpus_dir):

787
wai/live-conformance/route_record_verify.py Normal file
View file

@ -0,0 +1,787 @@
#!/usr/bin/env python3
"""Independent verifier of the route record corpus (extensions/route-record.md).
Written from the extension's text. It shares no code with wai-rs, which generates the corpus,
and never runs it. It checks the router's Ed25519 signature with
../claim-conformance/_ed25519.py, WAI's strict rule (SPEC §7 *Signatures*) written from RFC 8032
with the standard library, and verifies the hops' OER/2 receipts with the OER/2 reference
verifier, ../oer-conformance/oer2_verify.py (which uses the `cryptography` package). Everything
else is here:
- the documents (§2, §3): read strictly, held to the subset, their forms and digests, and RFC
8785's canonical form;
- routing (§4): the candidates, their estimates, every hard limit, the key list, the ranking,
the order, the decision and the allowed attempts;
- the record (§5): its members and forms, its signature, its binding to the documents, its
candidates, order and decision recomputed, and its attempts held to the order and the
billing rules;
- the hop receipts (§6), and the outcome of §7.3.
For every vector it requires the outcome the corpus states; of the corpus, that every reason of
§8 and every code it reuses appears in it, that §8 names every reason it raises, and that no
other file is there. Then it mutates copies of the corpus, one defect each, and requires every
copy to fail; and it drops each of its own checks in turn and requires the corpus to catch the
weakened verifier.
python3 -I route_record_verify.py <live-conformance/routes>
python3 -I route_record_verify.py --document <verification document>
Exit status 0 only if every vector verifies, every mutated copy fails and every weakened
verifier fails. Python 3.8 or later.
"""
import copy
import hashlib
import io
import json
import os
import re
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(HERE, "..", "claim-conformance"))
sys.path.insert(0, os.path.join(HERE, "..", "oer-conformance"))
import _ed25519 as ED # noqa: E402
import oer2_verify as OER # noqa: E402
EXTENSION = os.path.join(HERE, "..", "extensions", "route-record.md")
SIGN_DOMAIN = b"wai:route-record\x01" # §5.3
LIMITS = ["policy", "unit", "region", "residency", "training", "retention", "ga", "consent", "unit_price", "spend"] # §4.3
KEYS = ["price", "ttfb", "energy", "quality"] # §3.3
ERRORS = ["timeout", "server_error", "rate_limited", "refused", "unavailable"] # §5.2
CLASSES = ["CalibratedInstrument", "OnChipCounter", "ModelBased", "Estimator"] # §2
DECIDED_BY = ["none", "only_survivor", "targets", "price", "ttfb", "energy", "quality", "endpoint"] # §4.5
MAX = 2 ** 53 - 1
ENTRY = ["endpoint", "provider", "capability", "region", "residency", "trains_on_inputs", "retention_days", "ga",
"needs_consent", "unit", "divisor", "price", "ttfb_ms", "energy_uj", "energy_class", "quality"] # §3.2
CANDIDATE = ENTRY + ["via_fallback", "cost", "energy", "removed", "rank"] # §5.1
RECORD = ["kind", "version", "job", "operation", "capability", "fallback", "quantities", "request", "policy", "pricebook",
"candidates", "order", "decision", "accepted", "deadline", "attempts", "signer", "sig"] # §5.1
ATTEMPT = ["hop", "endpoint", "outcome", "error", "start", "end", "billed", "not_billed", "receipt"] # §5.2
SKIP = set()
class Refuse(Exception):
def __init__(self, reason, at):
super().__init__(reason)
self.reason, self.at = reason, at
def on(check):
return check not in SKIP
# ---------------------------------------------------------------- §2 encodings
class _Bad(ValueError):
pass
def _no_duplicates(pairs):
d = {}
for k, v in pairs:
if k in d:
raise _Bad("member %r twice" % k)
d[k] = v
return d
def _int(s):
# A number is an integer 0 to 2^53 - 1 in plain decimal: no sign (so `-0` too is out).
if s.startswith("-"):
raise _Bad("a negative number")
v = int(s)
if v > MAX:
raise _Bad("an integer past 2^53 - 1")
return v
def _not_a_number(s):
raise _Bad("not an integer: %s" % s)
def _walk(v):
stack = [(v, 1)]
while stack:
x, d = stack.pop()
yield x, d
if isinstance(x, dict):
for k, y in x.items():
stack.append((k, d + 1))
stack.append((y, d + 1))
elif isinstance(x, list):
stack.extend((y, d + 1) for y in x)
def read_document(text):
"""§2: a document, or None (`malformed`)."""
if not isinstance(text, str):
return None
try:
v = json.loads(text, object_pairs_hook=_no_duplicates, parse_int=_int, parse_float=_not_a_number,
parse_constant=_not_a_number)
except (ValueError, RecursionError):
return None
depth = 0
for x, d in _walk(v):
if isinstance(x, (dict, list)):
depth = max(depth, d)
if isinstance(x, str) and any("\ud800" <= c <= "\udfff" for c in x):
return None
if depth > 64:
return None
return v
def canonical(v):
"""RFC 8785 over §2's subset: members sorted by their UTF-16 code units, strings escaped as
§3.2.2.2 there, integers in plain decimal."""
if v is None:
return "null"
if v is True:
return "true"
if v is False:
return "false"
if isinstance(v, int):
return str(v)
if isinstance(v, str):
return json.dumps(v, ensure_ascii=False)
if isinstance(v, list):
return "[" + ",".join(canonical(x) for x in v) + "]"
keys = sorted(v, key=lambda k: k.encode("utf-16-be"))
return "{" + ",".join(json.dumps(k, ensure_ascii=False) + ":" + canonical(v[k]) for k in keys) + "}"
def digest(v):
return hashlib.sha256(canonical(v).encode("utf-8")).hexdigest()
def same(a, b):
"""Two values are the same JSON value: compared by their canonical forms, so `0` is not
`false` and `1` not `true`, as Python's `==` would have them."""
return canonical(a) == canonical(b)
TOKEN = re.compile(r"\A[A-Za-z0-9._:/-]{1,128}\Z")
UNIT = re.compile(r"\A[a-z][a-z0-9_]{0,63}\Z")
CAP = re.compile(r"\Awai\.[a-z0-9_.]{0,124}\Z")
OPERATION = re.compile(r"\A[a-z][a-z0-9]*\.[a-z][a-z0-9]*\Z")
CURRENCY = re.compile(r"\A[A-Z]{3}\Z")
RECEIPT_ID = re.compile(r"\A(?:sha256|blake3):[0-9a-f]{64}\Z")
def is_int(v, lo=0, hi=MAX):
return isinstance(v, int) and not isinstance(v, bool) and lo <= v <= hi
def is_token(v):
return isinstance(v, str) and TOKEN.match(v) is not None
def is_unit(v):
return isinstance(v, str) and UNIT.match(v) is not None
def is_cap(v):
return isinstance(v, str) and CAP.match(v) is not None
def is_hex(v, n):
return isinstance(v, str) and len(v) == n and all(c in "0123456789abcdef" for c in v)
def exactly(v, members):
return isinstance(v, dict) and set(v) == set(members) and len(v) == len(members)
def ascending(xs):
return all(xs[i].encode() < xs[i + 1].encode() for i in range(len(xs) - 1))
def null_or(v, f):
return v is None or f(v)
def tokens(v, top):
return isinstance(v, list) and 1 <= len(v) <= top and all(is_token(x) for x in v) and ascending(v)
def quantities_ok(v):
if not (isinstance(v, list) and 1 <= len(v) <= 16):
return False
if not all(exactly(q, ["unit", "quantity"]) and is_unit(q["unit"]) and is_int(q["quantity"], 1) for q in v):
return False
return ascending([q["unit"] for q in v])
# ---------------------------------------------------------------- §3 the documents
def request_ok(r):
if not exactly(r, ["kind", "job", "operation", "capability", "fallback", "quantities", "limits", "prefer", "deadline_us"]):
return False
if r["kind"] != "wai-route-request" or not is_token(r["job"]) or not isinstance(r["operation"], str) \
or not OPERATION.match(r["operation"]) or not is_cap(r["capability"]):
return False
if not null_or(r["fallback"], lambda f: is_cap(f) and f != r["capability"]):
return False
if not quantities_ok(r["quantities"]):
return False
lim = r["limits"]
if not exactly(lim, ["regions", "residency", "no_training", "zero_retention", "ga_only", "consent_on_file", "max_price", "spend_ceiling"]):
return False
if not all(null_or(lim[k], lambda x: tokens(x, 64)) for k in ("regions", "residency")):
return False
if not all(isinstance(lim[k], bool) for k in ("no_training", "zero_retention", "ga_only", "consent_on_file")):
return False
mp = lim["max_price"]
if not (isinstance(mp, list) and len(mp) <= 16 and all(
exactly(m, ["unit", "price", "divisor"]) and is_unit(m["unit"]) and is_int(m["price"]) and is_int(m["divisor"], 1)
for m in mp) and ascending([m["unit"] for m in mp])):
return False
if not null_or(lim["spend_ceiling"], lambda c: exactly(c, ["currency", "amount"]) and isinstance(c["currency"], str)
and CURRENCY.match(c["currency"]) is not None and is_int(c["amount"])):
return False
p = r["prefer"]
if not exactly(p, ["sort", "ttfb_ms", "quality"]):
return False
if not (null_or(p["sort"], lambda s: s in KEYS) and null_or(p["ttfb_ms"], is_int)
and null_or(p["quality"], lambda q: is_int(q, 0, 1000))):
return False
return null_or(r["deadline_us"], lambda d: is_int(d, 1))
def entry_ok(e):
if not exactly(e, ENTRY):
return False
return (is_token(e["endpoint"]) and is_token(e["provider"]) and is_cap(e["capability"])
and null_or(e["region"], is_token) and null_or(e["residency"], is_token)
and null_or(e["trains_on_inputs"], lambda b: isinstance(b, bool)) and null_or(e["retention_days"], is_int)
and null_or(e["ga"], lambda b: isinstance(b, bool)) and null_or(e["needs_consent"], lambda b: isinstance(b, bool))
and is_unit(e["unit"]) and is_int(e["divisor"], 1) and is_int(e["price"]) and null_or(e["ttfb_ms"], is_int)
and null_or(e["energy_uj"], is_int) and null_or(e["energy_class"], lambda c: c in CLASSES)
and (e["energy_uj"] is None) == (e["energy_class"] is None)
and null_or(e["quality"], lambda q: is_int(q, 0, 1000)))
def pricebook_ok(b):
if not exactly(b, ["kind", "currency", "issued", "entries"]) or b["kind"] != "wai-route-pricebook":
return False
if not (isinstance(b["currency"], str) and CURRENCY.match(b["currency"]) and is_int(b["issued"])):
return False
es = b["entries"]
if not (isinstance(es, list) and 1 <= len(es) <= 1024 and all(entry_ok(e) for e in es)):
return False
return ascending([e["endpoint"] for e in es])
def policy_ok(p):
if not exactly(p, ["kind", "name", "version", "keys", "max_attempts", "exclude"]) or p["kind"] != "wai-route-policy":
return False
if not (is_token(p["name"]) and is_int(p["version"], 1) and is_int(p["max_attempts"], 1, 16)):
return False
if not (isinstance(p["keys"], list) and len(p["keys"]) == 4 and sorted(k for k in p["keys"] if isinstance(k, str)) == sorted(KEYS)):
return False
ex = p["exclude"]
return isinstance(ex, list) and len(ex) <= 64 and all(is_token(x) for x in ex) and ascending(ex)
def read(text, ok, invalid, at):
v = read_document(text)
if v is None:
raise Refuse("malformed", at)
if on(invalid) and not ok(v):
raise Refuse(invalid, at)
return v
# ---------------------------------------------------------------- §4 routing
def quantity(req, unit):
for q in req["quantities"]:
if q["unit"] == unit:
return q["quantity"]
return None
def estimate(q, x, d):
"""§4.2: ⌊q × x / d⌋, null past 2^53 - 1."""
if q is None or x is None:
return None
e = q * x // d
return e if e <= MAX else None
def removals(req, book, pol, e, cost):
"""§4.3, in its order: (limit, value)."""
lim, out = req["limits"], []
if on("limit-policy") and e["provider"] in pol["exclude"]:
out.append(("policy", e["provider"]))
if on("limit-unit") and quantity(req, e["unit"]) is None:
out.append(("unit", e["unit"]))
for name, key in (("region", "regions"), ("residency", "residency")):
allowed = lim[key]
if on("limit-" + name) and allowed is not None and (e[name] is None or e[name] not in allowed):
out.append((name, e[name]))
if on("limit-training") and lim["no_training"] and e["trains_on_inputs"] is not False:
out.append(("training", e["trains_on_inputs"]))
if on("limit-retention") and lim["zero_retention"] and e["retention_days"] != 0:
out.append(("retention", e["retention_days"]))
if on("limit-ga") and lim["ga_only"] and e["ga"] is not True:
out.append(("ga", e["ga"]))
if on("limit-consent") and not lim["consent_on_file"] and e["needs_consent"] is not False:
out.append(("consent", e["needs_consent"]))
for m in lim["max_price"]:
if on("limit-unit-price") and m["unit"] == e["unit"] and e["price"] * m["divisor"] > m["price"] * e["divisor"]:
out.append(("unit_price", e["price"]))
c = lim["spend_ceiling"]
if on("limit-spend") and c is not None and (c["currency"] != book["currency"] or cost is None or cost > c["amount"]):
out.append(("spend", cost))
return out
def meets(req, c):
t, q = req["prefer"]["ttfb_ms"], req["prefer"]["quality"]
return (t is None or (c["ttfb_ms"] is not None and c["ttfb_ms"] <= t)) and \
(q is None or (c["quality"] is not None and c["quality"] >= q))
def key_list(req, pol):
"""§4.4."""
p, k = req["prefer"], []
if p["ttfb_ms"] is not None or p["quality"] is not None:
k.append("targets")
if p["sort"] is not None:
k.append(p["sort"])
k += [x for x in pol["keys"] if x != p["sort"]]
return k + ["endpoint"]
def compare(req, key, a, b):
"""-1 when a comes first under the key, 1 when b does, 0 when they do not differ."""
def nulls_last(x, y, high_first=False):
if x is None and y is None:
return 0
if x is None:
return 1
if y is None:
return -1
if x == y:
return 0
return (-1 if x > y else 1) if high_first else (-1 if x < y else 1)
if key == "targets":
ma, mb = meets(req, a), meets(req, b)
return 0 if ma == mb else (-1 if ma else 1)
if key == "price":
return nulls_last(a["cost"], b["cost"])
if key == "ttfb":
return nulls_last(a["ttfb_ms"], b["ttfb_ms"])
if key == "energy":
return nulls_last(a["energy"], b["energy"])
if key == "quality":
return nulls_last(a["quality"], b["quality"], high_first=True)
ea, eb = a["endpoint"].encode(), b["endpoint"].encode()
return 0 if ea == eb else (-1 if ea < eb else 1)
def route(req, book, pol):
"""§4: the candidates as the record states them, the order, the decision, the allowed."""
cands = []
for e in book["entries"]:
if e["capability"] == req["capability"]:
via = False
elif req["fallback"] is not None and e["capability"] == req["fallback"]:
via = True
else:
continue
q = quantity(req, e["unit"])
cost = estimate(q, e["price"], e["divisor"])
energy = estimate(q, e["energy_uj"], e["divisor"])
c = dict(e)
c.update(via_fallback=via, cost=cost, energy=energy,
removed=[{"limit": l, "value": v} for l, v in removals(req, book, pol, e, cost)], rank=None)
cands.append(c)
keys = key_list(req, pol)
survivors = [c for c in cands if not c["removed"]]
def first(a, b):
for k in keys:
r = compare(req, k, a, b)
if r:
return r, k
return 0, None
# An insertion sort under the total order of the key list.
order = []
for c in survivors:
i = len(order)
while i > 0 and first(c, order[i - 1])[0] < 0:
i -= 1
order.insert(i, c)
for n, c in enumerate(order):
c["rank"] = n + 1
if not order:
decision = {"decided_by": "none", "runner_up": None}
elif len(order) == 1:
decision = {"decided_by": "only_survivor", "runner_up": None}
else:
decision = {"decided_by": first(order[0], order[1])[1], "runner_up": order[1]["endpoint"]}
allowed = min(len(order), pol["max_attempts"])
return cands, [c["endpoint"] for c in order], decision, allowed
# ---------------------------------------------------------------- §5 the record
def scalar(v):
return not isinstance(v, (dict, list))
def candidate_form(c):
"""§5.1: each member of its own form. Relations between members (an energy figure and its
class, a cost and its price) are the recomputation's, at §7.2 step 6."""
if not exactly(c, CANDIDATE):
return False
is_bool = lambda b: isinstance(b, bool) # noqa: E731
return (is_token(c["endpoint"]) and is_token(c["provider"]) and is_cap(c["capability"])
and null_or(c["region"], is_token) and null_or(c["residency"], is_token)
and null_or(c["trains_on_inputs"], is_bool) and null_or(c["retention_days"], is_int)
and null_or(c["ga"], is_bool) and null_or(c["needs_consent"], is_bool)
and is_unit(c["unit"]) and is_int(c["divisor"], 1) and is_int(c["price"]) and null_or(c["ttfb_ms"], is_int)
and null_or(c["energy_uj"], is_int) and null_or(c["energy_class"], lambda x: x in CLASSES)
and null_or(c["quality"], lambda q: is_int(q, 0, 1000))
and isinstance(c["via_fallback"], bool) and null_or(c["cost"], is_int) and null_or(c["energy"], is_int)
and isinstance(c["removed"], list) and len(c["removed"]) <= len(LIMITS)
and all(exactly(x, ["limit", "value"]) and x["limit"] in LIMITS and scalar(x["value"]) for x in c["removed"])
and null_or(c["rank"], lambda n: is_int(n, 1)))
def attempt_form(i, a):
if not exactly(a, ATTEMPT) or a["outcome"] not in ("served", "failed"):
return False
served = a["outcome"] == "served"
error_ok = a["error"] is None if served else a["error"] in ERRORS
return (is_int(a["hop"]) and a["hop"] == i + 1 and is_token(a["endpoint"]) and error_ok and is_int(a["start"])
and is_int(a["end"]) and isinstance(a["billed"], bool) and a["not_billed"] in (None, "failed", "deadline_missed")
and isinstance(a["receipt"], str) and RECEIPT_ID.match(a["receipt"]) is not None)
def record_form(r):
if not exactly(r, RECORD):
return False
return (is_int(r["version"]) and r["version"] == 1 and is_token(r["job"]) and isinstance(r["operation"], str)
and OPERATION.match(r["operation"]) is not None and is_cap(r["capability"]) and null_or(r["fallback"], is_cap)
and quantities_ok(r["quantities"]) and is_hex(r["request"], 64)
and exactly(r["policy"], ["name", "version", "digest"]) and is_token(r["policy"]["name"])
and is_int(r["policy"]["version"], 1) and is_hex(r["policy"]["digest"], 64)
and exactly(r["pricebook"], ["digest", "currency", "issued"]) and is_hex(r["pricebook"]["digest"], 64)
and isinstance(r["pricebook"]["currency"], str) and CURRENCY.match(r["pricebook"]["currency"]) is not None
and is_int(r["pricebook"]["issued"])
and isinstance(r["candidates"], list) and len(r["candidates"]) <= 1024 and all(candidate_form(c) for c in r["candidates"])
and isinstance(r["order"], list) and len(r["order"]) <= 1024 and all(is_token(x) for x in r["order"])
and exactly(r["decision"], ["decided_by", "runner_up"]) and r["decision"]["decided_by"] in DECIDED_BY
and null_or(r["decision"]["runner_up"], is_token)
and is_int(r["accepted"]) and null_or(r["deadline"], is_int)
and isinstance(r["attempts"], list) and len(r["attempts"]) <= 16
and all(attempt_form(i, a) for i, a in enumerate(r["attempts"]))
and is_hex(r["signer"], 64) and is_hex(r["sig"], 128))
_VERIFIED = {}
def ed_verify(key, msg, sig):
k = (key, hashlib.sha256(msg).digest(), sig)
if k not in _VERIFIED:
_VERIFIED[k] = ED.verify(key, msg, sig)
return _VERIFIED[k]
def signing_bytes(r):
body = {k: v for k, v in r.items() if k != "sig"}
return SIGN_DOMAIN + canonical(body).encode("utf-8")
def billing(served, end, deadline):
"""§5.2: (billed, not_billed)."""
if not served:
return False, "failed"
if deadline is not None and end > deadline:
return False, "deadline_missed"
return True, None
# ---------------------------------------------------------------- §7 verification
def verify(doc):
try:
return verify_steps(doc)
except Refuse as f:
return {"outcome": "refuse", "reason": f.reason, "at": f.at}
def verify_steps(doc):
if not exactly(doc, ["request", "pricebook", "policy", "record", "receipts", "keys"]) \
or not all(isinstance(doc[k], str) for k in ("request", "pricebook", "policy", "record")) \
or not (doc["receipts"] is None or (isinstance(doc["receipts"], list) and all(isinstance(x, str) for x in doc["receipts"]))) \
or not (doc["keys"] is None or isinstance(doc["keys"], dict)) \
or (doc["receipts"] is not None and doc["keys"] is None):
raise Refuse("malformed", [])
# 1-3. The documents.
req = read(doc["request"], request_ok, "request_invalid", ["request"])
book = read(doc["pricebook"], pricebook_ok, "pricebook_invalid", ["pricebook"])
pol = read(doc["policy"], policy_ok, "policy_invalid", ["policy"])
# 4. The record.
rec = read_document(doc["record"])
if not isinstance(rec, dict) or rec.get("kind") != "wai-route-record":
raise Refuse("malformed", ["record"])
if on("version") and is_int(rec.get("version")) and rec["version"] != 1:
raise Refuse("unknown_version", ["record", "version"])
if on("record-form") and not record_form(rec):
raise Refuse("record_invalid", ["record"])
if on("signature"):
bad = ed_verify(bytes.fromhex(rec["signer"]), signing_bytes(rec), bytes.fromhex(rec["sig"]))
if bad:
raise Refuse(bad, ["record", "sig"])
# 5. The binding.
deadline = None if req["deadline_us"] is None else rec["accepted"] + req["deadline_us"]
if on("binding-request"):
for m, want in (("job", req["job"]), ("operation", req["operation"]), ("capability", req["capability"]),
("fallback", req["fallback"]), ("quantities", req["quantities"]), ("request", digest(req)),
("deadline", deadline)):
if not same(rec[m], want):
raise Refuse("request_mismatch", ["record", m])
if on("binding-policy") and not same(rec["policy"], {"name": pol["name"], "version": pol["version"], "digest": digest(pol)}):
raise Refuse("policy_mismatch", ["record", "policy"])
if on("binding-pricebook") and not same(rec["pricebook"], {"digest": digest(book), "currency": book["currency"], "issued": book["issued"]}):
raise Refuse("pricebook_mismatch", ["record", "pricebook"])
# 6. The candidates, the order, the decision.
cands, order, decision, allowed = route(req, book, pol)
stated = rec["candidates"]
for i in range(max(len(stated), len(cands))):
at = ["record", "candidates", i]
if i >= len(stated) or i >= len(cands):
raise Refuse("candidate_mismatch", at)
s, c = stated[i], cands[i]
if on("candidates") and not same({k: v for k, v in s.items() if k not in ("removed", "rank")},
{k: v for k, v in c.items() if k not in ("removed", "rank")}):
raise Refuse("candidate_mismatch", at)
if on("removals") and not same(s["removed"], c["removed"]):
raise Refuse("removal_mismatch", at)
if on("ranks") and not same(s["rank"], c["rank"]):
raise Refuse("rank_mismatch", at)
if on("order") and not same(rec["order"], order):
raise Refuse("rank_mismatch", ["record", "order"])
if on("decision") and not same(rec["decision"], decision):
raise Refuse("decision_mismatch", ["record", "decision"])
# 7. The attempts.
atts, prev_end = rec["attempts"], rec["accepted"]
for i, a in enumerate(atts):
at = ["record", "attempts", i]
served = a["outcome"] == "served"
if on("allowed") and i >= allowed:
raise Refuse("attempts_exceeded", at)
if on("removed") and a["endpoint"] not in order:
raise Refuse("attempt_removed", at)
if on("in-order") and (i >= len(order) or a["endpoint"] != order[i]):
raise Refuse("attempt_out_of_order", at)
if on("served-last") and served and i != len(atts) - 1:
raise Refuse("served_not_last", at)
if on("times") and (a["start"] < prev_end or a["end"] < a["start"]):
raise Refuse("times_invalid", at)
prev_end = a["end"]
billed, not_billed = billing(served, a["end"], deadline)
if on("billing") and not (same(a["billed"], billed) and same(a["not_billed"], not_billed)):
raise Refuse("billing_invalid", at)
last_served = bool(atts) and atts[-1]["outcome"] == "served"
if on("complete") and not last_served and len(atts) < allowed:
raise Refuse("attempts_incomplete", ["record", "attempts"])
# 8. The hop receipts.
unmetered = None
if doc["receipts"] is not None:
unmetered = hop_receipts(rec, doc["receipts"], doc["keys"])
served_attempt = atts[-1] if last_served else None
billed_hop = next((a["hop"] for a in atts if a["billed"]), None)
if deadline is None or served_attempt is None:
dl = "none"
else:
dl = "missed" if served_attempt["end"] > deadline else "met"
return {
"outcome": "accept",
"digest": digest(rec),
"job": req["job"],
"operation": req["operation"],
"served": None if served_attempt is None else {"endpoint": served_attempt["endpoint"], "hop": served_attempt["hop"]},
"billed": billed_hop,
"deadline": dl,
"order": order,
"decision": decision,
"removed": [{"endpoint": c["endpoint"], "limits": [x["limit"] for x in c["removed"]]} for c in cands if c["removed"]],
"receipts": "checked" if unmetered is not None else "not_checked",
"unmetered": unmetered,
}
def hop_receipts(rec, texts, keys):
"""§6: each attempt held to its hop's OER/2 receipt. The hops whose receipts state the
unmetered marker."""
read_ = []
for j, t in enumerate(texts):
try:
read_.append(OER.load(io.StringIO(t)))
except ValueError:
raise Refuse("receipt_invalid", ["receipts", j])
unmetered = []
for i, a in enumerate(rec["attempts"]):
at = ["record", "attempts", i, "receipt"]
r = next((x for x in read_ if isinstance(x, dict) and x.get("receipt_id") == a["receipt"]), None)
if r is None:
if on("receipt-given"):
raise Refuse("receipt_missing", at)
continue
if on("receipt-verifies") and OER.verify_receipt(r, keys):
raise Refuse("receipt_invalid", at)
status = 1 if a["outcome"] == "served" else -1
work = r.get("work") if isinstance(r.get("work"), dict) else {}
got = work.get("status")
status_ok = isinstance(got, int) and not isinstance(got, bool) and got == status
if on("receipt-matches") and (r.get("kind") != "job" or r.get("capability") != "oer.job." + rec["operation"] or not status_ok):
raise Refuse("receipt_mismatch", at)
if OER.is_unmetered(r):
unmetered.append(a["hop"])
return unmetered
# ---------------------------------------------------------------- the corpus
def check(doc, quiet=False):
fails = []
for v in doc.get("vectors", []):
try:
got = verify(v["document"])
except Exception as e: # a crash on a mutated copy is a failure to verify it
got = {"outcome": "crash", "error": "%s: %s" % (type(e).__name__, e)}
ok = got == v.get("outcome")
if not quiet:
what = got["outcome"] + (" %s at %s" % (got["reason"], got["at"]) if got["outcome"] == "refuse" else "")
print(("PASS" if ok else "FAIL") + " %s: %s" % (v.get("name"), what))
if not ok:
fails.append(v.get("name"))
return fails
def reasons_listed():
"""§8's table, in order, and the codes its paragraph names as reused."""
text = open(EXTENSION, encoding="utf-8").read()
start = text.index("\n## 8. Reasons")
section = text[start:text.find("\n## ", start + 1)]
added = [m.group(1) for m in re.finditer(r"^\| `([a-z_]+)` \|", section, re.M)]
reused = [r for r in re.findall(r"`([a-z_]+)`", section.split("also raises", 1)[1]) if r not in added]
return added, reused
def flip(text, at):
b = bytearray(text.encode("utf-8"))
i = at % len(b)
b[i] = ord("1") if b[i] != ord("1") else ord("2")
return b.decode("utf-8", "replace")
def mutations(doc):
"""Copies of the corpus, one defect each, that a verifier must reject."""
out = []
def copy_with(name, k, edit):
m = copy.deepcopy(doc)
edit(m["vectors"][k])
out.append(("%s: %s" % (doc["vectors"][k]["name"], name), m))
for k, v in enumerate(doc["vectors"]):
d, o = v["document"], v["outcome"]
if o["outcome"] != "accept":
r = o["reason"]
copy_with("its stated reason changed", k, lambda x, r=r: x["outcome"].__setitem__("reason", "record_invalid" if r != "record_invalid" else "malformed"))
copy_with("its stated place changed", k, lambda x: x["outcome"].__setitem__("at", ["record", "elsewhere"]))
continue
for member in ("request", "pricebook", "policy", "record"):
digits = [i for i, c in enumerate(d[member]) if c in "23456789"]
if digits:
copy_with("a digit of its %s changed" % member, k, lambda x, m=member, i=digits[len(digits) // 2]:
x["document"].__setitem__(m, x["document"][m][:i] + str(int(x["document"][m][i]) - 1) + x["document"][m][i + 1:]))
at_sig = d["record"].index('"sig":"') + len('"sig":"')
copy_with("its record's signature flipped", k, lambda x, i=at_sig: x["document"].__setitem__(
"record", d["record"][:i] + ("1" if d["record"][i] == "0" else "0") + d["record"][i + 1:]))
if d["receipts"]:
copy_with("its receipts dropped", k, lambda x: x["document"].__setitem__("receipts", []))
copy_with("its first receipt's figure changed", k, lambda x: x["document"]["receipts"].__setitem__(
0, x["document"]["receipts"][0].replace('"latency_s":"1.0"', '"latency_s":"2.0"', 1)))
copy_with("its keys swapped for another", k, lambda x: x["document"].__setitem__(
"keys", {n: "d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a" for n in x["document"]["keys"]}))
copy_with("its stated served hop changed", k, lambda x: x["outcome"].__setitem__("served", None if o["served"] else {"endpoint": "im.alpha", "hop": 1}))
copy_with("its stated billed hop changed", k, lambda x: x["outcome"].__setitem__("billed", None if o["billed"] else 1))
copy_with("its stated deadline changed", k, lambda x: x["outcome"].__setitem__("deadline", "met" if o["deadline"] != "met" else "missed"))
copy_with("its stated order reversed", k, lambda x: x["outcome"].__setitem__("order", list(reversed(o["order"])) + ["x"]))
copy_with("its stated decision changed", k, lambda x: x["outcome"]["decision"].__setitem__("decided_by", "price" if o["decision"]["decided_by"] != "price" else "ttfb"))
copy_with("its stated removals dropped", k, lambda x: x["outcome"].__setitem__("removed", o["removed"][1:] if o["removed"] else [{"endpoint": "x", "limits": ["ga"]}]))
changed = [] if o["unmetered"] is None else (None if o["unmetered"] == [] else [])
copy_with("its stated unmetered hops changed", k, lambda x, c=changed: x["outcome"].__setitem__("unmetered", c))
return out
CHECKS = ["request_invalid", "pricebook_invalid", "policy_invalid", "version", "record-form", "signature",
"binding-request", "binding-policy", "binding-pricebook", "candidates", "removals", "ranks", "order", "decision",
"allowed", "removed", "in-order", "served-last", "times", "billing", "complete",
"receipt-given", "receipt-verifies", "receipt-matches",
"limit-policy", "limit-unit", "limit-region", "limit-residency", "limit-training", "limit-retention", "limit-ga",
"limit-consent", "limit-unit-price", "limit-spend"]
def main(corpus_dir):
doc = json.load(open(os.path.join(corpus_dir, "vectors.json"), encoding="utf-8"))
fails = check(doc)
files = sorted(os.listdir(corpus_dir))
if files != ["README.md", "vectors.json"]:
fails.append("the corpus holds other files: %s" % files)
seen = {v["outcome"].get("reason") for v in doc["vectors"]}
added, reused = reasons_listed()
missing = [r for r in added + reused if r not in seen]
if missing:
fails.append("no vector is refused %s" % missing)
print("%s every reason of §8 and every code it reuses refuses a vector (%d added, %d reused)"
% ("PASS" if not missing else "FAIL", len(added), len(reused)))
unlisted = sorted(r for r in seen if r is not None and r not in added + reused)
if unlisted:
fails.append("§8 does not name %s, which the corpus raises" % unlisted)
print("%s §8 names every reason the corpus raises" % ("PASS" if not unlisted else "FAIL"))
copies = mutations(doc)
weak = [name for name, m in copies if not check(m, quiet=True)]
print("%s every mutated copy of the corpus fails (%d copies)" % ("PASS" if not weak else "FAIL", len(copies)))
fails += ["the mutated copy %r verifies" % w for w in weak]
undetected = []
for c in CHECKS:
SKIP.add(c)
try:
if not check(doc, quiet=True):
undetected.append(c)
finally:
SKIP.discard(c)
print("%s the corpus catches the verifier with each of its checks dropped (%d checks)" % ("PASS" if not undetected else "FAIL", len(CHECKS)))
fails += ["dropping the check %r goes unnoticed" % c for c in undetected]
for f in fails:
print("FAIL", f)
return 1 if fails else 0
def check_document(path):
"""One verification document: its outcome, printed. Status 0 only when it is accepted."""
got = verify(json.load(open(path, encoding="utf-8")))
print(json.dumps(got, indent=1))
return 0 if got["outcome"] == "accept" else 1
if __name__ == "__main__":
if len(sys.argv) == 3 and sys.argv[1] == "--document":
sys.exit(check_document(sys.argv[2]))
if len(sys.argv) != 2:
print("usage: route_record_verify.py <live-conformance/routes> | --document FILE", file=sys.stderr)
sys.exit(2)
sys.exit(main(sys.argv[1]))

23
wai/live-conformance/routes/README.md Normal file
View file

@ -0,0 +1,23 @@
# Route record corpus
The conformance corpus of [`wai.route.record`](../../extensions/route-record.md): one file,
`vectors.json`, written by `wai_route_record_vectors gen` from fixed seeds (extension §10).
Each vector is a verification document (§7.1) — the request, the price book, the policy and
the record, each as its text, the hops' OER/2 receipts of minor version 1 as their texts and
the key they are published under — with the outcome §7.3 gives it. The accepted vectors are
routed and sealed by the reference; each refused one is a control of one defect, its record
signed again over the defect where the defect is not in the signature, and the generator
refuses to write a corpus whose control is refused for any reason or at any place but its
own.
```sh
# wai-rs: regenerates byte for byte, and every vector gets its outcome
cargo run --no-default-features --features route_record --bin wai_route_record_vectors -- verify ../live-conformance/routes
# the second implementation, written from the extension's text (needs `cryptography`, which the
# OER/2 reference verifier uses for the hop receipts)
python3 -I ../route_record_verify.py live-conformance/routes
```
`keys` names the router's key and the hops' key; both come from fixed seeds and attest nothing.

2703
wai/live-conformance/routes/vectors.json Normal file
View file

File diff suppressed because one or more lines are too long

46
wai/oer-conformance/README.md
View file

@ -16,6 +16,7 @@ the reference verifier here, the portable C kernel in [`settle/`](settle/) that
| `vectors.json` | 9 | OER/2 (draft 0.1, as the extension profiles it): the last 150 decision receipts of one simulated day of Settlement World's gatewayless corridor, signed with the Ed25519 ledger key published in the file, an hourly bundle with its provenance root, an Iridium anchor, and seven tampered twins |
| `firmware_vectors.json` | 1 | 400 receipts written by the C kernel with its own SHA-256 and Ed25519 — pinned, and regenerated byte-for-byte in CI |
| `edge_vectors.json` | 99 | one case for each behaviour the upstream nine never exercise — see below |
| `job_vectors.json` | 38 | `job` receipts of minor version 1 (extension §3.1), each expectation written by hand — see below |
| `unreadable/` | 11 documents | one for each way the extension refuses a document whole — a number whose nearest binary64 is infinite (the midpoint above the largest, as a float and as an integer, and far past it), `NaN`, `Infinity`, `-Infinity`, an unpaired surrogate, a repeated member name (plain, and a second `joules_total` inside a sealed receipt), nesting 65 deep, and nesting 5000 deep |
| `sw_export.json` | — | the Settlement World export `generate_vectors.py` builds `vectors.json` from |
@ -90,16 +91,49 @@ version written as a number, an absent `kind`, lenient decimals, a receipt that
not an object, sums decided past the 28th digit, members of the wrong type, and
documents at the edge of what a parser can read. All are fixed in both.
## Minor version 1: the job vectors
`job_vectors.json` holds `job` receipts of minor version 1 (`"oer": "2.1"`, extension
§3.1), one hop of a routed media job each, named `oer.job.media.encode`:
- **valid** — a hop measured by its host's counter, its class `OnChipCounter` graded
`measured`; a model's figure graded `derived`, an estimate graded `projected`, a
calibrated instrument's graded `measured`; a hop with no class; the largest relative
uncertainty a class states; and the **unmetered marker** (`joules_total: null`, every
breakdown term null, graded `projected`) alone, after a metered hop in a failover
chain, and on a failed hop (`status` −1);
- **the gate** — a class, and the marker, under minor version 0, and a class under
`"2.10"`, which this text does not define: `canonical_failure` and `malformed` as
before minor version 1;
- **`unmetered_invalid`** — the marker beside a breakdown term, graded `measured`,
beside a class, and in a bundle;
- **`class_invalid`** — the legacy `HwShunt`, a counter class without its uncertainty or
its filtering state, a filtering state outside the three, a relative uncertainty of
zero, past 2³² − 1, as a string and as `true`, a window of zero, a model's figure
with evidence, a calibrated instrument with an empty reference or with a filtering
state, a class that is a string and a label outside the vocabulary (and a member the
class does not list, `canonical_failure`);
- **`class_grade_mismatch`** — a counter reading graded `derived`, an estimate graded
`measured`, a model's figure graded `simulated`, a calibrated instrument's graded
`projected` (and a class beside a grade outside the vocabulary, `grade_unknown`
alone);
- a **ledger** over a chain that holds the marker (`ledger_mismatch`), and a sealed
unmetered receipt given a figure afterwards (the two hash mismatches).
Every vector declares `unmetered`, the receipts it states the marker in, and a
verifier passes it only when it reports exactly those as unmetered.
## Running
```sh
# wai-rs — both fixtures, all three files, and every document in unreadable/
# wai-rs — the three fixtures, every vector file, and every document in unreadable/
cargo run --no-default-features --features oer --bin wai_oer_vectors -- verify ../oer-conformance
# the reference verifier, one file at a time (needs `cryptography`)
python3 oer2_verify.py vectors.json
python3 oer2_verify.py firmware_vectors.json
python3 oer2_verify.py edge_vectors.json
python3 oer2_verify.py job_vectors.json
python3 oer2_verify.py --unreadable unreadable # every document there must be refused
# the C kernel: rebuild and regenerate its vectors; the output is deterministic
@ -117,7 +151,8 @@ verifier with a perturbed field order rejects every receipt it writes. A second
fixture pins the stage list `bundle.provenance_root` addresses: every stage list in
the corpus is ASCII, where escaped and unescaped serialisations agree, so only a
non-ASCII fixture can hold the rule that the stage list is serialised exactly as the
body is.
body is. A third pins minor version 1's members: `energy.class` and its own members
out of order, against a literal, and the same input under `"2.0"`, which is refused.
## Differential fuzzing
@ -140,8 +175,9 @@ goes on to check the hash. A known divergence planted in either verifier is caug
## Regenerating
`python3 generate_vectors.py` and `python3 generate_edge_vectors.py` resolve their
inputs and outputs against their own directory and reproduce `vectors.json` and
`edge_vectors.json` byte for byte. The signing seed is published
`python3 generate_vectors.py`, `python3 generate_edge_vectors.py` and
`python3 generate_job_vectors.py` resolve their inputs and outputs against their own
directory and reproduce `vectors.json`, `edge_vectors.json` and `job_vectors.json` byte
for byte (the job vectors' seed is `sha256("oer job vectors: the hop key")`). The signing seed is published
(`sha256("settlement world node ledger vector key")`); it attests nothing and exists
only to make the vectors reproducible.

47
wai/oer-conformance/fuzz_differential.py
View file

@ -7,9 +7,15 @@ ledgers, published keys and signatures -- type swaps at any member, deletions, u
members, respelled decimals, repeated member names, literals no JSON library writes
(`-0`, `1e2`, lone surrogates) -- and requires the reference verifier here and wai-rs
(`wai_oer_diff`) to report the same thing for every document: the same reasons, the
same refusal, and no crash in either.
same refusal, and no crash in either. Minor version 1's job receipts (extension
section 3.1: the unmetered marker and the acquisition class) are mutated too, under
their own key.
python3 fuzz_differential.py [--seed N] [--cases N]
python3 fuzz_differential.py --write TEXTS --cases N # the documents and the
reference's verdicts, one JSON line each, for a wai_oer_diff run elsewhere
python3 fuzz_differential.py --compare TEXTS RUST # those verdicts against
wai_oer_diff's output
Exit status 0 when every document agrees. Deterministic for a given seed. Needs
`cryptography`; receipts hashed with blake3 are included only when the `blake3` module
@ -57,25 +63,34 @@ R = V01["receipts"][0]
BUNDLE = next(r for v in BASE["vectors"] for r in v["receipts"] if r.get("kind") == "bundle")
PRIV = ed25519.Ed25519PrivateKey.from_private_bytes(
hashlib.sha256(b"settlement world node ledger vector key").digest())
JOB = json.load(open(os.path.join(HERE, "job_vectors.json")))
JOB_KEYS = JOB["keys"]
JOB_PRIV = ed25519.Ed25519PrivateKey.from_private_bytes(hashlib.sha256(b"oer job vectors: the hop key").digest())
JOB_METERED = JOB["vectors"][0]["receipts"][0]
JOB_UNMETERED = JOB["vectors"][1]["receipts"][0]
JOB_CALIBRATED = JOB["vectors"][5]["receipts"][0]
VALUES = [None, True, False, 0, 1, -1, 5, 2**53 - 1, 2**53, -(2**53), 2**64, "", "x", "0", "1.5", "+1",
" 1", "1.", "01", "1e1000", "١", "NaN", "abstain", "escalate", "2.0", "3.0", "sha-256",
"sha3-512", "ed25519", [], {}, [0.5], {"a": 1}, [1, [2, [3]]], 0.5, 1e300,
Lit("-0"), Lit("1e2"), Lit("1.0"), Lit("-0.0"), Lit("2.99999999999999977"), Lit("9007199254740993"),
Lit('"\\u0000"'), Lit('"\\ud83d\\ude00"'), Lit('"\\ud800"'), Lit("1.7976931348623157e308"),
Lit("123456789012345678901234567890")]
Lit("123456789012345678901234567890"), "2.1", "2.10", "projected", "measured", "derived",
"OnChipCounter", "CalibratedInstrument", "ModelBased", "Estimator", "HwShunt", "Off", "Unknown",
{"label": "Estimator"}, {"label": "OnChipCounter", "relative_ppm": 1, "window_us": 1, "filtering": "On"},
2**32 - 1, 2**32]
if HAVE_BLAKE3:
VALUES.append("blake3")
def reseal(r):
def reseal(r, priv=None):
"""Re-seal after an edit where the body still canonicalises, so the checks past the
seal are reached; otherwise leave the stale seal, which is itself a case."""
try:
body = V.canonical_bytes(r)
r["receipt_id"] = V.h(r["hash_alg"], body)
r["chain"]["receipt_hash"] = V.h(r["hash_alg"], r["chain"]["prev_hash"].encode() + body)
r["chain"]["signature"] = PRIV.sign(r["chain"]["receipt_hash"].encode()).hex()
r["chain"]["signature"] = (priv or PRIV).sign(r["chain"]["receipt_hash"].encode()).hex()
except Exception:
pass
return r
@ -131,6 +146,11 @@ def document(receipts, extra=None, keys=None):
def case(rng):
if rng.random() < 0.25: # a job receipt of minor version 1, mutated, re-sealed half the time
r = mutate(rng, copy.deepcopy(rng.choice([JOB_METERED, JOB_UNMETERED, JOB_CALIBRATED])), rng.randint(1, 3))
if rng.random() < 0.5 and not isinstance(r, Pairs):
r = reseal(r, JOB_PRIV)
return document([r], keys=JOB_KEYS)
kind = rng.random()
if kind < 0.6: # one receipt or bundle, mutated, re-sealed half the time
r = mutate(rng, copy.deepcopy(rng.choice([R, BUNDLE])), rng.randint(1, 3))
@ -188,9 +208,22 @@ def main():
ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
ap.add_argument("--seed", type=int, default=1)
ap.add_argument("--cases", type=int, default=20000)
ap.add_argument("--write", metavar="TEXTS")
ap.add_argument("--compare", nargs=2, metavar=("TEXTS", "RUST"))
args = ap.parse_args()
if args.compare:
texts = [json.loads(l) for l in open(args.compare[0], encoding="utf-8")]
ours = [reference(t) for t in texts]
rust = open(args.compare[1], encoding="utf-8").read().splitlines()
return report(args.seed, texts, ours, rust)
rng = random.Random(args.seed)
texts = [case(rng) for _ in range(args.cases)]
if args.write:
with open(args.write, "w", encoding="utf-8") as f:
for t in texts:
f.write(json.dumps(t) + "\n")
print("wrote %d documents to %s" % (len(texts), args.write))
return
ours = [reference(t) for t in texts]
with tempfile.NamedTemporaryFile("w", suffix=".jsonl", delete=False) as f:
for t in texts:
@ -200,13 +233,17 @@ def main():
"--bin", "wai_oer_diff", "--", f.name],
cwd=os.path.join(HERE, "..", "wai-rs"), capture_output=True, text=True, check=True).stdout.splitlines()
os.unlink(f.name)
report(args.seed, texts, ours, rust)
def report(seed, texts, ours, rust):
assert len(rust) == len(ours), (len(rust), len(ours))
diverged = [i for i in range(len(ours)) if ours[i] != rust[i]]
crashes = sum(o.startswith("CRASH") for o in ours) + rust.count("CRASH")
shape = {k: sum(1 for o in ours if f(o)) for k, f in [
("refused", lambda o: o == "REFUSED"), ("valid", lambda o: o.endswith("=")),
("with reasons", lambda o: o not in ("REFUSED", "NOFILE") and not o.endswith("=") and "CRASH" not in o)]}
print("seed %d: %d documents (%s)%s" % (args.seed, len(ours), ", ".join("%d %s" % (n, k) for k, n in shape.items()),
print("seed %d: %d documents (%s)%s" % (seed, len(ours), ", ".join("%d %s" % (n, k) for k, n in shape.items()),
"" if HAVE_BLAKE3 else "; blake3 receipts left out (no blake3 module)"))
for i in diverged[:20]:
print("DIVERGED %d\n reference: %s\n wai-rs: %s\n document: %s" % (i, ours[i], rust[i], texts[i][:400]))

210
wai/oer-conformance/generate_job_vectors.py Normal file
View file

@ -0,0 +1,210 @@
#!/usr/bin/env python3
"""Job vectors for OER/2 minor version 1 (extensions/oer-receipts.md §3.1).
A `job` receipt states one hop of a routed media job: the operation it ran, under
`oer.job.*`, and its figure, which minor version 1 lets it state as the unmetered marker
(a null `energy.joules_total`) or beside the acquisition class it was read with
(`energy.class`). This file holds receipts of both forms, valid and with one defect each,
and the same members under minor version 0, where §3 still refuses them.
Every `expected` and every `unmetered` below is written by hand from the rules of
§3.1 and §7. None is copied from a verifier's output. A receipt whose defect leaves the
seal intact is re-sealed (fresh receipt id, receipt hash and Ed25519 signature under the
fixed test seed below), so the declared reason is the only one it can produce.
Run: python3 generate_job_vectors.py (writes job_vectors.json beside it)
"""
import copy, hashlib, json, os, sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import oer2_verify as V # canonical form + hashing only; the fixtures pin those independently
from cryptography.hazmat.primitives.asymmetric import ed25519
from cryptography.hazmat.primitives import serialization
SEED = hashlib.sha256(b"oer job vectors: the hop key").digest()
PRIV = ed25519.Ed25519PrivateKey.from_private_bytes(SEED)
PUB = PRIV.public_key().public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw).hex()
SIGNER = "ed25519:hop-01"
GENESIS = "sha256:" + "00" * 32
# A metered hop: an encode measured by the host's package counter.
METERED = {
"oer": "2.1", "kind": "job", "capability": "oer.job.media.encode", "intent": "replicate",
"subject": {"inputs": ["sha256:" + "11" * 32], "output": "blake3:" + "22" * 32},
"work": {"status": 1, "outcome": "completed", "latency_s": "1.250000"},
"energy": {"joules_total": "3.200000", "boundary": "the hop's host: CPU package and DRAM",
"breakdown": {"compute_j": "3.000000", "movement_j": "0.200000", "idle_alloc_j": "0",
"shared_alloc_j": "0", "upstream_j": "0"},
"method": "hardware_meter", "temperature_k": "300",
"class": {"label": "OnChipCounter", "relative_ppm": 20000, "window_us": 1250000, "filtering": "Unknown"}},
"provenance": {"how": "measured", "device": "hardware", "weakest_input": {"how": "measured", "input": "package counter"}},
"floor": {"bits_committed": 0, "bits_delivered": "0", "floor_j": "0", "decades_above_floor": "0"},
"hash_alg": "sha-256", "sig_alg": "ed25519",
"chain": {"prev_hash": GENESIS, "receipt_hash": "", "signer": SIGNER, "signature": ""},
}
# An unmetered hop: no meter on the host, so the figure is the marker.
UNMETERED = copy.deepcopy(METERED)
UNMETERED["energy"] = {"joules_total": None, "boundary": "the hop's host: CPU package and DRAM",
"breakdown": {"compute_j": None, "movement_j": None, "idle_alloc_j": None,
"shared_alloc_j": None, "upstream_j": None},
"method": "unmetered", "temperature_k": "300"}
UNMETERED["provenance"] = {"how": "projected", "device": "hardware", "weakest_input": {"how": "projected", "input": "no meter"}}
UNMETERED["subject"]["output"] = "blake3:" + "33" * 32
def seal(r, prev=GENESIS):
"""Recompute id, chained hash and signature so the receipt is otherwise valid."""
r["chain"]["prev_hash"] = prev
body = V.canonical_bytes(r)
r["receipt_id"] = V.h(r["hash_alg"], body)
r["chain"]["receipt_hash"] = V.h(r["hash_alg"], r["chain"]["prev_hash"].encode() + body)
r["chain"]["signature"] = PRIV.sign(r["chain"]["receipt_hash"].encode()).hex()
return dict(sorted(r.items(), key=lambda kv: V.FIELD_ORDER[""].index(kv[0])))
def edit(src, fn, sealed=True):
r = copy.deepcopy(src)
fn(r)
return seal(r) if sealed else r
def cls(**kw):
def f(r):
r["energy"]["class"] = kw
return f
def drop_class(r):
r["energy"].pop("class")
M = seal(copy.deepcopy(METERED))
U = seal(copy.deepcopy(UNMETERED))
vectors = []
def vec(name, receipts, expected, unmetered, **extra):
"""`unmetered` names, by hand, the receipts this vector states the marker in: True for
the vector's one receipt, or a list of receipt ids."""
if unmetered is True:
unmetered = [receipts[0]["receipt_id"]]
v = {"name": name, "receipts": receipts, "expected": expected, "unmetered": unmetered}
v.update(extra)
vectors.append(v)
# --- valid ---
vec("j01 a metered hop with its counter class", [M], [], [])
vec("j02 an unmetered hop states the marker", [U], [], [U["receipt_id"]])
U2 = seal(copy.deepcopy(UNMETERED), prev=M["chain"]["receipt_hash"])
vec("j03 a failover chain: a metered hop, then an unmetered one", [M, U2], [], [U2["receipt_id"]])
vec("j04 a model-based figure graded derived",
[edit(METERED, lambda r: (cls(label="ModelBased")(r), r["provenance"].update(how="derived"),
r["energy"].update(method="model_estimated")))], [], [])
vec("j05 an estimator's figure graded projected",
[edit(METERED, lambda r: (cls(label="Estimator")(r), r["provenance"].update(how="projected")))], [], [])
vec("j06 a calibrated instrument's figure graded measured",
[edit(METERED, cls(label="CalibratedInstrument", relative_ppm=5000, window_us=1250000,
calibration_ref="urn:example:calibration:0001"))], [], [])
vec("j07 a metered hop with no class", [edit(METERED, drop_class)], [], [])
vec("j08 the largest relative uncertainty a class states",
[edit(METERED, cls(label="OnChipCounter", relative_ppm=2 ** 32 - 1, window_us=1, filtering="On"))], [], [])
# --- minor version 0 keeps §3's members ---
vec("j09 a class under minor version 0", [edit(M, lambda r: r.update(oer="2.0"), sealed=False)], ["canonical_failure"], [])
vec("j10 the marker under minor version 0", [edit(UNMETERED, lambda r: r.update(oer="2.0"))], ["malformed"], [])
vec("j11 a class under a minor version this text does not define", [edit(M, lambda r: r.update(oer="2.10"), sealed=False)],
["canonical_failure"], [])
# --- the unmetered marker's rules ---
vec("j12 the marker beside a breakdown term",
[edit(UNMETERED, lambda r: r["energy"]["breakdown"].update(compute_j="1.0"))], ["unmetered_invalid"], True)
vec("j13 the marker graded measured", [edit(UNMETERED, lambda r: r["provenance"].update(how="measured"))],
["unmetered_invalid"], True)
vec("j14 the marker beside a class",
[edit(UNMETERED, cls(label="Estimator"))], ["unmetered_invalid"], True)
def unmetered_bundle(r):
r["kind"] = "bundle"
r["capability"] = "oer.bundle.job"
stages = [{"kind": "job", "receipt_hash": U["chain"]["receipt_hash"], "joules": "0"}]
r["bundle"] = {"stages": stages, "provenance_root": V.h("sha-256", V.stage_list_bytes(stages))}
r["efficiency"] = {"completed_units": 1}
vec("j15 an unmetered bundle", [edit(UNMETERED, unmetered_bundle)], ["unmetered_invalid"], True)
vec("j16 a ledger over a chain that holds the marker", [M, U2], ["ledger_mismatch"], [U2["receipt_id"]],
ledger={"meter_joules": "3.200000", "unallocated_joules": "0"})
# --- the class's rules ---
vec("j17 the legacy hardware label", [edit(METERED, cls(label="HwShunt"))], ["class_invalid"], [])
vec("j18 a counter class with no uncertainty", [edit(METERED, cls(label="OnChipCounter", filtering="Off"))],
["class_invalid"], [])
vec("j19 a counter class with no filtering state",
[edit(METERED, cls(label="OnChipCounter", relative_ppm=20000, window_us=1250000))], ["class_invalid"], [])
vec("j20 a filtering state outside the three",
[edit(METERED, cls(label="OnChipCounter", relative_ppm=20000, window_us=1250000, filtering="Maybe"))],
["class_invalid"], [])
vec("j21 a relative uncertainty of zero",
[edit(METERED, cls(label="OnChipCounter", relative_ppm=0, window_us=1250000, filtering="Off"))], ["class_invalid"], [])
vec("j22 a window of zero",
[edit(METERED, cls(label="OnChipCounter", relative_ppm=20000, window_us=0, filtering="Off"))], ["class_invalid"], [])
vec("j23 a relative uncertainty past 2^32 - 1",
[edit(METERED, cls(label="OnChipCounter", relative_ppm=2 ** 32, window_us=1250000, filtering="Off"))],
["class_invalid"], [])
vec("j24 a relative uncertainty written as a string",
[edit(METERED, cls(label="OnChipCounter", relative_ppm="20000", window_us=1250000, filtering="Off"))],
["class_invalid"], [])
vec("j25 a relative uncertainty written as true",
[edit(METERED, cls(label="OnChipCounter", relative_ppm=True, window_us=1250000, filtering="Off"))],
["class_invalid"], [])
vec("j26 a model's figure with evidence beside it",
[edit(METERED, lambda r: (cls(label="ModelBased", relative_ppm=20000)(r), r["provenance"].update(how="derived")))],
["class_invalid"], [])
vec("j27 a calibrated instrument with an empty certificate reference",
[edit(METERED, cls(label="CalibratedInstrument", relative_ppm=5000, window_us=1250000, calibration_ref=""))],
["class_invalid"], [])
vec("j28 a calibrated instrument with a filtering state",
[edit(METERED, cls(label="CalibratedInstrument", relative_ppm=5000, window_us=1250000, filtering="Off"))],
["class_invalid"], [])
vec("j29 a class that is a string", [edit(METERED, lambda r: r["energy"].update({"class": "OnChipCounter"}))],
["class_invalid"], [])
vec("j30 a label outside the vocabulary", [edit(METERED, cls(label="Guess"))], ["class_invalid"], [])
vec("j31 a member the class does not list",
[edit(M, cls(label="Estimator", source="nameplate"), sealed=False)], ["canonical_failure"], [])
# --- the class held to the grade (§2's correspondence) ---
vec("j32 a counter reading graded derived", [edit(METERED, lambda r: r["provenance"].update(how="derived"))],
["class_grade_mismatch"], [])
vec("j33 an estimator's figure graded measured", [edit(METERED, cls(label="Estimator"))],
["class_grade_mismatch"], [])
vec("j34 a model's figure graded simulated",
[edit(METERED, lambda r: (cls(label="ModelBased")(r), r["provenance"].update(how="simulated")))],
["class_grade_mismatch"], [])
vec("j35 a calibrated instrument's figure graded projected",
[edit(METERED, lambda r: (cls(label="CalibratedInstrument", relative_ppm=5000, window_us=1250000,
calibration_ref="urn:example:calibration:0001")(r),
r["provenance"].update(how="projected")))],
["class_grade_mismatch"], [])
vec("j36 a class beside a grade outside the vocabulary", [edit(METERED, lambda r: r["provenance"].update(how="guessed"))],
["grade_unknown"], [])
# A tampered figure after sealing is caught by the seal, as in minor version 0.
vec("j37 an unmetered receipt given a figure after it was sealed",
[edit(U, lambda r: (r["energy"].update(joules_total="0"), r["energy"]["breakdown"].update(
compute_j="0", movement_j="0", idle_alloc_j="0", shared_alloc_j="0", upstream_j="0")), sealed=False)],
["receipt_hash_mismatch", "receipt_id_mismatch"], [])
# A failed hop is completed work of no use: status -1, and here no meter either.
vec("j38 a failed hop, unmetered", [edit(UNMETERED, lambda r: r["work"].update(status=-1, outcome="failed"))], [], True)
doc = {"oer": "2.1", "draft": "0.1",
"source": "job receipts of minor version 1: the unmetered marker and the acquisition class (oer-receipts §3.1)",
"keys": {SIGNER: PUB}, "vectors": vectors}
with open(os.path.join(HERE, "job_vectors.json"), "w", encoding="utf-8") as f:
json.dump(doc, f, indent=1, ensure_ascii=False)
f.write("\n")
print("wrote job_vectors.json: %d vectors" % len(vectors))

2635
wai/oer-conformance/job_vectors.json Normal file
View file

File diff suppressed because it is too large Load diff

113
wai/oer-conformance/oer2_verify.py
View file

@ -26,32 +26,50 @@ FIELD_ORDER = {
"chain": ["prev_hash", "receipt_hash", "signer", "signature", "anchor"],
"chain.anchor": ["path", "delivered_at", "root"],
}
# Minor version 1 (§3.1): a receipt whose `oer` is exactly "2.1" may also carry the
# acquisition class of its figure, `energy.class`, and the unmetered marker, a `null`
# `energy.joules_total`. Every other receipt keeps the members above, so a 2.0 receipt
# reads exactly as it did.
MINOR_1 = "2.1"
FIELD_ORDER_MINOR_1 = dict(FIELD_ORDER)
FIELD_ORDER_MINOR_1["energy"] = FIELD_ORDER["energy"] + ["class"]
FIELD_ORDER_MINOR_1["energy.class"] = ["label", "relative_ppm", "window_us", "filtering", "calibration_ref"]
# The grade each class supports (§2), and the evidence each carries (§3.1).
CLASS_GRADE = {"CalibratedInstrument": "measured", "OnChipCounter": "measured", "ModelBased": "derived", "Estimator": "projected"}
CLASS_MEMBERS = {"CalibratedInstrument": {"label", "relative_ppm", "window_us", "calibration_ref"},
"OnChipCounter": {"label", "relative_ppm", "window_us", "filtering"},
"ModelBased": {"label"}, "Estimator": {"label"}}
FILTERING = {"Unknown", "Off", "On"}
KINDS = {"decision", "task", "reconstruction", "calibration", "job", "compile", "decode", "mitigate", "rearrange", "anchor", "bundle"}
GRADES = ["projected", "simulated", "derived", "measured"]
KBOLTZ, LN2 = Decimal("1.380649e-23"), Decimal("0.6931471805599453")
def _order(obj, path=""):
def _order(obj, path="", order=FIELD_ORDER):
if isinstance(obj, dict):
keys = FIELD_ORDER.get(path, sorted(obj.keys()))
keys = order.get(path, sorted(obj.keys()))
out = {}
for k in keys:
if k in obj:
out[k] = _order(obj[k], (path + "." + k) if path else k)
out[k] = _order(obj[k], (path + "." + k) if path else k, order)
for k in obj:
if k not in out:
raise ValueError("unknown field %s at %s" % (k, path or "root"))
return out
if isinstance(obj, list):
return [_order(x, path + "[]") for x in obj]
return [_order(x, path + "[]", order) for x in obj]
return obj
def field_order(receipt):
"""The members a receipt may carry: §3's, and for minor version 1 §3.1's too."""
return FIELD_ORDER_MINOR_1 if isinstance(receipt, dict) and receipt.get("oer") == MINOR_1 else FIELD_ORDER
def _dumps(v):
"""The one serialisation: compact separators, non-ASCII unescaped, UTF-8."""
return json.dumps(v, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def canonical_bytes(receipt):
body = {k: v for k, v in receipt.items() if k not in ("chain", "receipt_id")}
return _dumps(_order(body))
return _dumps(_order(body, "", field_order(receipt)))
def stage_list_bytes(stages):
"""The bytes `bundle.provenance_root` addresses: the stage list in member order,
@ -284,9 +302,18 @@ def _verify_receipt(r, keys):
return [e.args[0]]
raise
e = r["energy"]
total = sum(dec(v) for v in e["breakdown"].values())
if abs(total - dec(e["joules_total"])) > Decimal("1e-9") * max(Decimal(1), abs(total)):
reasons.append("sum_mismatch")
unmetered = is_unmetered(r)
if unmetered:
# The marker states no figure, so nothing sums to it: every term is null, the
# grade is the weakest, no class names how a figure was read, and no bundle's
# stages sum to it (§3.1).
if any(v is not None for v in e["breakdown"].values()) or r["provenance"]["how"] != "projected" \
or "class" in e or r.get("kind") == "bundle":
reasons.append("unmetered_invalid")
else:
total = sum(dec(v) for v in e["breakdown"].values())
if abs(total - dec(e["joules_total"])) > Decimal("1e-9") * max(Decimal(1), abs(total)):
reasons.append("sum_mismatch")
b = e.get("boundary")
if not (isinstance(b, str) and b):
reasons.append("boundary_missing")
@ -296,6 +323,12 @@ def _verify_receipt(r, keys):
reasons.append("floor_mismatch")
if r["provenance"]["how"] not in GRADES:
reasons.append("grade_unknown")
if r.get("oer") == MINOR_1 and "class" in e and not unmetered:
c = e["class"]
if not class_valid(c):
reasons.append("class_invalid")
elif r["provenance"]["how"] in GRADES and CLASS_GRADE[c["label"]] != r["provenance"]["how"]:
reasons.append("class_grade_mismatch")
# An abstention makes no verdict claim: no answer, and no record anything could
# later be served from (§3). Read leniently, so a strange shape is simply not an
# abstention rather than a different reason.
@ -314,7 +347,7 @@ def _verify_receipt(r, keys):
if root != r["bundle"]["provenance_root"]:
reasons.append("bundle_root_mismatch")
st = sum(dec(s["joules"]) for s in stages)
if abs(st - dec(e["joules_total"])) > Decimal("1e-9") * max(Decimal(1), abs(st)):
if not unmetered and abs(st - dec(e["joules_total"])) > Decimal("1e-9") * max(Decimal(1), abs(st)):
reasons.append("bundle_sum_mismatch")
eff = r.get("efficiency")
if not isinstance(eff, dict) or eff.get("completed_units") is None:
@ -324,6 +357,34 @@ def _verify_receipt(r, keys):
reasons.append(sig)
return reasons
def is_unmetered(r):
"""Whether a receipt states the unmetered marker (§3.1): minor version 1 and a null
`energy.joules_total`. A verifier reports it as unmetered, never as a figure."""
e = r.get("energy") if isinstance(r, dict) and r.get("oer") == MINOR_1 else None
return isinstance(e, dict) and "joules_total" in e and e["joules_total"] is None
def _uint(v, top):
return isinstance(v, int) and not isinstance(v, bool) and 1 <= v <= top
def class_valid(c):
"""`energy.class` as §3.1 states it: a label of the four classes a new profile may name
(never the legacy HwShunt), and exactly that label's evidence."""
if not isinstance(c, dict) or not isinstance(c.get("label"), str) or c["label"] not in CLASS_MEMBERS:
return False
if set(c) != CLASS_MEMBERS[c["label"]]:
return False
if c["label"] in ("ModelBased", "Estimator"):
return True
if not (_uint(c["relative_ppm"], 2**32 - 1) and _uint(c["window_us"], MAX_SAFE)):
return False
if c["label"] == "OnChipCounter":
return isinstance(c["filtering"], str) and c["filtering"] in FILTERING
return isinstance(c["calibration_ref"], str) and c["calibration_ref"] != ""
def unmetered_ids(receipts):
"""The receipts a verifier reports as unmetered, by receipt id, in order."""
return [r.get("receipt_id") if isinstance(r.get("receipt_id"), str) else "?" for r in receipts if is_unmetered(r)]
def _link(r, k):
ch = r.get("chain") if isinstance(r, dict) else None
v = ch.get(k) if isinstance(ch, dict) else None
@ -419,6 +480,16 @@ STAGE_FIXTURE = [{"joules": "2.5", "receipt_hash": "sha256:00", "kind": "décisi
STAGE_FIXTURE_BYTES = '[{"kind":"décision 😀","receipt_hash":"sha256:00","joules":"2.5"}]'
STAGE_FIXTURE_ID = "sha256:c1caa0cea7cf581f240346d9e9431d19cbe204c4daca157fea555fee6bf2638b"
# Minor version 1's members (§3.1), pinned the same way: `energy.class` and its members
# out of order, against a literal; and the same input under "2.0", which §3 refuses.
CLASS_FIXTURE = {
"sig_alg": "ed25519", "hash_alg": "sha-256", "oer": "2.1", "capability": "oer.job.media.encode", "kind": "job",
"energy": {"class": {"window_us": 1000000, "filtering": "Off", "relative_ppm": 20000, "label": "OnChipCounter"},
"boundary": "hop", "joules_total": "1.5"},
}
CLASS_FIXTURE_BYTES = '{"oer":"2.1","kind":"job","capability":"oer.job.media.encode","energy":{"joules_total":"1.5","boundary":"hop","class":{"label":"OnChipCounter","relative_ppm":20000,"window_us":1000000,"filtering":"Off"}},"hash_alg":"sha-256","sig_alg":"ed25519"}'
CLASS_FIXTURE_ID = "sha256:b473c94d166e9a262c4a151db7cad35f8e16b15b862d5bcfaa88750c83388b94"
def check_canonical_form():
"""Return a list of reasons the canonical form or the stage list no longer
@ -434,6 +505,16 @@ def check_canonical_form():
reasons.append("stage_form_mismatch")
if h("sha-256", got.encode("utf-8")) != STAGE_FIXTURE_ID:
reasons.append("stage_id_mismatch")
got = canonical_bytes(CLASS_FIXTURE).decode("utf-8")
if got != CLASS_FIXTURE_BYTES:
reasons.append("class_form_mismatch")
if h("sha-256", got.encode("utf-8")) != CLASS_FIXTURE_ID:
reasons.append("class_id_mismatch")
try:
canonical_bytes(dict(CLASS_FIXTURE, oer="2.0"))
reasons.append("class_admitted_in_2_0")
except ValueError:
pass
# The number rule reads -0 through load(); if that ever stops yielding a float,
# -0 would read as the integer zero and verify. Fail here, loudly, instead.
if not _forbidden_number(load(io.StringIO("-0"))):
@ -462,8 +543,15 @@ def run(vectors):
got.add("ledger_mismatch")
exp = v.get("expected")
expected = {x for x in exp if isinstance(x, str)} if isinstance(exp, list) else set()
results.append({"vector": str(v.get("name")), "expected": sorted(expected), "got": sorted(got),
"pass": got == expected})
res = {"vector": str(v.get("name")), "expected": sorted(expected), "got": sorted(got), "pass": got == expected}
# A vector that declares its unmetered receipts (§7) is held to them too: the
# marker is reported as unmetered, never as a figure.
if "unmetered" in v:
um = v["unmetered"]
res["unmetered_expected"] = [x for x in um if isinstance(x, str)] if isinstance(um, list) else []
res["unmetered_got"] = unmetered_ids([r for r in receipts if isinstance(r, dict)])
res["pass"] = res["pass"] and res["unmetered_got"] == res["unmetered_expected"]
results.append(res)
return results
def refuse_all(directory):
@ -504,5 +592,6 @@ if __name__ == "__main__":
ok = ok and not cf
for res in run(vec):
ok = ok and res["pass"]
print(("PASS " if res["pass"] else "FAIL ") + res["vector"] + " expected=" + ",".join(res["expected"]) + " got=" + ",".join(res["got"]))
um = (" unmetered expected=%s got=%s" % (",".join(res["unmetered_expected"]), ",".join(res["unmetered_got"]))) if "unmetered_got" in res else ""
print(("PASS " if res["pass"] else "FAIL ") + res["vector"] + " expected=" + ",".join(res["expected"]) + " got=" + ",".join(res["got"]) + um)
sys.exit(0 if ok else 1)

42
wai/registry/capabilities.json
View file

@ -1782,6 +1782,20 @@
"edition": null,
"registered": "2026-08-01"
},
{
"capability": "wai.route.record",
"kind": "record",
"status": "registered",
"media": null,
"role": null,
"tier": null,
"licence": null,
"replicate": null,
"sets": [],
"spec_section": "§5 Interactive, temporal & computed content",
"edition": null,
"registered": "2026-10-29"
},
{
"capability": "wai.semantic.clip",
"kind": "payload",
@ -3990,6 +4004,34 @@
"section": "§6",
"status": "defined"
},
{
"name": "wai-route-policy",
"kind": "document-kind",
"document": "extensions/route-record.md",
"section": "§3.3",
"status": "defined"
},
{
"name": "wai-route-pricebook",
"kind": "document-kind",
"document": "extensions/route-record.md",
"section": "§3.2",
"status": "defined"
},
{
"name": "wai-route-record",
"kind": "document-kind",
"document": "extensions/route-record.md",
"section": "§5.1",
"status": "defined"
},
{
"name": "wai-route-request",
"kind": "document-kind",
"document": "extensions/route-record.md",
"section": "§3.1",
"status": "defined"
},
{
"name": "wai-scene/1",
"kind": "scheme",

7
wai/vocab/v2.ttl
View file

@ -1298,6 +1298,13 @@ wai:wai.quantum.job a wai:Capability ;
wai:specSection "§5 Interactive, temporal & computed content" ;
wai:registeredOn "2026-08-01"^^xsd:date .
wai:wai.route.record a wai:Capability ;
rdfs:label "wai.route.record" ;
wai:kind wai:kind-record ;
wai:status wai:status-registered ;
wai:specSection "§5 Interactive, temporal & computed content" ;
wai:registeredOn "2026-10-29"^^xsd:date .
wai:wai.semantic.clip a wai:Capability ;
rdfs:label "wai.semantic.clip" ;
wai:kind wai:kind-payload ;

6
wai/wai-pack/src/pack.rs
View file

@ -96,8 +96,10 @@ pub struct PackOpts {
pub floor: FileSet,
/// The override a command line applies (§3.2).
pub overrides: Overrides,
/// The meter that brackets the pack, or none (the unmetered form).
pub meter: Option<Box<dyn wai::energy_meter::Meter>>,
/// The meter that brackets the pack, or none (the unmetered form). `Send`, so a host
/// that runs the pack on a worker thread hands the packer its meter, and the package
/// receipt carries the figure.
pub meter: Option<Box<dyn wai::energy_meter::Meter + Send>>,
/// What a meter's report names as its power reader.
pub power_reader: String,
/// The `issued_at` of a measurement claim (RFC 3339, UTC).

34
wai/wai-pack/tests/packs.rs
View file

@ -198,7 +198,7 @@ fn the_floor_s_cmaf_files_hold_every_frame() {
}
/// A meter for tests: a clock that advances one second between its reads, at 500 mW.
fn model_meter() -> Box<dyn wai::energy_meter::Meter> {
fn model_meter() -> Box<dyn wai::energy_meter::Meter + Send> {
let mut t = 0u64;
Box::new(wai::energy_meter::ModelMeter::new(
move || {
@ -217,6 +217,38 @@ fn relist(files: &mut BTreeMap<String, Vec<u8>>) {
files.insert("LISTING".into(), l);
}
/// A host that runs the pack on a worker thread hands the packer its meter
/// (`PackOpts::meter` is `Box<dyn Meter + Send>`), so the package receipt states the
/// figure, byte for byte the pack this thread makes with the same meter.
#[test]
fn a_pack_on_a_worker_thread_carries_its_meter() {
let d = corpus().join("packs/video-base-01");
let policy = std::fs::read(d.join("policy.toml")).unwrap();
let opts = || {
let key = ed25519_dalek::SigningKey::from_bytes(&[0x5a; 32]);
let id = wai::signer_id::did_key(&key.verifying_key().to_bytes());
let mut o = wai_pack::pack::PackOpts::new(key, id, held(&d.join("floor")));
o.meter = Some(model_meter());
o.power_reader = "model:500mW".into();
o.issued_at = "2026-10-06T00:00:00Z".into();
o
};
let mut src = BTreeMap::new();
src.insert("v".to_string(), wai_pack::fixture::generate("video-base-01").unwrap());
let here = wai_pack::pack::pack(&policy, &src, &mut opts()).unwrap();
let (mut o, p, s) = (opts(), policy.clone(), src.clone());
let there = std::thread::spawn(move || wai_pack::pack::pack(&p, &s, &mut o)).join().unwrap().unwrap();
assert_eq!(there.receipt.joules_micro, 500_000);
assert!(wai_pack::verify::verify_files(&there.files).unwrap().metered);
assert_eq!(there.files, here.files);
fn send<T: Send>() {}
send::<wai_pack::pack::PackOpts>();
// The fabric meter goes into the same slot (`--meter fabric`, a macOS build with the
// feature `fabric-meter`).
#[cfg(all(target_os = "macos", feature = "fabric-meter"))]
send::<wai_pack::fabric_meter::FabricMeter>();
}
/// Test 17: a metered pack writes its measurement claim and verifies; without the claim
/// it is refused (`measurement-claim-missing`), with another key's claim too
/// (`measurement-claim-invalid`); an unmetered pack needs none.

2
wai/wai-pack/tests/refusals.rs
View file

@ -29,7 +29,7 @@ fn policy_for(floor: &Files, edit: &dyn Fn(String) -> String) -> Vec<u8> {
edit(p).into_bytes()
}
fn pack(policy: &[u8], floor: Files, meter: Option<Box<dyn wai::energy_meter::Meter>>) -> String {
fn pack(policy: &[u8], floor: Files, meter: Option<Box<dyn wai::energy_meter::Meter + Send>>) -> String {
let mut o = wai_pack::pack::PackOpts::new(ed25519_dalek::SigningKey::from_bytes(&[0x5a; 32]), "did:key:test", floor);
o.meter = meter;
o.power_reader = "model".into();

25
wai/wai-rs/Cargo.toml
View file

@ -510,9 +510,18 @@ settlement = ["live_metering", "jwp_hash"]
# and ads; each source verified by its own extension (live_metering, settlement, video_package,
# live_ads, stitched_presentation); the invoice (WLI1) whose lines cite their records and whose
# joules are per scope and per class, never summed across either, recomputed by a verifier; its
# chain; and the ad and delivery reports. SHA-256, Ed25519 and the sources' own readers; no new
# dependency.
receipt_ledger = ["live_metering", "settlement", "live_ads", "video_package", "stitched_presentation"]
# chain; and the ad and delivery reports. A tariff of version 2 also prices media jobs (§4.5): a
# job source is its route record (route_record, kind 16) and its attempts' OER/2 receipts (kind
# 14), and an `int_motion` job's decode receipt (video_receipt, kind 15). SHA-256, Ed25519 and the
# sources' own readers; no new dependency.
receipt_ledger =["live_metering", "settlement", "live_ads", "video_package", "stitched_presentation", "route_record", "video_receipt"]
# Route record (extensions/route-record.md): `wai.route.record` — what a router considered for one
# media job and why it chose: the request, the price book and the routing policy as documents
# pinned by digest; the candidates, the limits that removed each, the ranking, the order and the
# decision, recomputed from them; the attempts held to the order and the billing rules, each to
# its hop's OER/2 receipt (oer); signed by the router. SHA-256, Ed25519 and the OER verifier; no
# new dependency.
route_record = ["oer"]
# Revenue share (extensions/revenue-share): `wai.delivery.revenue_share` — the share contract (WRC1)
# an operator and every owner it names sign, mapping each asset of a channel to a split of integer
# millionths, and the statement (WRS1) of one party and one period: the ledger's verified invoices
@ -1326,6 +1335,11 @@ name = "wai_settlement_vectors"
path = "src/bin/wai_settlement_vectors.rs"
required-features = ["settlement"]
[[bin]]
name = "wai_route_record_vectors"
path = "src/bin/wai_route_record_vectors.rs"
required-features = ["route_record"]
[[bin]]
name = "wai_receipt_ledger_vectors"
path = "src/bin/wai_receipt_ledger_vectors.rs"
@ -2372,6 +2386,11 @@ name = "fleet"
path = "tests/fleet.rs"
required-features = ["fleet"]
[[test]]
name = "route_record"
path = "tests/route_record.rs"
required-features = ["route_record"]
[[test]]
name = "stream_audit"
path = "tests/stream_audit.rs"

4
wai/wai-rs/src/bin/wai.rs
View file

@ -69,6 +69,10 @@ mod dx;
#[path = "wai_cli/door.rs"]
mod door;
// `wai package build|verify`: the reference packager, wai-pack, run as itself.
#[path = "wai_cli/package.rs"]
mod package;
// Every file a command writes, through one place, so `--dry-run` writes none.
#[path = "wai_cli/out.rs"]
mod out;

20
wai/wai-rs/src/bin/wai_cli/door.rs
View file

@ -564,6 +564,23 @@ gated!(encoder_list, "encoder.list", cfg(feature = "platform_encode"), |a| {
});
gated!(encoderecord_verify, "encoder.verify", cfg(feature = "platform_encode"), |a| Ok(crate::platform_encode_cli::record_cmd(&with(&["verify"], a))?));
gated!(route_verify, "route.verify", cfg(feature = "route_record"), |a| {
let [path] = a else { return Err(usage_of("route.verify")) };
let bytes = std::fs::read(path).map_err(|e| format!("{path}: {e}"))?;
let doc = wai::oer::load(&bytes).map_err(|e| OpError::of(1, "check-failed", format!("{path}: not a verification document: {e}")))?;
let r = wai::route_record::verify(&doc);
println!("{}", serde_json::to_string_pretty(&wai::route_record::outcome_json(&r)).expect("JSON"));
Ok(r.is_ok())
});
fn package_build(a: &[String]) -> R {
crate::package::build(a)
}
fn package_verify(a: &[String]) -> R {
crate::package::verify(a)
}
macro_rules! stage_op {
($name:ident, $sub:literal, $id:literal) => {
gated!($name, $id, cfg(all(feature = "staged", feature = "video_int", feature = "audio_int", feature = "splat_int")), |a| {
@ -607,6 +624,9 @@ pub const IMPLS: &[Impl] = &[
Impl { id: "score.decode", run: score_render, json: Json::Own },
Impl { id: "encoder.list", run: encoder_list, json: Json::Text },
Impl { id: "encoder.verify", run: encoderecord_verify, json: Json::Text },
Impl { id: "package.build", run: package_build, json: Json::Text },
Impl { id: "package.verify", run: package_verify, json: Json::Text },
Impl { id: "route.verify", run: route_verify, json: Json::Own },
Impl { id: "registry.list", run: registry_list, json: Json::Text },
Impl { id: "registry.export", run: registry_export, json: Json::Text },
Impl { id: "prior.list", run: prior_list, json: Json::Text },

76
wai/wai-rs/src/bin/wai_cli/package.rs Normal file
View file

@ -0,0 +1,76 @@
//! `wai package build` and `wai package verify` (the catalogue's `package.build` and
//! `package.verify`): the reference packager, `wai-pack` (extensions/packaging.md), run as
//! itself, so a package made or checked through this door is the packager's own, byte for
//! byte. The packer is found as `wai dev` finds the relay: `--packer BIN`, then
//! `$WAI_PACK_BIN`, then `wai-pack` beside this binary, then on `PATH`.
use std::path::PathBuf;
use std::process::Command;
use crate::dx::envelope::OpError;
/// The packer, and the arguments that are its own.
fn packer(args: &[String]) -> Result<(PathBuf, Vec<String>), OpError> {
let mut rest = Vec::new();
let mut given = None;
let mut i = 0;
while i < args.len() {
if args[i] == "--packer" {
given = Some(args.get(i + 1).ok_or_else(|| OpError::usage("--packer takes the packer's binary", ""))?.clone());
i += 2;
} else {
rest.push(args[i].clone());
i += 1;
}
}
if let Some(p) = given.map(PathBuf::from).or_else(|| std::env::var_os("WAI_PACK_BIN").map(PathBuf::from)) {
return if p.is_file() { Ok((p, rest)) } else { Err(OpError::from(format!("wai-pack: {} is not a file", p.display()))) };
}
let file = format!("wai-pack{}", std::env::consts::EXE_SUFFIX);
let mut dirs: Vec<PathBuf> = std::env::current_exe().ok().and_then(|e| e.parent().map(PathBuf::from)).into_iter().collect();
if let Some(path) = std::env::var_os("PATH") {
dirs.extend(std::env::split_paths(&path));
}
dirs.iter()
.map(|d| d.join(&file))
.find(|p| p.is_file())
.map(|p| (p, rest))
.ok_or_else(|| OpError::from("wai-pack is not beside `wai` nor on PATH: install the archive that holds it, or give --packer BIN".to_string()))
}
/// Run the packer: `Ok(true)` when it exits 0, `Ok(false)` when it reports a refusal (exit 1).
fn run(bin: &PathBuf, verb: &str, args: &[String]) -> Result<bool, OpError> {
let st = Command::new(bin).arg(verb).args(args).status().map_err(|e| OpError::internal(format!("running {}: {e}", bin.display())))?;
match st.code() {
Some(0) => Ok(true),
Some(1) => Ok(false),
c => Err(OpError::internal(format!("{} {verb} exited {c:?}", bin.display()))),
}
}
/// `package.build`: `wai-pack pack` with these arguments. A dry run names the directory it
/// would write and runs nothing.
pub fn build(args: &[String]) -> Result<bool, OpError> {
let (bin, rest) = packer(args)?;
let out = rest.iter().position(|a| a == "--out").and_then(|i| rest.get(i + 1)).ok_or_else(|| OpError::usage("package build needs --out <dir>", ""))?;
for f in ["--policy", "--key"] {
if !rest.iter().any(|a| a == f) {
return Err(OpError::usage(format!("package build needs {f}"), ""));
}
}
if crate::out::dry_run() {
crate::out::would_write(out);
println!("dry run: {} pack would write the package into {out}", bin.display());
return Ok(true);
}
run(&bin, "pack", &rest)
}
/// `package.verify`: `wai-pack verify` with these arguments.
pub fn verify(args: &[String]) -> Result<bool, OpError> {
let (bin, rest) = packer(args)?;
if rest.is_empty() {
return Err(OpError::usage("package verify needs the package's directory", ""));
}
run(&bin, "verify", &rest)
}

5
wai/wai-rs/src/bin/wai_oer_vectors.rs
View file

@ -8,7 +8,9 @@
//! The corpus holds vector files from two independent producers — `vectors.json`
//! from the reference generator (Python) and `firmware_vectors.json` from the
//! portable C kernel in `settle/` — plus `edge_vectors.json`, one hand-declared case
//! for each reason and number shape the upstream nine never exercise. This binary is the third
//! for each reason and number shape the upstream nine never exercise, and
//! `job_vectors.json`, the hand-declared cases of minor version 1's job receipts (the
//! unmetered marker and the acquisition class, §3.1). This binary is the third
//! implementation and checks all three files, plus the two fixtures — canonical
//! form and stage list — the vectors cannot check.
//!
@ -27,6 +29,7 @@ const FILES: &[(&str, usize)] = &[
("vectors.json", 9), // the upstream draft's vectors (reference generator)
("firmware_vectors.json", 1), // the portable C kernel's output
("edge_vectors.json", 99), // the reasons and number shapes the upstream nine never exercise
("job_vectors.json", 38), // minor version 1's job receipts: the unmetered marker and the class (§3.1)
];
/// Documents a verifier must refuse whole (§4), in `unreadable/`: numbers whose

55
wai/wai-rs/src/bin/wai_route_record_vectors.rs Normal file
View file

@ -0,0 +1,55 @@
//! Conformance corpus for the route record (`live-conformance/routes/`,
//! extensions/route-record.md §10).
//!
//! wai_route_record_vectors gen <corpus-dir> write vectors.json, from fixed seeds
//! wai_route_record_vectors verify <corpus-dir> vectors.json regenerates byte for byte, and
//! every vector gets the outcome it states
//!
//! The generator and this verifier are one implementation. The corpus's second
//! implementation is `live-conformance/route_record_verify.py`, written from the extension's
//! text and sharing no code with this binary.
use std::path::Path;
use wai::route_record::corpus;
fn main() {
let a: Vec<String> = std::env::args().collect();
match (a.get(1).map(String::as_str), a.get(2)) {
(Some("gen"), Some(d)) => {
let p = Path::new(d).join("vectors.json");
let text = corpus::generate().unwrap_or_else(|e| {
eprintln!("generate: {e}");
std::process::exit(1);
});
if let Err(e) = std::fs::create_dir_all(d).and_then(|_| std::fs::write(&p, text)) {
eprintln!("write {}: {e}", p.display());
std::process::exit(1);
}
println!("wrote {}", p.display());
}
(Some("verify"), Some(d)) => std::process::exit(verify(Path::new(d))),
_ => {
eprintln!("usage: wai_route_record_vectors gen|verify <corpus-dir>");
std::process::exit(2);
}
}
}
fn verify(dir: &Path) -> i32 {
let p = dir.join("vectors.json");
let doc = match std::fs::read_to_string(&p) {
Ok(s) => s,
Err(e) => {
eprintln!("read {}: {e}", p.display());
return 1;
}
};
let same = corpus::generate().is_ok_and(|g| g == doc);
println!("{} vectors.json regenerates byte for byte from its seeds", if same { "PASS" } else { "FAIL" });
let fails = corpus::verify_corpus(&doc);
for f in &fails {
println!("FAIL {f}");
}
println!("{} every vector gets the outcome the corpus states", if fails.is_empty() { "PASS" } else { "FAIL" });
(!same || !fails.is_empty()) as i32
}

55
wai/wai-rs/src/catalogue.rs
View file

@ -309,6 +309,7 @@ const S_ENROLMENT: &str = r#"{"type":"object","required":["node","role","organis
const S_ENROLMENTS: &str = r#"{"type":"object","required":["enrolments"],"description":"Each node the fleet service holds enrolled: its key, role, label and records"}"#;
const S_REPORT: &str = r#"{"type":"object","required":["report","outcome"],"description":"The report's digest and its outcome as extensions/fleet.md §7.3 states it"}"#;
const S_REPORT_EXPORT: &str = r#"{"type":"object","required":["file","outcome"],"description":"The bundle's file and its outcome, recomputed here"}"#;
const S_ROUTE: &str = r#"{"type":"object","required":["outcome"],"description":"The outcome of extensions/route-record.md §7.3: accept with the record's digest, the served and billed hops, the deadline, the order, the decision, the removals and the unmetered hops; or refuse with the reason and where"}"#;
// The channel operations' outputs (extensions/channel-lineup.md §9).
const S_CHANNEL_IMPORT: &str = r#"{"type":"object","required":["channel","receipt","receipt_hash","schedule_root","lineup","lineup_digest","cues"],"description":"The files written (the receipt, the lineup, the cues), the receipt's identity and schedule root, the lineup's digest, and their counts"}"#;
const S_CHANNEL_GUIDE: &str = r#"{"type":"object","required":["file","format","schedule_root","lineup"],"description":"The guide's file and format (xmltv or now-next), the sources it names, and its range or instant"}"#;
@ -552,6 +553,46 @@ encode avc|hevc <in.y4m> [-o out.wai] [--encoder auto|platform|va|gpu-driver|sof
needs: Needs::All(&["platform_encode"]),
..OP
},
// ─── packaging, by the reference packager (wai/wai-pack, extensions/packaging.md) ───
Op {
id: "package.build",
summary: "A package packed by the reference packager wai-pack from a policy, a source per component and a supplied floor, with its signed package receipt",
cli: Some("package build"),
usage: "package build --policy <policy.toml> --out <dir> --key <seed.hex> [--source <id>=<file>]... [--floor <dir>]
[--signer-id <id>] [--meter none|model:<mW>] [--packer <wai-pack>] [wai-pack pack's other flags]
the reference packager's pack (extensions/packaging.md): the policy's components packed into <dir>, new or empty,
with the package receipt (package.receipt.json); the packer is --packer, $WAI_PACK_BIN, wai-pack beside wai,
or wai-pack on PATH",
effect: Effect::WritesProject,
stability: Stability::Preview,
input: &[
flag("policy", "--policy", Ty::Path, true, "The packaging policy (extensions/packaging.md §3)."),
flag("out", "--out", Ty::Path, true, "A new or empty directory, written with the package."),
flag("key", "--key", Ty::Path, true, "The packager's signing key: a 32-byte seed, in hexadecimal."),
flag("source", "--source", Ty::Strings, false, "A component's source, <id>=<file>; each component's own by default."),
flag("floor", "--floor", Ty::Path, false, "A supplied floor's directory (floor/ beside the policy by default)."),
flag("signer_id", "--signer-id", Ty::String, false, "The receipt's signer id (the key's did:key by default)."),
flag("meter", "--meter", Ty::String, false, "none, or model:<milliwatts>: the meter of the receipt's figure."),
flag("packer", "--packer", Ty::Path, false, "The packer's binary (wai-pack)."),
],
receipt: true,
..OP
},
Op {
id: "package.verify",
summary: "A package verified by the reference packager wai-pack: every reason it is refused, or its counts and pack digest",
cli: Some("package verify"),
usage: "package verify <dir> [--parent <dir>] [--packer <wai-pack>] the reference packager's verify
(extensions/packaging.md §9): the package's receipt, stage indexes, objects and floor; with --parent, a re-pack against
the package its receipt names; exit 1 when refused",
stability: Stability::Preview,
input: &[
pos("dir", Ty::Path, true, "The package's directory."),
flag("parent", "--parent", Ty::Path, false, "The package a re-pack's receipt names."),
flag("packer", "--packer", Ty::Path, false, "The packer's binary (wai-pack)."),
],
..OP
},
// ─── the registry and parameter sets ───
Op {
id: "registry.list",
@ -918,6 +959,19 @@ http vectors diff|fuzz <dir> --seed S --count N --out F",
confirm: Some(Confirm::AnyFile),
..OP
},
// ─── a routed media job (extensions/route-record.md) ───
Op {
id: "route.verify",
summary: "A route record checked offline against its request, price book and routing policy, and against its hops' OER/2 receipts where given",
cli: Some("route verify"),
usage: "route verify <document.json> a verification document (extensions/route-record.md §7.1): the record's outcome
(§7.3) as JSON, recomputed from the request, the price book and the policy; exit 1 when it is refused",
stability: Stability::Preview,
input: &[pos("document", Ty::Path, true, "The verification document: the request, price book, policy and record, and the hops' receipts.")],
output: Output::Json(S_ROUTE),
needs: Needs::All(&["route_record"]),
..OP
},
// ─── linear channels (extensions/channel-lineup.md §9) ───
Op {
id: "channel.import",
@ -1774,6 +1828,7 @@ pub fn feature_on(feature: &str) -> Option<bool> {
"mcp" => cfg!(feature = "mcp"),
"app" => cfg!(feature = "app"),
"fleet" => cfg!(feature = "fleet"),
"route_record" => cfg!(feature = "route_record"),
"revenue_share" => cfg!(feature = "revenue_share"),
"channel_lineup" => cfg!(feature = "channel_lineup"),
"opportunity_report" => cfg!(feature = "opportunity_report"),

8
wai/wai-rs/src/codecs/registry.rs
View file

@ -240,6 +240,7 @@ pub const CAP_DELIVERY_SETTLEMENT: &str = "wai.delivery.settlement";
pub const CAP_DELIVERY_LEDGER: &str = "wai.delivery.ledger";
pub const CAP_DELIVERY_AUDIT: &str = "wai.delivery.audit";
pub const CAP_DELIVERY_FLEET: &str = "wai.delivery.fleet";
pub const CAP_ROUTE_RECORD: &str = "wai.route.record";
pub const CAP_DELIVERY_REVENUE_SHARE: &str = "wai.delivery.revenue_share";
pub const CAP_DELIVERY_OPPORTUNITY: &str = "wai.delivery.opportunity";
pub const CAP_MEDIA_TAKEDOWN: &str = "wai.media.takedown";
@ -369,6 +370,7 @@ const ROWS: &[(&str, &str, Option<&str>, &str)] = &[
(CAP_DELIVERY_LEDGER, S_INTERACTIVE, None, "2026-10-25"),
(CAP_DELIVERY_AUDIT, S_INTERACTIVE, None, "2026-10-25"),
(CAP_DELIVERY_FLEET, S_INTERACTIVE, None, "2026-10-27"),
(CAP_ROUTE_RECORD, S_INTERACTIVE, None, "2026-10-29"),
(CAP_DELIVERY_REVENUE_SHARE, S_INTERACTIVE, None, "2026-10-29"),
(CAP_DELIVERY_OPPORTUNITY, S_INTERACTIVE, None, "2026-10-29"),
(CAP_MEDIA_TAKEDOWN, S_INTERACTIVE, None, "2026-10-21"),
@ -465,6 +467,7 @@ const RECORDS: &[&str] = &[
CAP_DELIVERY_LEDGER,
CAP_DELIVERY_AUDIT,
CAP_DELIVERY_FLEET,
CAP_ROUTE_RECORD,
CAP_DELIVERY_REVENUE_SHARE,
CAP_DELIVERY_OPPORTUNITY,
CAP_MEDIA_TAKEDOWN,
@ -689,6 +692,7 @@ const LOC_CARRIAGE: &str = "extensions/loc-carriage.md";
const STREAM_TERMINATION: &str = "extensions/stream-termination.md";
const KEY_RELEASE: &str = "extensions/key-release.md";
const CONFORMANCE_RESULTS: &str = "extensions/conformance-results.md";
const ROUTE_RECORD: &str = "extensions/route-record.md";
const fn wire(name: &'static str, kind: WireKind, document: &'static str, section: &'static str,
status: WireStatus) -> WireIdentifier {
@ -848,6 +852,10 @@ pub const WIRE_IDENTIFIERS: &[WireIdentifier] = &[
wire("WFR1", WireKind::Magic, FLEET, "§6.1", WireStatus::Defined),
wire("wai-fleet-bundle", WireKind::DocumentKind, FLEET, "§7.1", WireStatus::Defined),
wire(".wai-fleet.json", WireKind::Path, FLEET, "§7.1", WireStatus::Defined),
wire("wai-route-request", WireKind::DocumentKind, ROUTE_RECORD, "§3.1", WireStatus::Defined),
wire("wai-route-pricebook", WireKind::DocumentKind, ROUTE_RECORD, "§3.2", WireStatus::Defined),
wire("wai-route-policy", WireKind::DocumentKind, ROUTE_RECORD, "§3.3", WireStatus::Defined),
wire("wai-route-record", WireKind::DocumentKind, ROUTE_RECORD, "§5.1", WireStatus::Defined),
wire("WRC1", WireKind::Magic, REVENUE_SHARE, "§3.1", WireStatus::Defined),
wire("WRS1", WireKind::Magic, REVENUE_SHARE, "§7.1", WireStatus::Defined),
wire("WOR1", WireKind::Magic, OPPORTUNITY, "§7.1", WireStatus::Defined),

59
wai/wai-rs/src/energy_meter.rs
View file

@ -114,8 +114,8 @@ enum CounterSource {
/// `wrap_at`. The read and the clock are injected, so the delta, wraparound and
/// window arithmetic is exercised in tests without needing the hardware present.
pub struct CounterMeter {
read: Box<dyn FnMut() -> Option<u64>>,
clock_micros: Box<dyn FnMut() -> u64>,
read: Box<dyn FnMut() -> Option<u64> + Send>,
clock_micros: Box<dyn FnMut() -> u64 + Send>,
wrap_at: u64,
relative_ppm: NonZeroU32,
source: CounterSource,
@ -143,8 +143,8 @@ impl CounterMeter {
/// hardware unit is coarser than a microjoule hides that from the meter;
/// `relative_ppm` must cover it.
pub fn new(
read: impl FnMut() -> Option<u64> + 'static,
clock_micros: impl FnMut() -> u64 + 'static,
read: impl FnMut() -> Option<u64> + Send + 'static,
clock_micros: impl FnMut() -> u64 + Send + 'static,
wrap_at: u64,
relative_ppm: NonZeroU32,
) -> CounterMeter {
@ -204,8 +204,8 @@ impl CounterMeter {
/// Refuses an empty reference: without one, the "calibrated" in the class
/// name is an assertion nobody can check.
pub fn calibrated_instrument(
read: impl FnMut() -> Option<u64> + 'static,
clock_micros: impl FnMut() -> u64 + 'static,
read: impl FnMut() -> Option<u64> + Send + 'static,
clock_micros: impl FnMut() -> u64 + Send + 'static,
wrap_at: u64,
relative_ppm: NonZeroU32,
calibration_ref: impl Into<String>,
@ -351,7 +351,7 @@ impl Meter for CounterMeter {
/// (which the receipts refuse) or a saturated maximum (which is not the
/// model's output).
pub struct ModelMeter {
clock_micros: Box<dyn FnMut() -> u64>,
clock_micros: Box<dyn FnMut() -> u64 + Send>,
milliwatts: u64,
t0: Option<u64>,
}
@ -359,7 +359,7 @@ pub struct ModelMeter {
impl ModelMeter {
/// `clock_micros` returns a monotonic microsecond timestamp; `milliwatts`
/// is the calibrated marginal power of the work being measured.
pub fn new(clock_micros: impl FnMut() -> u64 + 'static, milliwatts: u64) -> ModelMeter {
pub fn new(clock_micros: impl FnMut() -> u64 + Send + 'static, milliwatts: u64) -> ModelMeter {
ModelMeter { clock_micros: Box::new(clock_micros), milliwatts, t0: None }
}
@ -395,18 +395,45 @@ impl Meter for ModelMeter {
#[cfg(test)]
mod tests {
use super::*;
use std::cell::RefCell;
use std::rc::Rc;
/// A scripted counter: successive reads return successive values.
fn scripted(values: Vec<Option<u64>>) -> impl FnMut() -> Option<u64> {
let q = Rc::new(RefCell::new(values.into_iter()));
move || q.borrow_mut().next().flatten()
fn scripted(values: Vec<Option<u64>>) -> impl FnMut() -> Option<u64> + Send {
let mut q = values.into_iter();
move || q.next().flatten()
}
fn scripted_clock(values: Vec<u64>) -> impl FnMut() -> u64 {
let q = Rc::new(RefCell::new(values.into_iter()));
move || q.borrow_mut().next().unwrap_or(0)
fn scripted_clock(values: Vec<u64>) -> impl FnMut() -> u64 + Send {
let mut q = values.into_iter();
move || q.next().unwrap_or(0)
}
/// Every meter here is `Send`, so a host hands one to a worker thread that runs the
/// work while another thread reads the counter (the package receipt's meter slot is
/// `Box<dyn Meter + Send>`).
#[test]
fn every_meter_crosses_threads() {
fn send<T: Send>(_: &T) {}
let counter = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0));
let c = std::sync::Arc::clone(&counter);
let mut m = CounterMeter::new(move || Some(c.load(std::sync::atomic::Ordering::SeqCst)), scripted_clock((0..64).map(|i| i * 100_000).collect()), u64::MAX, NonZeroU32::new(20_000).unwrap());
send(&m);
let model = ModelMeter::new(scripted_clock(vec![0, 1_000_000]), 500);
send(&model);
// The span runs on a worker thread; the counter advances from this one.
m.start();
let worker = std::thread::spawn(move || {
for _ in 0..8 {
m.sample();
}
m
});
counter.store(1_000_000, std::sync::atomic::Ordering::SeqCst);
let mut m = worker.join().unwrap();
let boxed: Box<dyn Meter + Send> = Box::new(model);
let back = std::thread::spawn(move || boxed).join().unwrap();
assert_eq!(back.kind(), AcquisitionKind::ModelBased);
assert_eq!(m.kind(), AcquisitionKind::OnChipCounter);
let _ = m.stop();
}
fn ppm(n: u32) -> NonZeroU32 {

14
wai/wai-rs/src/lib.rs
View file

@ -740,6 +740,14 @@ pub mod settlement;
/// cbindgen:ignore
#[cfg(feature = "receipt_ledger")]
pub mod receipt_ledger;
// Route record (extensions/route-record.md): `wai.route.record` — a router's account of one media
// job: the candidates a price book offered, the limits that removed each, the ranking under a
// named policy, the order, the decision and the attempts with their hop receipts, recomputed by a
// verifier from the request, the price book and the policy. Rust-only, like the other record
// modules.
/// cbindgen:ignore
#[cfg(feature = "route_record")]
pub mod route_record;
// Revenue share (extensions/revenue-share.md): `wai.delivery.revenue_share` — a share contract an
// operator and every owner it names sign (a channel, its carriers, each asset's split of integer
// millionths, the rule for a break between programmes), and the statement of one party and one
@ -821,10 +829,10 @@ pub mod prior_wire;
#[cfg(feature = "scene_map")]
pub mod scene_map;
// The strict JSON reader the two mappings above share, and the fleet's reader of a relay's session
// summaries. Rust-only.
// The strict JSON reader the two mappings above share, the fleet's reader of a relay's session
// summaries, and the reader of a `wai.video.int_motion` receipt's JSON. Rust-only.
/// cbindgen:ignore
#[cfg(any(feature = "prior_wire", feature = "scene_map", feature = "conform_result", feature = "fleet", feature = "channel_lineup"))]
#[cfg(any(feature = "prior_wire", feature = "scene_map", feature = "conform_result", feature = "fleet", feature = "video_receipt", feature = "channel_lineup"))]
pub(crate) mod strict_json;
// Conformance results (extensions/conformance-results.md): `conform-result/1`, the signed outcome
// of a conformance run, which `wai conform` writes and `wai verify` checks; and its corpus.

273
wai/wai-rs/src/oer.rs
View file

@ -51,6 +51,20 @@ pub const FIELD_ORDER: &[(&str, &[&str])] = &[
("chain.anchor", &["path", "delivered_at", "root"]),
];
/// Minor version 1 (extensions/oer-receipts.md §3.1): a receipt whose `oer` is exactly
/// this string may also carry its figure's acquisition class, `energy.class`, and the
/// unmetered marker, a `null` `energy.joules_total`. Every other receipt keeps §3's
/// members, so a `2.0` receipt reads exactly as it did.
pub const MINOR_1: &str = "2.1";
/// The members `energy` holds under minor version 1, and those of `energy.class`.
pub const ENERGY_MINOR_1: &[&str] = &["joules_total", "boundary", "breakdown", "method", "temperature_k", "class"];
pub const CLASS_MEMBERS: &[&str] = &["label", "relative_ppm", "window_us", "filtering", "calibration_ref"];
/// The grade each acquisition class supports (§2), for the labels a class may carry.
pub const CLASS_GRADE: &[(&str, &str)] =
&[("CalibratedInstrument", "measured"), ("OnChipCounter", "measured"), ("ModelBased", "derived"), ("Estimator", "projected")];
pub const KINDS: &[&str] = &["decision", "task", "reconstruction", "calibration", "job", "compile",
"decode", "mitigate", "rearrange", "anchor", "bundle"];
@ -209,15 +223,32 @@ pub fn py_json(v: &Value, ensure_ascii: bool) -> String {
// --- canonical form --------------------------------------------------------------
fn field_order(path: &str) -> Option<&'static [&'static str]> {
fn field_order(path: &str, minor_1: bool) -> Option<&'static [&'static str]> {
if minor_1 {
match path {
"energy" => return Some(ENERGY_MINOR_1),
"energy.class" => return Some(CLASS_MEMBERS),
_ => {}
}
}
FIELD_ORDER.iter().find(|(p, _)| *p == path).map(|(_, keys)| *keys)
}
/// Whether a receipt is of minor version 1 (§3.1), whose members include `energy.class`.
pub fn is_minor_1(receipt: &Value) -> bool {
receipt.get("oer").and_then(Value::as_str) == Some(MINOR_1)
}
/// Reorder `v` into the envelope's canonical key order, refusing unknown fields.
pub fn order(v: &Value, path: &str) -> Result<Value, String> {
order_in(v, path, false)
}
/// [`order`] under minor version 1's members when `minor_1` (§3.1).
fn order_in(v: &Value, path: &str, minor_1: bool) -> Result<Value, String> {
match v {
Value::Object(m) => {
let keys: Vec<String> = match field_order(path) {
let keys: Vec<String> = match field_order(path, minor_1) {
Some(ks) => ks.iter().map(|s| s.to_string()).collect(),
None => {
let mut ks: Vec<String> = m.keys().cloned().collect();
@ -229,7 +260,7 @@ pub fn order(v: &Value, path: &str) -> Result<Value, String> {
for k in &keys {
if let Some(x) = m.get(k) {
let child = if path.is_empty() { k.clone() } else { format!("{path}.{k}") };
out.insert(k.clone(), order(x, &child)?);
out.insert(k.clone(), order_in(x, &child, minor_1)?);
}
}
for k in m.keys() {
@ -241,7 +272,7 @@ pub fn order(v: &Value, path: &str) -> Result<Value, String> {
Ok(Value::Object(out))
}
Value::Array(xs) => Ok(Value::Array(
xs.iter().map(|x| order(x, &format!("{path}[]"))).collect::<Result<_, _>>()?,
xs.iter().map(|x| order_in(x, &format!("{path}[]"), minor_1)).collect::<Result<_, _>>()?,
)),
other => Ok(other.clone()),
}
@ -336,7 +367,7 @@ pub fn canonical_bytes(receipt: &Value) -> Result<Vec<u8>, String> {
body.insert(k.clone(), v.clone());
}
}
Ok(py_json(&order(&Value::Object(body), "")?, false).into_bytes())
Ok(py_json(&order_in(&Value::Object(body), "", is_minor_1(receipt))?, false).into_bytes())
}
/// The bytes `bundle.provenance_root` addresses: the stage list in member order,
@ -515,6 +546,31 @@ pub(crate) fn rounds_to_micro(v: &Value, uj: u64) -> Option<bool> {
Some(off.cmp(&Dec::int(1)) != Ordering::Greater)
}
/// A quantity, in joules, as whole microjoules: `v · 10⁶` rounded to the nearest, a half to the
/// even one, exactly. `None` when `v` is not a quantity (§4), is negative, or is past what a
/// `u64` holds. The receipt ledger states a job hop's figure so (receipt-ledger §5.1).
pub fn micro_joules(v: &Value) -> Option<u64> {
let d = Dec::from_value(v)?;
if d.m < BigInt::from(0u8) {
return None;
}
// d.m × 10^(6 − d.s), rounded half to even.
let shift = 6 - d.s;
let n = if shift >= 0 {
&d.m * BigInt::from(10u8).pow(u32::try_from(shift).ok()?)
} else {
let div = BigInt::from(10u8).pow(u32::try_from(-shift).ok()?);
let (q, r) = (&d.m / &div, &d.m % &div);
let twice = &r * BigInt::from(2u8);
match twice.cmp(&div) {
Ordering::Less => q,
Ordering::Greater => q + 1,
Ordering::Equal => if (&q % BigInt::from(2u8)) == BigInt::from(0u8) { q } else { q + 1 },
}
};
u64::try_from(n).ok()
}
// --- verification ----------------------------------------------------------------
fn get<'a>(v: &'a Value, path: &[&str]) -> Result<&'a Value, String> {
@ -603,14 +659,32 @@ fn verify_receipt_inner(r: &Value, keys: &Map<String, Value>) -> Result<Vec<Stri
let breakdown = get(r, &["energy", "breakdown"])?
.as_object()
.ok_or("malformed: breakdown")?;
let mut total = Dec::int(0);
for v in breakdown.values() {
total = total.add(&Dec::from_value(v).ok_or("malformed: breakdown")?);
}
let jt = dec(r, &["energy", "joules_total"])?;
if exceeds(&total, &jt, &Dec::int(1).max(total.abs())) {
reasons.push("sum_mismatch".into());
}
let unmetered = is_unmetered(r);
let energy = get(r, &["energy"])?;
// The figure, or `None` for the unmetered marker (§3.1), which states none.
let jt = if unmetered {
// Nothing sums to the marker: every term is null, the grade is the weakest, no
// class names how a figure was read, and no bundle's stages sum to it.
let how = get(r, &["provenance", "how"])?;
if breakdown.values().any(|v| !v.is_null())
|| how.as_str() != Some("projected")
|| energy.get("class").is_some()
|| r.get("kind").and_then(Value::as_str) == Some("bundle")
{
reasons.push("unmetered_invalid".into());
}
None
} else {
let mut total = Dec::int(0);
for v in breakdown.values() {
total = total.add(&Dec::from_value(v).ok_or("malformed: breakdown")?);
}
let jt = dec(r, &["energy", "joules_total"])?;
if exceeds(&total, &jt, &Dec::int(1).max(total.abs())) {
reasons.push("sum_mismatch".into());
}
Some(jt)
};
if !get(r, &["energy"])?.get("boundary").and_then(Value::as_str).is_some_and(|b| !b.is_empty()) {
reasons.push("boundary_missing".into());
}
@ -623,9 +697,27 @@ fn verify_receipt_inner(r: &Value, keys: &Map<String, Value>) -> Result<Vec<Stri
if off.cmp(&floor.abs()) == Ordering::Greater {
reasons.push("floor_mismatch".into());
}
if !get(r, &["provenance", "how"])?.as_str().is_some_and(|g| GRADES.contains(&g)) {
let how = get(r, &["provenance", "how"])?;
if !how.as_str().is_some_and(|g| GRADES.contains(&g)) {
reasons.push("grade_unknown".into());
}
// The class, held to its own rules and to the grade it supports (§3.1, §2).
if is_minor_1(r)
&& !unmetered
&& let Some(c) = energy.get("class")
{
match class_label(c) {
None => reasons.push("class_invalid".into()),
Some(label) => {
let supported = CLASS_GRADE.iter().find(|(l, _)| *l == label).map(|(_, g)| *g);
if let Some(g) = how.as_str().filter(|g| GRADES.contains(g))
&& supported != Some(g)
{
reasons.push("class_grade_mismatch".into());
}
}
}
}
// An abstention makes no verdict claim: no answer, and no record anything could
// later be served from (§3). Read leniently, as the reference does, so a strange
// shape is simply not an abstention rather than a different reason.
@ -649,8 +741,11 @@ fn verify_receipt_inner(r: &Value, keys: &Map<String, Value>) -> Result<Vec<Stri
for s in stages {
st = st.add(&dec(s, &["joules"])?);
}
// Every tolerance scales by the figure's magnitude (§6): max(1, |st|).
if exceeds(&st, &jt, &Dec::int(1).max(st.abs())) {
// Every tolerance scales by the figure's magnitude (§6): max(1, |st|). The
// unmetered marker states no figure for the stages to sum to (§3.1).
if let Some(jt) = &jt
&& exceeds(&st, jt, &Dec::int(1).max(st.abs()))
{
reasons.push("bundle_sum_mismatch".into());
}
let completed = r.get("efficiency").and_then(|e| e.get("completed_units"));
@ -665,6 +760,54 @@ fn verify_receipt_inner(r: &Value, keys: &Map<String, Value>) -> Result<Vec<Stri
Ok(reasons)
}
/// Whether a receipt states the unmetered marker (§3.1): minor version 1 and a `null`
/// `energy.joules_total`. A verifier reports such a receipt as unmetered, never as a
/// figure, whatever else it reports of it.
pub fn is_unmetered(r: &Value) -> bool {
is_minor_1(r) && r.get("energy").and_then(|e| e.as_object()).and_then(|e| e.get("joules_total")).is_some_and(Value::is_null)
}
/// The receipts a verifier reports as unmetered, by receipt id (`?` for one with no
/// string id), in order.
pub fn unmetered_ids(receipts: &[Value]) -> Vec<String> {
receipts
.iter()
.filter(|r| is_unmetered(r))
.map(|r| r.get("receipt_id").and_then(Value::as_str).unwrap_or("?").to_string())
.collect()
}
/// `energy.class` as §3.1 states it, or `None`: an object holding a label of the four
/// classes a new profile may name (never the legacy `HwShunt`), and exactly that label's
/// evidence — for `OnChipCounter` a relative uncertainty in `1..=2^32−1` ppm, a window in
/// `1..=2^53−1` µs and a filtering state; for `CalibratedInstrument` the same two and a
/// non-empty certificate reference; for `ModelBased` and `Estimator` none.
pub fn class_label(c: &Value) -> Option<&'static str> {
let o = c.as_object()?;
let label = CLASS_GRADE.iter().map(|(l, _)| *l).find(|l| o.get("label").and_then(Value::as_str) == Some(*l))?;
let members: &[&str] = match label {
"OnChipCounter" => &["label", "relative_ppm", "window_us", "filtering"],
"CalibratedInstrument" => &["label", "relative_ppm", "window_us", "calibration_ref"],
_ => &["label"],
};
if o.len() != members.len() || !members.iter().all(|m| o.contains_key(*m)) {
return None;
}
if members.len() == 1 {
return Some(label);
}
// An integer, never a boolean: serde_json keeps the two apart.
let uint = |k: &str, top: u64| o.get(k).and_then(Value::as_u64).is_some_and(|n| (1..=top).contains(&n));
if !uint("relative_ppm", u32::MAX as u64) || !uint("window_us", MAX_SAFE_INT) {
return None;
}
let ok = match label {
"OnChipCounter" => o.get("filtering").and_then(Value::as_str).is_some_and(|f| ["Unknown", "Off", "On"].contains(&f)),
_ => o.get("calibration_ref").and_then(Value::as_str).is_some_and(|s| !s.is_empty()),
};
ok.then_some(label)
}
/// Every reason one receipt fails. Empty means valid.
///
/// A receipt missing a member the checks need reports `malformed: <member>` rather
@ -799,9 +942,19 @@ pub fn run(doc: &Value) -> Result<Vec<VectorResult>, String> {
.and_then(Value::as_array)
.map(|a| a.iter().filter_map(Value::as_str).map(str::to_string).collect())
.unwrap_or_default();
// A vector that declares its unmetered receipts (§7) is held to them too.
let unmetered_ok = match v.get("unmetered") {
None => true,
Some(u) => {
let want: Vec<String> =
u.as_array().map(|a| a.iter().filter_map(Value::as_str).map(str::to_string).collect()).unwrap_or_default();
let objects: Vec<Value> = receipts.iter().filter(|r| r.is_object()).cloned().collect();
unmetered_ids(&objects) == want
}
};
out.push(VectorResult {
name: py_str(v.get("name")),
pass: got == expected,
pass: got == expected && unmetered_ok,
expected: expected.into_iter().collect(),
got: got.into_iter().collect(),
});
@ -863,6 +1016,34 @@ pub fn check_canonical_form() -> Vec<String> {
Err(e) => reasons.push(format!("canonical_failure: {e}")),
}
// Minor version 1's members (§3.1), pinned the same way: `energy.class` and its
// members out of order, against a literal; and the same input under `2.0`, which §3
// refuses.
let class_fixture: Value = serde_json::from_str(
r#"{"sig_alg":"ed25519","hash_alg":"sha-256","oer":"2.1","capability":"oer.job.media.encode","kind":"job",
"energy":{"class":{"window_us":1000000,"filtering":"Off","relative_ppm":20000,"label":"OnChipCounter"},
"boundary":"hop","joules_total":"1.5"}}"#,
)
.expect("class fixture parses");
const CLASS_BYTES: &str = r#"{"oer":"2.1","kind":"job","capability":"oer.job.media.encode","energy":{"joules_total":"1.5","boundary":"hop","class":{"label":"OnChipCounter","relative_ppm":20000,"window_us":1000000,"filtering":"Off"}},"hash_alg":"sha-256","sig_alg":"ed25519"}"#;
const CLASS_ID: &str = "sha256:b473c94d166e9a262c4a151db7cad35f8e16b15b862d5bcfaa88750c83388b94";
match canonical_bytes(&class_fixture) {
Ok(b) => {
if b != CLASS_BYTES.as_bytes() {
reasons.push("class_form_mismatch".into());
}
if hash("sha-256", &b).ok().as_deref() != Some(CLASS_ID) {
reasons.push("class_id_mismatch".into());
}
}
Err(e) => reasons.push(format!("canonical_failure: {e}")),
}
let mut minor_0 = class_fixture.clone();
minor_0["oer"] = Value::from("2.0");
if canonical_bytes(&minor_0).is_ok() {
reasons.push("class_admitted_in_2_0".into());
}
// The number rule reads `-0` through serde_json, which gives it no `as_i64` only
// while `arbitrary_precision` is off. If feature unification ever turns it on,
// `-0` would read as the integer zero and verify: fail here, loudly, instead.
@ -1007,7 +1188,7 @@ mod tests {
#[test]
fn every_published_vector_produces_exactly_its_reasons() {
for (file, want) in [("vectors.json", 9), ("firmware_vectors.json", 1), ("edge_vectors.json", 99)] {
for (file, want) in [("vectors.json", 9), ("firmware_vectors.json", 1), ("edge_vectors.json", 99), ("job_vectors.json", 38)] {
let results = run(&corpus(file)).expect("vector file");
assert_eq!(results.len(), want, "{file}: a wrong path must not look like a pass");
for r in &results {
@ -1016,6 +1197,62 @@ mod tests {
}
}
/// Minor version 1 (§3.1): the unmetered marker is reported as unmetered and is no
/// figure; a vector is held to the receipts it declares unmetered; and the members it
/// adds stay outside minor version 0.
#[test]
fn the_unmetered_marker_is_reported_as_unmetered_never_as_a_figure() {
let doc = corpus("job_vectors.json");
let keys = doc["keys"].as_object().unwrap();
let v = doc["vectors"].as_array().unwrap().iter().find(|v| v["name"].as_str().unwrap().starts_with("j02")).unwrap();
let r = &v["receipts"][0];
assert!(is_unmetered(r) && verify_receipt(r, keys).is_empty());
assert_eq!(unmetered_ids(std::slice::from_ref(r)), vec![r["receipt_id"].as_str().unwrap().to_string()]);
// A ledger cannot sum a figure the receipt does not state.
assert!(!verify_ledger(std::slice::from_ref(r), &serde_json::json!({"meter_joules": "0", "unallocated_joules": "0"})));
// Declaring it metered fails the vector, though its reasons are right.
let mut d = doc.clone();
let i = d["vectors"].as_array().unwrap().iter().position(|x| x["name"] == v["name"]).unwrap();
d["vectors"][i]["unmetered"] = serde_json::json!([]);
assert!(!run(&d).unwrap()[i].pass);
// Under minor version 0 the marker is no quantity, and the class no member.
let mut r0 = r.clone();
r0["oer"] = Value::from("2.0");
assert!(!is_unmetered(&r0));
assert!(verify_receipt(&r0, keys).iter().any(|x| x.starts_with("malformed")));
}
#[test]
fn a_figure_reads_as_whole_microjoules_rounded_half_to_even() {
let uj = |t: &str| micro_joules(&Value::from(t));
assert_eq!(uj("31.104"), Some(31_104_000));
assert_eq!(uj("0.0000005"), Some(0), "a half rounds to the even neighbour, 0");
assert_eq!(uj("0.0000015"), Some(2), "and here to 2");
assert_eq!(uj("0.00000150000001"), Some(2));
assert_eq!(uj("0.00000149999999"), Some(1));
assert_eq!(uj("1e-6"), Some(1));
assert_eq!(uj("2.5e3"), Some(2_500_000_000));
assert_eq!(uj("-1"), None);
assert_eq!(uj("18446744073709.551615"), Some(u64::MAX));
assert_eq!(uj("18446744073709.551616"), None);
assert_eq!(micro_joules(&Value::from(7)), Some(7_000_000));
assert_eq!(micro_joules(&Value::Null), None);
}
#[test]
fn a_class_is_read_as_its_label_with_exactly_its_evidence() {
let ok = |v: Value| class_label(&v);
assert_eq!(ok(serde_json::json!({"label": "ModelBased"})), Some("ModelBased"));
assert_eq!(ok(serde_json::json!({"label": "OnChipCounter", "relative_ppm": 1, "window_us": 1, "filtering": "On"})), Some("OnChipCounter"));
assert_eq!(ok(serde_json::json!({"label": "OnChipCounter", "relative_ppm": 1, "window_us": 1})), None, "no filtering state");
assert_eq!(ok(serde_json::json!({"label": "OnChipCounter", "relative_ppm": true, "window_us": 1, "filtering": "On"})), None);
assert_eq!(ok(serde_json::json!({"label": "OnChipCounter", "relative_ppm": 4294967296u64, "window_us": 1, "filtering": "On"})), None);
assert_eq!(ok(serde_json::json!({"label": "CalibratedInstrument", "relative_ppm": 1, "window_us": 1, "calibration_ref": ""})), None);
assert_eq!(ok(serde_json::json!({"label": "HwShunt"})), None, "the legacy label");
assert_eq!(ok(serde_json::json!({"label": "Estimator", "relative_ppm": 1})), None, "evidence beside an unrefined class");
assert_eq!(ok(serde_json::json!("Estimator")), None);
}
#[test]
fn a_missing_member_is_reported_not_raised() {
let doc = corpus("vectors.json");

2
wai/wai-rs/src/opportunity_report/corpus.rs
View file

@ -531,6 +531,7 @@ pub fn world() -> Result<World, String> {
// The ledger: the room's receipts for every break, billed for the day.
let source = ads_source(&main, &opened);
let t = Tariff {
version: ledger::VERSION,
seller: pk("operator"),
buyer: pk("buyer"),
start: OCT,
@ -540,6 +541,7 @@ pub fn world() -> Result<World, String> {
delivery: [0; 4],
package: [0; 3],
ads: PRICES,
job: vec![],
scope: vec![(4, pk("decider"))],
seller_sig: [0; 64],
buyer_sig: [0; 64],

388
wai/wai-rs/src/receipt_ledger.rs
View file

@ -49,6 +49,8 @@ pub const CAPABILITY: &str = "wai.delivery.ledger";
pub const TARIFF_MAGIC: [u8; 4] = *b"WLT1";
pub const INVOICE_MAGIC: [u8; 4] = *b"WLI1";
pub const VERSION: u8 = 1;
/// The version of a tariff that prices jobs, and of its invoices (§4.5).
pub const VERSION_2: u8 = 2;
pub const TARIFF_DOMAIN: &[u8] = b"wai:ledger-tariff\x01";
pub const ACCEPT_DOMAIN: &[u8] = b"wai:ledger-accept\x01";
pub const INVOICE_DOMAIN: &[u8] = b"wai:ledger-invoice\x01";
@ -66,9 +68,11 @@ pub mod source_kind {
pub const DELIVERY: u8 = 2;
pub const PACKAGE: u8 = 3;
pub const ADS: u8 = 4;
/// Version 2 (§4.5).
pub const JOB: u8 = 5;
pub fn name(k: u8) -> &'static str {
["", "room", "delivery", "package", "ads"].get(k as usize).copied().unwrap_or("")
["", "room", "delivery", "package", "ads", "job"].get(k as usize).copied().unwrap_or("")
}
}
@ -83,6 +87,19 @@ pub mod record_kind {
pub const CLAIM: u8 = 7;
pub const POD_INPUT: u8 = 8;
pub const STITCHED_CLAIM: u8 = 9;
/// Defined by the texts that cite them, registered here (§5.2): revenue-share §7.4's run
/// and invoice, opportunity-report §7.3's cue event and room policy. No ledger source's.
pub const RUN: u8 = 10;
pub const INVOICE: u8 = 11;
pub const CUE: u8 = 12;
pub const ROOM_POLICY: u8 = 13;
/// Version 2: a job source's hop receipts, output receipts and route record (§5.2).
pub const OER_RECEIPT: u8 = 14;
pub const VIDEO_RECEIPT: u8 = 15;
pub const ROUTE_RECORD: u8 = 16;
/// The first kind reserved for private and provisional use (§5.2): 128 to 255. A WAI record
/// never states one (`private_kind`).
pub const PRIVATE: u8 = 128;
}
/// The energy scopes (§6.6).
@ -91,9 +108,11 @@ pub mod scope {
pub const DELIVERY_HOPS: u8 = 2;
pub const PACKAGING: u8 = 3;
pub const AD_DECISIONS: u8 = 4;
/// Version 2 (§6.6).
pub const JOBS: u8 = 5;
pub fn name(s: u8) -> &'static str {
["", "room_hops", "delivery_hops", "packaging", "ad_decisions"].get(s as usize).copied().unwrap_or("")
["", "room_hops", "delivery_hops", "packaging", "ad_decisions", "jobs"].get(s as usize).copied().unwrap_or("")
}
}
@ -146,11 +165,14 @@ pub enum Reason {
QuantityNotRecorded,
CitationMismatch,
InvoiceMismatch,
PrivateKind,
OutputInvalid,
JobFigureInvalid,
}
impl Reason {
/// §10's order.
pub const ALL: [Reason; 15] = [
pub const ALL: [Reason; 18] = [
Reason::TariffInvalid,
Reason::InvoiceInvalid,
Reason::TariffMismatch,
@ -166,6 +188,9 @@ impl Reason {
Reason::QuantityNotRecorded,
Reason::CitationMismatch,
Reason::InvoiceMismatch,
Reason::PrivateKind,
Reason::OutputInvalid,
Reason::JobFigureInvalid,
];
/// The codes of other extensions this one raises itself, besides those of its sources.
@ -188,6 +213,9 @@ impl Reason {
Reason::QuantityNotRecorded => "quantity_not_recorded",
Reason::CitationMismatch => "citation_mismatch",
Reason::InvoiceMismatch => "invoice_mismatch",
Reason::PrivateKind => "private_kind",
Reason::OutputInvalid => "output_invalid",
Reason::JobFigureInvalid => "job_figure_invalid",
}
}
}
@ -282,8 +310,8 @@ impl<'a> Rd<'a> {
}
}
/// The first steps of reading (§4.3, §6.8): the magic and the version.
fn read_head(b: &[u8], magic: &[u8; 4]) -> Result<(), &'static str> {
/// The first steps of reading (§4.3, §6.8): the magic and the version, 1 or 2.
fn read_head(b: &[u8], magic: &[u8; 4]) -> Result<u8, &'static str> {
if b.len() < 4 {
return Err(TRUNCATED);
}
@ -292,11 +320,27 @@ fn read_head(b: &[u8], magic: &[u8; 4]) -> Result<(), &'static str> {
}
match b.get(4) {
None => Err(TRUNCATED),
Some(&VERSION) => Ok(()),
Some(&v) if v == VERSION || v == VERSION_2 => Ok(v),
Some(_) => Err(UNKNOWN_VERSION),
}
}
/// A job price (§4.5): a capability, a unit (1 `jobs`, 2 `subpixels`), a divisor and a price.
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord)]
pub struct JobPrice {
pub capability: String,
pub unit: u8,
pub divisor: u64,
pub price: u64,
}
/// The job units, in unit order (§4.5).
pub const JOB_UNITS: [&str; 2] = ["jobs", "subpixels"];
fn job_capability(c: &str) -> bool {
(1..=128).contains(&c.len()) && c.starts_with("wai.") && c.bytes().all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'_' || b == b'.')
}
fn ascending<T: PartialOrd>(v: &[T]) -> bool {
v.windows(2).all(|w| w[0] < w[1])
}
@ -308,6 +352,8 @@ fn ascending<T: PartialOrd>(v: &[T]) -> bool {
/// A tariff (§4.1).
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Tariff {
/// 1, or 2 for a tariff that prices jobs (§4.5).
pub version: u8,
pub seller: [u8; 32],
pub buyer: [u8; 32],
pub start: u64,
@ -321,6 +367,8 @@ pub struct Tariff {
pub package: [u64; 3],
/// Per presented impression, per billable impression.
pub ads: [u64; 2],
/// Version 2: the job prices, in strictly ascending order of (capability, unit) (§4.5).
pub job: Vec<JobPrice>,
/// (kind, id), strictly ascending.
pub scope: Vec<(u8, [u8; 32])>,
pub seller_sig: [u8; 64],
@ -332,7 +380,7 @@ impl Tariff {
pub fn body(&self) -> Vec<u8> {
let mut o = Vec::with_capacity(256 + 33 * self.scope.len());
o.extend_from_slice(&TARIFF_MAGIC);
o.push(VERSION);
o.push(self.version);
o.extend_from_slice(&self.seller);
o.extend_from_slice(&self.buyer);
o.extend_from_slice(&self.start.to_le_bytes());
@ -344,6 +392,16 @@ impl Tariff {
for p in self.delivery.iter().chain(&self.package).chain(&self.ads) {
o.extend_from_slice(&p.to_le_bytes());
}
if self.version == VERSION_2 {
o.extend_from_slice(&u16::try_from(self.job.len()).expect("at most 65535 job prices").to_le_bytes());
for j in &self.job {
o.push(u8::try_from(j.capability.len()).expect("a capability of at most 255 bytes"));
o.extend_from_slice(j.capability.as_bytes());
o.push(j.unit);
o.extend_from_slice(&j.divisor.to_le_bytes());
o.extend_from_slice(&j.price.to_le_bytes());
}
}
o.extend_from_slice(&u16::try_from(self.scope.len()).expect("at most 65535 scope entries").to_le_bytes());
for (k, id) in &self.scope {
o.push(*k);
@ -376,7 +434,7 @@ impl Tariff {
/// refusal names `tariff`, or `tariff` and the key whose signature failed.
pub fn parse(b: &[u8]) -> Result<Tariff, Refusal> {
let at = || json!(["tariff"]);
read_head(b, &TARIFF_MAGIC).map_err(|r| (r, at()))?;
let version = read_head(b, &TARIFF_MAGIC).map_err(|r| (r, at()))?;
let mut r = Rd { b, p: 5 };
let read = |r: &mut Rd| -> Result<Tariff, &'static str> {
let (seller, buyer) = (r.arr()?, r.arr()?);
@ -395,6 +453,16 @@ impl Tariff {
*p = r.u64()?;
}
let ads = [r.u64()?, r.u64()?];
let mut job = Vec::new();
if version == VERSION_2 {
let n = r.u16()? as usize;
for _ in 0..n {
let len = r.u8()? as usize;
let cap = r.take(len)?;
let capability = String::from_utf8(cap.to_vec()).map_err(|_| MALFORMED)?;
job.push(JobPrice { capability, unit: r.u8()?, divisor: r.u64()?, price: r.u64()? });
}
}
let n = r.u16()? as usize;
let mut scope = Vec::with_capacity(n.min(b.len() / 33));
for _ in 0..n {
@ -404,7 +472,7 @@ impl Tariff {
if !r.done() {
return Err(MALFORMED);
}
Ok(Tariff { seller, buyer, start, end, currency, room, delivery, package, ads, scope, seller_sig, buyer_sig })
Ok(Tariff { version, seller, buyer, start, end, currency, room, delivery, package, ads, job, scope, seller_sig, buyer_sig })
};
let t = read(&mut r).map_err(|e| (e, at()))?;
let rules = t.start < t.end
@ -414,7 +482,10 @@ impl Tariff {
&& t.seller != t.buyer
&& !t.scope.is_empty()
&& ascending(&t.scope)
&& t.scope.iter().all(|(k, id)| (1..=4).contains(k) && *id != [0; 32]);
&& t.scope.iter().all(|(k, id)| (1..=if t.version == VERSION_2 { 5 } else { 4 }).contains(k) && *id != [0; 32])
&& t.job.len() <= 255
&& t.job.windows(2).all(|w| (w[0].capability.as_bytes(), w[0].unit) < (w[1].capability.as_bytes(), w[1].unit))
&& t.job.iter().all(|j| job_capability(&j.capability) && (1..=JOB_UNITS.len() as u8).contains(&j.unit) && j.divisor >= 1);
if !rules {
return Err((Reason::TariffInvalid.code(), at()));
}
@ -429,16 +500,22 @@ impl Tariff {
self.scope.binary_search(&(kind, *id)).is_ok()
}
/// The price of a unit (§6.3).
/// The price of a unit (§6.3): for a job line, its job price's (§4.5).
pub fn price(&self, kind: u8, unit: u8) -> u64 {
let i = unit as usize - 1;
match kind {
source_kind::ROOM => self.room[i] as u64,
source_kind::DELIVERY => self.delivery[i],
source_kind::PACKAGE => self.package[i],
source_kind::JOB => self.job[i].price,
_ => self.ads[i],
}
}
/// A unit's divisor (§6.3): for a job line, its job price's (§4.5).
pub fn divisor(&self, kind: u8, unit: u8) -> u128 {
if kind == source_kind::JOB { self.job[unit as usize - 1].divisor as u128 } else { divisor(kind, unit) }
}
}
// ---------------------------------------------------------------------------------------
@ -477,6 +554,8 @@ pub struct Row {
/// An invoice (§6.1).
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Invoice {
/// Its tariff's version, 1 or 2.
pub version: u8,
pub seller: [u8; 32],
pub buyer: [u8; 32],
pub tariff: [u8; 32],
@ -487,7 +566,8 @@ pub struct Invoice {
pub records: Vec<(u8, [u8; 32])>,
pub lines: Vec<Line>,
pub energy: Vec<Row>,
pub unmetered: [u32; 4],
/// Scopes 1 to 4, and in version 2 scope 5; a version-1 invoice states no fifth.
pub unmetered: [u32; 5],
pub amount: u64,
pub sig: [u8; 64],
}
@ -497,7 +577,7 @@ impl Invoice {
pub fn body(&self) -> Vec<u8> {
let mut o = Vec::with_capacity(200 + 33 * self.records.len() + LINE_LEN * self.lines.len());
o.extend_from_slice(&INVOICE_MAGIC);
o.push(VERSION);
o.push(self.version);
for k in [&self.seller, &self.buyer, &self.tariff, &self.prev] {
o.extend_from_slice(k);
}
@ -528,7 +608,8 @@ impl Invoice {
o.extend_from_slice(&e.joules.to_le_bytes());
o.extend_from_slice(&e.max_ppm.to_le_bytes());
}
for u in self.unmetered {
let scopes = if self.version == VERSION_2 { 5 } else { 4 };
for u in &self.unmetered[..scopes] {
o.extend_from_slice(&u.to_le_bytes());
}
o.extend_from_slice(&self.amount.to_le_bytes());
@ -554,7 +635,8 @@ impl Invoice {
/// Read an invoice (§6.8): its rules, then its signature.
pub fn parse(b: &[u8]) -> Result<Invoice, &'static str> {
read_head(b, &INVOICE_MAGIC)?;
let version = read_head(b, &INVOICE_MAGIC)?;
let v2 = version == VERSION_2;
let mut r = Rd { b, p: 5 };
let (seller, buyer, tariff, prev) = (r.arr()?, r.arr()?, r.arr()?, r.arr()?);
let (start, end) = (r.u64()?, r.u64()?);
@ -583,26 +665,32 @@ impl Invoice {
for _ in 0..n {
energy.push(Row { scope: r.u8()?, tag: r.u8()?, figures: r.u32()?, joules: r.u64()?, max_ppm: r.u32()? });
}
let unmetered = [r.u32()?, r.u32()?, r.u32()?, r.u32()?];
let unmetered = [r.u32()?, r.u32()?, r.u32()?, r.u32()?, if v2 { r.u32()? } else { 0 }];
let amount = r.u64()?;
let sig = r.arr()?;
if !r.done() {
return Err(MALFORMED);
}
let i = Invoice { seller, buyer, tariff, prev, start, end, currency, records, lines, energy, unmetered, amount, sig };
let i = Invoice { version, seller, buyer, tariff, prev, start, end, currency, records, lines, energy, unmetered, amount, sig };
if i.energy.iter().any(|e| e.tag == 0) {
return Err(CLASSES_SUMMED);
}
if i.records.iter().any(|(k, _)| *k >= record_kind::PRIVATE) {
return Err(Reason::PrivateKind.code());
}
let keys: Vec<_> = i.lines.iter().map(Line::key).collect();
let rows: Vec<_> = i.energy.iter().map(|e| (e.scope, e.tag)).collect();
let rules = i.start < i.end
&& capitals(&i.currency)
&& ascending(&i.records)
&& i.records.iter().all(|(k, _)| (1..=9).contains(k))
&& i.records.iter().all(|(k, _)| (1..=9).contains(k) || (v2 && (record_kind::OER_RECEIPT..=record_kind::ROUTE_RECORD).contains(k)))
&& ascending(&keys)
&& i.lines.iter().all(|l| unit_name(l.source_kind, l.unit).is_some() && l.quantity > 0 && l.cited > 0)
&& i.lines.iter().all(|l| {
let unit_ok = if l.source_kind == source_kind::JOB { v2 && l.unit >= 1 } else { unit_name(l.source_kind, l.unit).is_some() };
unit_ok && l.quantity > 0 && l.cited > 0
})
&& ascending(&rows)
&& i.energy.iter().all(|e| (1..=4).contains(&e.scope) && (2..=5).contains(&e.tag) && e.figures > 0);
&& i.energy.iter().all(|e| (1..=if v2 { 5 } else { 4 }).contains(&e.scope) && (2..=5).contains(&e.tag) && e.figures > 0);
if !rules {
return Err(Reason::InvoiceInvalid.code());
}
@ -759,6 +847,25 @@ pub struct StitchedFacts {
pub statement: Value,
}
/// A verified job source (§5.1, version 2).
#[derive(Clone, Debug)]
pub struct JobFacts {
/// The route record's digest, and its signer: the router.
pub record: [u8; 32],
pub router: [u8; 32],
/// The job's id, which the router names it by.
pub job: String,
pub accepted: u64,
/// Each attempt's OER/2 receipt, by the 32 bytes its id names, in attempt order.
pub hops: Vec<[u8; 32]>,
/// The billed hop, if any: its serving capability and its receipt's identity.
pub billed: Option<(String, [u8; 32])>,
/// The billed hop's figure (§5.1).
pub figure: Option<Figure>,
/// Each output receipt: its receipt hash and its `work_pixel_ops`.
pub outputs: Vec<([u8; 32], u64)>,
}
/// What a verified source gives (§5.2).
#[derive(Clone, Debug)]
pub enum Facts {
@ -767,6 +874,7 @@ pub enum Facts {
Package(PackageFacts),
Ads(AdsFacts),
Stitched(StitchedFacts),
Job(JobFacts),
}
impl Facts {
@ -778,6 +886,7 @@ impl Facts {
Facts::Package(p) => vec![(record_kind::PACKAGE, p.receipt_hash)],
Facts::Ads(a) => a.receipts.iter().map(|r| (record_kind::AD_RECEIPT, r.identity)).collect(),
Facts::Stitched(x) => x.records.iter().map(|r| (record_kind::POD_INPUT, r.digest)).collect(),
Facts::Job(j) => vec![(record_kind::ROUTE_RECORD, j.record)],
}
}
@ -793,6 +902,10 @@ impl Facts {
Facts::Package(_) => {}
Facts::Ads(a) => s.extend(a.claims.iter().map(|x| (record_kind::CLAIM, *x))),
Facts::Stitched(x) => s.extend(x.claims.iter().map(|c| (record_kind::STITCHED_CLAIM, *c))),
Facts::Job(j) => {
s.extend(j.hops.iter().map(|h| (record_kind::OER_RECEIPT, *h)));
s.extend(j.outputs.iter().map(|(h, _)| (record_kind::VIDEO_RECEIPT, *h)));
}
}
s
}
@ -1159,9 +1272,89 @@ pub fn signers(t: &Tariff) -> Vec<[u8; 32]> {
t.scope.iter().filter(|(k, _)| *k == source_kind::ADS).map(|(_, id)| *id).collect()
}
/// §5.1: verify one source by its extension, a stitched source with `signers` (§4.4). A
/// refusal's place is the extension's own, relative to the source.
/// The 32 bytes an OER/2 receipt id names (§5.2): `sha256:` or `blake3:` and 64 hex digits.
fn oer_identity(id: &str) -> Option<[u8; 32]> {
let h = id.strip_prefix("sha256:").or_else(|| id.strip_prefix("blake3:"))?;
unhex(h)?.try_into().ok()
}
/// A job hop's figure (§5.1): the unmetered marker, or its microjoules beside its class.
fn job_figure(r: &Value) -> Option<Figure> {
use crate::energy_class::{CounterFiltering, Uncertainty};
if crate::oer::is_unmetered(r) {
return Some(Figure { joules: 0, class: None });
}
let e = r.get("energy")?;
let c = e.get("class")?;
let label = crate::oer::class_label(c)?;
let joules = crate::oer::micro_joules(e.get("joules_total")?).filter(|j| *j >= 1)?;
let u = || Uncertainty::new(c["relative_ppm"].as_u64()? as u32, c["window_us"].as_u64()?).ok();
let class = match label {
"OnChipCounter" => EnergyProvenance::on_chip_counter(u()?, CounterFiltering::from_label(c["filtering"].as_str()?)?).ok()?,
"CalibratedInstrument" => EnergyProvenance::calibrated_instrument(u()?, c["calibration_ref"].as_str()?).ok()?,
"ModelBased" => EnergyProvenance::ModelBased,
_ => EnergyProvenance::Estimator,
};
Some(Figure { joules, class: Some(class) })
}
fn job_source(o: &Map<String, Value>) -> Result<Facts, Refusal> {
use crate::route_record as rr;
if !has_exactly(o, &["kind", "request", "pricebook", "policy", "record", "receipts", "keys", "outputs"]) {
return Err(bad_source());
}
let mut doc = o.clone();
doc.remove("kind");
let outputs = doc.remove("outputs").and_then(|v| v.as_array().cloned()).ok_or_else(bad_source)?;
let doc = Value::Object(doc);
if doc["receipts"].is_null() || doc["keys"].is_null() {
return Err(bad_source());
}
rr::verify(&doc).map_err(|f| (f.reason.code(), Value::Array(f.at)))?;
let record = crate::oer::load(doc["record"].as_str().expect("a text").as_bytes()).expect("an accepted record reads");
let texts: Vec<Value> = doc["receipts"].as_array().expect("a list").iter().map(|t| crate::oer::load(t.as_str().expect("a text").as_bytes()).expect("read")).collect();
let attempts = record["attempts"].as_array().expect("a list");
let hops: Vec<[u8; 32]> = attempts.iter().map(|a| oer_identity(a["receipt"].as_str().expect("an id")).expect("an id")).collect();
let (mut billed, mut figure) = (None, None);
if let Some(a) = attempts.iter().find(|a| a["billed"] == true) {
let endpoint = a["endpoint"].as_str().expect("a token");
let cap = record["candidates"].as_array().expect("a list").iter().find(|c| c["endpoint"] == endpoint).and_then(|c| c["capability"].as_str()).expect("a survivor");
let id = a["receipt"].as_str().expect("an id");
let j = texts.iter().position(|r| r.get("receipt_id").and_then(Value::as_str) == Some(id)).expect("a checked receipt");
figure = Some(job_figure(&texts[j]).ok_or((Reason::JobFigureInvalid.code(), json!(["receipts", j])))?);
billed = Some((cap.to_string(), oer_identity(id).expect("an id")));
}
let mut outs = Vec::with_capacity(outputs.len());
for (i, x) in outputs.iter().enumerate() {
let bad = || (Reason::OutputInvalid.code(), json!(["outputs", i]));
let x = x.as_object().filter(|x| has_exactly(x, &["receipt", "codestream"])).ok_or_else(bad_source)?;
let (Some(text), Some(code)) = (x["receipt"].as_str(), bytes_of(&x["codestream"])) else { return Err(bad_source()) };
let r = crate::video_receipt::VideoReceipt::from_json(text).map_err(|_| bad())?;
if !r.verify_reconstruction_wiv1(&code) {
return Err(bad());
}
outs.push((r.receipt_hash(), r.work_pixel_ops));
}
Ok(Facts::Job(JobFacts {
record: rr::digest(&record),
router: unhex(record["signer"].as_str().expect("a key")).expect("a key").try_into().expect("32 bytes"),
job: record["job"].as_str().expect("a token").to_string(),
accepted: record["accepted"].as_u64().expect("a time"),
hops,
billed,
figure,
outputs: outs,
}))
}
/// §5.1: verify one source by its extension, a stitched source with `signers` (§4.4), a job
/// source only under a tariff of version 2. A refusal's place is the extension's own, relative
/// to the source.
pub fn verify_source(v: &Value, signers: &[[u8; 32]]) -> Result<Facts, Refusal> {
verify_source_of(v, signers, VERSION_2)
}
fn verify_source_of(v: &Value, signers: &[[u8; 32]], version: u8) -> Result<Facts, Refusal> {
let o = v.as_object().ok_or_else(bad_source)?;
match o.get("kind").and_then(Value::as_str) {
Some("room") => room_source(o),
@ -1169,6 +1362,7 @@ pub fn verify_source(v: &Value, signers: &[[u8; 32]]) -> Result<Facts, Refusal>
Some("package") => package_source(o),
Some("ads") => ads_source(o),
Some("stitched") => stitched_source(o, signers),
Some("job") if version == VERSION_2 => job_source(o),
_ => Err(bad_source()),
}
}
@ -1215,6 +1409,14 @@ pub fn check_source(t: &Tariff, start: u64, end: u64, f: &Facts) -> Result<(), R
}
// Its records are accepted only under the tariff's signers (§4.4), and it states no time.
Facts::Stitched(_) => {}
Facts::Job(j) => {
if !t.in_scope(source_kind::JOB, &j.router) {
return Err(Reason::OutsideScope);
}
if !(start <= j.accepted && j.accepted < end) {
return Err(Reason::OutsidePeriod);
}
}
Facts::Ads(a) => {
if !a.receipts.iter().all(|r| t.in_scope(source_kind::ADS, &r.decider)) {
return Err(Reason::OutsideScope);
@ -1242,6 +1444,8 @@ fn overlap(f: &Facts, g: &Facts) -> bool {
a.statement.party == b.statement.party && overlaps((a.statement.start, a.statement.end), (b.statement.start, b.statement.end))
}
(Facts::Ads(a), Facts::Ads(b)) => a.room == b.room && a.receipts.iter().any(|x| b.receipts.iter().any(|y| x.cue == y.cue && x.session == y.session)),
// Two records of one router and one job route one job twice (§5.3).
(Facts::Job(a), Facts::Job(b)) => a.router == b.router && a.job == b.job,
_ => false,
}
}
@ -1250,11 +1454,20 @@ fn overlap(f: &Facts, g: &Facts) -> bool {
pub fn check_across(facts: &[Facts], previous: Option<&Invoice>) -> Result<(), (usize, Reason)> {
let billed: BTreeSet<(u8, [u8; 32])> = previous.map(|p| p.records.iter().copied().collect()).unwrap_or_default();
let mut heads: BTreeSet<(u8, [u8; 32])> = BTreeSet::new();
// One hop serves one job, and one output is one job's (§5.2).
let mut job_records: BTreeSet<(u8, [u8; 32])> = BTreeSet::new();
for (k, f) in facts.iter().enumerate() {
let hs = f.heads();
if hs.iter().any(|h| heads.contains(h)) || hs.iter().collect::<BTreeSet<_>>().len() != hs.len() {
return Err((k, Reason::RecordDuplicate));
}
if let Facts::Job(_) = f {
let own: BTreeSet<(u8, [u8; 32])> = f.records().into_iter().filter(|(kind, _)| *kind != record_kind::ROUTE_RECORD).collect();
if own.iter().any(|r| job_records.contains(r)) {
return Err((k, Reason::RecordDuplicate));
}
job_records.extend(own);
}
if facts[..k].iter().any(|g| overlap(f, g)) {
return Err((k, Reason::RecordOverlap));
}
@ -1277,7 +1490,7 @@ pub struct Computed {
/// Each line, and the records it cites.
pub lines: Vec<(Line, Cited)>,
pub energy: Vec<Row>,
pub unmetered: [u32; 4],
pub unmetered: [u32; 5],
pub amount: u64,
/// Per scope and tag, the sum of its figures, for the summed-classes check.
pub sums: BTreeMap<(u8, u8), u128>,
@ -1353,6 +1566,21 @@ pub fn compute(t: &Tariff, facts: &[Facts]) -> Result<Computed, Overflow> {
add(source_kind::ADS, r.decider, unit, counted.len() as u128, &cites);
}
}
// A job is billed at the prices of its serving capability, from its billed hop (§6.3).
Facts::Job(j) => {
let Some((cap, hop)) = &j.billed else { continue };
for (k, p) in t.job.iter().enumerate().filter(|(_, p)| p.capability == *cap) {
let unit = u8::try_from(k + 1).expect("at most 255 job prices");
let mut cites = vec![(record_kind::ROUTE_RECORD, j.record), (record_kind::OER_RECEIPT, *hop)];
let n = if p.unit == 1 {
1
} else {
cites.extend(j.outputs.iter().filter(|(_, w)| *w > 0).map(|(h, _)| (record_kind::VIDEO_RECEIPT, *h)));
j.outputs.iter().map(|(_, w)| *w as u128).sum()
};
add(source_kind::JOB, j.router, unit, n, &cites);
}
}
}
}
let mut lines = Vec::with_capacity(q.len());
@ -1360,7 +1588,7 @@ pub fn compute(t: &Tariff, facts: &[Facts]) -> Result<Computed, Overflow> {
for ((kind, source, unit), (n, cited)) in q {
let quantity = u64::try_from(n).map_err(|_| Overflow)?;
let price = t.price(kind, unit);
let a = n * price as u128 / divisor(kind, unit);
let a = n * price as u128 / t.divisor(kind, unit);
let line_amount = u64::try_from(a).map_err(|_| Overflow)?;
amount += a;
let line = Line { source_kind: kind, source, unit, quantity, price, amount: line_amount, cited: u32::try_from(cited.len()).map_err(|_| Overflow)?, cites: cites_digest(&cited) };
@ -1391,6 +1619,7 @@ pub fn compute(t: &Tariff, facts: &[Facts]) -> Result<Computed, Overflow> {
match f {
Facts::Package(p) => figures.push((scope::PACKAGING, p.figure.clone())),
Facts::Ads(a) => figures.extend(a.receipts.iter().map(|r| (scope::AD_DECISIONS, r.figure.clone()))),
Facts::Job(j) => figures.extend(j.figure.iter().map(|f| (scope::JOBS, f.clone()))),
_ => {}
}
}
@ -1398,11 +1627,11 @@ pub fn compute(t: &Tariff, facts: &[Facts]) -> Result<Computed, Overflow> {
Ok(Computed { records: records.into_iter().collect(), lines, energy, unmetered, amount: u64::try_from(amount).map_err(|_| Overflow)?, sums })
}
type RowsOf = (Vec<Row>, [u32; 4], BTreeMap<(u8, u8), u128>);
type RowsOf = (Vec<Row>, [u32; 5], BTreeMap<(u8, u8), u128>);
/// Rows, unmetered counts and sums per (scope, tag) of figures.
fn rows_of(figures: &[(u8, Figure)]) -> Result<RowsOf, Overflow> {
let mut unmetered = [0u32; 4];
let mut unmetered = [0u32; 5];
let mut acc: BTreeMap<(u8, u8), (u32, u128, u32)> = BTreeMap::new();
for (s, f) in figures {
match f.tag() {
@ -1440,6 +1669,7 @@ fn summed_across(row: &Row, sums: &BTreeMap<(u8, u8), u128>) -> bool {
/// The invoice of `computed`, unsigned.
pub fn invoice_of(t: &Tariff, prev: [u8; 32], start: u64, end: u64, c: &Computed) -> Invoice {
Invoice {
version: t.version,
seller: t.seller,
buyer: t.buyer,
tariff: t.digest(),
@ -1685,7 +1915,7 @@ fn place(prefix: Value, rest: Value) -> Value {
pub fn verify_sources(t: &Tariff, start: u64, end: u64, sources: &[Value], previous: Option<&Invoice>) -> Result<Vec<Facts>, Refusal> {
let mut facts = Vec::with_capacity(sources.len());
for (k, s) in sources.iter().enumerate() {
let f = verify_source(s, &signers(t)).map_err(|(r, at)| (r, place(json!(["source", k]), at)))?;
let f = verify_source_of(s, &signers(t), t.version).map_err(|(r, at)| (r, place(json!(["source", k]), at)))?;
check_source(t, start, end, &f).map_err(|r| (r.code(), json!(["source", k])))?;
facts.push(f);
}
@ -1709,7 +1939,7 @@ pub fn verify(doc: &Value) -> Result<Accepted, Refusal> {
// Step 2.
let inv = Invoice::parse(&ib).map_err(|r| (r, json!(["invoice"])))?;
// Step 3.
if inv.tariff != t.digest() || inv.seller != t.seller || inv.buyer != t.buyer || inv.currency != t.currency {
if inv.tariff != t.digest() || inv.version != t.version || inv.seller != t.seller || inv.buyer != t.buyer || inv.currency != t.currency {
return Err((Reason::TariffMismatch.code(), json!(["invoice"])));
}
if !(t.start <= inv.start && inv.end <= t.end) {
@ -1753,16 +1983,23 @@ pub fn accepted_json(a: &Accepted) -> Value {
.lines
.iter()
.map(|(l, cited)| {
json!({
let mut v = json!({
"source_kind": source_kind::name(l.source_kind), "source": hex(&l.source),
"unit": unit_name(l.source_kind, l.unit).unwrap_or(""), "quantity": l.quantity, "price": l.price, "amount": l.amount,
"cites": cited.iter().map(|(k, id)| json!([k, hex(id)])).collect::<Vec<_>>(),
})
});
// A job line names its unit and capability by its job price (§7.3).
if l.source_kind == source_kind::JOB {
let p = &a.tariff.job[l.unit as usize - 1];
v["unit"] = json!(JOB_UNITS[p.unit as usize - 1]);
v["capability"] = json!(p.capability);
}
v
})
.collect();
let energy: Vec<Value> = i.energy.iter().map(|r| json!({"scope": scope::name(r.scope), "class": class_label(r.tag), "figures": r.figures, "joules": r.joules, "max_ppm": r.max_ppm})).collect();
let (mut unmetered, mut coverage) = (Map::new(), Map::new());
for s in 1..=4u8 {
for s in 1..=if i.version == VERSION_2 { 5u8 } else { 4 } {
let n = i.unmetered[s as usize - 1];
let has_row = i.energy.iter().any(|r| r.scope == s);
unmetered.insert(scope::name(s).into(), n.into());
@ -1803,6 +2040,7 @@ mod tests {
fn tariff() -> Tariff {
Tariff {
version: VERSION,
seller: key(1).verifying_key().to_bytes(),
buyer: key(2).verifying_key().to_bytes(),
start: 10,
@ -1812,6 +2050,7 @@ mod tests {
delivery: [4; 4],
package: [5; 3],
ads: [6; 2],
job: vec![],
scope: vec![(1, [9; 32]), (2, [8; 32])],
seller_sig: [0; 64],
buyer_sig: [0; 64],
@ -1819,6 +2058,84 @@ mod tests {
.sign(&key(1), &key(2))
}
fn tariff_v2() -> Tariff {
let mut t = tariff();
t.version = VERSION_2;
t.job = vec![
JobPrice { capability: "wai.video.av1".into(), unit: 1, divisor: 1, price: 7 },
JobPrice { capability: "wai.video.int_motion".into(), unit: 2, divisor: 1_000_000, price: 9 },
];
t.scope.push((source_kind::JOB, [7; 32]));
t.scope.sort();
t.sign(&key(1), &key(2))
}
/// A tariff of version 2 reads back with its job prices, and every altered byte is refused;
/// its rules refuse prices out of order, a unit outside the two, a divisor of 0, a capability
/// outside WAI's form, and a router in a tariff of version 1 (§4.3, §4.5).
#[test]
fn a_tariff_of_version_2_reads_back_with_its_job_prices() {
let t = tariff_v2();
let b = t.to_bytes();
assert_eq!(Tariff::parse(&b).unwrap(), t);
for at in 0..b.len() {
let mut m = b.clone();
m[at] ^= 0x01;
assert!(Tariff::parse(&m).is_err(), "byte {at} altered and accepted");
}
let refused = |f: &dyn Fn(&mut Tariff)| {
let mut x = tariff_v2();
f(&mut x);
Tariff::parse(&x.sign(&key(1), &key(2)).to_bytes()).unwrap_err().0
};
assert_eq!(refused(&|x| x.job.swap(0, 1)), "tariff_invalid");
assert_eq!(refused(&|x| x.job[0].unit = 3), "tariff_invalid");
assert_eq!(refused(&|x| x.job[0].divisor = 0), "tariff_invalid");
assert_eq!(refused(&|x| x.job[0].capability = "video.av1".into()), "tariff_invalid");
assert_eq!(refused(&|x| { x.version = VERSION; x.job.clear(); }), "tariff_invalid", "kind 5 in a tariff of version 1");
// Version 1 is written as it always was: no job field.
let v1 = tariff();
assert_eq!(v1.to_bytes()[4], 1);
assert_eq!(Tariff::parse(&v1.to_bytes()).unwrap().job, vec![]);
}
/// An invoice states five unmetered counts in version 2 and four in version 1 (§6.1), and a
/// job line's unit is at least 1 (§6.8).
#[test]
fn an_invoice_of_version_2_states_the_jobs_scope() {
let t = tariff_v2();
let c = Computed { records: vec![], lines: vec![], energy: vec![], unmetered: [0, 0, 0, 0, 3], amount: 0, sums: BTreeMap::new() };
let i = invoice_of(&t, [0; 32], 10, 20, &c).sign(&key(1));
assert_eq!(Invoice::parse(&i.to_bytes()).unwrap(), i);
let mut v1 = i.clone();
v1.version = VERSION;
v1.unmetered[4] = 0;
let v1 = v1.sign(&key(1));
assert_eq!(v1.to_bytes().len() + 4, i.to_bytes().len(), "one u32 fewer");
assert_eq!(Invoice::parse(&v1.to_bytes()).unwrap(), v1);
let mut line = i.clone();
line.lines.push(Line { source_kind: source_kind::JOB, source: [7; 32], unit: 0, quantity: 1, price: 1, amount: 1, cited: 1, cites: [0; 32] });
assert_eq!(Invoice::parse(&line.sign(&key(1)).to_bytes()).unwrap_err(), "invoice_invalid");
// The kinds an invoice may list (§5.2): the ledger's, never another text's, never a private one.
let with = |k: u8, v: &Invoice| {
let mut x = v.clone();
x.records.push((k, [3; 32]));
x.records.sort();
Invoice::parse(&x.sign(&key(1)).to_bytes()).map(|_| ())
};
for k in [record_kind::OER_RECEIPT, record_kind::VIDEO_RECEIPT, record_kind::ROUTE_RECORD] {
assert_eq!(with(k, &i), Ok(()), "kind {k} in version 2");
assert_eq!(with(k, &v1), Err("invoice_invalid"), "kind {k} in version 1");
}
for k in [record_kind::RUN, record_kind::INVOICE, record_kind::CUE, record_kind::ROOM_POLICY, 17, 127] {
assert_eq!(with(k, &i), Err("invoice_invalid"), "kind {k} is no ledger source's");
}
for k in [128, 200, 255] {
assert_eq!(with(k, &i), Err("private_kind"), "kind {k} in version 2");
assert_eq!(with(k, &v1), Err("private_kind"), "kind {k} in version 1");
}
}
#[test]
fn a_tariff_reads_back_and_every_altered_byte_is_refused() {
let t = tariff();
@ -1842,6 +2159,7 @@ mod tests {
let t = tariff();
let cited: BTreeSet<(u8, [u8; 32])> = [(record_kind::SESSION, [7; 32])].into_iter().collect();
Invoice {
version: VERSION,
seller: t.seller,
buyer: t.buyer,
tariff: t.digest(),
@ -1852,7 +2170,7 @@ mod tests {
records: vec![(record_kind::SESSION, [7; 32]), (record_kind::HOP, [6; 32])],
lines: vec![Line { source_kind: 1, source: [9; 32], unit: 1, quantity: 1_500_000, price: 3, amount: 4, cited: 1, cites: cites_digest(&cited) }],
energy: vec![Row { scope: 1, tag: 4, figures: 1, joules: 5, max_ppm: 20_000 }],
unmetered: [0, 0, 1, 0],
unmetered: [0, 0, 1, 0, 0],
amount: 4,
sig: [0; 64],
}
@ -1876,8 +2194,8 @@ mod tests {
assert_eq!(Invoice::parse(&unit.sign(&key(1)).to_bytes()).unwrap_err(), "invoice_invalid");
assert_eq!(Invoice::parse(&[b.clone(), vec![0]].concat()).unwrap_err(), "malformed");
let mut v = b.clone();
v[4] = 2;
assert_eq!(Invoice::parse(&v).unwrap_err(), "unknown_version");
v[4] = 3;
assert_eq!(Invoice::parse(&v).unwrap_err(), "unknown_version", "version 3 is not defined (2 is, §6.1)");
let mut s = b;
let n = s.len();
s[n - 1] ^= 1;
@ -1944,6 +2262,6 @@ mod tests {
Row { scope: 2, tag: 2, figures: 1, joules: 7, max_ppm: 0 },
]
);
assert_eq!(unmetered, [0, 0, 1, 0]);
assert_eq!(unmetered, [0, 0, 1, 0, 0]);
}
}

94
wai/wai-rs/src/receipt_ledger/corpus.rs
View file

@ -266,6 +266,7 @@ fn scope_of(f: &Facts) -> Vec<(u8, [u8; 32])> {
Facts::Package(p) => vec![(source_kind::PACKAGE, p.signer)],
Facts::Ads(a) => a.receipts.iter().map(|r| (source_kind::ADS, r.decider)).collect(),
Facts::Stitched(x) => x.records.iter().map(|r| (source_kind::ADS, r.signer)).collect(),
Facts::Job(j) => vec![(source_kind::JOB, j.router)],
}
}
@ -274,6 +275,7 @@ fn tariff_of(buyer: &str, room: [u32; ROOM_PRICES], scope: Vec<(u8, [u8; 32])>)
scope.sort();
scope.dedup();
Tariff {
version: super::VERSION,
seller: pk("seller"),
buyer: pk(buyer),
start: MONTH.0,
@ -283,6 +285,7 @@ fn tariff_of(buyer: &str, room: [u32; ROOM_PRICES], scope: Vec<(u8, [u8; 32])>)
delivery: DELIVERY_PRICES,
package: [500_000, 1_000, 2_000_000],
ads: [1_500, 1_000],
job: vec![],
scope,
seller_sig: [0; 64],
buyer_sig: [0; 64],
@ -290,6 +293,23 @@ fn tariff_of(buyer: &str, room: [u32; ROOM_PRICES], scope: Vec<(u8, [u8; 32])>)
.sign(&key("seller"), &key(buyer))
}
/// A job source (§5.1, version 2) of a route-record vector, with its output receipts.
fn job_of(routes: &Value, name: &str, outputs: Vec<Value>) -> Result<Value, String> {
let d = &named(&routes["vectors"], name)?["document"];
Ok(json!({
"kind": "job", "request": d["request"], "pricebook": d["pricebook"], "policy": d["policy"], "record": d["record"],
"receipts": d["receipts"], "keys": d["keys"], "outputs": outputs,
}))
}
/// An output of a job: a `wai.video.int_motion` decode receipt over `payload`, sealed by the
/// sink that decoded it (unmetered) under an id that names the job, with the payload it
/// re-derives from (integer-payloads §15).
fn output_of(receipt_payload: &[u8], codestream: &[u8], job: &str) -> Value {
let r = crate::video_receipt::VideoReceipt::seal_decode_wiv1(&key("output-sink"), format!("did:example:sink/{job}"), receipt_payload, 0, None, None).expect("a corpus payload decodes");
json!({"receipt": r.to_json(), "codestream": hex(codestream)})
}
/// The delivery prices: millionths of a euro cent per 10^9 bytes, by object class.
const DELIVERY_PRICES: [u64; 4] = [20_000_000, 50_000_000, 80_000_000, 5_000_000];
@ -333,6 +353,10 @@ pub fn generate(wai: &Path) -> Result<String, String> {
let adsdoc = read_json(wai, "live-conformance/ads/vectors.json")?;
let present = read_json(wai, "live-conformance/presentation/vectors.json")?;
let stitched = read_json(wai, "live-conformance/stitched/vectors.json")?;
let routes = read_json(wai, "live-conformance/routes/vectors.json")?;
let read_bytes = |rel: &str| std::fs::read(wai.join(rel)).map_err(|e| format!("{rel}: {e}"));
let clip = read_bytes("wai-rs/tests/vectors/int_video/int_video_bitstream.wiv1")?;
let other_clip = read_bytes("wai-rs/tests/vectors/int_video/video_real_bitstream.wiv1")?;
let lesson_room = {
let r = &settle["rooms"]["lesson"];
json!({"descriptor": r["descriptor"], "grants": r["grants"]})
@ -364,6 +388,20 @@ pub fn generate(wai: &Path) -> Result<String, String> {
pool.insert("stitched_hls", stitched_of(wai, &stitched, "hls_stitched")?);
pool.insert("stitched_form1", stitched_of(wai, &stitched, "form1_prefetch")?);
pool.insert("stitched_squeezeback", stitched_of(wai, &stitched, "squeezeback")?);
// Media jobs (version 2): routed by the route corpus's router, an int_motion job's output
// receipted by the sink that decoded it.
let out = |job: &str| vec![output_of(&clip, &clip, job)];
pool.insert("job_int_motion", job_of(&routes, "served_first_try", out("job-0001"))?);
pool.insert("job_failover", job_of(&routes, "failover_after_a_server_error", vec![])?);
pool.insert("job_late", job_of(&routes, "served_after_the_deadline", out("job-0003"))?);
pool.insert("job_unmetered", job_of(&routes, "served_by_an_unmetered_hop", out("job-0005"))?);
pool.insert("job_failed", job_of(&routes, "every_allowed_survivor_failed", vec![])?);
pool.insert("job_unclassed", job_of(&routes, "served_by_a_hop_whose_figure_states_no_class", out("job-0006"))?);
pool.insert("job_billing_refused", job_of(&routes, "a_failed_hop_billed", vec![])?);
pool.insert("job_bad_output", job_of(&routes, "served_first_try", vec![output_of(&clip, &other_clip, "job-0001")])?);
// Another record of job-0001, served by another endpoint: its own hop receipt, one job.
pool.insert("job_routed_again", job_of(&routes, "an_unknown_energy_ranks_last", vec![])?);
pool.insert("job_unmetered_claims_another_output", job_of(&routes, "served_by_an_unmetered_hop", out("job-0001"))?);
// The rooms' prices are the lesson receipts'; the tariff's scope is every party of the
// accepted invoices.
@ -442,7 +480,7 @@ pub fn generate(wai: &Path) -> Result<String, String> {
let t_s_narrow = t_s_narrow.sign(&seller, &key("buyer-s"));
let inv_s_narrow = sealed(&t_s, [0; 32], a0, a1, &refs(&day_s), |i| i.tariff = t_s_narrow.digest()).to_bytes();
let vectors = vec![
let mut vectors = vec![
Vector { name: "a_day", about: "one day: a lesson room, two delivering parties and a viewer that forwarded sealed objects under no tariff of its own, a package, ads under a policy that requires claims and under one that does not", tariff: tb.clone(), invoice: ab.clone(), previous: None, sources: day_a.to_vec() },
Vector { name: "the_next_invoice", about: "the next invoice of the chain: a relay's session whose energy is unmetered, and a second package", tariff: tb.clone(), invoice: bb.clone(), previous: Some(ab.clone()), sources: day_b.to_vec() },
Vector { name: "room_hops_of_three_classes", about: "a room whose hops are a model, a counter and an estimate: one row each", tariff: t_m.to_bytes(), invoice: mb.clone(), previous: None, sources: vec!["lesson_mixed_classes"] },
@ -456,7 +494,7 @@ pub fn generate(wai: &Path) -> Result<String, String> {
Vector { name: "tariff_period_empty", about: "a tariff whose period is empty, signed by both", tariff: tariff_period.to_bytes(), invoice: ab.clone(), previous: None, sources: day_a.to_vec() },
// The invoice.
Vector { name: "invoice_trailing_byte", about: "an invoice with a byte after its signature", tariff: tb.clone(), invoice: [ab.clone(), vec![0]].concat(), previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_version", about: "an invoice of version 2", tariff: tb.clone(), invoice: { let mut b = ab.clone(); b[4] = 2; b }, previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_version", about: "an invoice of version 3, which this revision does not define", tariff: tb.clone(), invoice: { let mut b = ab.clone(); b[4] = 3; b }, previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_signature", about: "an invoice whose signature does not verify", tariff: tb.clone(), invoice: flip(ab.clone(), ab.len() - 1), previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_lines_unordered", about: "an invoice whose first two lines are swapped", tariff: tb.clone(), invoice: sealed(&t, [0; 32], a0, a1, &refs(&day_a), |i| i.lines.swap(0, 1)).to_bytes(), previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_row_of_no_class", about: "an energy row of tag 0: joules of no one class", tariff: tb.clone(), invoice: sealed(&t, [0; 32], a0, a1, &refs(&day_a), |i| i.energy.insert(0, Row { scope: 1, tag: 0, figures: 1, joules: 1, max_ppm: 0 })).to_bytes(), previous: None, sources: day_a.to_vec() },
@ -500,6 +538,56 @@ pub fn generate(wai: &Path) -> Result<String, String> {
}, previous: None, sources: vec!["lesson_mixed_classes"] },
];
// Version 2: a tariff that prices jobs (§4.5), and its invoices.
let router: [u8; 32] = routes["keys"]["router"].as_str().and_then(unhex).and_then(|b| b.try_into().ok()).ok_or("the route corpus's router")?;
let job_prices = vec![
JobPrice { capability: "wai.video.av1".into(), unit: 1, divisor: 1, price: 2_000_000 },
JobPrice { capability: "wai.video.int_motion".into(), unit: 1, divisor: 1, price: 500_000 },
JobPrice { capability: "wai.video.int_motion".into(), unit: 2, divisor: 1_000_000, price: 30_000 },
];
let v2 = |scope: Vec<(u8, [u8; 32])>, job: Vec<JobPrice>| -> Tariff {
let mut t2 = tariff_of("buyer-j", room_prices, scope);
t2.version = VERSION_2;
t2.job = job;
t2.sign(&seller, &key("buyer-j"))
};
let package_scope = first_scope("package_base");
let t_j = v2(vec![package_scope, (source_kind::JOB, router)], job_prices.clone());
let day_j = ["package_base", "job_int_motion", "job_failover", "job_late", "job_unmetered", "job_failed"];
let inv_j = make_invoice(&seller, &t_j, None, MONTH.0, MONTH.1, &src(&day_j)).map_err(|e| format!("invoice j: {e:?}"))?;
let (tjb, jb) = (t_j.to_bytes(), inv_j.to_bytes());
let t_j_narrow = v2(vec![package_scope], job_prices.clone());
let inv_j_narrow = sealed(&t_j_narrow, [0; 32], MONTH.0, MONTH.1, &refs(&day_j), |_| {});
let mut swapped = job_prices.clone();
swapped.swap(1, 2);
let t_j_unordered = v2(vec![package_scope, (source_kind::JOB, router)], swapped);
let mut v1_jobs = t.clone();
v1_jobs.scope.push((source_kind::JOB, router));
v1_jobs.scope.sort();
let v1_jobs = v1_jobs.sign(&seller, &key("buyer"));
let routed_at = named(&routes["vectors"], "served_first_try")?["document"]["record"].as_str().and_then(|r| serde_json::from_str::<Value>(r).ok()).and_then(|r| r["accepted"].as_u64()).ok_or("the record's acceptance")?;
let job_line = |i: &Invoice, unit: u8| i.lines.iter().position(|l| l.source_kind == source_kind::JOB && l.unit == unit).expect("a job line");
vectors.extend([
Vector { name: "jobs_routed_and_served", about: "version 2: a month of media jobs routed by one router beside a package: an int_motion job billed per job and per subpixel of its output's receipt, a failover billed at the fallback's price, a job served past its deadline and one every allowed hop failed, billed nothing, and a job whose serving hop was unmetered", tariff: tjb.clone(), invoice: jb.clone(), previous: None, sources: day_j.to_vec() },
Vector { name: "job_prices_unordered", about: "a tariff of version 2 whose job prices are out of order", tariff: t_j_unordered.to_bytes(), invoice: jb.clone(), previous: None, sources: day_j.to_vec() },
Vector { name: "job_scope_in_a_version_1_tariff", about: "a tariff of version 1 naming a router in its scope", tariff: v1_jobs.to_bytes(), invoice: ab.clone(), previous: None, sources: day_a.to_vec() },
Vector { name: "job_source_under_a_version_1_tariff", about: "a job source given with an invoice of version 1", tariff: tb.clone(), invoice: ab.clone(), previous: None, sources: [&day_a[..], &["job_int_motion"][..]].concat() },
Vector { name: "version_1_invoice_under_a_version_2_tariff", about: "an invoice of version 1 naming a tariff of version 2", tariff: tjb.clone(), invoice: sealed(&t_j, [0; 32], MONTH.0, MONTH.1, &refs(&day_j), |i| { i.version = VERSION; i.unmetered[4] = 0; i.records.retain(|(k, _)| *k <= 9); i.lines.retain(|l| l.source_kind <= 4); i.energy.retain(|r| r.scope <= 4); }).to_bytes(), previous: None, sources: day_j.to_vec() },
Vector { name: "job_record_refused", about: "a job whose route record bills a failed hop, refused by the route record", tariff: tjb.clone(), invoice: jb.clone(), previous: None, sources: vec!["job_billing_refused"] },
Vector { name: "job_output_not_its_codestream", about: "a job whose output receipt does not re-derive from the codestream given with it", tariff: tjb.clone(), invoice: jb.clone(), previous: None, sources: vec!["job_bad_output"] },
Vector { name: "job_figure_with_no_class", about: "a job whose serving hop's receipt states a figure with no acquisition class", tariff: tjb.clone(), invoice: jb.clone(), previous: None, sources: vec!["job_unclassed"] },
Vector { name: "router_outside_scope", about: "a tariff of version 2 whose scope does not name the router", tariff: t_j_narrow.to_bytes(), invoice: inv_j_narrow.to_bytes(), previous: None, sources: day_j.to_vec() },
Vector { name: "job_outside_period", about: "a job accepted before the invoice's period", tariff: tjb.clone(), invoice: sealed(&t_j, [0; 32], routed_at + 1, MONTH.1, &refs(&["job_int_motion"]), |_| {}).to_bytes(), previous: None, sources: vec!["job_int_motion"] },
Vector { name: "job_given_twice", about: "one job source given twice", tariff: tjb.clone(), invoice: jb.clone(), previous: None, sources: [&day_j[..], &["job_int_motion"][..]].concat() },
Vector { name: "output_claimed_by_two_jobs", about: "a job whose output receipt another job of the invoice contributes", tariff: tjb.clone(), invoice: jb.clone(), previous: None, sources: vec!["job_int_motion", "job_unmetered_claims_another_output"] },
Vector { name: "job_routed_twice", about: "a second route record of one router for one job", tariff: tjb.clone(), invoice: jb.clone(), previous: None, sources: [&day_j[..], &["job_routed_again"][..]].concat() },
Vector { name: "job_line_bills_more_than_recorded", about: "a job line one subpixel above what its output receipts state", tariff: tjb.clone(), invoice: sealed(&t_j, [0; 32], MONTH.0, MONTH.1, &refs(&day_j), |i| { let k = job_line(i, 3); i.lines[k].quantity += 1; }).to_bytes(), previous: None, sources: day_j.to_vec() },
Vector { name: "invoice_record_of_a_private_kind", about: "an invoice of version 2 listing a record of kind 200, which §5.2 reserves for private use", tariff: tjb.clone(), invoice: sealed(&t_j, [0; 32], MONTH.0, MONTH.1, &refs(&day_j), |i| { i.records.push((200, [0x5a; 32])); i.records.sort(); }).to_bytes(), previous: None, sources: day_j.to_vec() },
Vector { name: "version_1_invoice_record_of_a_private_kind", about: "an invoice of version 1 listing a record of kind 128", tariff: tb.clone(), invoice: sealed(&t, [0; 32], a0, a1, &refs(&day_a), |i| { i.records.push((record_kind::PRIVATE, [0x5a; 32])); i.records.sort(); }).to_bytes(), previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_record_of_another_texts_kind", about: "an invoice listing a record of kind 10, a revenue-share run, which no ledger source contributes", tariff: tjb.clone(), invoice: sealed(&t_j, [0; 32], MONTH.0, MONTH.1, &refs(&day_j), |i| { i.records.push((record_kind::RUN, [0x5a; 32])); i.records.sort(); }).to_bytes(), previous: None, sources: day_j.to_vec() },
Vector { name: "job_energy_row_altered", about: "the jobs' counter row one microjoule above its hop's figure", tariff: tjb.clone(), invoice: sealed(&t_j, [0; 32], MONTH.0, MONTH.1, &refs(&day_j), |i| { if let Some(r) = i.energy.iter_mut().find(|r| r.scope == scope::JOBS && r.tag == 4) { r.joules += 1; } }).to_bytes(), previous: None, sources: day_j.to_vec() },
]);
let mut sources = Map::new();
for (n, v) in &pool {
sources.insert((*n).into(), v.clone());
@ -513,7 +601,7 @@ pub fn generate(wai: &Path) -> Result<String, String> {
"sources": v.sources, "outcome": outcome,
}));
}
let keys = json!({"seller": hex(&pk("seller")), "buyer": hex(&pk("buyer")), "buyer_m": hex(&pk("buyer-m")), "buyer_s": hex(&pk("buyer-s")), "cdn_a": hex(&pk("cdn-a")), "cdn_b": hex(&pk("cdn-b"))});
let keys = json!({"seller": hex(&pk("seller")), "buyer": hex(&pk("buyer")), "buyer_m": hex(&pk("buyer-m")), "buyer_s": hex(&pk("buyer-s")), "buyer_j": hex(&pk("buyer-j")), "cdn_a": hex(&pk("cdn-a")), "cdn_b": hex(&pk("cdn-b")), "output_sink": hex(&pk("output-sink"))});
let doc = json!({
"corpus": CAPABILITY,
"extension": "extensions/receipt-ledger.md",

2
wai/wai-rs/src/revenue_share/corpus.rs
View file

@ -71,6 +71,7 @@ fn tariff(buyer: &str, currency: [u8; 3], ads: [u64; 2], scope: &[[u8; 32]]) ->
scope.sort();
scope.dedup();
Tariff {
version: ledger::VERSION,
seller: pk("operator"),
buyer: pk(buyer),
start: OCT,
@ -80,6 +81,7 @@ fn tariff(buyer: &str, currency: [u8; 3], ads: [u64; 2], scope: &[[u8; 32]]) ->
delivery: [0; 4],
package: [0; 3],
ads,
job: vec![],
scope,
seller_sig: [0; 64],
buyer_sig: [0; 64],

1223
wai/wai-rs/src/route_record.rs Normal file
View file

File diff suppressed because it is too large Load diff

601
wai/wai-rs/src/route_record/corpus.rs Normal file
View file

@ -0,0 +1,601 @@
//! The conformance corpus of the route record (`live-conformance/routes/`,
//! extensions/route-record.md §10): one file, `vectors.json`, written here from fixed seeds.
//!
//! Each vector is a verification document (§7.1) with the outcome §7.3 gives it. The
//! accepted ones are routed and sealed by [`super::seal`] from documents written here; each
//! refused one is a control of one defect, its record signed again over the defect where the
//! defect is not in the signature, and each declares the reason and the place it must be
//! refused at: [`generate`] refuses to write a corpus whose control reaches another, so a
//! control cannot be mistaken for the defect it names. The hop receipts are OER/2 receipts
//! of minor version 1 (oer-receipts §3.1), sealed here under the published hop key.
use ed25519_dalek::{Signer, SigningKey};
use serde_json::{json, Value};
use sha2::{Digest, Sha256};
use super::{canonical, hex, outcome_json, resign, seal, verify, Attempt, PriceBook, Policy, Request};
/// The name the hop key is published under (oer-receipts §5).
pub const HOP_SIGNER: &str = "ed25519:route-corpus-hops";
const ACCEPTED: u64 = 1_793_000_000_000_000;
fn key(label: &str) -> SigningKey {
let mut h = Sha256::new();
h.update(b"wai:route-record/corpus/key\x00");
h.update(label.as_bytes());
SigningKey::from_bytes(&h.finalize().into())
}
fn router() -> SigningKey {
key("router")
}
fn hops() -> SigningKey {
key("hops")
}
fn sha(s: &str) -> String {
format!("sha256:{}", hex(&Sha256::digest(s.as_bytes())))
}
// ───────────────────────────── the documents ─────────────────────────────
fn request() -> Value {
json!({
"kind": "wai-route-request",
"job": "job-0001",
"operation": "media.encode",
"capability": "wai.video.int_motion",
"fallback": "wai.video.av1",
"quantities": [{"unit": "seconds", "quantity": 4}, {"unit": "subpixels", "quantity": 34_560_000}],
"limits": {
"regions": ["eu-central", "eu-west"],
"residency": ["eu"],
"no_training": true,
"zero_retention": true,
"ga_only": true,
"consent_on_file": false,
"max_price": [{"unit": "seconds", "price": 150_000_000, "divisor": 1}],
"spend_ceiling": {"currency": "EUR", "amount": 2_000_000_000},
},
"prefer": {"sort": null, "ttfb_ms": null, "quality": null},
"deadline_us": 30_000_000,
})
}
#[allow(clippy::too_many_arguments)]
fn entry(
endpoint: &str,
provider: &str,
cap: &str,
place: (Option<&str>, Option<&str>),
rules: (Option<bool>, Option<u64>, Option<bool>, Option<bool>),
price: (&str, u64, u64),
ttfb_ms: Option<u64>,
energy: Option<(u64, &str)>,
quality: Option<u64>,
) -> Value {
json!({
"endpoint": endpoint, "provider": provider, "capability": cap,
"region": place.0, "residency": place.1,
"trains_on_inputs": rules.0, "retention_days": rules.1, "ga": rules.2, "needs_consent": rules.3,
"unit": price.0, "divisor": price.1, "price": price.2,
"ttfb_ms": ttfb_ms, "energy_uj": energy.map(|e| e.0), "energy_class": energy.map(|e| e.1), "quality": quality,
})
}
const IM: &str = "wai.video.int_motion";
const AV1: &str = "wai.video.av1";
const EU_W: (Option<&str>, Option<&str>) = (Some("eu-west"), Some("eu"));
const EU_C: (Option<&str>, Option<&str>) = (Some("eu-central"), Some("eu"));
const CLEAN: (Option<bool>, Option<u64>, Option<bool>, Option<bool>) = (Some(false), Some(0), Some(true), Some(false));
fn entries() -> Vec<Value> {
vec![
entry("av1.alpha", "prov-alpha", AV1, EU_W, CLEAN, ("seconds", 1, 120_000_000), Some(900), Some((2_500_000, "ModelBased")), Some(820)),
entry("av1.beta", "prov-beta", AV1, EU_W, CLEAN, ("seconds", 1, 200_000_000), Some(400), None, Some(900)),
entry("im.alpha", "prov-alpha", IM, EU_C, CLEAN, ("subpixels", 1_000_000, 30_000), Some(300), Some((900_000, "OnChipCounter")), Some(950)),
entry("im.delta", "prov-delta", IM, EU_W, (Some(false), None, Some(false), Some(false)), ("subpixels", 1_000_000, 20_000), Some(250), None, Some(900)),
entry("im.epsilon", "prov-epsilon", IM, EU_W, CLEAN, ("frames", 1, 5_000), Some(200), None, Some(900)),
entry("im.excluded", "prov-x", IM, EU_W, CLEAN, ("subpixels", 1_000_000, 10_000), Some(100), None, Some(990)),
entry("im.gamma", "prov-gamma", IM, (Some("us-east"), None), (None, Some(30), Some(true), Some(false)), ("subpixels", 1_000_000, 15_000), Some(150), None, None),
entry("im.zeta", "prov-zeta", IM, EU_C, (Some(false), Some(0), Some(true), None), ("subpixels", 1_000_000, 45_000), Some(350), Some((700_000, "Estimator")), Some(800)),
entry("opus.alpha", "prov-alpha", "wai.audio.opus", EU_W, CLEAN, ("seconds", 1, 1_000), Some(50), None, Some(900)),
]
}
fn pricebook_of(entries: Vec<Value>) -> Value {
json!({"kind": "wai-route-pricebook", "currency": "EUR", "issued": ACCEPTED - 3_600_000_000, "entries": entries})
}
fn pricebook() -> Value {
pricebook_of(entries())
}
fn policy(max_attempts: u64) -> Value {
json!({"kind": "wai-route-policy", "name": "lowest-cost", "version": 3, "keys": ["price", "quality", "ttfb", "energy"], "max_attempts": max_attempts, "exclude": ["prov-x"]})
}
// ───────────────────────────── the hop receipts ─────────────────────────────
/// An OER/2 `job` receipt of minor version 1 for a hop: metered with its class and grade, or
/// the unmetered marker.
fn receipt(operation: &str, endpoint: &str, status: i64, figure: Option<(&str, Value, &str)>) -> Value {
let (energy, how) = match figure {
Some((joules, class, how)) => (
json!({"joules_total": joules, "boundary": "the hop's host", "breakdown": {"compute_j": joules, "movement_j": "0", "idle_alloc_j": "0", "shared_alloc_j": "0", "upstream_j": "0"},
"method": "hardware_meter", "temperature_k": "300", "class": class}),
how,
),
None => (
json!({"joules_total": null, "boundary": "the hop's host", "breakdown": {"compute_j": null, "movement_j": null, "idle_alloc_j": null, "shared_alloc_j": null, "upstream_j": null},
"method": "unmetered", "temperature_k": "300"}),
"projected",
),
};
let output = if status == 1 { format!("blake3:{}", hex(blake3::hash(endpoint.as_bytes()).as_bytes())) } else { String::new() };
let mut r = json!({
"oer": "2.1",
"receipt_id": "",
"kind": "job",
"capability": format!("oer.job.{operation}"),
"intent": "replicate",
"subject": {"inputs": [sha(endpoint)], "output": output},
"work": {"status": status, "outcome": if status == 1 { "completed" } else { "failed" }, "latency_s": "1.0"},
"energy": energy,
"provenance": {"how": how, "device": "hardware", "weakest_input": {"how": how, "input": endpoint}},
"floor": {"bits_committed": 0, "bits_delivered": "0", "floor_j": "0", "decades_above_floor": "0"},
"chain": {"prev_hash": format!("sha256:{}", "00".repeat(32)), "receipt_hash": "", "signer": HOP_SIGNER, "signature": ""},
"hash_alg": "sha-256",
"sig_alg": "ed25519",
});
reseal_receipt(&mut r);
r
}
fn reseal_receipt(r: &mut Value) {
let body = crate::oer::canonical_bytes(r).expect("a receipt of minor version 1");
r["receipt_id"] = json!(crate::oer::hash("sha-256", &body).expect("sha-256"));
let mut chained = r["chain"]["prev_hash"].as_str().expect("a hash").as_bytes().to_vec();
chained.extend_from_slice(&body);
let rh = crate::oer::hash("sha-256", &chained).expect("sha-256");
r["chain"]["signature"] = json!(hex(&hops().sign(rh.as_bytes()).to_bytes()));
r["chain"]["receipt_hash"] = json!(rh);
}
fn counter() -> Value {
json!({"label": "OnChipCounter", "relative_ppm": 20_000, "window_us": 4_000_000, "filtering": "Unknown"})
}
fn metered(endpoint: &str) -> Value {
receipt("media.encode", endpoint, 1, Some(("31.104", counter(), "measured")))
}
fn model(endpoint: &str) -> Value {
receipt("media.encode", endpoint, 1, Some(("10.0", json!({"label": "ModelBased"}), "derived")))
}
fn failed(endpoint: &str) -> Value {
receipt("media.encode", endpoint, -1, None)
}
fn id(r: &Value) -> String {
r["receipt_id"].as_str().expect("an id").to_string()
}
fn text(r: &Value) -> String {
serde_json::to_string(r).expect("JSON")
}
// ───────────────────────────── vectors ─────────────────────────────
/// A job's documents, its attempts' receipts and its sealed record.
#[derive(Clone)]
struct Job {
request: Value,
pricebook: Value,
policy: Value,
record: Value,
receipts: Option<Vec<Value>>,
}
impl Job {
fn new(request: Value, pricebook: Value, policy: Value, attempts: &[(&str, Option<&'static str>, u64, u64, Value)], receipts: bool) -> Job {
let r = Request::read(&canonical(&request)).expect("the request reads");
let b = PriceBook::read(&canonical(&pricebook)).expect("the price book reads");
let p = Policy::read(&canonical(&policy)).expect("the policy reads");
// Each hop's receipt names the job it served, so no two jobs share a receipt.
let bound: Vec<Value> = attempts
.iter()
.map(|(e, _, _, _, rc)| {
let mut rc = rc.clone();
rc["subject"]["inputs"] = json!([sha(&format!("{}/{e}", r.job))]);
reseal_receipt(&mut rc);
rc
})
.collect();
let atts: Vec<Attempt> = attempts
.iter()
.zip(&bound)
.map(|((e, err, s, t, _), rc)| Attempt { endpoint: e.to_string(), error: *err, start: ACCEPTED + s, end: ACCEPTED + t, receipt: id(rc) })
.collect();
let record = seal(&router(), &r, &b, &p, ACCEPTED, &atts);
let rs = receipts.then_some(bound);
Job { request, pricebook, policy, record, receipts: rs }
}
fn document(&self) -> Value {
self.document_with(&canonical(&self.record))
}
fn document_with(&self, record: &str) -> Value {
json!({
"request": canonical(&self.request),
"pricebook": canonical(&self.pricebook),
"policy": canonical(&self.policy),
"record": record,
"receipts": self.receipts.as_ref().map(|rs| rs.iter().map(text).collect::<Vec<_>>()),
"keys": self.receipts.as_ref().map(|_| json!({HOP_SIGNER: hex(&hops().verifying_key().to_bytes())})),
})
}
/// The same job with its record edited and signed again.
fn edited(&self, f: impl FnOnce(&mut Value)) -> Job {
let mut j = self.clone();
f(&mut j.record);
j.record = resign(j.record, &router());
j
}
}
/// The base request under another job id.
fn job(id: &str, mut r: Value) -> Value {
r["job"] = json!(id);
r
}
fn base() -> Job {
Job::new(request(), pricebook(), policy(3), &[("im.alpha", None, 1_000, 4_000_000, metered("im.alpha"))], true)
}
fn failover() -> Job {
Job::new(
job("job-0002", request()),
pricebook(),
policy(3),
&[("im.alpha", Some("server_error"), 1_000, 2_000_000, failed("im.alpha")), ("av1.alpha", None, 2_000_100, 9_000_000, model("av1.alpha"))],
true,
)
}
/// A request with no price cap, so `av1.beta` survives too: three survivors.
fn three_survivors(prefer: Value) -> Value {
let mut r = request();
r["limits"]["max_price"] = json!([]);
r["prefer"] = prefer;
r
}
fn all_failed() -> Job {
Job::new(
job("job-0004", three_survivors(json!({"sort": null, "ttfb_ms": null, "quality": null}))),
pricebook(),
policy(2),
&[("im.alpha", Some("timeout"), 1_000, 3_000_000, failed("im.alpha")), ("av1.alpha", Some("unavailable"), 3_000_000, 3_500_000, failed("av1.alpha"))],
true,
)
}
struct Vector {
name: &'static str,
document: Value,
/// For a control: the reason and the place it must be refused at.
expect: Option<(&'static str, Value)>,
}
fn accept(name: &'static str, j: &Job) -> Vector {
Vector { name, document: j.document(), expect: None }
}
fn refuse(name: &'static str, document: Value, reason: &'static str, at: Value) -> Vector {
Vector { name, document, expect: Some((reason, at)) }
}
fn set(doc: &mut Value, member: &str, f: impl FnOnce(&mut Value)) {
let mut v: Value = serde_json::from_str(doc[member].as_str().expect("a text")).expect("JSON");
f(&mut v);
doc[member] = json!(canonical(&v));
}
fn vectors() -> Vec<Vector> {
let mut v = Vec::new();
let b = base();
let fo = failover();
let af = all_failed();
// ── accepted ──
v.push(accept("served_first_try", &b));
v.push(accept("failover_after_a_server_error", &fo));
v.push(accept(
"served_after_the_deadline",
&Job::new(job("job-0003", request()), pricebook(), policy(3), &[("im.alpha", None, 1_000, 31_000_000, metered("im.alpha"))], true),
));
v.push(accept("every_allowed_survivor_failed", &af));
let mut tight = request();
tight["limits"]["spend_ceiling"]["amount"] = json!(1_000);
v.push(accept("no_candidate_survives", &Job::new(tight, pricebook(), policy(3), &[], true)));
let mut usd = request();
usd["limits"]["spend_ceiling"]["currency"] = json!("USD");
v.push(accept("a_ceiling_in_another_currency", &Job::new(usd, pricebook(), policy(3), &[], true)));
v.push(accept(
"decided_by_the_targets",
&Job::new(three_survivors(json!({"sort": null, "ttfb_ms": 350, "quality": null})), pricebook(), policy(3), &[("im.alpha", None, 1_000, 4_000_000, metered("im.alpha"))], true),
));
v.push(accept(
"decided_by_the_sort",
&Job::new(three_survivors(json!({"sort": "ttfb", "ttfb_ms": null, "quality": null})), pricebook(), policy(3), &[("im.alpha", None, 1_000, 4_000_000, metered("im.alpha"))], true),
));
v.push(accept(
"a_quality_floor_no_survivor_meets",
&Job::new(three_survivors(json!({"sort": null, "ttfb_ms": null, "quality": 990})), pricebook(), policy(3), &[("im.alpha", None, 1_000, 4_000_000, metered("im.alpha"))], true),
));
v.push(accept(
"an_unknown_energy_ranks_last",
&Job::new(three_survivors(json!({"sort": "energy", "ttfb_ms": null, "quality": null})), pricebook(), policy(3), &[("av1.alpha", None, 1_000, 6_000_000, model("av1.alpha"))], true),
));
v.push(accept(
"decided_by_quality",
&Job::new(three_survivors(json!({"sort": "quality", "ttfb_ms": null, "quality": null})), pricebook(), policy(3), &[("im.alpha", None, 1_000, 4_000_000, metered("im.alpha"))], true),
));
// A price tie the policy's second key decides: quality, before the time to first byte that
// would decide it the other way.
let mut tie = entries();
let mut slow = tie[2].clone();
slow["endpoint"] = json!("im.alpha-q");
slow["quality"] = json!(990);
slow["ttfb_ms"] = json!(600);
tie.insert(3, slow);
v.push(accept("decided_by_the_policys_second_key", &Job::new(request(), pricebook_of(tie), policy(3), &[("im.alpha-q", None, 1_000, 4_000_000, metered("im.alpha-q"))], true)));
let mut twin = entries();
let mut copy = twin[2].clone();
copy["endpoint"] = json!("im.alpha2");
twin.insert(3, copy);
v.push(accept("decided_by_the_endpoint", &Job::new(request(), pricebook_of(twin), policy(3), &[("im.alpha", None, 1_000, 4_000_000, metered("im.alpha"))], true)));
let mut direct = request();
direct["fallback"] = Value::Null;
v.push(accept("only_survivor", &Job::new(direct, pricebook(), policy(3), &[("im.alpha", None, 1_000, 4_000_000, metered("im.alpha"))], true)));
let mut consent = request();
consent["limits"]["consent_on_file"] = json!(true);
consent["fallback"] = Value::Null;
v.push(accept("consent_on_file_admits_an_endpoint", &Job::new(consent, pricebook(), policy(3), &[("im.alpha", None, 1_000, 4_000_000, metered("im.alpha"))], true)));
// A served hop with no meter, and one whose figure states no class: both valid records,
// which the receipt ledger bills from (receipt-ledger §6.6).
v.push(accept(
"served_by_an_unmetered_hop",
&Job::new(job("job-0005", request()), pricebook(), policy(3), &[("im.alpha", None, 1_000, 4_000_000, receipt("media.encode", "im.alpha", 1, None))], true),
));
let unclassed = {
let mut r = metered("im.alpha");
r["energy"].as_object_mut().unwrap().remove("class");
reseal_receipt(&mut r);
r
};
v.push(accept("served_by_a_hop_whose_figure_states_no_class", &Job::new(job("job-0006", request()), pricebook(), policy(3), &[("im.alpha", None, 1_000, 4_000_000, unclassed)], true)));
v.push(accept(
"receipts_not_checked",
&Job::new(request(), pricebook(), policy(3), &[("im.alpha", None, 1_000, 4_000_000, metered("im.alpha"))], false),
));
// ── refused: reading ──
let d = b.document();
let rec = |p: &[Value]| {
let mut a = vec![json!("record")];
a.extend_from_slice(p);
Value::Array(a)
};
v.push(refuse("record_not_json", b.document_with("{\"kind\":"), "malformed", json!(["record"])));
let dup = canonical(&b.record).replacen("\"job\":\"job-0001\"", "\"job\":\"job-0001\",\"job\":\"job-0002\"", 1);
v.push(refuse("record_with_a_member_twice", b.document_with(&dup), "malformed", json!(["record"])));
let float = canonical(&b.record).replacen("\"version\":1", "\"version\":1.0", 1);
v.push(refuse("record_with_a_float", b.document_with(&float), "malformed", json!(["record"])));
let kind = b.edited(|r| r["kind"] = json!("wai-route-recording"));
v.push(refuse("record_of_another_kind", kind.document(), "malformed", json!(["record"])));
let mut no_keys = d.clone();
no_keys["keys"] = Value::Null;
v.push(refuse("receipts_without_their_keys", no_keys, "malformed", json!([])));
let mut req_float = d.clone();
req_float["request"] = json!(canonical(&b.request).replacen("\"deadline_us\":30000000", "\"deadline_us\":3e7", 1));
v.push(refuse("request_with_an_exponent", req_float, "malformed", json!(["request"])));
v.push(refuse("record_version_2", b.edited(|r| r["version"] = json!(2)).document(), "unknown_version", rec(&[json!("version")])));
// ── refused: the documents ──
let mut x = d.clone();
set(&mut x, "request", |r| r["fallback"] = json!(IM));
v.push(refuse("fallback_is_the_capability", x, "request_invalid", json!(["request"])));
let mut x = d.clone();
set(&mut x, "request", |r| r["quantities"] = json!([{"unit": "subpixels", "quantity": 34_560_000}, {"unit": "seconds", "quantity": 4}]));
v.push(refuse("quantities_out_of_order", x, "request_invalid", json!(["request"])));
let mut x = d.clone();
set(&mut x, "pricebook", |p| p["entries"].as_array_mut().unwrap().swap(0, 1));
v.push(refuse("price_book_out_of_order", x, "pricebook_invalid", json!(["pricebook"])));
let mut x = d.clone();
set(&mut x, "pricebook", |p| p["entries"][1]["energy_class"] = json!("Estimator"));
v.push(refuse("a_class_with_no_figure", x, "pricebook_invalid", json!(["pricebook"])));
let mut x = d.clone();
set(&mut x, "policy", |p| p["keys"] = json!(["price", "ttfb", "energy"]));
v.push(refuse("policy_without_a_key", x, "policy_invalid", json!(["policy"])));
let mut x = d.clone();
set(&mut x, "policy", |p| p["max_attempts"] = json!(0));
v.push(refuse("policy_of_no_attempt", x, "policy_invalid", json!(["policy"])));
// ── refused: the record's form and signature ──
v.push(refuse("hop_numbered_from_2", b.edited(|r| r["attempts"][0]["hop"] = json!(2)).document(), "record_invalid", json!(["record"])));
v.push(refuse("served_with_an_error", b.edited(|r| r["attempts"][0]["error"] = json!("timeout")).document(), "record_invalid", json!(["record"])));
v.push(refuse("a_candidate_member_dropped", b.edited(|r| { r["candidates"][0].as_object_mut().unwrap().remove("quality"); }).document(), "record_invalid", json!(["record"])));
let mut upper = b.record.clone();
upper["signer"] = json!(upper["signer"].as_str().unwrap().to_uppercase());
v.push(refuse("signer_not_lowercase_hex", b.document_with(&canonical(&upper)), "record_invalid", json!(["record"])));
let mut small = b.record.clone();
small["signer"] = json!(hex(&crate::sig::IDENTITY));
v.push(refuse("signer_of_small_order", b.document_with(&canonical(&small)), "bad_key", rec(&[json!("sig")])));
let mut flipped = b.record.clone();
let s = flipped["sig"].as_str().unwrap().to_string();
flipped["sig"] = json!(format!("{}{}", if s.starts_with('0') { "1" } else { "0" }, &s[1..]));
v.push(refuse("signature_flipped", b.document_with(&canonical(&flipped)), "bad_signature", rec(&[json!("sig")])));
// ── refused: the binding ──
let mut x = d.clone();
set(&mut x, "request", |r| r["job"] = json!("job-0002"));
v.push(refuse("another_jobs_request", x, "request_mismatch", rec(&[json!("job")])));
v.push(refuse("deadline_restated", b.edited(|r| r["deadline"] = json!(ACCEPTED + 60_000_000)).document(), "request_mismatch", rec(&[json!("deadline")])));
let mut x = d.clone();
set(&mut x, "request", |r| r["limits"]["ga_only"] = json!(false));
v.push(refuse("a_limit_changed_after", x, "request_mismatch", rec(&[json!("request")])));
let mut x = d.clone();
set(&mut x, "policy", |p| p["version"] = json!(4));
v.push(refuse("another_version_of_the_policy", x, "policy_mismatch", rec(&[json!("policy")])));
let mut x = d.clone();
set(&mut x, "pricebook", |p| p["entries"][2]["price"] = json!(29_000));
v.push(refuse("a_price_changed_after", x, "pricebook_mismatch", rec(&[json!("pricebook")])));
// ── refused: the candidates, the order and the decision ──
v.push(refuse("a_price_restated", b.edited(|r| r["candidates"][2]["price"] = json!(25_000)).document(), "candidate_mismatch", rec(&[json!("candidates"), json!(2)])));
v.push(refuse("a_cost_restated", b.edited(|r| r["candidates"][2]["cost"] = json!(1_000_000)).document(), "candidate_mismatch", rec(&[json!("candidates"), json!(2)])));
v.push(refuse("a_candidate_left_out", b.edited(|r| { r["candidates"].as_array_mut().unwrap().pop(); }).document(), "candidate_mismatch", rec(&[json!("candidates"), json!(7)])));
let opus = {
let pb = PriceBook::read(&canonical(&pricebook())).unwrap();
let e = pb.entries.iter().find(|e| e.endpoint == "opus.alpha").unwrap().clone();
super::Candidate { entry: e, via_fallback: false, cost: Some(4_000), energy: None, removed: vec![], rank: None }.to_value()
};
v.push(refuse("another_capability_listed", b.edited(|r| r["candidates"].as_array_mut().unwrap().push(opus)).document(), "candidate_mismatch", rec(&[json!("candidates"), json!(8)])));
v.push(refuse("a_removal_left_out", b.edited(|r| { r["candidates"][6]["removed"].as_array_mut().unwrap().remove(1); }).document(), "removal_mismatch", rec(&[json!("candidates"), json!(6)])));
v.push(refuse("a_removed_candidate_stated_a_survivor", b.edited(|r| { r["candidates"][3]["removed"] = json!([]); r["candidates"][3]["rank"] = json!(3); }).document(), "removal_mismatch", rec(&[json!("candidates"), json!(3)])));
v.push(refuse("a_limit_added", b.edited(|r| r["candidates"][2]["removed"] = json!([{"limit": "ga", "value": true}])).document(), "removal_mismatch", rec(&[json!("candidates"), json!(2)])));
v.push(refuse("ranks_swapped", b.edited(|r| { r["candidates"][0]["rank"] = json!(1); r["candidates"][2]["rank"] = json!(2); r["order"] = json!(["av1.alpha", "im.alpha"]); }).document(), "rank_mismatch", rec(&[json!("candidates"), json!(0)])));
v.push(refuse("order_reversed", b.edited(|r| r["order"] = json!(["av1.alpha", "im.alpha"])).document(), "rank_mismatch", rec(&[json!("order")])));
v.push(refuse("decided_by_restated", b.edited(|r| r["decision"]["decided_by"] = json!("ttfb")).document(), "decision_mismatch", rec(&[json!("decision")])));
v.push(refuse("runner_up_restated", b.edited(|r| r["decision"]["runner_up"] = json!("av1.beta")).document(), "decision_mismatch", rec(&[json!("decision")])));
// ── refused: the attempts ──
let third = af.edited(|r| {
let mut a = r["attempts"][1].clone();
a["hop"] = json!(3);
a["endpoint"] = json!("av1.beta");
a["start"] = json!(ACCEPTED + 4_000_000);
a["end"] = json!(ACCEPTED + 5_000_000);
r["attempts"].as_array_mut().unwrap().push(a);
});
let mut third = third;
if let Some(rs) = third.receipts.as_mut() {
rs.push(failed("av1.beta"));
}
v.push(refuse("past_the_allowed_attempts", third.document(), "attempts_exceeded", rec(&[json!("attempts"), json!(2)])));
v.push(refuse("a_removed_candidate_tried", b.edited(|r| r["attempts"][0]["endpoint"] = json!("im.gamma")).document(), "attempt_removed", rec(&[json!("attempts"), json!(0)])));
v.push(refuse("a_non_candidate_tried", b.edited(|r| r["attempts"][0]["endpoint"] = json!("opus.alpha")).document(), "attempt_removed", rec(&[json!("attempts"), json!(0)])));
v.push(refuse("the_runner_up_tried_first", b.edited(|r| r["attempts"][0]["endpoint"] = json!("av1.alpha")).document(), "attempt_out_of_order", rec(&[json!("attempts"), json!(0)])));
v.push(refuse(
"served_then_tried_again",
fo.edited(|r| {
let a = &mut r["attempts"][0];
a["outcome"] = json!("served");
a["error"] = Value::Null;
a["billed"] = json!(true);
a["not_billed"] = Value::Null;
})
.document(),
"served_not_last",
rec(&[json!("attempts"), json!(0)]),
));
v.push(refuse("ended_before_it_started", b.edited(|r| r["attempts"][0]["end"] = json!(ACCEPTED + 500)).document(), "times_invalid", rec(&[json!("attempts"), json!(0)])));
v.push(refuse("started_before_the_hop_before_ended", fo.edited(|r| r["attempts"][1]["start"] = json!(ACCEPTED + 1_500_000)).document(), "times_invalid", rec(&[json!("attempts"), json!(1)])));
v.push(refuse("started_before_acceptance", b.edited(|r| r["attempts"][0]["start"] = json!(ACCEPTED - 1)).document(), "times_invalid", rec(&[json!("attempts"), json!(0)])));
v.push(refuse("a_failed_hop_billed", fo.edited(|r| { r["attempts"][0]["billed"] = json!(true); r["attempts"][0]["not_billed"] = Value::Null; }).document(), "billing_invalid", rec(&[json!("attempts"), json!(0)])));
let late = Job::new(request(), pricebook(), policy(3), &[("im.alpha", None, 1_000, 31_000_000, metered("im.alpha"))], true);
v.push(refuse("a_late_hop_billed", late.edited(|r| { r["attempts"][0]["billed"] = json!(true); r["attempts"][0]["not_billed"] = Value::Null; }).document(), "billing_invalid", rec(&[json!("attempts"), json!(0)])));
v.push(refuse("a_served_hop_not_billed", b.edited(|r| { r["attempts"][0]["billed"] = json!(false); r["attempts"][0]["not_billed"] = json!("deadline_missed"); }).document(), "billing_invalid", rec(&[json!("attempts"), json!(0)])));
let mut gave_up = af.edited(|r| { r["attempts"].as_array_mut().unwrap().pop(); });
if let Some(rs) = gave_up.receipts.as_mut() {
rs.pop();
}
v.push(refuse("gave_up_before_the_allowed_attempts", gave_up.document(), "attempts_incomplete", rec(&[json!("attempts")])));
let mut none = b.edited(|r| r["attempts"] = json!([]));
none.receipts = Some(vec![]);
v.push(refuse("no_attempt_while_survivors_were_allowed", none.document(), "attempts_incomplete", rec(&[json!("attempts")])));
// ── refused: the hop receipts ──
let ra = |i: usize| rec(&[json!("attempts"), json!(i), json!("receipt")]);
let mut x = b.clone();
x.receipts = Some(vec![]);
v.push(refuse("a_hop_receipt_not_given", x.document(), "receipt_missing", ra(0)));
let mut x = fo.clone();
x.receipts = Some(vec![fo.receipts.as_ref().unwrap()[1].clone()]);
v.push(refuse("the_failed_hops_receipt_not_given", x.document(), "receipt_missing", ra(0)));
let mut x = b.clone();
let mut tampered = x.receipts.as_ref().unwrap()[0].clone();
tampered["energy"]["joules_total"] = json!("3.104");
x.receipts = Some(vec![tampered]);
v.push(refuse("a_hop_receipt_altered_after_its_seal", x.document(), "receipt_invalid", ra(0)));
let mut x = b.clone();
let mut graded = x.receipts.as_ref().unwrap()[0].clone();
graded["provenance"]["how"] = json!("derived");
reseal_receipt(&mut graded);
let rid = id(&graded);
x.receipts = Some(vec![graded]);
let x = x.edited(|r| r["attempts"][0]["receipt"] = json!(rid));
v.push(refuse("a_counter_reading_graded_derived", x.document(), "receipt_invalid", ra(0)));
let mut x = b.document();
x["receipts"] = json!(["{\"oer\":"]);
v.push(refuse("a_hop_receipt_not_json", x, "receipt_invalid", json!(["receipts", 0])));
let decode = receipt("media.decode", "im.alpha", 1, Some(("31.104", counter(), "measured")));
let mut x = b.edited(|r| r["attempts"][0]["receipt"] = json!(id(&decode)));
x.receipts = Some(vec![decode]);
v.push(refuse("a_receipt_of_another_operation", x.document(), "receipt_mismatch", ra(0)));
let served_status = receipt("media.encode", "im.alpha", 1, None);
let mut x = fo.edited(|r| r["attempts"][0]["receipt"] = json!(id(&served_status)));
x.receipts = Some(vec![served_status, fo.receipts.as_ref().unwrap()[1].clone()]);
v.push(refuse("a_failed_hops_receipt_states_it_completed", x.document(), "receipt_mismatch", ra(0)));
v
}
/// The corpus, as `vectors.json` holds it.
pub fn generate() -> Result<String, String> {
let mut out = Vec::new();
for vec in vectors() {
let got = outcome_json(&verify(&vec.document));
if let Some((reason, at)) = &vec.expect
&& (got["outcome"] != "refuse" || got["reason"] != *reason || got["at"] != *at)
{
return Err(format!("{}: the control must be refused {reason} at {at}, and is {got}", vec.name));
}
if vec.expect.is_none() && got["outcome"] != "accept" {
return Err(format!("{}: must be accepted, and is {got}", vec.name));
}
out.push(json!({"name": vec.name, "document": vec.document, "outcome": got}));
}
let doc = json!({
"corpus": "wai.route.record",
"extension": "extensions/route-record.md",
"keys": {"router": hex(&router().verifying_key().to_bytes()), "hops": {HOP_SIGNER: hex(&hops().verifying_key().to_bytes())}},
"vectors": out,
});
Ok(serde_json::to_string_pretty(&doc).expect("JSON") + "\n")
}
/// Every vector whose stated outcome is not the one [`verify`] gives it.
pub fn verify_corpus(text: &str) -> Vec<String> {
let doc: Value = match serde_json::from_str(text) {
Ok(d) => d,
Err(e) => return vec![format!("vectors.json: {e}")],
};
let mut fails = Vec::new();
for v in doc["vectors"].as_array().into_iter().flatten() {
let got = outcome_json(&verify(&v["document"]));
if got != v["outcome"] {
fails.push(format!("{}: stated {}, got {got}", v["name"], v["outcome"]));
}
}
if doc["vectors"].as_array().is_none_or(|a| a.is_empty()) {
fails.push("vectors.json holds no vector".into());
}
fails
}

2
wai/wai-rs/src/strict_json.rs
View file

@ -33,6 +33,7 @@ impl Json {
_ => None,
}
}
#[cfg_attr(not(any(feature = "prior_wire", feature = "scene_map", feature = "conform_result", feature = "fleet")), allow(dead_code))]
pub fn as_str(&self) -> Option<&str> {
match self {
Json::Str(s) => Some(s),
@ -46,6 +47,7 @@ impl Json {
_ => None,
}
}
#[cfg_attr(not(any(feature = "prior_wire", feature = "scene_map", feature = "conform_result", feature = "fleet")), allow(dead_code))]
pub fn as_obj(&self) -> Option<&[(String, Json)]> {
match self {
Json::Obj(m) => Some(m),

436
wai/wai-rs/src/video_receipt.rs
View file

@ -9,7 +9,10 @@
//! before the payload was pinned identifies the tool's JSON form instead; the
//! deprecated [`VideoReceipt::seal_decode`] and
//! [`VideoReceipt::verify_reconstruction`] still read it, so such a receipt keeps
//! verifying byte for byte.
//! verifying byte for byte. [`VideoReceipt::from_json`] reads the JSON
//! [`VideoReceipt::to_json`] writes, so a holder of that JSON runs
//! [`VideoReceipt::verify`] and [`VideoReceipt::verify_reconstruction_wiv1`] with
//! nothing else.
//!
//! **What makes this profile different from its siblings.** In every other
//! cargo class the reconstruction is deterministic *by construction* and taken
@ -125,6 +128,35 @@ pub fn decode_wiv1(payload: &[u8]) -> Result<DecodedClip, &'static str> {
})
}
/// The profile string the receipt's JSON names.
pub const PROFILE: &str = "wai.video.int_motion/receipt";
/// Why [`VideoReceipt::from_json`] refused a text.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum ReadError {
/// Not the JSON `to_json` writes: what was wrong, by member.
Malformed(&'static str),
/// The class or its evidence, as [`crate::energy_class::label_from_json`] refuses it.
Class(crate::energy_class::AcquisitionError),
/// `uj_per_kilopixel_op` is not the receipt's own rate.
RateMismatch,
/// `receipt_hash` is not the receipt's own.
ReceiptHashMismatch,
}
impl std::fmt::Display for ReadError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
ReadError::Malformed(m) => write!(f, "malformed: {m}"),
ReadError::Class(e) => write!(f, "class: {e}"),
ReadError::RateMismatch => f.write_str("uj_per_kilopixel_op is not the receipt's"),
ReadError::ReceiptHashMismatch => f.write_str("receipt_hash is not the receipt's"),
}
}
}
impl std::error::Error for ReadError {}
#[derive(Clone, Debug, PartialEq)]
pub struct VideoReceipt {
/// Identity: BLAKE3 over the codestream bytes exactly as they were read — the
@ -376,6 +408,98 @@ impl VideoReceipt {
}
}
/// Read a receipt from the JSON [`Self::to_json`] writes: its inverse.
///
/// The text is read strictly (RFC 8259, no member named twice), and it must hold
/// exactly the members `to_json` writes for the receipt's class, each of its form:
/// lowercase hexadecimal of the field's length, the geometry and figures as plain
/// integers, the frames in decode order. The class is read by
/// [`crate::energy_class::label_from_json`], so the JSON cannot carry a class the
/// signed encoding refuses. Two members are derived and held to the receipt they
/// describe: `uj_per_kilopixel_op` must be what `to_json` writes for it (`null` when
/// unmetered), spelled as `to_json` spells it, and `receipt_hash` the receipt's
/// own. Reading checks no signature: [`Self::verify`] does, and
/// [`Self::verify_reconstruction_wiv1`] against the codestream.
pub fn from_json(text: &str) -> Result<VideoReceipt, ReadError> {
use crate::strict_json::{self, Json};
let strict = strict_json::read(text.as_bytes()).map_err(|_| ReadError::Malformed("not JSON read strictly"))?;
let v: Value = serde_json::from_str(text).map_err(|_| ReadError::Malformed("not JSON"))?;
let o = v.as_object().ok_or(ReadError::Malformed("not an object"))?;
if o.get("profile").and_then(Value::as_str) != Some(PROFILE) {
return Err(ReadError::Malformed("profile"));
}
let energy_provenance = crate::energy_class::label_from_json(o).map_err(ReadError::Class)?;
// Exactly the members to_json writes for this class.
let mut want: Vec<&str> = vec![
"profile", "codestream_hash", "clip_hash", "h", "w", "frames", "merkle_root", "work_pixel_ops", "joules_micro",
"uj_per_kilopixel_op", "energy_provenance", "parent_receipt_hash", "signer_pubkey", "signer_id", "sig", "receipt_hash",
];
match &energy_provenance {
Some(Provenance::OnChipCounter { .. }) => want.extend(["energy_uncertainty", "energy_counter_filtering"]),
Some(Provenance::CalibratedInstrument { .. }) => want.extend(["energy_uncertainty", "energy_calibration_ref"]),
_ => {}
}
if o.len() != want.len() || !want.iter().all(|k| o.contains_key(*k)) {
return Err(ReadError::Malformed("members"));
}
fn hexn<const N: usize>(v: &Value) -> Option<[u8; N]> {
let s = v.as_str()?;
if s.len() != 2 * N || !s.bytes().all(|c| c.is_ascii_digit() || (b'a'..=b'f').contains(&c)) {
return None;
}
let mut out = [0u8; N];
for (i, b) in out.iter_mut().enumerate() {
*b = u8::from_str_radix(&s[2 * i..2 * i + 2], 16).ok()?;
}
Some(out)
}
// A plain integer, as to_json writes it: read from its spelling, so `5.0` or `5e0` is
// not 5.
let int = |k: &'static str| -> Result<u64, ReadError> {
strict.get(k).and_then(Json::as_u64).ok_or(ReadError::Malformed(k))
};
let h32 = |k: &'static str| hexn::<32>(&o[k]).ok_or(ReadError::Malformed(k));
let frames = o["frames"]
.as_array()
.ok_or(ReadError::Malformed("frames"))?
.iter()
.enumerate()
.map(|(i, f)| hexn::<32>(f).map(|hash| FrameHash { index: i as u64, hash }))
.collect::<Option<Vec<_>>>()
.ok_or(ReadError::Malformed("frames"))?;
let parent_receipt_hash = match &o["parent_receipt_hash"] {
Value::Null => None,
v => Some(hexn::<32>(v).ok_or(ReadError::Malformed("parent_receipt_hash"))?),
};
let r = VideoReceipt {
codestream_hash: h32("codestream_hash")?,
clip_hash: h32("clip_hash")?,
h: u32::try_from(int("h")?).map_err(|_| ReadError::Malformed("h"))?,
w: u32::try_from(int("w")?).map_err(|_| ReadError::Malformed("w"))?,
frames,
merkle_root: h32("merkle_root")?,
work_pixel_ops: int("work_pixel_ops")?,
joules_micro: int("joules_micro")?,
energy_provenance,
parent_receipt_hash,
signer_pubkey: h32("signer_pubkey")?,
signer_id: o["signer_id"].as_str().ok_or(ReadError::Malformed("signer_id"))?.to_string(),
sig: hexn::<64>(&o["sig"]).ok_or(ReadError::Malformed("sig"))?,
};
// The derived rate, spelled as to_json spells it.
let rate = match r.uj_per_kilopixel_op() {
Some((x, _)) => Json::Num(format!("{x:.6}")),
None => Json::Null,
};
if strict.get("uj_per_kilopixel_op") != Some(&rate) {
return Err(ReadError::RateMismatch);
}
if h32("receipt_hash")? != r.receipt_hash() {
return Err(ReadError::ReceiptHashMismatch);
}
Ok(r)
}
pub fn to_json(&self) -> String {
let hx = |b: &[u8]| b.iter().map(|x| format!("{x:02x}")).collect::<String>();
let objs: Vec<String> =
@ -405,6 +529,8 @@ impl VideoReceipt {
),
_ => String::new(),
};
// A JSON string: the id as it is when it needs no escape, escaped when it does.
let signer_id = serde_json::Value::from(self.signer_id.as_str()).to_string();
format!(
concat!(
"{{\"profile\":\"wai.video.int_motion/receipt\",",
@ -413,7 +539,7 @@ impl VideoReceipt {
"\"work_pixel_ops\":{},\"joules_micro\":{},\"uj_per_kilopixel_op\":{},",
"\"energy_provenance\":{}{},",
"\"parent_receipt_hash\":{},\"signer_pubkey\":\"{}\",",
"\"signer_id\":\"{}\",\"sig\":\"{}\",\"receipt_hash\":\"{}\"}}"
"\"signer_id\":{},\"sig\":\"{}\",\"receipt_hash\":\"{}\"}}"
),
hx(&self.codestream_hash),
hx(&self.clip_hash),
@ -428,13 +554,317 @@ impl VideoReceipt {
evidence,
parent,
hx(&self.signer_pubkey),
self.signer_id,
signer_id,
hx(&self.sig),
hx(&self.receipt_hash())
)
}
}
// ───────────────────────────── the encode receipt (integer-payloads §15.3) ─────────────────────────────
const DOMAIN_ENCODE: &[u8] = b"wai:video-encode-receipt\x01";
const DOMAIN_ENCODE_ID: &[u8] = b"wai:video-encode-receipt-id\x01";
const DOMAIN_SFRAME: &[u8] = b"wai:video-sframe\x01";
const DOMAIN_SOURCE: &[u8] = b"wai:video-source\x01";
/// The encode receipt's profile string (integer-payloads §15.3).
pub const ENCODE_PROFILE: &str = "wai.video.int_motion/encode-receipt";
/// BLAKE3 over one source frame's RGB bytes, bound to its index (§15.3).
pub fn source_frame_hash(index: u64, rgb: &[u8]) -> FrameHash {
let mut h = blake3::Hasher::new();
h.update(DOMAIN_SFRAME);
h.update(&index.to_le_bytes());
h.update(rgb);
FrameHash { index, hash: *h.finalize().as_bytes() }
}
/// The source commitment: geometry, then every source frame hash in order (§15.3).
pub fn source_hash(h: u32, w: u32, frames: &[FrameHash]) -> [u8; 32] {
let mut hs = blake3::Hasher::new();
hs.update(DOMAIN_SOURCE);
hs.update(&h.to_le_bytes());
hs.update(&w.to_le_bytes());
hs.update(&(frames.len() as u64).to_le_bytes());
for f in frames {
hs.update(&f.index.to_le_bytes());
hs.update(&f.hash);
}
*hs.finalize().as_bytes()
}
/// An encoder's statement of a `WIV1` payload it wrote (integer-payloads §15.3): the source
/// frames it encoded, its parameters, the error the payload decodes with, and the encode's
/// energy and class. [`VideoEncodeReceipt::seal_encode`] encodes and seals one;
/// [`VideoEncodeReceipt::verify_encode`] re-derives it from the payload and the source.
#[derive(Clone, Debug, PartialEq)]
pub struct VideoEncodeReceipt {
pub codestream_hash: [u8; 32],
pub source_hash: [u8; 32],
pub h: u32,
pub w: u32,
pub source_frames: Vec<FrameHash>,
pub params: crate::int_video_enc::EncParams,
/// `Σ (source − decode)²` over every sample: recomputed by [`Self::verify_encode`].
pub sse: u64,
pub work_pixel_ops: u64,
/// The encode's microjoules (`0` = unmetered).
pub joules_micro: u64,
/// How `joules_micro` was acquired; `None` iff unmetered, and never the legacy
/// `HwShunt` (energy-measurement §2.6, *New profiles*).
pub energy_provenance: Option<Provenance>,
pub parent_receipt_hash: Option<[u8; 32]>,
pub signer_pubkey: [u8; 32],
pub signer_id: String,
pub sig: [u8; 64],
}
impl VideoEncodeReceipt {
fn signing_payload(&self) -> Vec<u8> {
let mut o = DOMAIN_ENCODE.to_vec();
o.extend_from_slice(&self.codestream_hash);
o.extend_from_slice(&self.source_hash);
o.extend_from_slice(&self.h.to_be_bytes());
o.extend_from_slice(&self.w.to_be_bytes());
o.extend_from_slice(&(self.source_frames.len() as u64).to_be_bytes());
o.extend_from_slice(&self.params.block.to_be_bytes());
o.extend_from_slice(&self.params.search.to_be_bytes());
o.push(self.params.mv_precision);
o.extend_from_slice(&self.params.q.to_be_bytes());
o.extend_from_slice(&self.sse.to_be_bytes());
o.extend_from_slice(&self.work_pixel_ops.to_be_bytes());
o.extend_from_slice(&self.joules_micro.to_be_bytes());
write_optional(self.energy_provenance.as_ref(), &mut o);
match self.parent_receipt_hash {
Some(h) => {
o.push(1);
o.extend_from_slice(&h);
}
None => o.push(0),
}
for f in &self.source_frames {
o.extend_from_slice(&f.index.to_be_bytes());
o.extend_from_slice(&f.hash);
}
o.extend_from_slice(&self.signer_pubkey);
let id = self.signer_id.as_bytes();
o.extend_from_slice(&(id.len() as u32).to_be_bytes());
o.extend_from_slice(id);
o
}
/// Encode `frames` (RGB8, `h × w` each) under `params` with the reference encoder, and
/// seal the receipt of the payload it writes. Returns the receipt and the payload.
#[allow(clippy::too_many_arguments)]
pub fn seal_encode(
signer: &SigningKey,
signer_id: impl Into<String>,
frames: &[Vec<u8>],
h: u32,
w: u32,
params: crate::int_video_enc::EncParams,
joules_micro: u64,
energy_provenance: Option<Provenance>,
parent_receipt_hash: Option<[u8; 32]>,
) -> Result<(VideoEncodeReceipt, Vec<u8>), crate::int_payload::IntPayloadError> {
let enc = crate::int_video_enc::encode(frames, h, w, &params)?;
let source_frames: Vec<FrameHash> = frames.iter().enumerate().map(|(i, f)| source_frame_hash(i as u64, f)).collect();
let mut r = VideoEncodeReceipt {
codestream_hash: *blake3::hash(&enc.wiv1).as_bytes(),
source_hash: source_hash(h, w, &source_frames),
h,
w,
work_pixel_ops: work_pixel_ops(source_frames.len() as u64, h as u64, w as u64),
source_frames,
params,
sse: enc.sse,
joules_micro,
energy_provenance,
parent_receipt_hash,
signer_pubkey: signer.verifying_key().to_bytes(),
signer_id: signer_id.into(),
sig: [0u8; 64],
};
r.sig = signer.sign(&r.signing_payload()).to_bytes();
Ok((r, enc.wiv1))
}
/// Consistent (§15.3): the source commitment and the work follow from the frames, the
/// frames are numbered from 0, the figure and its class cohere under the new-profile rule,
/// and the signature verifies strictly.
pub fn verify(&self) -> bool {
if source_hash(self.h, self.w, &self.source_frames) != self.source_hash
|| work_pixel_ops(self.source_frames.len() as u64, self.h as u64, self.w as u64) != self.work_pixel_ops
|| self.source_frames.iter().enumerate().any(|(i, f)| f.index != i as u64)
{
return false;
}
let class_ok = match &self.energy_provenance {
None => self.joules_micro == 0,
Some(p) => crate::energy_class::check_new_profile(self.joules_micro, p).is_ok(),
};
class_ok && crate::sig::verify_strict(&self.signer_pubkey, &self.signing_payload(), &self.sig).is_ok()
}
/// Re-derived (§15.3) from the payload and the source frames: the payload's identity,
/// its header's geometry and parameters, every source frame's hash, and the error the
/// decode has against the source; then [`Self::verify`].
pub fn verify_encode(&self, payload: &[u8], source: &[Vec<u8>]) -> bool {
if *blake3::hash(payload).as_bytes() != self.codestream_hash || source.len() != self.source_frames.len() {
return false;
}
let Ok(v) = crate::int_payload::wire::parse_wiv1(payload, Some(0)) else { return false };
if (v.h, v.w, v.n_frames as usize) != (self.h, self.w, self.source_frames.len())
|| (v.block, v.mv_precision, v.q) != (self.params.block, self.params.mv_precision, self.params.q)
{
return false;
}
if source.iter().enumerate().any(|(i, f)| source_frame_hash(i as u64, f) != self.source_frames[i]) {
return false;
}
let Ok(clip) = decode_wiv1(payload) else { return false };
let mut sse: u128 = 0;
for (src, dec) in source.iter().zip(&clip.frames) {
if src.len() != dec.len() {
return false;
}
for (a, b) in src.iter().zip(dec) {
let d = *a as i64 - *b as i64;
sse += (d * d) as u128;
}
}
sse == self.sse as u128 && self.verify()
}
/// Sign the receipt again over whatever it states: for controls of one defect, which a
/// test or a corpus seals so that the defect is the only thing wrong with it.
#[doc(hidden)]
pub fn resign(&mut self, signer: &SigningKey) {
self.signer_pubkey = signer.verifying_key().to_bytes();
self.sig = signer.sign(&self.signing_payload()).to_bytes();
}
pub fn receipt_hash(&self) -> [u8; 32] {
let mut h = blake3::Hasher::new();
h.update(DOMAIN_ENCODE_ID);
h.update(&self.signing_payload());
h.update(&self.sig);
*h.finalize().as_bytes()
}
/// The JSON form (§15.3).
pub fn to_json(&self) -> String {
let hx = |b: &[u8]| b.iter().map(|x| format!("{x:02x}")).collect::<String>();
let mut o = serde_json::Map::new();
o.insert("profile".into(), ENCODE_PROFILE.into());
o.insert("codestream_hash".into(), hx(&self.codestream_hash).into());
o.insert("source_hash".into(), hx(&self.source_hash).into());
o.insert("h".into(), self.h.into());
o.insert("w".into(), self.w.into());
o.insert("source_frames".into(), self.source_frames.iter().map(|f| Value::from(hx(&f.hash))).collect::<Vec<_>>().into());
o.insert("block".into(), self.params.block.into());
o.insert("search".into(), self.params.search.into());
o.insert("mv_precision".into(), self.params.mv_precision.into());
o.insert("q".into(), self.params.q.into());
o.insert("sse".into(), self.sse.into());
o.insert("work_pixel_ops".into(), self.work_pixel_ops.into());
o.insert("joules_micro".into(), self.joules_micro.into());
match &self.energy_provenance {
None => {
o.insert("energy_provenance".into(), Value::Null);
}
Some(p) => {
for (k, v) in crate::energy_class::json_fields(p) {
o.insert(k.into(), v);
}
}
}
o.insert("parent_receipt_hash".into(), self.parent_receipt_hash.map_or(Value::Null, |h| hx(&h).into()));
o.insert("signer_pubkey".into(), hx(&self.signer_pubkey).into());
o.insert("signer_id".into(), self.signer_id.as_str().into());
o.insert("sig".into(), hx(&self.sig).into());
o.insert("receipt_hash".into(), hx(&self.receipt_hash()).into());
Value::Object(o).to_string()
}
/// Read the JSON form (§15.3, read as §15.2 reads a decode receipt's).
pub fn from_json(text: &str) -> Result<VideoEncodeReceipt, ReadError> {
use crate::strict_json::{self, Json};
let strict = strict_json::read(text.as_bytes()).map_err(|_| ReadError::Malformed("not JSON read strictly"))?;
let v: Value = serde_json::from_str(text).map_err(|_| ReadError::Malformed("not JSON"))?;
let o = v.as_object().ok_or(ReadError::Malformed("not an object"))?;
if o.get("profile").and_then(Value::as_str) != Some(ENCODE_PROFILE) {
return Err(ReadError::Malformed("profile"));
}
let energy_provenance = crate::energy_class::label_from_json(o).map_err(ReadError::Class)?;
let mut want: Vec<&str> = vec![
"profile", "codestream_hash", "source_hash", "h", "w", "source_frames", "block", "search", "mv_precision", "q", "sse", "work_pixel_ops",
"joules_micro", "energy_provenance", "parent_receipt_hash", "signer_pubkey", "signer_id", "sig", "receipt_hash",
];
match &energy_provenance {
Some(Provenance::OnChipCounter { .. }) => want.extend(["energy_uncertainty", "energy_counter_filtering"]),
Some(Provenance::CalibratedInstrument { .. }) => want.extend(["energy_uncertainty", "energy_calibration_ref"]),
_ => {}
}
if o.len() != want.len() || !want.iter().all(|k| o.contains_key(*k)) {
return Err(ReadError::Malformed("members"));
}
let int = |k: &'static str| strict.get(k).and_then(Json::as_u64).ok_or(ReadError::Malformed(k));
let small = |k: &'static str| int(k).and_then(|n| u32::try_from(n).map_err(|_| ReadError::Malformed(k)));
let h32 = |k: &'static str| hex32(&o[k]).ok_or(ReadError::Malformed(k));
let source_frames = o["source_frames"]
.as_array()
.ok_or(ReadError::Malformed("source_frames"))?
.iter()
.enumerate()
.map(|(i, f)| hex32(f).map(|hash| FrameHash { index: i as u64, hash }))
.collect::<Option<Vec<_>>>()
.ok_or(ReadError::Malformed("source_frames"))?;
let parent_receipt_hash = match &o["parent_receipt_hash"] {
Value::Null => None,
v => Some(hex32(v).ok_or(ReadError::Malformed("parent_receipt_hash"))?),
};
let mv_precision = u8::try_from(int("mv_precision")?).map_err(|_| ReadError::Malformed("mv_precision"))?;
let r = VideoEncodeReceipt {
codestream_hash: h32("codestream_hash")?,
source_hash: h32("source_hash")?,
h: small("h")?,
w: small("w")?,
source_frames,
params: crate::int_video_enc::EncParams { block: small("block")?, search: small("search")?, mv_precision, q: small("q")? },
sse: int("sse")?,
work_pixel_ops: int("work_pixel_ops")?,
joules_micro: int("joules_micro")?,
energy_provenance,
parent_receipt_hash,
signer_pubkey: h32("signer_pubkey")?,
signer_id: o["signer_id"].as_str().ok_or(ReadError::Malformed("signer_id"))?.to_string(),
sig: hex64(&o["sig"]).ok_or(ReadError::Malformed("sig"))?,
};
if h32("receipt_hash")? != r.receipt_hash() {
return Err(ReadError::ReceiptHashMismatch);
}
Ok(r)
}
}
fn hex_n(v: &Value, n: usize) -> Option<Vec<u8>> {
let s = v.as_str()?;
if s.len() != 2 * n || !s.bytes().all(|c| c.is_ascii_digit() || (b'a'..=b'f').contains(&c)) {
return None;
}
(0..n).map(|i| u8::from_str_radix(&s[2 * i..2 * i + 2], 16).ok()).collect()
}
fn hex32(v: &Value) -> Option<[u8; 32]> {
hex_n(v, 32)?.try_into().ok()
}
fn hex64(v: &Value) -> Option<[u8; 64]> {
hex_n(v, 64)?.try_into().ok()
}
#[cfg(test)]
mod tests {
use super::*;

4
wai/wai-rs/tests/cli_ops.rs
View file

@ -534,6 +534,10 @@ fn samples(d: &Path, origin_port: u16) -> Vec<(&'static str, Vec<String>, bool)>
("conformance.run", s(&["conform", "verifier", "--suite", "jwp", "--impl", &format!("'{}' verdicts {{corpus}}", env!("CARGO_BIN_EXE_wai_jwp_vectors")), "--root", wai_dir().to_str().unwrap(), "--out", "jwp.wai-conform.json"]), true),
("agent.setup", s(&["setup-agent", "--dir", "agentproj"]), true),
("mcp.start", s(&["mcp", "--project", "."]), false),
// The packer given is this binary, which is no packer: the dry run names the package's
// directory and runs nothing, and a real run writes nothing here. A pack through this
// door with wai-pack itself runs in wai-pack.yml.
("package.build", s(&["package", "build", "--packer", me, "--policy", "p.toml", "--out", "pkg", "--key", "k.hex"]), false),
];
v.extend(more);
v

136
wai/wai-rs/tests/route_record.rs Normal file
View file

@ -0,0 +1,136 @@
//! `wai.route.record` (extensions/route-record.md): the committed corpus
//! `live-conformance/routes/vectors.json` regenerates byte for byte from its seeds and is the
//! only file there besides its README; every vector gets the outcome it states; every reason
//! the extension adds and every code it reuses refuses a vector; the accepted vectors are the
//! cases §10 names; a record a router seals verifies, and an edit of any of its documents or of
//! the record is refused; and §8 lists exactly the reasons the reference reports.
#![cfg(feature = "route_record")]
use std::collections::BTreeSet;
use std::path::PathBuf;
use serde_json::Value;
use wai::route_record::{self, corpus, Reason};
fn wai_dir() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("..")
}
fn committed() -> String {
std::fs::read_to_string(wai_dir().join("live-conformance/routes/vectors.json")).expect("the corpus")
}
fn doc() -> Value {
serde_json::from_str(&committed()).unwrap()
}
fn vector(d: &Value, name: &str) -> Value {
d["vectors"].as_array().unwrap().iter().find(|v| v["name"] == name).unwrap_or_else(|| panic!("no vector {name}")).clone()
}
#[test]
fn the_committed_corpus_regenerates_from_its_seeds() {
assert!(committed() == corpus::generate().expect("the corpus generates"), "live-conformance/routes/vectors.json differs from `wai_route_record_vectors gen`");
let mut files: Vec<String> = std::fs::read_dir(wai_dir().join("live-conformance/routes")).unwrap().flatten().map(|e| e.file_name().to_string_lossy().into_owned()).collect();
files.sort();
assert_eq!(files, ["README.md", "vectors.json"]);
}
#[test]
fn every_vector_gets_its_stated_outcome() {
let fails = corpus::verify_corpus(&committed());
assert!(fails.is_empty(), "{}", fails.join("\n"));
}
#[test]
fn the_corpus_refuses_every_reason() {
let d = doc();
let seen: BTreeSet<&str> = d["vectors"].as_array().unwrap().iter().filter_map(|v| v["outcome"]["reason"].as_str()).collect();
for r in Reason::ALL.iter().chain(Reason::REUSED.iter()) {
assert!(seen.contains(r.code()), "nothing in the corpus is refused {r}");
}
}
/// The accepted vectors are §10's: a billed first try, a failover whose failed hop is unmetered
/// and not billed, a hop past the deadline not billed, every allowed survivor failed, no
/// survivor at all, each way a decision is decided, and every limit removing a candidate.
#[test]
fn the_accepted_vectors_are_the_cases_section_10_names() {
let d = doc();
let o = |n: &str| vector(&d, n)["outcome"].clone();
let first = o("served_first_try");
assert_eq!((first["served"]["endpoint"].as_str(), first["billed"].as_u64(), first["deadline"].as_str()), (Some("im.alpha"), Some(1), Some("met")));
assert_eq!(first["unmetered"], serde_json::json!([]));
let fo = o("failover_after_a_server_error");
assert_eq!((fo["served"]["hop"].as_u64(), fo["billed"].as_u64()), (Some(2), Some(2)));
assert_eq!(fo["unmetered"], serde_json::json!([1]), "the failed hop's receipt states the unmetered marker");
let late = o("served_after_the_deadline");
assert_eq!((late["billed"].clone(), late["deadline"].as_str()), (Value::Null, Some("missed")));
let failed = o("every_allowed_survivor_failed");
assert_eq!((failed["served"].clone(), failed["billed"].clone()), (Value::Null, Value::Null));
assert_eq!(failed["order"].as_array().unwrap().len(), 3, "three survivors, two allowed");
assert_eq!(o("no_candidate_survives")["decision"]["decided_by"], "none");
for (n, by) in [
("served_first_try", "price"),
("decided_by_the_targets", "targets"),
("decided_by_the_sort", "ttfb"),
("an_unknown_energy_ranks_last", "energy"),
("decided_by_the_endpoint", "endpoint"),
("decided_by_quality", "quality"),
("decided_by_the_policys_second_key", "quality"),
("only_survivor", "only_survivor"),
] {
assert_eq!(o(n)["decision"]["decided_by"], by, "{n}");
}
assert_eq!(o("an_unknown_energy_ranks_last")["order"].as_array().unwrap().last().unwrap(), "av1.beta", "the survivor with no energy figure ranks last");
assert_eq!(o("receipts_not_checked")["receipts"], "not_checked");
let mut limits = BTreeSet::new();
for v in d["vectors"].as_array().unwrap().iter().filter(|v| v["outcome"]["outcome"] == "accept") {
for r in v["outcome"]["removed"].as_array().unwrap() {
for l in r["limits"].as_array().unwrap() {
limits.insert(l.as_str().unwrap().to_string());
}
}
}
assert_eq!(limits, route_record::LIMITS.iter().map(|l| l.to_string()).collect::<BTreeSet<_>>(), "every limit removes a candidate");
}
/// A record verifies only against the documents it was routed from: each of them, and the
/// record, changed by one byte is refused.
#[test]
fn a_record_is_bound_to_its_documents() {
let d = doc();
let good = vector(&d, "served_first_try")["document"].clone();
assert!(route_record::verify(&good).is_ok());
for member in ["request", "pricebook", "policy", "record"] {
let t = good[member].as_str().unwrap();
// A digit of a figure, changed: still a document, no longer the one routed.
let at = t.find(|c: char| c.is_ascii_digit() && c != '0').unwrap();
let mut b = t.as_bytes().to_vec();
b[at] = if b[at] == b'9' { b'8' } else { b[at] + 1 };
let mut x = good.clone();
x[member] = Value::from(String::from_utf8(b).unwrap());
assert!(route_record::verify(&x).is_err(), "{member} changed");
}
}
/// `wai route verify` (the catalogue's `route.verify`) prints a document's outcome, the
/// corpus's, and exits 0 when the record is accepted and 1 when it is refused.
#[test]
fn route_verify_prints_the_outcome_and_exits_by_it() {
let d = doc();
let dir = std::env::temp_dir().join(format!("wai-route-verify-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
for (name, code) in [("failover_after_a_server_error", 0), ("a_failed_hop_billed", 1), ("a_hop_receipt_not_given", 1)] {
let v = vector(&d, name);
let p = dir.join(format!("{name}.json"));
std::fs::write(&p, serde_json::to_vec(&v["document"]).unwrap()).unwrap();
let o = std::process::Command::new(env!("CARGO_BIN_EXE_wai")).args(["route", "verify"]).arg(&p).output().unwrap();
assert_eq!(o.status.code(), Some(code), "{name}: {}", String::from_utf8_lossy(&o.stderr));
let got: Value = serde_json::from_slice(&o.stdout).unwrap();
assert_eq!(got, v["outcome"], "{name}");
}
let o = std::process::Command::new(env!("CARGO_BIN_EXE_wai")).args(["route", "verify"]).arg(dir.join("none.json")).output().unwrap();
assert_ne!(o.status.code(), Some(0), "a document that is not there is no accepted record");
std::fs::remove_dir_all(&dir).ok();
}

123
wai/wai-rs/tests/video_receipt.rs
View file

@ -348,3 +348,126 @@ fn decode_wiv1_reports_refusal_codes() {
assert_eq!(code(&fixture("wiv1_unread_word.bin")), "stream_noncanonical");
assert_eq!(code(&raw("int_video_bitstream.json")), "bad_magic");
}
/// `VideoReceipt::from_json` is the inverse of `to_json`: the JSON of a receipt of every
/// class (unmetered, the on-chip counter, a calibrated instrument, a model, the legacy
/// label), chained or not, reads back as the same receipt; and a holder of that JSON and
/// the payload, and nothing else, verifies it and re-derives it.
#[test]
#[allow(deprecated)] // the legacy HwShunt label is read back too
fn a_receipt_reads_back_from_its_json() {
use wai::video_receipt::ReadError;
let payload = raw("int_video_bitstream.wiv1");
let cal = Provenance::calibrated_instrument(Uncertainty::new(5_000, 2_800_000).unwrap(), "urn:example:cert \"7\"").unwrap();
let seal = |uj, p, parent| VideoReceipt::seal_decode_wiv1(&key(7), "did:example:meter", &payload, uj, p, parent).unwrap();
let first = seal(4_200, Some(chip()), None);
for r in [
first.clone(),
seal(0, None, None),
seal(4_200, Some(cal), None),
seal(4_200, Some(Provenance::ModelBased), Some(first.receipt_hash())),
seal(4_200, Some(Provenance::HwShunt), None),
VideoReceipt::seal_decode_wiv1(&key(9), "did:example:a \"quoted\" id\\", &payload, 0, None, None).unwrap(),
] {
let j = r.to_json();
let back = VideoReceipt::from_json(&j).unwrap_or_else(|e| panic!("{e}: {j}"));
assert_eq!(back, r);
assert!(back.verify() && back.verify_reconstruction_wiv1(&payload));
assert_eq!(back.to_json(), j, "to_json(from_json(j)) is j");
// Whitespace between the members is the reader's business, not the receipt's.
let spaced = serde_json::to_string_pretty(&serde_json::from_str::<serde_json::Value>(&j).unwrap()).unwrap();
assert!(spaced.contains("\"uj_per_kilopixel_op\": null") || r.joules_micro > 0);
if r.joules_micro == 0 {
assert_eq!(VideoReceipt::from_json(&spaced), Ok(r.clone()), "a re-indented unmetered receipt");
}
}
let j = first.to_json();
let refused = |t: String| VideoReceipt::from_json(&t).unwrap_err();
// Not what to_json writes.
assert_eq!(refused(j.replacen("\"h\":48", "\"h\":48,\"h\":49", 1)), ReadError::Malformed("not JSON read strictly"), "a member named twice");
assert_eq!(refused(j.replacen("{\"profile\"", "{\"extra\":1,\"profile\"", 1)), ReadError::Malformed("members"));
assert_eq!(refused(j.replacen(",\"energy_counter_filtering\":\"Unknown\"", "", 1)), ReadError::Malformed("members"), "the class's evidence dropped");
assert_eq!(refused(j.replacen("wai.video.int_motion/receipt", "wai.video.int_motion/other", 1)), ReadError::Malformed("profile"));
assert_eq!(refused(j.replacen("\"h\":48", "\"h\":48.0", 1)), ReadError::Malformed("h"), "a plain integer is read from its spelling");
assert_eq!(refused(j.replacen("\"w\":48", "\"w\":4294967296", 1)), ReadError::Malformed("w"));
let upper = format!("\"clip_hash\":\"{}\"", hex(&first.clip_hash).to_uppercase());
assert_eq!(refused(j.replacen(&format!("\"clip_hash\":\"{}\"", hex(&first.clip_hash)), &upper, 1)), ReadError::Malformed("clip_hash"));
assert!(matches!(refused(j.replacen("\"OnChipCounter\"", "\"OnChipCountr\"", 1)), ReadError::Class(_)));
// The derived members are held to the receipt.
let rate = format!("{:.6}", first.uj_per_kilopixel_op().unwrap().0);
assert_eq!(refused(j.replacen(&format!("\"uj_per_kilopixel_op\":{rate}"), "\"uj_per_kilopixel_op\":1.000000", 1)), ReadError::RateMismatch);
assert_eq!(refused(j.replacen(&format!("\"uj_per_kilopixel_op\":{rate}"), &format!("\"uj_per_kilopixel_op\":{rate}0"), 1)), ReadError::RateMismatch, "spelled as to_json spells it");
let rh = hex(&first.receipt_hash());
let mut other = rh.clone().into_bytes();
other[0] = if other[0] == b'0' { b'1' } else { b'0' };
assert_eq!(refused(j.replacen(&rh, std::str::from_utf8(&other).unwrap(), 1)), ReadError::ReceiptHashMismatch);
// A field restated reads, and then fails the receipt's own checks.
let restated = VideoReceipt::from_json(&j.replacen("\"joules_micro\":4200", "\"joules_micro\":4201", 1));
assert!(matches!(restated, Err(ReadError::RateMismatch) | Err(ReadError::ReceiptHashMismatch)));
}
/// Three frames of a gradient that moves a pixel a frame, 16 × 16.
fn moving_source() -> Vec<Vec<u8>> {
(0..3u32)
.map(|f| (0..16u32 * 16 * 3).map(|i| { let (p, c) = (i / 3, i % 3); ((p % 16 + f) * 9 + (p / 16) * 5 + c * 40) as u8 }).collect())
.collect()
}
fn enc_params() -> wai::int_video_enc::EncParams {
wai::int_video_enc::EncParams { block: 8, search: 2, mv_precision: 1, q: 4 }
}
/// The encode receipt (integer-payloads §15.3): sealed over the payload the reference
/// encoder writes, re-derived from that payload and the source alone (the source's hashes,
/// the header's parameters, the error recomputed), and refused when any of them is not the
/// receipt's; its JSON form reads back; its class is never the legacy label.
#[test]
#[allow(deprecated)] // the legacy label is refused here
fn an_encode_receipt_is_re_derived_from_its_payload_and_source() {
use wai::video_receipt::{ReadError, VideoEncodeReceipt};
let src = moving_source();
let model = Some(Provenance::ModelBased);
let (r, payload) = VideoEncodeReceipt::seal_encode(&key(7), "did:example:encoder", &src, 16, 16, enc_params(), 1_200, model.clone(), None).unwrap();
assert!(r.verify() && r.verify_encode(&payload, &src));
assert_eq!(r.work_pixel_ops, 3 * 16 * 16 * 3);
let enc = wai::int_video_enc::encode(&src, 16, 16, &enc_params()).unwrap();
assert_eq!((r.sse, payload.clone()), (enc.sse, enc.wiv1), "the error and the payload are the encoder's");
// The payload decodes to frames whose decode receipt is the payload's.
let dec = VideoReceipt::seal_decode_wiv1(&key(7), "did:example:meter", &payload, 0, None, None).unwrap();
assert_eq!(dec.codestream_hash, r.codestream_hash);
// Each part it rests on, changed, is refused.
let mut other = src.clone();
other[1][7] ^= 1;
assert!(!r.verify_encode(&payload, &other), "another source");
let mut p2 = payload.clone();
let at = p2.len() - 3;
p2[at] ^= 1;
assert!(!r.verify_encode(&p2, &src), "another payload");
let restated = |f: &dyn Fn(&mut VideoEncodeReceipt)| { let mut t = r.clone(); f(&mut t); t };
// Each control is signed again over its defect, so the defect alone is what is refused.
let signed = |f: &dyn Fn(&mut VideoEncodeReceipt)| { let mut t = restated(f); t.resign(&key(7)); assert!(t.verify(), "a consistent control"); t };
assert!(!signed(&|t| t.sse += 1).verify_encode(&payload, &src), "an error the decode does not have");
assert!(!signed(&|t| t.params.q = 5).verify_encode(&payload, &src), "a parameter the payload does not have");
let others: Vec<wai::video_receipt::FrameHash> = other.iter().enumerate().map(|(i, f)| wai::video_receipt::source_frame_hash(i as u64, f)).collect();
let claims_other = signed(&|t| { t.source_frames = others.clone(); t.source_hash = wai::video_receipt::source_hash(16, 16, &others); });
assert!(!claims_other.verify_encode(&payload, &src), "a source the receipt does not state, though the decode's error against it is the stated one");
assert!(!restated(&|t| t.params.search = 3).verify(), "search is attested: inside the signature");
assert!(!restated(&|t| t.joules_micro = 1_201).verify(), "the figure is signed");
assert!(!restated(&|t| t.source_frames.swap(0, 2)).verify(), "the frames in order");
// The class: the unmetered marker, a refined class, never the legacy label.
let (u, _) = VideoEncodeReceipt::seal_encode(&key(7), "did:example:encoder", &src, 16, 16, enc_params(), 0, None, None).unwrap();
assert!(u.verify());
let (legacy, _) = VideoEncodeReceipt::seal_encode(&key(7), "did:example:encoder", &src, 16, 16, enc_params(), 1_200, Some(Provenance::HwShunt), None).unwrap();
assert!(!legacy.verify(), "a new profile refuses HwShunt");
let (zero_classed, _) = VideoEncodeReceipt::seal_encode(&key(7), "did:example:encoder", &src, 16, 16, enc_params(), 0, model.clone(), None).unwrap();
assert!(!zero_classed.verify(), "a class beside the marker");
// The JSON form reads back as the receipt, and refuses what it does not write.
for x in [r.clone(), u.clone(), VideoEncodeReceipt::seal_encode(&key(9), "did:example:e", &src, 16, 16, enc_params(), 4_000_000, Some(chip()), Some(r.receipt_hash())).unwrap().0] {
let j = x.to_json();
assert_eq!(VideoEncodeReceipt::from_json(&j), Ok(x.clone()), "{j}");
}
let j = r.to_json();
assert_eq!(VideoEncodeReceipt::from_json(&j.replacen("\"sse\":", "\"sse\":1,\"sse\":", 1)), Err(ReadError::Malformed("not JSON read strictly")));
assert_eq!(VideoEncodeReceipt::from_json(&j.replacen("encode-receipt", "receipt", 1)), Err(ReadError::Malformed("profile")));
assert!(matches!(VideoEncodeReceipt::from_json(&j.replacen(&format!("\"sse\":{}", r.sse), &format!("\"sse\":{}", r.sse + 1), 1)), Err(ReadError::ReceiptHashMismatch)));
}

5
wai/wai-studio/src/actions.generated.js
View file

@ -1,7 +1,7 @@
// GENERATED by scripts/actions.mjs from the WAI operation catalogue. Do not edit: change the
// catalogue (wai-rs/src/catalogue.rs, exported by `wai ops export`), then run
// `node scripts/actions.mjs`. `node scripts/actions.mjs --check` fails when this file differs.
export const CATALOGUE = {"format":"wai.ops/1","toolchain":"0.4.0","sha256":"f63a5d9f34e021915cabe55a4fed30fbb0eea967d5beede72bc2b6aa9ab84efd"};
export const CATALOGUE = {"format":"wai.ops/1","toolchain":"0.4.0","sha256":"7168ac79a55b09ed96bb81d0ada8602e01358164135d0de4f6b2077830ef8497"};
export const ACTIONS = [
{"id":"toolchain.test","noun":"toolchain","verb":"test","menu":"Toolchain","item":"Test","palette":"toolchain test","summary":"WAI's four steps (name, dispatch, rebuild, receipt) on a picture made here, its receipt checked offline","placement":"local","effect":"writes-project","stability":"stable","toolset":"core","cli":"wai hello","mcp":null,"route":null,"call":"toolchain_test"},
{"id":"project.create","noun":"project","verb":"create","menu":"Project","item":"Create","palette":"project create","summary":"A new project from one of the examples, written into a new or empty directory; --list names the templates","placement":"local","effect":"writes-project","stability":"stable","toolset":"core","cli":"wai create","mcp":null,"route":null,"call":"project_create"},
@ -17,6 +17,8 @@ export const ACTIONS = [
{"id":"score.decode","noun":"score","verb":"decode","menu":"Score","item":"Decode","palette":"score decode","summary":"A score's reference mixdown and its digests (pcm_sha256, mixdown_blake3, score_hash)","placement":"local","effect":"writes-project","stability":"stable","toolset":"core","cli":"wai score render","mcp":null,"route":null,"call":"score_decode"},
{"id":"encoder.list","noun":"encoder","verb":"list","menu":"Encoder","item":"List","palette":"encoder list","summary":"What each encoder interface offers on this device, for AVC and HEVC","placement":"local","effect":"reads","stability":"preview","toolset":"core","cli":"wai encoders","mcp":null,"route":null,"call":"encoder_list"},
{"id":"encoder.verify","noun":"encoder","verb":"verify","menu":"Encoder","item":"Verify","palette":"encoder verify","summary":"A signed encode record checked, against the stream and its source where given","placement":"local","effect":"reads","stability":"preview","toolset":"core","cli":"wai encode-record verify","mcp":null,"route":null,"call":"encoder_verify"},
{"id":"package.build","noun":"package","verb":"build","menu":"Package","item":"Build","palette":"package build","summary":"A package packed by the reference packager wai-pack from a policy, a source per component and a supplied floor, with its signed package receipt","placement":"local","effect":"writes-project","stability":"preview","toolset":"core","cli":"wai package build","mcp":null,"route":null,"call":"package_build"},
{"id":"package.verify","noun":"package","verb":"verify","menu":"Package","item":"Verify","palette":"package verify","summary":"A package verified by the reference packager wai-pack: every reason it is refused, or its counts and pack digest","placement":"local","effect":"reads","stability":"preview","toolset":"core","cli":"wai package verify","mcp":null,"route":null,"call":"package_verify"},
{"id":"registry.list","noun":"registry","verb":"list","menu":"Registry","item":"List","palette":"registry list","summary":"Every registered capability with its fidelity, licence class, replicate class and role, and whether this build decodes it","placement":"local","effect":"reads","stability":"stable","toolset":"core","cli":"wai capabilities","mcp":"registry_list","route":null,"call":"registry_list"},
{"id":"registry.export","noun":"registry","verb":"export","menu":"Registry","item":"Export","palette":"registry export","summary":"Write, or with --check compare, the registry's machine-readable export and the generated table of REGISTRY.md","placement":"local","effect":"writes-project","stability":"stable","toolset":"maintainer","cli":"wai registry export","mcp":null,"route":null,"call":"registry_export"},
{"id":"prior.list","noun":"prior","verb":"list","menu":"Prior","item":"List","palette":"prior list","summary":"The listing of a parameter set (wai.prior.set) on stdout, and its pin on stderr","placement":"local","effect":"reads","stability":"stable","toolset":"core","cli":"wai prior-set","mcp":null,"route":null,"call":"prior_list"},
@ -43,6 +45,7 @@ export const ACTIONS = [
{"id":"enrolment.list","noun":"enrolment","verb":"list","menu":"Enrolment","item":"List","palette":"enrolment list","summary":"The nodes a fleet service holds enrolled, each with its role, label and how many records it gave","placement":"client","effect":"reads","stability":"preview","toolset":"core","cli":"wai enrolment list","mcp":null,"route":null,"call":"enrolment_list"},
{"id":"report.create","noun":"report","verb":"create","menu":"Report","item":"Create","palette":"report create","summary":"The fleet report of a period, signed by a fleet service over the enrolments and records it holds","placement":"client","effect":"writes-target","stability":"preview","toolset":"core","cli":"wai report create","mcp":null,"route":null,"call":"report_create"},
{"id":"report.export","noun":"report","verb":"export","menu":"Report","item":"Export","palette":"report export","summary":"A fleet report's bundle fetched from a fleet service, its report recomputed here, then written to a file","placement":"client","effect":"writes-project","stability":"preview","toolset":"core","cli":"wai report export","mcp":null,"route":null,"call":"report_export"},
{"id":"route.verify","noun":"route","verb":"verify","menu":"Route","item":"Verify","palette":"route verify","summary":"A route record checked offline against its request, price book and routing policy, and against its hops' OER/2 receipts where given","placement":"local","effect":"reads","stability":"preview","toolset":"core","cli":"wai route verify","mcp":null,"route":null,"call":"route_verify"},
{"id":"channel.import","noun":"channel","verb":"import","menu":"Channel","item":"Import","palette":"channel import","summary":"A linear channel's feeds (an XMLTV schedule, Media RSS catalogues or the JSON form, and a Media Playlist's SCTE-35 date ranges) imported into its signed receipt, its signed lineup and its breaks' cues","placement":"local","effect":"writes-project","stability":"preview","toolset":"core","cli":"wai channel import","mcp":null,"route":null,"call":"channel_import"},
{"id":"channel.guide","noun":"channel","verb":"guide","menu":"Channel","item":"Guide","palette":"channel guide","summary":"A channel's guide, an XMLTV document or a now/next document, rendered from its receipt and its lineup and naming both","placement":"local","effect":"writes-project","stability":"preview","toolset":"core","cli":"wai channel guide","mcp":null,"route":null,"call":"channel_guide"},
{"id":"channel.verify","noun":"channel","verb":"verify","menu":"Channel","item":"Verify","palette":"channel verify","summary":"A channel's receipt, its lineup and a guide verified together: the guide recomputed from the two and compared, each programme with whether it aired","placement":"local","effect":"reads","stability":"preview","toolset":"core","cli":"wai channel verify","mcp":null,"route":null,"call":"channel_verify"},