wai: energy-aware content steering, each client's pathway chosen by the joules its hops signed (L15) #330

Merged

dcharlot merged 10 commits from wai/steering into main 2026-10-19 06:00:24 -04:00

Conversation 0 Commits 10 Files changed 21 +7932 -11

dcharlot commented 2026-10-19 02:34:30 -04:00

Owner

Copy link

wai: energy-aware content steering (L15)

This adds wai.live.steering. A steering server picks each client's pathway (a CDN, a relay tree, an origin path, or a clone of one) from the joules that pathway's hops signed in their per-hop records (WMH1, live-metering §3). It does so within the operator's quality, latency and capacity limits. Each decision is signed and anyone can recompute it. It is delivered as the steering manifest that HLS and DASH players already read.

What is in the PR:

• the spec, wai/extensions/energy-steering.md;
• the reference module, wai/wai-rs/src/energy_steering.rs, behind the feature energy_steering (which implies live_metering);
• the corpus, wai/live-conformance/steering/;
• an independent verifier, wai/live-conformance/energy_steering_verify.py;
• a CI step, the registry entries, SPEC §5, the jwp-receipts §7.5 authority row and a CHANGELOG entry.

Standards read, with pinned versions

• draft-pantos-hls-rfc8216bis-22 (1 May 2026), §4.4.6.6 EXT-X-CONTENT-STEERING and §7. In this revision the steering manifest itself has moved out of the HLS draft into the next document.
• draft-pantos-content-steering-04 (1 May 2026), §2 to §7: pathway IDs, VERSION, TTL, RELOAD-URI, PATHWAY-PRIORITY, PATHWAY-CLONES and the clone's URI-REPLACEMENT. The clone keys are HOST and PARAMS; there is no QUERY-PARAMETERS.
• ETSI TS 103 998 V1.1.1 (2024-01), clauses 5 to 8:
  • the ContentSteering element with @defaultServiceLocation, @queryBeforeStart and @clientRequirement;
  • the DCSM, whose keys are the same as content-steering-04's;
  • _DASH_pathway, a comma-separated list, optionally in quotes;
  • _DASH_throughput, the n-th item for the n-th pathway.
• CTA-5004-B: bl and mtp are Inner Lists of Integers with object-type tokens as parameters. I read this from the CMCD library version the telemetry differential locks.
• RFC 9651 §4.2, for parsing the CMCD dictionary.

The decision rule

• The window. Each pathway's evidence is read over a window that ends on the last record period that has been over for at least the policy's lag.
• Exact tiling. Every hop's records must tile that window exactly. So every pathway is read over the same span, and a record that is left out makes the pathway stale. It can never make it look cheaper.
• The figure. A pathway's figure is J/D. J is the sum of its hops' microjoules. D is either its edge hops' media bytes out, or their microseconds of delivery at or above the floor tier. Every record must share one acquisition class.
• No figure. A pathway has no figure when:
  • any of its records carries the unmetered marker;
  • its records are of two classes;
  • it is stale;
  • it delivered nothing.
• The order. Pathways are ordered:
  1. by rank: calibrated instrument, then on-chip counter, then model, then estimate, then no figure;
  2. within one class, by figure, compared by exact cross-products (J_p·D_q < J_q·D_p) with no division and no rounding;
  3. then by pathway ID, bytewise.
• Choosing the first pathway, in this order:
  1. hold a client whose buffer (CMCD bl) is below the policy's minimum, or whose throughput sustains no tier above the floor;
  2. if no pathway is within the limits, keep the client where it is;
  3. a session whose draw falls in a pathway's minimum-share band goes there; the draw is SHA-256 of the policy digest and the session, fixed for the session;
  4. keep the client's current pathway unless the best pathway of the same rank beats it by more than the hysteresis margin (J_w·D_c·(10⁶+h) < J_c·D_w·10⁶);
  5. otherwise the best pathway within the limits.
• The output.
  • The priority is the first pathway, then the others within the limits in order, then the rest.
  • The TTL is the short one when the client was held or nothing qualified, and the normal one otherwise.

Unknown energy. A pathway with no figure is ranked after every pathway with one, and among themselves those are ordered only by ID. Hysteresis holds a client only against a pathway of the same rank. The guards and the share bands place a client whatever the evidence, and a pathway's band is its configured share whether or not its energy is known. So being unknown can never raise a pathway's rank or its share. Not measuring is never rewarded, and an operator that wants a pathway preferred has to meter it. A decision that gives an unmetered pathway a figure is refused unmetered_as_zero. One that sums two classes into a figure is refused classes_summed. One that puts a pathway with no figure first where the rule would not is refused unknown_preferred.

Records

• WSP1, the steering policy. It is signed by the room authority, or by a key the origin's key log lists with the new authority live/steer.

• WSD1, the steering decision. It names:

  • the policy digest;
  • each pathway's evidence records, by digest;
  • the client's report;
  • the server's session and loads;
  • each pathway's status, class, J, D and limit bits;
  • the TTL, the reason the first pathway is first, and the priority.

  Its signer must also hold energy/<class> for each class it restates.

• Carriage. The manifest names the decision's URI in one extra member, science.transaction.wai-decision. Both content-steering-04 §4 and TS 103 998 clause 6.1 have clients ignore unknown keys. The manifest is a function of the decision, and the member holds the decision's digest, so there is no loop between them.

Mappings

• The HLS steering manifest and the DASH DCSM are the same bytes: compact JSON in a fixed member order, with PATHWAY-CLONES written for every clone in the policy.
• The playlist line is #EXT-X-CONTENT-STEERING:SERVER-URI="…",PATHWAY-ID="<initial>".
• The MPD line is <ContentSteering defaultServiceLocation="…" queryBeforeStart="…"> with & escaped as &.
• The policy's character rules leave nothing in any of these that needs JSON or XML escaping.

Refusal reasons (§10)

• New in this extension:
  • policy_invalid, clone_unknown, decision_invalid;
  • evidence_not_on_pathway, evidence_outside_window, evidence_mismatch;
  • unmetered_as_zero, unknown_preferred, decision_mismatch, manifest_mismatch.
• Reused from live-metering:
  • classes_summed;
  • hop_not_authorised (a hop key the key log does not list) and hop_revoked;
  • signer_not_authorised, signer_revoked, record_missing.

Corpus: live-conformance/steering/

I put it beside metering/ because its evidence is live-metering's WMH1 records, which the Python side reads with live_metering_verify.py.

• Contents:

  • one room with its key log;
  • 11 policies, 3 of which are refused;
  • the per-hop records, signed in four "worlds" of figures;
  • 27 steering requests over HLS and DASH, with CMCD in the query and in the header, plus malformed values;
  • 39 vectors.

• Accepted vectors (17):

  • three pathways metered by on-chip counters;
  • mixed classes, ordered by class and not by figure;
  • one unmetered hop; hops of two classes in one pathway; stale evidence;
  • a client at its floor; a short buffer;
  • hysteresis holding, then a period later switching;
  • clones;
  • a session placed by its share band, and one outside the band;
  • capacity, latency, the per-second basis, DASH, and no pathway within the limits.

  Each carries its manifest, plus the HLS tag and DASH element of its policy, byte for byte.

• Refused vectors (22): one per new reason plus the reused codes, each a control of exactly one defect.

• The figures are synthetic, and the README says so. The corpus claims no energy saving.

Checks run

All cargo work ran on the build host through ~/kani-logs/remote/rb. The full set was re-run after the host rebooted mid-run, with its logs on the build volume:

command	exit
cargo test --no-default-features --features energy_steering --lib energy_steering:: (10 unit tests)	0
cargo test --no-default-features --features energy_steering --test energy_steering (4)	0
wai_energy_steering_vectors verify ../live-conformance/steering (regenerates byte for byte; every request and vector)	0
cargo test --no-default-features --lib codecs::registry:: (14)	0
cargo run --no-default-features --bin wai -- registry export --check	0
cargo test --no-default-features --features live_metering --test live_metering and --lib live_metering::	0
cargo clippy --no-default-features --features energy_steering --lib --tests --bins: no finding in the steering code; the warnings that remain are older ones elsewhere in the crate	0
touch cbindgen.toml && cargo build --no-default-features --features energy_steering --lib, then include/wai.h compared: unchanged	0

Locally (standard library or shell only):

command	exit
python3 -I live-conformance/energy_steering_verify.py live-conformance/steering: 27 requests and 39 vectors verify, §10 matches, 24 of 24 mutated copies fail	0
scripts/check-tests-in-ci.py, check-spec-refs.py, check-wai-section-refs.py, check-no-vanity-metrics.sh, check-comparative-prose.sh, check-orphan-sources.py, check-workflow-concurrency.sh, check-lockfiles.sh	0 each
scripts/check-public-names.sh with the private denylist	0

The Python verifier agreed with wai-rs on every request and vector on its first run.

Design choices for a reviewer to check

1. Ordering across classes. Classes are never compared by figure. Across classes the order follows energy-measurement §2's order of evidential strength. As a result, a calibrated pathway with a higher figure ranks above an estimated pathway with a lower one. That rewards measuring better. The alternative was to treat such pathways as incomparable, which leaves no total order.
2. Mixed classes within one pathway. A pathway whose hops are metered by different classes has no figure: the rules forbid summing across classes. This penalises pathways metered by a mix of classes until they use one class.
3. Exact tiling. Evidence must tile a grid-aligned window, so hops must cut their records on the policy's period (live-metering §3.8 allows any cut). This is what stops a server from making a pathway look cheaper by leaving records out. The cost is that a hop flushing off-grid is always stale.
4. Hops belong to one pathway. A hop cannot be shared between pathways, because no record states how its joules split. A hop common to every pathway is simply left out.
5. Capacity is the server's own count of sessions it steers to the pathway first. The draw and the loads are stated in the decision but only recomputed, not verified. An audit over the decision log is what checks them (§9.2).
6. The per-second basis charges every joule to delivery at or above the floor tier. It is an upper bound on any allocation by tier, stated as such.
7. The report is stated, not the raw request, so tokens in the request query are not published. The client's throughput and buffer are taken as the client sent them.
8. The CMCD reader has its own complete RFC 9651 parser in both implementations, base64 validity of byte sequences included. telemetry::sf is a subset and refuses parameters on inner-list items, which CTA-5004-B's bl uses.

Not done yet

• A live steering server. This PR has the decision function and the record, but no HTTP service serving manifests and .wsd records.
• Feeding decisions from live WMH1 records. No hop yet cuts its records on a policy's grid, and joule-moq-relay signs one record per session with the unmetered marker.
• PER-VARIANT-URIS and PER-RENDITION-URIS (HLS-only clone keys) are not written by this revision.
• No player has been run against these manifests yet. The bytes follow the drafts' grammars, but no player e2e has been run.

🤖 Generated with Claude Code

## wai: energy-aware content steering (L15) This adds `wai.live.steering`. A steering server picks each client's pathway (a CDN, a relay tree, an origin path, or a clone of one) from the joules that pathway's hops signed in their per-hop records (`WMH1`, live-metering §3). It does so within the operator's quality, latency and capacity limits. Each decision is signed and anyone can recompute it. It is delivered as the steering manifest that HLS and DASH players already read. **What is in the PR:** - the spec, `wai/extensions/energy-steering.md`; - the reference module, `wai/wai-rs/src/energy_steering.rs`, behind the feature `energy_steering` (which implies `live_metering`); - the corpus, `wai/live-conformance/steering/`; - an independent verifier, `wai/live-conformance/energy_steering_verify.py`; - a CI step, the registry entries, SPEC §5, the jwp-receipts §7.5 authority row and a CHANGELOG entry. ### Standards read, with pinned versions - `draft-pantos-hls-rfc8216bis-22` (1 May 2026), §4.4.6.6 `EXT-X-CONTENT-STEERING` and §7. In this revision the steering manifest itself has moved out of the HLS draft into the next document. - `draft-pantos-content-steering-04` (1 May 2026), §2 to §7: pathway IDs, `VERSION`, `TTL`, `RELOAD-URI`, `PATHWAY-PRIORITY`, `PATHWAY-CLONES` and the clone's `URI-REPLACEMENT`. The clone keys are `HOST` and `PARAMS`; there is no `QUERY-PARAMETERS`. - ETSI TS 103 998 V1.1.1 (2024-01), clauses 5 to 8: - the `ContentSteering` element with `@defaultServiceLocation`, `@queryBeforeStart` and `@clientRequirement`; - the DCSM, whose keys are the same as content-steering-04's; - `_DASH_pathway`, a comma-separated list, optionally in quotes; - `_DASH_throughput`, the n-th item for the n-th pathway. - CTA-5004-B: `bl` and `mtp` are Inner Lists of Integers with object-type tokens as parameters. I read this from the CMCD library version the telemetry differential locks. - RFC 9651 §4.2, for parsing the CMCD dictionary. ### The decision rule - **The window.** Each pathway's evidence is read over a window that ends on the last record period that has been over for at least the policy's lag. - **Exact tiling.** Every hop's records must tile that window exactly. So every pathway is read over the same span, and a record that is left out makes the pathway *stale*. It can never make it look cheaper. - **The figure.** A pathway's figure is J/D. *J* is the sum of its hops' microjoules. *D* is either its edge hops' media bytes out, or their microseconds of delivery at or above the floor tier. Every record must share one acquisition class. - **No figure.** A pathway has no figure when: - any of its records carries the unmetered marker; - its records are of two classes; - it is stale; - it delivered nothing. - **The order.** Pathways are ordered: 1. by rank: calibrated instrument, then on-chip counter, then model, then estimate, then no figure; 2. within one class, by figure, compared by exact cross-products (J_p·D_q < J_q·D_p) with no division and no rounding; 3. then by pathway ID, bytewise. - **Choosing the first pathway**, in this order: 1. hold a client whose buffer (CMCD `bl`) is below the policy's minimum, or whose throughput sustains no tier above the floor; 2. if no pathway is within the limits, keep the client where it is; 3. a session whose draw falls in a pathway's minimum-share band goes there; the draw is SHA-256 of the policy digest and the session, fixed for the session; 4. keep the client's current pathway unless the best pathway of the same rank beats it by more than the hysteresis margin (J_w·D_c·(10⁶+h) < J_c·D_w·10⁶); 5. otherwise the best pathway within the limits. - **The output.** - The priority is the first pathway, then the others within the limits in order, then the rest. - The TTL is the short one when the client was held or nothing qualified, and the normal one otherwise. **Unknown energy.** A pathway with no figure is ranked after every pathway with one, and among themselves those are ordered only by ID. Hysteresis holds a client only against a pathway of the same rank. The guards and the share bands place a client whatever the evidence, and a pathway's band is its configured share whether or not its energy is known. So being unknown can never raise a pathway's rank or its share. Not measuring is never rewarded, and an operator that wants a pathway preferred has to meter it. A decision that gives an unmetered pathway a figure is refused `unmetered_as_zero`. One that sums two classes into a figure is refused `classes_summed`. One that puts a pathway with no figure first where the rule would not is refused `unknown_preferred`. ### Records - **`WSP1`, the steering policy.** It is signed by the room authority, or by a key the origin's key log lists with the new authority `live/steer`. - **`WSD1`, the steering decision.** It names: - the policy digest; - each pathway's evidence records, by digest; - the client's report; - the server's session and loads; - each pathway's status, class, J, D and limit bits; - the TTL, the reason the first pathway is first, and the priority. Its signer must also hold `energy/<class>` for each class it restates. - **Carriage.** The manifest names the decision's URI in one extra member, `science.transaction.wai-decision`. Both content-steering-04 §4 and TS 103 998 clause 6.1 have clients ignore unknown keys. The manifest is a function of the decision, and the member holds the decision's digest, so there is no loop between them. ### Mappings - The HLS steering manifest and the DASH DCSM are the same bytes: compact JSON in a fixed member order, with `PATHWAY-CLONES` written for every clone in the policy. - The playlist line is `#EXT-X-CONTENT-STEERING:SERVER-URI="…",PATHWAY-ID="<initial>"`. - The MPD line is `<ContentSteering defaultServiceLocation="…" queryBeforeStart="…">` with `&` escaped as `&amp;`. - The policy's character rules leave nothing in any of these that needs JSON or XML escaping. ### Refusal reasons (§10) - **New in this extension:** - `policy_invalid`, `clone_unknown`, `decision_invalid`; - `evidence_not_on_pathway`, `evidence_outside_window`, `evidence_mismatch`; - `unmetered_as_zero`, `unknown_preferred`, `decision_mismatch`, `manifest_mismatch`. - **Reused from live-metering:** - `classes_summed`; - `hop_not_authorised` (a hop key the key log does not list) and `hop_revoked`; - `signer_not_authorised`, `signer_revoked`, `record_missing`. ### Corpus: `live-conformance/steering/` I put it beside `metering/` because its evidence is live-metering's `WMH1` records, which the Python side reads with `live_metering_verify.py`. - **Contents:** - one room with its key log; - 11 policies, 3 of which are refused; - the per-hop records, signed in four "worlds" of figures; - 27 steering requests over HLS and DASH, with CMCD in the query and in the header, plus malformed values; - 39 vectors. - **Accepted vectors (17):** - three pathways metered by on-chip counters; - mixed classes, ordered by class and not by figure; - one unmetered hop; hops of two classes in one pathway; stale evidence; - a client at its floor; a short buffer; - hysteresis holding, then a period later switching; - clones; - a session placed by its share band, and one outside the band; - capacity, latency, the per-second basis, DASH, and no pathway within the limits. Each carries its manifest, plus the HLS tag and DASH element of its policy, byte for byte. - **Refused vectors (22):** one per new reason plus the reused codes, each a control of exactly one defect. - The figures are synthetic, and the README says so. The corpus claims no energy saving. ### Checks run All cargo work ran on the build host through `~/kani-logs/remote/rb`. The full set was re-run after the host rebooted mid-run, with its logs on the build volume: | command | exit | |---|---| | `cargo test --no-default-features --features energy_steering --lib energy_steering::` (10 unit tests) | 0 | | `cargo test --no-default-features --features energy_steering --test energy_steering` (4) | 0 | | `wai_energy_steering_vectors verify ../live-conformance/steering` (regenerates byte for byte; every request and vector) | 0 | | `cargo test --no-default-features --lib codecs::registry::` (14) | 0 | | `cargo run --no-default-features --bin wai -- registry export --check` | 0 | | `cargo test --no-default-features --features live_metering --test live_metering` and `--lib live_metering::` | 0 | | `cargo clippy --no-default-features --features energy_steering --lib --tests --bins`: no finding in the steering code; the warnings that remain are older ones elsewhere in the crate | 0 | | `touch cbindgen.toml && cargo build --no-default-features --features energy_steering --lib`, then `include/wai.h` compared: unchanged | 0 | Locally (standard library or shell only): | command | exit | |---|---| | `python3 -I live-conformance/energy_steering_verify.py live-conformance/steering`: 27 requests and 39 vectors verify, §10 matches, 24 of 24 mutated copies fail | 0 | | `scripts/check-tests-in-ci.py`, `check-spec-refs.py`, `check-wai-section-refs.py`, `check-no-vanity-metrics.sh`, `check-comparative-prose.sh`, `check-orphan-sources.py`, `check-workflow-concurrency.sh`, `check-lockfiles.sh` | 0 each | | `scripts/check-public-names.sh` with the private denylist | 0 | The Python verifier agreed with `wai-rs` on every request and vector on its first run. ### Design choices for a reviewer to check 1. **Ordering across classes.** Classes are never compared by figure. Across classes the order follows energy-measurement §2's order of evidential strength. As a result, a calibrated pathway with a higher figure ranks above an estimated pathway with a lower one. That rewards measuring better. The alternative was to treat such pathways as incomparable, which leaves no total order. 2. **Mixed classes within one pathway.** A pathway whose hops are metered by different classes has no figure: the rules forbid summing across classes. This penalises pathways metered by a mix of classes until they use one class. 3. **Exact tiling.** Evidence must tile a grid-aligned window, so hops must cut their records on the policy's period (live-metering §3.8 allows any cut). This is what stops a server from making a pathway look cheaper by leaving records out. The cost is that a hop flushing off-grid is always stale. 4. **Hops belong to one pathway.** A hop cannot be shared between pathways, because no record states how its joules split. A hop common to every pathway is simply left out. 5. **Capacity is the server's own count of sessions** it steers to the pathway first. The draw and the loads are stated in the decision but only recomputed, not verified. An audit over the decision log is what checks them (§9.2). 6. **The per-second basis** charges every joule to delivery at or above the floor tier. It is an upper bound on any allocation by tier, stated as such. 7. **The report is stated, not the raw request**, so tokens in the request query are not published. The client's throughput and buffer are taken as the client sent them. 8. **The CMCD reader** has its own complete RFC 9651 parser in both implementations, base64 validity of byte sequences included. `telemetry::sf` is a subset and refuses parameters on inner-list items, which CTA-5004-B's `bl` uses. ### Not done yet - **A live steering server.** This PR has the decision function and the record, but no HTTP service serving manifests and `.wsd` records. - **Feeding decisions from live `WMH1` records.** No hop yet cuts its records on a policy's grid, and `joule-moq-relay` signs one record per session with the unmetered marker. - **PER-VARIANT-URIS and PER-RENDITION-URIS** (HLS-only clone keys) are not written by this revision. - **No player has been run against these manifests yet.** The bytes follow the drafts' grammars, but no player e2e has been run. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 8 commits 2026-10-19 02:34:30 -04:00

wai: energy-aware content steering, the extension text (L15) ... eb3fef7118

wai.live.steering: a signed steering policy (WSP1) and a signed steering
decision (WSD1) that chooses each client's pathway from the per-hop
records' joules, compared only within a class, a pathway with no figure
ranked after every pathway with one, and writes the HLS and DASH
steering manifests exactly.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai-rs: energy_steering, the reference module, its corpus generator and tests (L15) ... d8817912f3

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: the steering corpus, live-conformance/steering/ (L15) ... 1166f270d3

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: energy_steering_verify.py, the steering corpus's second implementation (L15) ... cfc617cb24

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: steering's RFC 9651 parsers refuse a byte sequence that is not base64 ... 39467caf74

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: steering in CI, SPEC §5, the registry, jwp-receipts §7.5 and the CHANGELOG (L15) ... e54023cb20

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: registry export, wai.live.steering and the magics WSP1 and WSD1 ... 7c194ec559

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai-rs: energy_steering is clippy-clean ... dccec57e39

parse_dictionary returns an Option, the first pathway's steps take the
best pathway by a let rather than an expect, and the corpus helper with
eight arguments says so.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot added 2 commits 2026-10-19 06:00:11 -04:00

Merge origin/main into wai/steering: energy steering beside live ingest ... 92d9c2ecf7

Main gained #325 (the browser client, the hardening and live-ingest). REGISTRY keeps WRP1
and WSD1, SPEC §5 both paragraphs, the CHANGELOG both entries; capabilities.json is
regenerated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai registry export: steering beside the WRP1 proofs ... 224080a6fd

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit 06711536ed into main 2026-10-19 06:00:24 -04:00

dcharlot referenced this pull request from a commit 2026-10-19 06:00:24 -04:00

Merge pull request 'wai: energy-aware content steering, each client's pathway chosen by the joules its hops signed (L15)' (#330) from wai/steering into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!330

No description provided.