Some checks failed
wai-web wasm32 / does this change touch it (pull_request) Has been cancelled
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Has been cancelled
wai-web reach builds / wai-web reach · does this change touch it (pull_request) Has been cancelled
wai-web reach builds / wai-web reach builds · the package of record, every output as recorded (pull_request) Has been cancelled
wai-web reach builds / wai-web reach builds · this tree's wai-web, every rung on its engine line (pull_request) Has been cancelled
WAI mbt2018_mean integer decode (regenerated vectors) / does this change touch it (pull_request) Has been cancelled
WAI mbt2018_mean integer decode (regenerated vectors) / regenerate mbt2018_mean integer vectors → Rust decode, required (pull_request) Has been cancelled
wai-quantum WebGPU member / does this change touch it (pull_request) Has been cancelled
wai-quantum WebGPU member / shader checks · browser build · wasip2 inertness · MSRV (pull_request) Has been cancelled
wai-quantum WebGPU member / member · software Vulkan (linux) (pull_request) Has been cancelled
wai-quantum WebGPU member / member · Metal (macos) (pull_request) Has been cancelled
wai-quantum WebGPU member / member · discrete GPU, Vulkan (jetson-hub) (pull_request) Has been cancelled
wai-quantum WebGPU member / member · integrated GPU, Vulkan (framework-128gb) (pull_request) Has been cancelled
wai-quantum WebGPU member / member · browser WebGPU (macos) (pull_request) Has been cancelled
wai-quantum studio and desktop / does this change touch it (pull_request) Has been cancelled
wai-quantum studio and desktop / studio · browser (pull_request) Has been cancelled
wai-quantum studio and desktop / desktop app · macos (pull_request) Has been cancelled
wai-pack / wai-pack · does this change touch it (pull_request) Has been cancelled
wai-pack / wai-pack · linux (pull_request) Has been cancelled
wai-pack / wai-pack · linux-arm64 (pull_request) Has been cancelled
wai-pack / wai-pack · macos (pull_request) Has been cancelled
wai-pack / wai-pack · independent verifier (Python) (pull_request) Has been cancelled
wai-pack / wai-pack · the floor through a media prober (pull_request) Has been cancelled
wai-pack / wai-pack · dependency licences (pull_request) Has been cancelled
wai-ops-gpu integer operators on GPUs / does this change touch it (pull_request) Has been cancelled
wai-ops-gpu integer operators on GPUs / builds · no adapter is not a pass (linux) (pull_request) Has been cancelled
wai-ops-gpu integer operators on GPUs / Metal GPU and browser WebGPU (macos) (pull_request) Has been cancelled
WAI neural sink gates / does this change touch it (pull_request) Has been cancelled
WAI neural sink gates / model-free refusals of the neural sink (pull_request) Has been cancelled
WAI JPEG-AI whole decode (pinned inputs, required) / does this change touch it (pull_request) Has been cancelled
WAI JPEG-AI whole decode (pinned inputs, required) / fetch pinned JPEG-AI inputs → whole decode + pixel hash, required (pull_request) Has been cancelled
WAI energy receipts / does this change touch it (pull_request) Has been cancelled
WAI energy receipts / energy receipts · linux (pull_request) Has been cancelled
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Has been cancelled
WAI energy receipts / energy receipts · macos (pull_request) Has been cancelled
WAI conformance / does this change touch it (pull_request) Has been cancelled
WAI conformance / cargo-class corpora · linux (pull_request) Has been cancelled
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Has been cancelled
WAI conformance / cargo-class corpora · macos (pull_request) Has been cancelled
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Has been cancelled
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Has been cancelled
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Has been cancelled
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Has been cancelled
WAI conformance / cross-arch byte-equality · linux (pull_request) Has been cancelled
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Has been cancelled
WAI conformance / cross-arch byte-equality · macos (pull_request) Has been cancelled
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Has been cancelled
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Has been cancelled
standards workspaces build + test / which workspaces this change touches (pull_request) Has been cancelled
standards workspaces build + test / macOS energy backends (pull_request) Has been cancelled
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Has been cancelled
wai-quantum WebGPU member / member · DX12 and Vulkan (cbio-windows) (pull_request) Has been cancelled
joule-moq-web browser client / joule-moq-web · does this change touch it (pull_request) Has been cancelled
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Has been cancelled
joule-moq-room room service / joule-moq-room · does this change touch it (pull_request) Has been cancelled
joule-moq-room room service / joule-moq-room, every target (pull_request) Has been cancelled
Co-Authored-By: Claude Opus 5.5 <[email protected]>
454 lines
27 KiB
Markdown
454 lines
27 KiB
Markdown
# WAI Extension: QoE Aggregation
|
||
|
||
> Status: Draft. Registers the record `wai.telemetry.aggregate`: quality-of-experience
|
||
> figures the service receives only as sums. A sink reports each session's **measures**, the
|
||
> session metrics of [delivery-telemetry](delivery-telemetry.md) §7 (its §7.2) cast as
|
||
> Prio3 measurements, through the Distributed Aggregation Protocol of
|
||
> `draft-ietf-ppm-dap-20` (DAP): it secret-shares each measurement between a **leader** and a **helper** aggregator, and
|
||
> encrypts each share to its aggregator. The aggregators check each measurement with the range
|
||
> proof of its Prio3 VDAF (`draft-irtf-cfrg-vdaf-22`) without either of them seeing it, sum the
|
||
> valid ones, and release an aggregate share only for a batch of at least the task's minimum
|
||
> size, which this extension never lets fall below fifty reports. Each aggregator signs a
|
||
> **statement** of its aggregate share with the batch's parameters (`WQS1`), and the collector,
|
||
> the service, signs an **aggregate receipt** (`WQA1`) that carries the task, both statements
|
||
> and the result: a reader recomputes the result from the two shares and knows what was
|
||
> summed, over how many reports, and by whom. With two aggregators, neither learns a
|
||
> measurement as long as one of them follows the protocol. A room's quality-of-experience cell
|
||
> ([live-metering](live-metering.md) §4.7) is computed the same way. Leader and helper are
|
||
> roles any party can run. No new transport: DAP runs over HTTP, as its own text defines.
|
||
> The reference implementation is the `telemetry_aggregate` module of `wai-rs` (feature
|
||
> `telemetry_aggregate`), with a reference leader, helper and collector that run in its tests;
|
||
> the corpus is [`live-conformance/aggregate/`](../live-conformance/aggregate/), and its
|
||
> second implementation is `live-conformance/telemetry_aggregate_verify.py`.
|
||
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
|
||
>
|
||
> Texts read for this revision: `draft-ietf-ppm-dap-20` (all of it; §4 to §8 and Appendix B
|
||
> are used here) and `draft-irtf-cfrg-vdaf-22` (§4 to §7, §9, Appendices A to C), both
|
||
> Internet-Drafts, with the test vectors the VDAF draft's [TestVectors] reference names; RFC
|
||
> 9180 (§4, §5.1, §6.1, §7.1, Appendix A.1); RFC 9861 (§2.1, §5); RFC 5869; RFC 8032; FIPS
|
||
> 180-4; CTA-5004-B through [delivery-telemetry](delivery-telemetry.md). A section number of a
|
||
> draft in this text is that revision's.
|
||
|
||
## 1. Scope
|
||
|
||
In scope: the measures and the measurement each takes from a session's metrics (§3); the DAP
|
||
task a measure runs under, the WAI layout of its `task_info`, its id and the rules a task
|
||
meets (§4); the roles of DAP as this extension uses them (§5); the statement an aggregator
|
||
signs (§6); the aggregate receipt (§7); its verification (§8); where aggregation replaces
|
||
the suppression of [delivery-telemetry](delivery-telemetry.md) §8 (its §8.4), where that
|
||
suppression still applies, and a room's cell (§9).
|
||
|
||
Out of scope: running aggregators in production; how a deployment chooses its aggregators
|
||
and agrees a task with them, which DAP leaves out of band (`draft-ietf-ppm-dap-20` §4.2);
|
||
client authentication and anonymizing proxies (DAP §8.3, §8.4); differential privacy (DAP
|
||
§8.5); any VDAF but the four Prio3 variants of §3.
|
||
|
||
Relationship to the core spec: `wai.telemetry.aggregate` is a **record** ([SPEC](../SPEC.md)
|
||
§5): a signed record a WAI tool emits and verifies. It names no payload format and is not
|
||
dispatchable.
|
||
|
||
## 2. Terms
|
||
|
||
- **DAP**, **task**, **report**, **report share**, **input share**, **leader**, **helper**,
|
||
**collector**, **batch**, **batch bucket**, **batch interval**, **minimum batch size**,
|
||
**aggregate share**, **aggregate result**: as `draft-ietf-ppm-dap-20` §1.2.1 defines them.
|
||
This extension's collector is the content's own service.
|
||
- **Prio3**, **measurement**, **verification**, **verifier share**: as
|
||
`draft-irtf-cfrg-vdaf-22` defines them (§5, §7).
|
||
- **Session metrics**: the figures [delivery-telemetry](delivery-telemetry.md) §7 computes
|
||
from one session's log (its §7.2).
|
||
- **Measure**: a quantity of §3, aggregated by one task.
|
||
- **Statement**: an aggregator's signed account of the aggregate share it released (§6).
|
||
- **Aggregate receipt**: the collector's signed record of an aggregate result (§7).
|
||
|
||
## 3. Measures
|
||
|
||
§3.1 A measure takes one figure, or a few, from a session's metrics and aggregates them with
|
||
one Prio3 variant of `draft-irtf-cfrg-vdaf-22` §7.4, with two aggregators and one proof:
|
||
|
||
| code | measure | VDAF (§7.4 there) | configuration (DAP Appendix B) | measurement, from the session's metrics |
|
||
|---|---|---|---|---|
|
||
| 1 | `outcomes` | Prio3SumVec | `length` 4, `max_measurement` 1, `chunk_length` 2 | `[started, start_failed, exit_before_start, vst_ms ≠ null]`, each 0 or 1 |
|
||
| 2 | `stalls` | Prio3Sum | `max_measurement` 65535 | `stalls`, at most 65535 |
|
||
| 3 | `durations` | Prio3SumVec | `length` 3, `max_measurement` 4294967295, `chunk_length` 10 | `[stall_ms, engaged_ms, vst_ms]`, `vst_ms` 0 when `null`, each at most 4294967295 |
|
||
| 4 | `start_time` | Prio3Histogram | `length` 8, `chunk_length` 3 | the bucket of `vst_ms`: 0 to 5 for a time below 250, 500, 1000, 2000, 4000 and 8000 ms, the first that holds; 6 for a later start; 7 for `null` |
|
||
|
||
§3.2 A figure past its measure's largest value is reported as that value, and a sum counts
|
||
it at that value. Each largest value is past what one session's figures reach in practice: a
|
||
stall count of 65535, and 49 days in milliseconds.
|
||
|
||
§3.3 The range proof is the VDAF's: Prio3Sum and Prio3SumVec encode each value as the bits
|
||
of `draft-irtf-cfrg-vdaf-22` §7.4.2 and prove each bit is 0 or 1, so a value past the
|
||
largest cannot be encoded, and Prio3Histogram proves its vector has one 1 and every other
|
||
element 0 (§7.4.4 there). An aggregator that verifies a report whose measurement is out of
|
||
range finds the proof fails and rejects the report, `vdaf_verify_error` (DAP §4.5.3.2),
|
||
without learning the measurement.
|
||
|
||
§3.4 Prio3Count (VDAF §7.4.1) is implemented with the others and passes the draft's
|
||
vectors; no measure of this revision uses it.
|
||
|
||
## 4. The task
|
||
|
||
### 4.1 `task_info`
|
||
|
||
A measure runs under one DAP task (`draft-ietf-ppm-dap-20` §4.2). Its configuration's
|
||
`task_info`, which DAP leaves to the deployment, is laid out as follows, 153 bytes:
|
||
|
||
| field | size | meaning |
|
||
|---|---|---|
|
||
| `label` | 8 | `wai-qoe1` (ASCII) |
|
||
| `measure` | 1 | the measure's code (§3.1) |
|
||
| `scope` | 32 | 32 zero bytes for a task of the service's own; the room id for a room's (§9.2) |
|
||
| `salt` | 16 | chosen by the task's author, so that two tasks of one configuration have two ids |
|
||
| `leader` | 32 | the Ed25519 key the leader signs its statements with |
|
||
| `helper` | 32 | the helper's |
|
||
| `collector` | 32 | the collector's, which signs the aggregate receipt |
|
||
|
||
The task configuration enters the authenticated data of every input share and of every
|
||
aggregate share (DAP §4.4.2.1, §4.6.7), so a report is bound at upload to the measure, the
|
||
scope and the three keys, and an aggregator that holds another configuration cannot decrypt
|
||
it.
|
||
|
||
### 4.2 The task id
|
||
|
||
The task id is SHA-256 of the 29 bytes `wai:telemetry-aggregate/task` (ASCII) and 0x00,
|
||
followed by the TaskConfiguration's encoding. DAP admits a task id derived from the task's
|
||
parameters (§8.6.1 there); the salt makes it unpredictable.
|
||
|
||
### 4.3 Rules
|
||
|
||
A task configuration is refused, in this order:
|
||
|
||
1. `task_invalid` when it does not read as a TaskConfiguration; its `task_info` is not 153
|
||
bytes beginning `wai-qoe1`; its `time_precision` is 0; its `batch_mode` is not
|
||
time-interval (1) or its `batch_config` is not empty; or an extension is other than
|
||
`task_interval` (1) holding one Interval, or the extensions are not in ascending order of
|
||
type;
|
||
2. `measure_unknown` when `measure` is no code of §3.1;
|
||
3. `measure_mismatch` when `vdaf_type` and `vdaf_configuration` are not the measure's: its
|
||
VDAF identifier (VDAF Table 19) and its DAP Appendix B encoding;
|
||
4. `floor_too_low` when `min_batch_size` is below 50, the floor of delivery-telemetry §8
|
||
(its §8.4);
|
||
5. `aggregator_alone` when `leader` and `helper` are one key.
|
||
|
||
The time-interval batch mode is the only one, because a leader-selected batch is one the
|
||
leader composes (DAP §8.2); a task's `time_precision` **SHOULD** be an hour or more, which
|
||
also coarsens the report times an aggregator sees (DAP §8.4).
|
||
|
||
### 4.4 The verification key
|
||
|
||
The aggregators share the VDAF verification key of the task, kept from every client (DAP
|
||
§8.6.2). The reference derives it as that section shows: HKDF-Expand of
|
||
HKDF-Extract(`verify_key`, a seed the aggregators agree) with the task id. Its verification
|
||
key id is 0.
|
||
|
||
## 5. The protocol
|
||
|
||
§5.1 **The sink.** When a session ends or is abandoned, the sink computes its metrics
|
||
(delivery-telemetry §7.2) and, for each measure the service runs, a report: a fresh random
|
||
report id, the time of the session's `end` or `abandon` in units of the task's
|
||
`time_precision`, and the measurement of §3.1 sharded and encrypted as DAP §4.4.2.1 does, with
|
||
the application context `dap-20` followed by the task id. It uploads the reports to the
|
||
leader (DAP §4.4.2). A session gives at most one report to each task. Its reports carry no
|
||
`sid`, no principal and no extension.
|
||
|
||
§5.2 **The aggregators.** The leader and the helper run DAP §4.5 and §4.6 as written, over
|
||
HTTP, with the HPKE suite of DAP §7 (DHKEM(X25519, HKDF-SHA256), HKDF-SHA256, AES-128-GCM):
|
||
|
||
- each decrypts its own input share, checks it (§4.5.3.3, §4.5.3.4) and runs Prio3's
|
||
one-round verification with its peer by the ping-pong exchange of VDAF §5.7.1, so a report
|
||
whose proof fails is rejected by both, `vdaf_verify_error`;
|
||
- each commits a valid report's output share to the batch bucket of its time, once
|
||
(`report_replayed` for a second), and keeps per bucket the aggregate share, the count and
|
||
the checksum, the XOR of the SHA-256 of each report id (§4.5.4.3);
|
||
- each refuses to release an aggregate share for a batch below the task's minimum batch
|
||
size: the leader fails the collection job and the helper the aggregate share request,
|
||
`invalidBatchSize` (§4.6.1, §4.6.4); the helper also refuses a report count or checksum not
|
||
its own, `batchMismatch`, and a batch already collected, `batchOverlap`;
|
||
- each encrypts its aggregate share to the collector (§4.6.7) and signs a statement of it (§6).
|
||
|
||
A batch below the minimum therefore yields no aggregate as long as one aggregator follows the
|
||
protocol: the result needs both shares.
|
||
|
||
§5.3 **The collector.** The collector opens both aggregate shares and both statements,
|
||
requires each statement's share and count to be the ones DAP delivered, unshards the shares
|
||
(VDAF §7.2.5) and signs the aggregate receipt (§7).
|
||
|
||
§5.4 **What each party learns.** The leader and the helper each hold one share of each
|
||
measurement: a share alone, and the verification exchange, reveal nothing about it but
|
||
whether it is valid (VDAF §7.1; DAP §2.2). Each sees the reports' ids, times and lengths and
|
||
which reports were rejected. The collector, and a reader of a receipt, learn the aggregate,
|
||
its report count and its batch, nothing per session. Two aggregators that pool their shares
|
||
learn every measurement: the deployment chooses aggregators that do not, and DAP §8.7 says
|
||
why their infrastructure should not be shared.
|
||
|
||
## 6. The aggregate share statement
|
||
|
||
### 6.1 Layout
|
||
|
||
When an aggregator releases its aggregate share for a batch, it signs a statement, `WQS1`:
|
||
|
||
| field | size | meaning |
|
||
|---|---|---|
|
||
| `magic` | 4 | `WQS1` (ASCII) |
|
||
| `version` | 1 | 1 |
|
||
| `role` | 1 | 2 for the leader, 3 for the helper (DAP §4.1) |
|
||
| `task_id` | 32 | the task id (§4.2) |
|
||
| `collection` | 32 | SHA-256 of the CollectionJobReq the share answers (DAP §4.6.1) |
|
||
| `batch` | 16 | the batch: its interval, `start` and `duration` (u64 each) in units of the task's `time_precision` |
|
||
| `report_count` | 8 | u64: the reports aggregated in the batch |
|
||
| `checksum` | 32 | the batch's checksum (DAP §4.5.4.3) |
|
||
| `agg_share` | 4 + n | u32 length, then the aggregator's aggregate share, encoded as VDAF §7.2.7.5 encodes it |
|
||
| `signer` | 32 | the aggregator's key, the one `task_info` names for its role |
|
||
| `sig` | 64 | Ed25519 by `signer` over the 30 bytes `wai:telemetry-aggregate/share` (ASCII) and 0x01, followed by every byte before `sig`, verified strictly (SPEC §7 *Signatures*) |
|
||
|
||
Integers are big-endian. A statement is read exactly: its magic, version 1, a role of 2 or 3,
|
||
and no byte past its end; otherwise it is refused `statement_invalid`.
|
||
|
||
### 6.2 When an aggregator signs
|
||
|
||
An aggregator signs a statement only for a batch whose aggregate share it releases under DAP
|
||
§4.6: a task that meets §4.3, a batch of at least the minimum batch size, and, at the helper,
|
||
a report count and checksum that are its own. Its `batch` is the batch selector's interval,
|
||
`report_count` and `checksum` are its batch's, and `agg_share` is the share it encrypted to
|
||
the collector.
|
||
|
||
### 6.3 How a statement travels
|
||
|
||
A statement carries an aggregate share in clear, so it travels as the share does: sealed to
|
||
the collector with SealBase (RFC 9180 §6.1) under the collector's HPKE configuration, with
|
||
the info string `wai-telemetry-aggregate statement` (ASCII) followed by the sender's role and
|
||
the collector's (0x00), and the AggregateShareAad of DAP §4.6.7 as authenticated data. The
|
||
leader that relays the helper's learns nothing from it. How the sealed statement reaches the
|
||
collector beside DAP's own messages is the deployment's.
|
||
|
||
## 7. The aggregate receipt
|
||
|
||
### 7.1 Layout
|
||
|
||
The collector signs the aggregate receipt, `WQA1`:
|
||
|
||
| field | size | meaning |
|
||
|---|---|---|
|
||
| `magic` | 4 | `WQA1` (ASCII) |
|
||
| `version` | 1 | 1 |
|
||
| `task` | 2 + n | u16 length, then the TaskConfiguration (DAP §4.2) |
|
||
| `collection` | 4 + n | u32 length, then the CollectionJobReq (DAP §4.6.1) |
|
||
| `interval` | 16 | the CollectionJobResp's interval: the smallest interval holding the batch's reports |
|
||
| `leader` | 4 + n | u32 length, then the leader's statement |
|
||
| `helper` | 4 + n | u32 length, then the helper's statement |
|
||
| `result` | 2 + 16 × k | u16 count, then each element of the aggregate result as a u128 |
|
||
| `signer` | 32 | the collector's key, the one `task_info` names |
|
||
| `sig` | 64 | Ed25519 by `signer` over the 32 bytes `wai:telemetry-aggregate/receipt` (ASCII) and 0x01, followed by every byte before `sig`, verified strictly |
|
||
|
||
The receipt's **digest** is SHA-256 over every byte before `sig`.
|
||
|
||
### 7.2 Reading
|
||
|
||
A reader reads exactly the bytes it is given: `truncated` when they end before a field does,
|
||
`malformed` for another magic or a byte past `sig`, and `unknown_version` for a version other
|
||
than 1. A `helper` of length zero reads, and is refused at §8.1 step 5.
|
||
|
||
## 8. Verification
|
||
|
||
### 8.1 Steps
|
||
|
||
A verifier is given the receipt and, where it has them, the batch's reports: each report's
|
||
id, its time and the task it was uploaded to. It runs these steps and stops at the first
|
||
refusal, which it reports with the reason and where:
|
||
|
||
1. **The receipt** reads (§7.2), at `receipt`.
|
||
2. **The task** reads as a TaskConfiguration and meets §4.3, at `task`: `task_invalid`,
|
||
`measure_unknown`, `measure_mismatch`, `floor_too_low`, `aggregator_alone`.
|
||
3. **The collector**: `signer` is the `collector` key of `task_info`, otherwise
|
||
`wrong_signer`; the signature verifies (`bad_key`, `bad_signature`), at `receipt`.
|
||
4. **The collection** reads as a CollectionJobReq with a time-interval query of a duration of
|
||
at least 1, an empty aggregation parameter and no extension (Prio3's only aggregation
|
||
parameter is the empty one, VDAF §7.2.3): otherwise `collection_invalid`, at
|
||
`collection`.
|
||
5. **The statements**: both are present, and their signers differ and their roles differ,
|
||
otherwise `aggregator_alone`; each reads (§6.1, `statement_invalid`); the first has role 2
|
||
and the leader's key and the second role 3 and the helper's, otherwise `wrong_signer`;
|
||
each signature verifies (`bad_key`, `bad_signature`), at `leader` or `helper`.
|
||
6. **The binding**: each statement's `task_id` is the task's id (§4.2) and its `collection`
|
||
the SHA-256 of the receipt's `collection`, otherwise `task_mismatch`.
|
||
7. **The batch**: each statement's batch has a duration of at least 1 and falls within the
|
||
query's interval, and the receipt's `interval` has a duration of at least 1 and falls
|
||
within the batch (DAP §4.1.1), otherwise `batch_invalid`.
|
||
8. **Agreement**: the two statements state one batch, one report count and one checksum,
|
||
otherwise `statements_disagree`.
|
||
9. **The floor**: the report count is at least the task's `min_batch_size`, otherwise
|
||
`batch_too_small`.
|
||
10. **The result**: each `agg_share` is an aggregate share of the task's VDAF (its length,
|
||
and each element below the field's modulus), otherwise `share_invalid`; the result is
|
||
the two shares unsharded (VDAF §7.2.5), otherwise `result_mismatch`.
|
||
11. **The reports**, where given: each was uploaded to the receipt's task, each id appears
|
||
once, each time falls within `interval`, and their number and checksum are the
|
||
statements', otherwise `batch_mismatch`, at `reports`.
|
||
|
||
### 8.2 Outcome
|
||
|
||
A receipt that passes is **accepted**: its digest; the task id, the measure, the VDAF and its
|
||
configuration, the scope, the time precision and the minimum batch size; the batch and the
|
||
interval; the report count; the result; and the leader's, the helper's and the collector's
|
||
keys. What was summed is the measure's VDAF over that task's reports, over the report count
|
||
the two aggregators each signed, by the aggregators the task named.
|
||
|
||
An accepted receipt shows that two keys each signed its share of the batch and that the
|
||
result is their sum. It does not show that the aggregators ran verification honestly: DAP
|
||
assures a correct result only when both do (DAP §8). A statement makes each aggregator's
|
||
part attributable to its key.
|
||
|
||
## 9. Where aggregation replaces suppression
|
||
|
||
### 9.1 Event and stream reporting
|
||
|
||
[delivery-telemetry](delivery-telemetry.md) §8 suppresses a published cell of fewer than 50
|
||
sessions (its §8.4); the service holds each session's observations and refrains. Under this
|
||
extension the service never holds a session's measures: it holds aggregates, each over at
|
||
least the task's minimum batch size, enforced by each aggregator (§5.2) and checked by every
|
||
reader (§8.1 step 9). For a figure a measure of §3 aggregates, a deployment **MAY** report it
|
||
by DAP instead of in an Event-mode report, and the published figure is the receipt's.
|
||
|
||
§8.4's suppression still applies where reporting is a stream rather than a sum after the
|
||
fact:
|
||
|
||
- Request-mode reports, which travel with each request to the host that serves it
|
||
(delivery-telemetry §8.3): a host acts on them per request, and a deployment that keeps
|
||
them to the delivery class does so with delivery-only Request mode (delivery-telemetry
|
||
§4.3);
|
||
- Event-mode reports a service consumes as they arrive, for one session's playback while it
|
||
plays, which no batch can wait for;
|
||
- a figure no measure of §3 aggregates.
|
||
|
||
### 9.2 A room's cell
|
||
|
||
A room's clients **MAY** report their sessions' quality of experience under this extension
|
||
instead of in the room's logs document ([live-metering](live-metering.md) §6.1). The room's
|
||
`outcomes`, `stalls` and `durations` tasks have the room id as their `scope`. Over one batch,
|
||
the cell of live-metering §4.7 is, from the three accepted receipts of that batch and scope:
|
||
|
||
| cell field | from |
|
||
|---|---|
|
||
| `started`, `start_failed`, `exit_before_start`, `vst_count` | `outcomes`, elements 0 to 3 |
|
||
| `stalls` | `stalls` |
|
||
| `stall_ms`, `engaged_ms`, `vst_ms` | `durations`, elements 0 to 2 |
|
||
|
||
each with the report count of its receipt. A set of receipts that is not one of each of the
|
||
three measures, or that names more than one scope, a zero scope, or more than one batch, is
|
||
refused `cell_mismatch`. Within the clamps of §3.2 these are the figures live-metering §4.7
|
||
computes from the same sessions' logs. The room-session receipt then names no logs (`qoe`
|
||
zero) and its cell is suppressed, as that section requires of a receipt with fewer than 50
|
||
logs; the room's cell is the aggregate receipts', whose floor the aggregators enforce.
|
||
|
||
## 10. What a report reveals
|
||
|
||
The key classes of delivery-telemetry §3 (its §3.4) state, for each key of the CMCD mapping,
|
||
what it reveals and to whom. A DAP report reveals to each aggregator its id, its time, its length, which task it
|
||
belongs to and whether its proof verified, and to no party its measurement (§5.4). This
|
||
extension sets no consent requirement for the service's own mechanics: using a distributor's
|
||
WAI service is the person's agreement to how that service works, its aggregation among them.
|
||
Where a regime asks for a lawful basis for processing quality-of-experience data, the service
|
||
records it in a `lawful-basis` claim ([jwp-receipts](jwp-receipts.md) §2.4.6).
|
||
|
||
## 11. Reasons
|
||
|
||
The codes this extension adds, each with where it is raised:
|
||
|
||
| reason | raised at |
|
||
|---|---|
|
||
| `task_invalid` | §4.3, §8.1 step 2 |
|
||
| `measure_unknown` | §4.3, §8.1 step 2 |
|
||
| `measure_mismatch` | §4.3, §8.1 step 2 |
|
||
| `floor_too_low` | §4.3, §8.1 step 2 |
|
||
| `wrong_signer` | §8.1 steps 3 and 5 |
|
||
| `collection_invalid` | §8.1 step 4 |
|
||
| `statement_invalid` | §6.1, §8.1 step 5 |
|
||
| `aggregator_alone` | §4.3, §8.1 step 5 |
|
||
| `task_mismatch` | §8.1 step 6 |
|
||
| `batch_invalid` | §8.1 step 7 |
|
||
| `statements_disagree` | §8.1 step 8 |
|
||
| `batch_too_small` | §8.1 step 9 |
|
||
| `share_invalid` | §8.1 step 10 |
|
||
| `result_mismatch` | §8.1 step 10 |
|
||
| `batch_mismatch` | §8.1 step 11 |
|
||
| `cell_mismatch` | §9.2 |
|
||
| `measurement_key_in_delivery` | delivery-telemetry §4 (its §4.3) |
|
||
|
||
A verifier of this extension also raises `truncated`, `malformed` and `unknown_version` for a
|
||
receipt that does not read, and `bad_key` and `bad_signature` for its signatures. The
|
||
reference aggregators raise DAP's report errors (`draft-ietf-ppm-dap-20` §4.1: among them
|
||
`vdaf_verify_error`, `hpke_decrypt_error`, `report_replayed`) and its problem types (§3.6
|
||
there: among them `invalidBatchSize`, `batchMismatch`, `batchOverlap`, `batchInvalid`), as
|
||
DAP names them.
|
||
|
||
## 12. Conformance
|
||
|
||
[`live-conformance/aggregate/`](../live-conformance/aggregate/) holds:
|
||
|
||
- `vdaf-22/`: the Prio3Count, Prio3Sum, Prio3SumVec and Prio3Histogram test vectors and the
|
||
XofTurboShake128 vector of `draft-irtf-cfrg-vdaf-22` Appendix C, as published at the
|
||
commit its [TestVectors] reference names;
|
||
- `vectors.json`, written from fixed seeds by `wai_telemetry_aggregate_vectors gen`:
|
||
- the keys: each aggregator's and the collector's HPKE configuration and private key, the
|
||
three statement keys and a stranger's, and the seed of the verification keys (test
|
||
material from fixed seeds, as every corpus here);
|
||
- fifty sessions' metrics, and a **flow** for each measure: its task, the reports (fifty
|
||
sessions, one with an out-of-range measurement, and in one flow a report replayed and in
|
||
another one whose helper share does not decrypt), the upload request and the reports it
|
||
refused, the aggregation job's request and response with each report's outcome, the
|
||
collection request, the aggregate share request, the helper's aggregate share, the
|
||
collection response, both sealed statements and the result; the `outcomes` flow also
|
||
holds the collections each aggregator refuses (a batch below the minimum, a checksum not
|
||
the batch's, a batch past the query, a batch collected twice);
|
||
- **receipts**: the four flows' receipts, accepted, and refused ones, each a control of one
|
||
defect: every reason of §11 the receipt raises, the reading and signature codes, a
|
||
receipt signed for a task the reports were not uploaded to, a batch below the minimum,
|
||
a leader alone and a leader signing as both aggregators;
|
||
- **cells**: the room's cell from its three receipts, with the plaintext cell of
|
||
live-metering §4.7 over the same sessions, which it equals, and a set of receipts of two
|
||
scopes;
|
||
- **delivery-only** reports (delivery-telemetry §4.3): observations with their
|
||
delivery-only reports and header shards, and reports that carry a key past the delivery
|
||
class.
|
||
|
||
A conforming verifier gives every vector its outcome. `wai_telemetry_aggregate_vectors
|
||
verify` and the test `tests/telemetry_aggregate.rs` check that `vectors.json` regenerates
|
||
byte for byte, that `vdaf-22/` is the files as published and that `wai-rs` passes every one,
|
||
that every receipt, cell and delivery-only report gets its outcome, that every reason of §11
|
||
and every reused code refuses a vector, that §11 lists exactly the reference's codes, and
|
||
that delivery-telemetry §3 (its §3.4) classes exactly the keys the reference does.
|
||
`live-conformance/telemetry_aggregate_verify.py` is the second implementation, written from
|
||
this text and the drafts it cites: it shares no code with `wai-rs`; it evaluates Prio3 itself
|
||
(the fields, TurboSHAKE128 and the XOF, the FLP in the Lagrange basis) and passes the draft's
|
||
vectors; it opens every report with its own HPKE (X25519, HKDF-SHA256 and AES-128-GCM written
|
||
from RFC 7748, RFC 5869 and the AES and GCM specifications), recombines each measurement and
|
||
requires it to be its session's, runs both aggregators' verification and requires the
|
||
aggregation job's request and response byte for byte, recomputes every batch bucket, every
|
||
refusal, every aggregate share and every result; verifies every receipt and cell with WAI's
|
||
strict Ed25519 rule (`claim-conformance/_ed25519.py`); reads the key classes from
|
||
delivery-telemetry §3 (its §3.4) and this text's reasons from §11; and requires every mutated copy of
|
||
the corpus to fail.
|
||
|
||
## 13. Security and privacy considerations
|
||
|
||
- **Non-collusion.** Privacy rests on the two aggregators not pooling their shares
|
||
(§5.4). A statement attributes each share to a key, so a reader sees which two parties a
|
||
result depended on; it cannot see whether they colluded.
|
||
- **Correctness.** The range proof stops a sink from poisoning a sum with an out-of-range
|
||
value (§3.3); it does not stop a sink from reporting a false value in range (DAP §2.2). An
|
||
aggregator can still corrupt a result by an arbitrary amount with a wrong share; the
|
||
statement makes that share its signed statement.
|
||
- **Small batches.** A batch at the floor reveals what fifty sessions sum to. A deployment
|
||
that publishes overlapping figures across tasks of one population should consider what
|
||
their differences reveal; DAP collects each batch once (§2.3 there), which this extension
|
||
keeps, and §8.5 there describes noise an aggregator can add.
|
||
- **Linking a session's reports.** A session sends one report to each task, at once; an
|
||
aggregator that sees their arrival together can link them, though not their values. A
|
||
deployment that wants them apart sends them at different times or through a proxy (DAP
|
||
§8.4).
|
||
- **Sybil reports.** Reports are not authenticated here; one party can submit many, and so
|
||
approach a batch's floor with its own (DAP §8.1). The floor is a floor of reports, not of
|
||
people.
|
||
- **Verification key.** A sink that knew the verification key could forge a report that
|
||
passes verification (DAP §8.6.2); it is the aggregators' alone.
|