Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions

wai: a carousel schedule extends, what a receiver needs rides in band, a node schedules its own instance, and Raptor #658

Closed
dcharlot wants to merge 4 commits from wai/carousel-monotone-inband into main
pull from: wai/carousel-monotone-inband
merge into: Transaction-Science:main
Conversation 0 Commits 4 Files changed 30 +14779 -3030
30 changed files with 14779 additions and 3030 deletions
Download patch file Download diff file Expand all files Collapse all files

5
wai/SPEC.md
View file

@ -1801,14 +1801,15 @@ multicast and the applications of hybrid broadcast receivers:
And for **when** each object passes on a bearer with no return path — `wai.bearer.carousel`: a
publisher's signed schedule, a jwp-receipts claim of class `schedule` beside the group receipt
and the instance's transmission claim, that names for each object its forward error correction
(RaptorQ, RFC 6330, or Compact No-Code, RFC 5445) with that code's object transmission
(RaptorQ, RFC 6330; Raptor, RFC 5053; or Compact No-Code, RFC 5445) with that code's object transmission
information, the repair symbols each pass adds, whether a later pass repeats its symbols or sends
new ones, and the times and count of its passes. A verifier holds the parameters to each code's
bounds and the entries to the group, its objects' content hashes and, where it holds them, their
lengths; sets aside schedules that contradict each other, or whose instance is undeclared or
two-way; and derives, for a loss bound stated in parts per million, the pass after which a
receiver whose losses stay within it holds each object, whatever symbols it loses: a RaptorQ
block at K′ + h distinct symbols, the count RFC 6330 states its decoding-failure bound at, and
block at K′ + h distinct symbols, the count RFC 6330 states its decoding-failure bound at, a Raptor
block at K and a margin this text sets by measurement, with no bound, as RFC 5053 states none, and
a symbol a repeating pass missed counted as missed again. That is the deadline a one-way bearer
can promise, checked with no receiver; a statement that an object or a group is held by an
instant holds or is refused (`availability_after_deadline`, `availability_not_within_schedule`,

32
wai/changes/2026-11-05-carousel-extension-in-band-cache-schedule.md Normal file
View file

@ -0,0 +1,32 @@
### A later schedule adds passes, what a receiver needs rides on the bearer, and a node schedules its own one-way sending
- **Adding passes is no contradiction** ([extensions/carousel.md](extensions/carousel.md) §4.5, §6.2 step 4): along
their chain, a later schedule of one instance that keeps every earlier entry, with at least its passes at the same
instants sending the same symbols, extends the one before it, and may add objects. The last stands and the earlier
are labelled `extended`. An object available stays available at the same pass, a statement that held still holds,
and a plan listens to what it listened to and completes where it completed. A schedule that drops an object, or
withdraws, moves or changes a pass, still sets aside every schedule of its instance (`schedule_contradicted`). A
timeliness plan reads the same schedule, and labels it the same. A single pass's period, which is not read, may be
given by the schedule that extends it.
- **In-band repetition** (carousel §8.1, §8.2): on an instance whose schedule stands, the publisher sends on the
instance the group receipt, its claims from the head of their chain to the instance's last schedule, and the head
of its origin's key log, at least once per the schedule's interval: the least period of its entries of two passes or
more, or, where every entry has one pass, the least duration. A verifier given what a receiver acquired checks that
it suffices: `inband_chain_incomplete`, `inband_not_scheduled` or `inband_key_log_missing` otherwise, and the
interval as `inband_ms`. A log on the bearer still anchors nothing.
- **A node's schedule** ([extensions/cache-authority.md](extensions/cache-authority.md) §5.6, carousel §4.3): a node
states its plan for its own cache instance in a `schedule` claim signed by its own key, which needs `schedule` from
its operator. The verdict of cache-authority §7 now reads schedules: a publisher's by carousel §6.2 step 4, a node's
beside its own instance, and another key's schedule of that instance is set aside
(`cache_schedule_signer_mismatch`) and contradicts nothing. A statement of carousel §6.4 offered with a loss bound
and an overhead is held to the schedule that stands there. A node's schedule credits nothing, and carousel's verdict
and timeliness' plan read the publisher's schedules alone.
- **A §9 line with a ladder** (cache-authority §9): the lines of a timeliness entry derived from all three kinds of
held-at statement read every rung of a ladder, and state its list of instants as timeliness §6.3 does.
- **Breaking:** `api:rust:wai::carousel::ClaimVerdict::Extended` and `api:rust:wai::timeliness::ClaimVerdict::Extended`: a
schedule a later one extends is a new label, not `stands`. A match over either enum takes the new arm.
- **Breaking:** `api:rust:wai::cache_authority::ClaimVerdict::Stands` and `api:rust:wai::cache_authority::ClaimVerdict::Extended`:
the cache verdict now reads schedules. A match over the enum takes the new arms; a schedule that was `context_mismatch`
there now gets one of them.
- **Breaking:** `api:rust:wai::cache_authority::Inputs::held` and `api:rust:wai::cache_authority::Verdict::held`: the
statement of carousel §6.4 the verdict holds to a schedule. A caller that builds `Inputs` field by field sets `held: None`.

17
wai/changes/2026-11-05-carousel-raptor-rfc-5053.md Normal file
View file

@ -0,0 +1,17 @@
### A schedule names Raptor (RFC 5053), the code MBMS download uses
- **The code** ([extensions/carousel.md](extensions/carousel.md) §4.1, §5.5): an entry's `fec` may be `raptor` (FEC
Encoding ID 1), with RFC 5053's F, T, Z, N and Al. They are held to its fields and to its block sizes: from 4 source
symbols, the fewest its §5.7 indexes, to 8,192, with ESIs in 16 bits. A receiver of 3GPP MBMS download supports the
code (TS 26.346 §7.2.2), so a schedule of such a session can state what is sent.
- **What is held** (carousel §5.3, §6.3, §7): RFC 5053 states no bound on a decoder's failure, and an independent
implementation of it failed on 14 to 31 blocks in 100 at *K* + 2. So a Raptor block is held at *K* and a margin this
text sets by measurement, 8, 16 or 28 symbols at overhead 0, 1 or 2, and `failure_one_in` is `null`. `tests/carousel.rs`
decodes each Raptor object a case holds available with that implementation, a development dependency, from what a
receiver within the case's loss bound holds, and holds a fixed sample of decoders to the measurement.
- **`no-code` stays the default** (carousel §11), where the IETF's disclosures on RFC 5053 stay stated for implementers.
- **Breaking:** `record:corpus:wai/live-conformance/carousel/vectors.json`: the case `fec_unknown` named `raptor`, which is
now a code, and names `lt` instead. A verifier takes `raptor` as a code at jwp-receipts §3.1 step 2.
- **Breaking:** `api:rust:wai::claim_receipt::Fec::Raptor`, `api:rust:wai::claim_receipt::Oti::Raptor`,
`api:rust:wai::claim_receipt::schedule::Fec::Raptor` and `api:rust:wai::claim_receipt::schedule::Oti::Raptor`: the new
code. A match over either enum takes the new arm.

28
wai/claim-conformance/claim_verify.py
View file

@ -1568,9 +1568,12 @@ def fabric_rules(f, joules):
SCHEDULE_ENTRY = ["leaf_index", "content_hash", "fec", "oti", "repair", "esi", "first_ms", "period_ms",
"duration_ms", "count"]
SCHEDULE_OTI = {"raptorq": ["transfer_length", "symbol_size", "source_blocks", "sub_blocks", "alignment"],
"raptor": ["transfer_length", "symbol_size", "source_blocks", "sub_blocks", "alignment"],
"no-code": ["transfer_length", "symbol_size", "max_block_length"]}
K_PRIME_MAX = 56_403 # RFC 6330 §5.1.2
RAPTORQ_MAX_F = 942_574_504_275 # RFC 6330 §3.3.2 as its erratum 5548 corrects it
RAPTOR_K_MAX = 8_192 # RFC 5053 §5.1.2
RAPTOR_K_MIN = 4 # RFC 5053 §5.7: the fewest source symbols with a systematic index
LAST_INSTANT_MS = 253_402_300_799_999 # 9999-12-31T23:59:59.999Z
@ -1579,7 +1582,7 @@ def ceil_div(a, b):
def schedule_blocks(e):
"""carousel §5.1, §5.2: an entry's source blocks as [(K, how many)], the larger size first
"""carousel §5.1, §5.2, §5.5: an entry's source blocks as [(K, how many)], the larger size first
and a size no block has left out; None when its code does not admit its parameters."""
o = e["oti"]
if e["fec"] == "raptorq":
@ -1594,6 +1597,18 @@ def schedule_blocks(e):
kl, ks = ceil_div(kt, z), kt // z
zl = kt - ks * z
sizes = [(kl, zl), (ks, z - zl)]
elif e["fec"] == "raptor":
f, t, z, n, al = (o[k] for k in SCHEDULE_OTI["raptor"])
if not (1 <= f < 1 << 45 and 1 <= al <= 255 and 1 <= t <= 65_535 and t % al == 0
and 1 <= z <= 65_535 and 1 <= n <= 255 and n <= t // al):
return None
kt = ceil_div(f, t)
if kt // z < RAPTOR_K_MIN or ceil_div(kt, z) > RAPTOR_K_MAX:
return None
# RFC 5053 §5.3.1.2: (KL, KS, ZL, ZS) = Partition[Kt, Z].
kl, ks = ceil_div(kt, z), kt // z
zl = kt - ks * z
sizes = [(kl, zl), (ks, z - zl)]
else:
ln, es, bl = (o[k] for k in SCHEDULE_OTI["no-code"])
if not (1 <= ln < 1 << 48 and 1 <= es <= 65_535 and 1 <= bl < 1 << 32):
@ -1610,8 +1625,8 @@ def schedule_blocks(e):
def schedule_bytes(e, blocks):
"""carousel §5.4: count · Σ m · T for RaptorQ, count · L for Compact No-Code."""
if e["fec"] == "raptorq":
"""carousel §5.4: count · Σ m · T for RaptorQ and Raptor, count · L for Compact No-Code."""
if e["fec"] != "no-code":
return e["count"] * sum(c * (k + e["repair"]) for k, c in blocks) * e["oti"]["symbol_size"]
return e["count"] * e["oti"]["transfer_length"]
@ -1624,7 +1639,8 @@ def schedule_entry_rules(e):
named = e["repair"] == 0 and e["esi"] == "repeat"
else:
m = blocks[0][0] + e["repair"]
named = (m if e["esi"] == "repeat" else e["count"] * m) <= 1 << 24
field = 16 if e["fec"] == "raptor" else 24 # RFC 5053 §3.1, RFC 6330 §3.2
named = (m if e["esi"] == "repeat" else e["count"] * m) <= 1 << field
need(named, 4, "schedule_symbols_invalid")
c, p, d = e["count"], e["period_ms"], e["duration_ms"]
formed = (c >= 1 and d >= 1 and (c < 2 or p >= d) and e["first_ms"] + (c - 1) * p + d <= LAST_INSTANT_MS
@ -2479,8 +2495,8 @@ def carousel_claims(path):
doc = json.load(f)
fail = n = 0
checks = []
for case in doc["verdicts"]:
if case["outcome"]["outcome"] != "verdict":
for case in doc["verdicts"] + doc["inband"]:
if case["outcome"]["outcome"] == "refuse":
continue
for wire, stated in zip(case["claims"], case["outcome"]["claims"]):
m = re.fullmatch(r"reject step=(\d) reason=(\S+)", stated)

4
wai/extensions/bearer-receipts.md
View file

@ -466,7 +466,9 @@ reception reporting (TS 26.346 clause 9.4) lists each file received by its `file
`Content-MD5` where the table carried it, and is unsigned: it names files, not objects of a
group, and a reconciliation travels beside it. TS 26.517 defines no reception report, and
TS 26.502 leaves one for further study (clause 4.2.5). A session whose receivers report
declares `multicast`; one without a path back, `unidirectional`.
declares `multicast`; one without a path back, `unidirectional`. A receiver of MBMS download
supports the Compact No-Code and Raptor schemes (TS 26.346 clause 7.2.2), which a schedule of such a
session states as `no-code` and `raptor` ([carousel](carousel.md) §4.1).
### 9.7 MoQ over multicast (`draft-ramadan-moq-multicast-00`)

131
wai/extensions/cache-authority.md
View file

@ -15,7 +15,8 @@
> with the caches' instances beside the publisher's, crediting a cache per instance and only for
> what it says it sent (§7); and counts, of a population of receiver keys stated for a region, how
> many held an object by a deadline on their own signed word, with a miss rate in parts per million
> held to a target (§8). WAI defines no transport. The reference implementation is the
> held to a target (§8). A node states its plan for its own instance in a [carousel](carousel.md)
> `schedule` claim signed by its own key (§5.6). WAI defines no transport. The reference implementation is the
> `cache_authority` module of `wai-rs` (feature `cache_authority`), with the class's body in
> `claim_receipt::bearer`; the corpus is
> [`live-conformance/cache-authority/`](../live-conformance/cache-authority/), and its second
@ -30,9 +31,10 @@
## 1. Scope
In scope: the authorities a node that serves from what it holds signs under, who gives them, and
how a verifier places a server key (§4); the cache-transmission claim (§5); what a receiver of a
peer-served object does (§6); the bearer verdict with a cache's instances (§7); and the count of a
stated population's receivers that held an object by a deadline (§8).
how a verifier places a server key (§4); the cache-transmission claim and a node's schedule of its
own instance (§5); what a receiver of a peer-served object does (§6); the bearer verdict with a
cache's instances and their schedules (§7); and the count of a stated population's receivers that
held an object by a deadline (§8).
Out of scope: a transport; how a node chooses what to keep, or an asker whom to fetch from, which
are [holdings](holdings.md) §5.4's; a record of service beside settlement's acknowledgement and
@ -95,7 +97,8 @@ another key signed. Under it the key signs its holdings records (holdings §4) a
declarations of what it served (settlement §4), and is the `upstream` of the acknowledgements its
receivers sign (settlement §3). A key that also holds `cache-transmission` signs
cache-transmission claims (§5), each declaring a bearer instance of its own and the objects of one
group it sent over it. That is all either covers.
group it sent over it, and one that holds `schedule` as well schedules those instances (§5.6). That
is all either covers.
### 4.2 What neither lets a key claim
@ -117,9 +120,9 @@ when it stores and serves byte for byte. jwp-receipts §4.3 binds it either way.
### 4.3 Who gives them
The **node's operator's origin** lists, in its key log, the nodes it runs with `cache/serve`, and
with `cache-transmission` those that declare instances of their own, as it lists its relays with
`live/relay` and `delivery-gap`:
The **node's operator's origin** lists, in its key log, the nodes it runs with `cache/serve`, with
`cache-transmission` those that declare instances of their own, and with `schedule` those that
schedule them, as it lists its relays with `live/relay` and `delivery-gap`:
- **Integrity needs no grant.** A node changes no byte, so jwp-receipts §3 verifies what it serves
against the publisher's receipts, whoever it is. `edge/derive` comes from the source's origin
@ -292,6 +295,47 @@ nor `delivery`. Where `energy` is not `null`, its figure is a second item that n
`cache-transmission` and `energy/<class>`, as a transmission's does (bearer-receipts §4.4). The
claim's signer is placed as a server by §4.4, reported beside.
### 5.6 A schedule of a cache instance
A node that sends over a cache instance to a plan **MAY** state the plan in a `schedule` claim
(carousel §4), so that a receiver of its instance plans as a receiver of the publisher's does and
turns its radio off between the passes that add nothing (carousel §7). Its `subject` is the group
receipt's link hash and its `bearer_id` the instance's, and it is signed by the node's key, never
the group's. Its body, rules and blocks are carousel §4.1, §4.2 and §5's; a later schedule extends it
as carousel §4.5 says; and its availability and the statements held to it are carousel §6.3 and
§6.4's. It attests that the node scheduled the passes it names. It does not attest that they were
sent, which is its cache-transmission claim's, that any receiver heard them, or that the node had any
right of the publisher's to send them (§5.4).
**Against the group.** At jwp-receipts §3.1 step 6 in the verdict of §7, a schedule signed by the
group's key is the publisher's and is checked by carousel §4.3. One signed by another key is a
node's and is checked by carousel §4.3 but for its step 2, raising that text's codes at its other
steps. Carousel's verdict and timeliness' plan read the publisher's schedules alone.
**Where it stands.** At §7.2 step 4, in this order:
1. every node's schedule of an instance is set aside `bearer_contradicted` when an accepted
transmission claim has that `bearer_id`, whether its declarations stand or were set aside, or when
the instance is a contradicted cache instance: it is not the node's to plan;
2. every node's schedule of an instance is set aside `bearer_not_declared` when no cache instance
stands with that `bearer_id`;
3. a schedule whose signer is not the instance's server is set aside
`cache_schedule_signer_mismatch`: a node schedules only the instance it declares, and another
key's schedule of it contradicts nothing;
4. of the server's schedules, in the order of its chain, when one does not extend the one before it
(carousel §4.5) every one of them is set aside `schedule_contradicted`, and the instance itself
stands. A node that withdraws, moves or changes a pass has stated neither plan.
Otherwise the last of them **stands**, and each before it with another body is **extended**.
A cache-transmission claim says what was sent and comes after it (§5), and a schedule says what will
be sent and may come before, so either may come first in the node's chain. A schedule may name
objects the node's claim does not. **It credits nothing**: a cache is credited only for what its
claim says it sent (§7.2 step 6), and a schedule changes no count, no status and no placement.
**Authority.** Confirmed under jwp-receipts §7, the claim needs `schedule` (carousel §4.4), which the
node's operator gives beside `cache-transmission` (§4.3).
## 6. A peer-served object at the receiver
- **The verdict is unchanged.** The receiver verifies the object under its object receipt and the
@ -313,7 +357,8 @@ claim's signer is placed as a server by §4.4, reported beside.
Bearer-receipts §6.1's — the group receipt, the object receipts with envelopes where held, the
claims in each signer's chain order, optionally a sink's own class and instance, a statement
offered and `count_only` — with cache-transmission claims among the claims, and the heads of §4.4.
offered and `count_only` — with cache-transmission and schedule claims among the claims, the heads of
§4.4, and optionally a statement of carousel §6.4 with its loss bound and overhead (§7.3).
A verifier that implements bearer-receipts and not this extension refuses a cache-transmission
claim (`unknown_class` before this registration, `context_mismatch` at bearer-receipts §6.2 step 3
@ -326,8 +371,8 @@ unknown class is reported, as jwp-receipts §2.4.4 requires, never counted.
with its reason, and nothing is derived from it.
2. **The claims**, in order: jwp-receipts §3.1 steps 1 to 5, then step 6: a transmission claim by
bearer-receipts §4.3, a reconciliation by bearer-receipts §5.3, a cache-transmission claim by
§5.3 here. A claim of any other class is `context_mismatch`. A refused claim takes no further
part.
§5.3 here, a schedule by carousel §4.3 or §5.6 here. A claim of any other class is
`context_mismatch`. A refused claim takes no further part.
3. **The publisher's declarations**: bearer-receipts §6.2 step 4.
4. **The caches' instances**: the accepted cache-transmission claims, by `bearer_id`. Every claim of
an instance is set aside `bearer_contradicted` when:
@ -339,6 +384,9 @@ unknown class is reported, as jwp-receipts §2.4.4 requires, never counted.
Two with the same signer and body are one, whatever their `issued_at` and `seq`. Otherwise the
instance **stands**; its **server** is the claims' signer, placed by §4.4.
Then the schedules: those signed by the group's key by carousel §6.2 step 4, over step 3's
declarations, and a node's by §5.6.
5. **The reconciliations**, each accepted one in order, are counted as bearer-receipts §6.2 step 5
counts them, over the publisher's declarations and the caches' instances that stand. One is set
aside, and counted nowhere, at the first of these that holds:
@ -366,8 +414,8 @@ unknown class is reported, as jwp-receipts §2.4.4 requires, never counted.
A claim's label is bearer-receipts §6.2's (`declared`, `counted`,
`joined reason=reconciliation_duplicate_sink`, `set-aside reason=…`,
`reject step=N reason=…`), and `cache-declared` for a cache-transmission claim whose instance
stands.
`reject step=N reason=…`), `cache-declared` for a cache-transmission claim whose instance
stands, and `stands` or `extended` for a schedule.
### 7.3 Status, statements and a sink's own instance
@ -385,6 +433,13 @@ stands.
declaration stands for it, `declared` when the classes agree and `bearer_contradicted` when they
do not; where a cache instance stands for it, `cache-declared` when the classes agree and
`bearer_contradicted` when they do not; otherwise `bearer_not_declared`.
- **A statement held to a schedule**: carousel §6.4's, that object *i* or the group is held by *t* on
an instance, offered with a loss bound *L* and an overhead *h*, is checked against the schedule that
stands on that instance, the publisher's or a node's. It holds, or it is refused with carousel
§6.4's reason, `object_not_scheduled` where no schedule stands there, and `loss_bound_invalid` or
`overhead_invalid` where *L* or *h* is out of range, as carousel §6.2 checks them; only the
statement is refused. On a node's instance it speaks for the node's schedule, and says no more than
carousel §6.4 says of the publisher's.
Never delivered on the publisher's word (bearer-receipts §6.3) holds unchanged, and never
delivered on a node's.
@ -555,9 +610,10 @@ a loss bound; a relying party may hold both. The asker's plan of whom to fetch f
- **holdings**: a holder signs with the key its operator lists to be placed (§4.5; holdings §9). A
record is the third kind of held-at statement §8.3 reads.
- **carousel**: availability (carousel §6.3) is the publisher's plan for a receiver within a loss
bound; a node repairs what that plan did not cover, and §8 counts the outcome. A cache instance
has no schedule: a schedule is signed by the group's key (carousel §4.3 step 2), and a
cache-transmission claim and a schedule share no instance (§7.2 step 4).
bound; a node repairs what that plan did not cover, and §8 counts the outcome. A node's plan for
its own instance is a schedule of its own (§5.6), which shares no instance with the publisher's: a
publisher's schedule stands beside a publisher's declaration, and a node's beside its own
cache-transmission claim (§7.2 step 4).
- **bearer-receipts**: the verdict extends its §6.2 here, as carousel's does. Its §7 bound applies
to each cache instance, and its §8 holds for a node's key: a receiver with no path back places a
node only by heads provisioned in advance, and a key log arriving over the node's instance
@ -579,17 +635,18 @@ a loss bound; a relying party may hold both. The asker's plan of whom to fetch f
the statement it answers (its §6.4) read holdings records alone. A verifier that holds this
text's inputs (§7.1, §8.2) and a timeliness claim **MAY** derive the lines of the claim's entry
for object *i* by timeliness §6.3 from held-at statements of all three kinds:
- the holdings records' exactly as timeliness §6.2 step 5 derives them, for the primary and its
fallback whatever envelopes the verifier holds, with a line for each holder with a record that
stands and has an entry for the group or the fallback's group;
- the holdings records' exactly as timeliness §6.2 step 5 derives them, for the primary and each
rung of its fallback whatever envelopes the verifier holds, with a line for each holder with a
record that stands and has an entry for the group or a rung's group;
- those §8.3 derives from acknowledgements and counted reconciliations, for every object of the
group the verifier holds `delivered-verified`, the primary and a fallback in the group among
them, with a line for each key with one. A fallback in another group is held by records alone,
group the verifier holds `delivered-verified`, the primary and a rung in the group among
them, with a line for each key with one. A rung in another group is held by records alone,
since this text judges one group.
The claim is checked as §8.2 step 6 checks a deadline's, and a claim refused there, or with no
entry for *i*, derives no lines (`deadline_not_stated`). A fallback rendition is another object
with its own content hash, so it has its own tally and is never a hit for the primary.
entry for *i*, derives no lines (`deadline_not_stated`). A rung is another object with its own
content hash, so it has its own tally and is never a hit for the primary. A line's form is
timeliness §6.3's: a ladder written as an array states the least instant of each rung.
- **rung-price**: it owns price, direction and a price list's `footprint`, in the region codes of
its §2, which §8.1 uses. It carries no deadline and no miss target, so the tally takes the
deadline, the target and the population as stated inputs and reports the population's digest (and
@ -628,6 +685,7 @@ The codes this extension adds, each with where it is raised:
| `cache_transmission_leaf_count_mismatch` | §5.3 step 3 |
| `cache_transmission_objects_not_held` | §5.3 step 4 |
| `cache_transmission_root_mismatch` | §5.3 step 4 |
| `cache_schedule_signer_mismatch` | §5.6 |
| `population_invalid` | §8.2 |
| `target_invalid` | §8.2 |
| `object_not_verified` | §8.2 |
@ -637,7 +695,9 @@ The extension also raises `key_revoked`, `key_retired` and `authority_not_grante
§7.7), at the levels §4.4 gives them, and `key_log_not_held` (deriving-edge §9), at §4.4;
`bearer_contradicted`, `bearer_not_declared`, `sink_not_confirmed`,
`reconciliation_duplicate_sink`, `delivery_not_reconciled` and `delivery_exceeds_reconciled`
(bearer-receipts §10) at §7.2 and §7.3; `self_acknowledged` (settlement §11) at §7.2 step 5;
(bearer-receipts §10) at §7.2 and §7.3; a schedule's codes of carousel §4.2 and §4.3 at jwp-receipts
§3.1 steps 4 and 6, `schedule_contradicted` and `schedule_on_bidirectional` (carousel §9) at §7.2 step
4, and carousel §9's statement and bound codes at §7.3; `self_acknowledged` (settlement §11) at §7.2 step 5;
`object_out_of_range` (holdings §7), `context_invalid` (jwp-receipts §3.1) and
`deadline_not_stated` (timeliness §9) at §8.2; settlement §3.6's codes and `record_missing` for an
acknowledgement, holdings §7's codes for a holdings record, and `held_before_group` (timeliness §9)
@ -656,8 +716,10 @@ regions are ISO 3166-1's user-assigned codes. It has:
sent from a group, every object with no object receipt held, the last object of a group of nine,
labelled leaves, a metered instance, a chained second claim; a chain broken and a parent not
held; one negative control for each rule of §5.2 and each step of §5.3, an unknown bearer class,
an extra member and an energy figure of 0; a context that is not a group receipt; and a claim with
no group, which stops at step 5.
an extra member and an energy figure of 0; a context that is not a group receipt; a claim with
no group, which stops at step 5; and a node's schedule (§5.6) and a publisher's, a node's chained
after its claim, and one of a node's refused at each of carousel §4.3's steps it keeps and at
§4.2 rule 3.3.
- **Placements**: each step of §4.4, the least origin, a serving role other than `cache/serve`, a
retired key at its `until` and a millisecond past it, a revocation beating a confirmation and
outliving another role, a head not held, a head under another origin, and a revocation beside a
@ -668,8 +730,11 @@ regions are ISO 3166-1's user-assigned codes. It has:
standing and contradicted; two caches on one instance; one claim twice and a cache contradicting
itself; each reason of §7.2 step 5; placements of every level beside unchanged counts, and a
retired node's two claims of one instance on either side of its `until`; a sink on a cache
instance; the energy bound before and after keys are counted, and unmetered; another class; and a
group that fails.
instance; the energy bound before and after keys are counted, and unmetered; another class; a
group that fails; and a node's schedule that stands before and after its claim, is extended, is
contradicted by its own node, is set aside for each step of §5.6 and credits nothing, a stranger's
beside it, a publisher's that stands and one on a node's instance, with a statement held to each
that holds, comes too late, is not within the schedule, has no entry, and is out of bounds.
- **Tallies**: a target met exactly and missed by one ppm, a rate rounded up; an acknowledgement
ending at the deadline and one and two microseconds past it; a reconciliation signed at the
deadline, a microsecond past it, below a second, with an offset and at second 60; evidence before
@ -687,7 +752,9 @@ regions are ISO 3166-1's user-assigned codes. It has:
derived.
- **Lines** (§9): from an acknowledgement and from a counted reconciliation; from a holdings record
whose object's envelope is not held; with a fallback in another group that a holder's record
alone names; and with a fallback in the group, held by an acknowledgement and a reconciliation.
alone names; with a fallback in the group, held by an acknowledgement and a reconciliation; and with
a ladder of a rung in the group and a rung in another, the first rung and the second each held
alone, and the first late with the second in time.
The population bound of 2<sup>20</sup> keys is carried by no case; both implementations check it
themselves.
@ -699,7 +766,8 @@ appears, and that §11 lists exactly the reference's codes.
`live-conformance/cache_authority_verify.py` is the second implementation, written from this text:
it shares no code with `wai-rs`; it verifies groups and objects with the JWP corpus's second
implementation (`jwp-conformance/jwp_verify.py`), claims through steps 1 to 5 with the claim
corpus's (`claim-conformance/claim_verify.py`), reconciliations with the bearer corpus's
corpus's (`claim-conformance/claim_verify.py`), schedules against their group and their extensions and
availability with the carousel corpus's (`live-conformance/carousel_verify.py`), reconciliations with the bearer corpus's
(`live-conformance/bearer_verify.py`), acknowledgements with the settlement corpus's
(`live-conformance/settlement_verify.py`), key logs with the signer corpus's
(`signer-conformance/signer_verify.py`), held-at statements with the timeliness corpus's
@ -710,7 +778,8 @@ checks in turn and requires the corpus to catch the weakened verifier.
## 13. Security and privacy considerations
- **A node's claim is its word.** A false one is evidence against its key.
- **A node's claim is its word.** A false one is evidence against its key, and so is a schedule of
passes it did not send.
- **A placement is as good as the heads named.** It is the relying party's choice of logs, held
under jwp-receipts §7.2 rules 1 to 3 only; a head that is not the head of an origin it anchored
(jwp-receipts §7.3) is its own say-so. A revocation the relying party has not retrieved is not in

282
wai/extensions/carousel.md
View file

@ -9,28 +9,34 @@
> transmission claim of [bearer-receipts](bearer-receipts.md) §4. From it a verifier derives,
> for each object, when a receiver whose losses stay within a stated bound holds it (§6): the
> deadline a one-way bearer can promise, with the decoding failure the FEC code bounds. A
> receiver derives the passes it must listen to and when it may turn its radio off (§7). WAI
> defines no transport and no code: the codes are RaptorQ (RFC 6330) and Compact No-Code
> (RFC 5445), and the schedule declares their parameters. The reference implementation is the
> receiver derives the passes it must listen to and when it may turn its radio off (§7). A later
> schedule adds passes to an earlier one without contradicting it (§4.5), and the receipts a
> receiver needs ride on the instance itself, so one that joins mid-carousel recovers them there
> (§8.1, §8.2). WAI
> defines no transport and no code: the codes are RaptorQ (RFC 6330), Raptor (RFC 5053) and Compact
> No-Code (RFC 5445), and the schedule declares their parameters. The reference implementation is the
> `carousel` module of `wai-rs` (feature `carousel`), with the class's body in
> `claim_receipt::schedule`; the corpus is
> [`live-conformance/carousel/`](../live-conformance/carousel/), and its second implementation
> is `live-conformance/carousel_verify.py`.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
>
> Texts read for this revision: RFC 6330 (RaptorQ) with its verified erratum 5548, RFC 5445
> (Compact No-Code) and RFC 5052 §9.1 (block partitioning). A clause number of one of them in
> this text is that RFC's. §11 also cites the IETF's IPR disclosures on RFC 5053, RFC 6330 and
> their drafts, read as the IETF datatracker shows them, and RFC 8179 §4.
> Texts read for this revision: RFC 6330 (RaptorQ) with its verified erratum 5548, RFC 5053 (Raptor),
> RFC 5445 (Compact No-Code), RFC 5052 §9.1 (block partitioning) and 3GPP TS 26.346 V19.3.0 §7.2.2 and
> §7.2.3. A clause number of one of them in this text is that text's. §11 also cites the IETF's IPR
> disclosures on RFC 5053, RFC 6330 and their drafts, read as the IETF datatracker shows them, and
> RFC 8179 §4.
## 1. Scope
In scope: the schedule claim (§4); how an object is divided into source blocks and symbols
under each code (§5); when an object is held by a receiver whose losses stay within a bound,
and the statements a verifier may check against it (§6); the passes a receiver listens to (§7).
In scope: the schedule claim and how a later one extends an earlier one (§4); how an object is
divided into source blocks and symbols under each code (§5); when an object is held by a receiver
whose losses stay within a bound, and the statements a verifier may check against it (§6); the
passes a receiver listens to (§7); and what a publisher repeats on the instance for a receiver
that joins mid-carousel, and whether what it acquired suffices (§8.1, §8.2).
Out of scope: encoding and decoding symbols, which are the codes' own (RFC 6330 §5, RFC 5445
§3); a transport, and how symbols are packed into packets; a model of a channel. Every
Out of scope: encoding and decoding symbols, which are the codes' own (RFC 6330 §5, RFC 5053 §5,
RFC 5445 §3); a transport, and how symbols are packed into packets; a model of a channel. Every
statement here holds for every pattern of loss within the stated bound (§2). Whether losses on a
channel are independent, and how often a bound is exceeded, is a judgement about that
channel, which a relying party makes and a schedule does not carry. Nothing here states
@ -57,8 +63,8 @@ and is not dispatchable.
pass, it fails to receive at most ⌊*L* · *m* / 1,000,000⌋, whichever they are. It is a bound
on symbols: a relying party whose bound is stated on packets carrying several symbols each
converts it.
- **Overhead** *h*: 0, 1 or 2, the encoding symbols beyond *K′* a RaptorQ block is held at
(§5.3).
- **Overhead** *h*: 0, 1 or 2, the encoding symbols beyond *K′* a RaptorQ block is held at. A
Raptor block is held at a margin beyond *K* that *h* selects (§5.3).
## 3. Why a schedule beside the receipt
@ -103,8 +109,8 @@ Each entry of `objects` has exactly these members:
|---|---|---|
| `leaf_index` | integer | The object's position in the group (jwp-receipts §2.2). |
| `content_hash` | 32 B hex | The object's content hash (jwp-receipts §2.1): what a receiver that decodes the object checks it against. |
| `fec` | string | The code: `raptorq` (RFC 6330, FEC Encoding ID 6) or `no-code` (RFC 5445, FEC Encoding ID 0). |
| `oti` | object | The code's object transmission information, as §5 reads it. For `raptorq`, exactly `transfer_length` (F), `symbol_size` (T), `source_blocks` (Z), `sub_blocks` (N) and `alignment` (Al), RFC 6330 §3.3.2 and §3.3.3. For `no-code`, exactly `transfer_length`, `symbol_size` (the encoding symbol length) and `max_block_length` (the maximum source block length, in symbols), RFC 5445 §3.2.2.2. Each an integer. |
| `fec` | string | The code: `raptorq` (RFC 6330, FEC Encoding ID 6), `raptor` (RFC 5053, FEC Encoding ID 1) or `no-code` (RFC 5445, FEC Encoding ID 0). |
| `oti` | object | The code's object transmission information, as §5 reads it. For `raptorq`, exactly `transfer_length` (F), `symbol_size` (T), `source_blocks` (Z), `sub_blocks` (N) and `alignment` (Al), RFC 6330 §3.3.2 and §3.3.3. For `raptor`, the same members, RFC 5053 §3.2.2 and §3.2.3. For `no-code`, exactly `transfer_length`, `symbol_size` (the encoding symbol length) and `max_block_length` (the maximum source block length, in symbols), RFC 5445 §3.2.2.2. Each an integer. |
| `repair` | integer | The repair symbols of each source block sent in each pass, beyond its source symbols (§5.4). |
| `esi` | string | `repeat`: every pass sends the same ESIs. `advance`: each pass sends ESIs no earlier pass sent (§5.4). |
| `first_ms` | integer | The instant the first pass starts. |
@ -124,7 +130,7 @@ Checked at jwp-receipts §3.1 step 4, in this order:
2. `objects` has at least one entry, and its `leaf_index` values strictly increase
(`schedule_objects_malformed`): one entry for each object scheduled, in group order.
3. Each entry in order, the first of these to fail:
1. its `oti` is one its code admits, §5.1 or §5.2 (`schedule_oti_invalid`);
1. its `oti` is one its code admits, §5.1, §5.2 or §5.5 (`schedule_oti_invalid`);
2. its symbols are ones its code can name, §5.4 (`schedule_symbols_invalid`);
3. its passes are well formed (`schedule_passes_invalid`): `count` and `duration_ms` are at
least 1; with `count` at least 2, `period_ms` is at least `duration_ms`, so one pass of
@ -144,7 +150,9 @@ of those it holds. In order:
1. the group's link hash is the claim's `subject` (`schedule_subject_mismatch`);
2. the claim's `signer_pubkey` is the group's (`schedule_signer_mismatch`): only the publisher
schedules its own transmission;
schedules its own transmission. A node schedules the instance it declares as its own under
[cache-authority](cache-authority.md) §5.6, in the verdict that holds that instance, which
replaces this step and no other; every other verdict reads the publisher's schedules alone;
3. every `leaf_index` is less than the group's `leaf_count` (`schedule_leaf_out_of_range`);
4. no entry's first pass starts before the group does (`schedule_before_group`): ⌊`first_ms`
/ 1000⌋ is at least the seconds jwp-receipts §2.4.3 reads from `group_start`, its fraction
@ -159,7 +167,40 @@ A schedule checked without the group stops at step 5 and is not counted by §6.
Confirmed under jwp-receipts §7, a schedule claim needs the authority `schedule`, which is not
pinned-only and is not `delivery`. An origin gives it to the keys that publish its groups, as it
gives `transmission`.
gives `transmission`, and a node's operator to the nodes that schedule their own instances
(cache-authority §4.3).
### 4.5 Extending a schedule
A publisher **MAY** add to a schedule it has published, after a fade or for a longer transmission,
by signing a later schedule of the same instance that extends it. Schedule *B* **extends** schedule
*A* of one `bearer_id` when *B* has, for every entry of *A*, an entry of the same `leaf_index` that
extends it; and an entry extends another when its passes begin with the other's, at the same
instants and sending the same symbols:
- its `count` is at least the other's;
- its `period_ms` is the other's, unless the other has one pass, whose period is not read (§4.2
rule 3.3);
- each of its other members is the other's.
*B* **MAY** also have entries for objects *A* has none for. Every schedule extends itself, and a
schedule that extends one that extends a third extends the third. *B* is held to §4.2 and §4.3 on
its own, so rule 3.3 and §5.4's bound on ESIs hold for its greater count.
Nothing *A* states is weakened by *B*, at any loss bound and overhead:
- each pass of *A* is a pass of *B*, and §5.4 gives it the same ESIs, which depend only on the
pass and on *m*;
- an entry available under *A* (§6.3) is available under *B* at the same pass. One not available
under *A* may be under *B*: by a later pass under `advance`, while under `repeat` and `no-code` the
first pass decides and the answer is the same;
- a statement that holds under *A* (§6.4) holds under *B*, and one refused
`availability_not_within_schedule` or `object_not_scheduled` may hold;
- a plan under *B* (§7) listens to every pass a plan under *A* listens to from the same instant
with the same holding, and is complete at the same instant wherever that plan is.
What does not extend an earlier schedule, and contradicts it, is §6.2 step 4's: a schedule that
drops an object, or withdraws, moves or changes a pass.
## 5. Blocks and symbols
@ -210,6 +251,18 @@ A block of *K* source symbols is **held** when a receiver can recover it:
1 in 100, 1 in 10,000 and 1 in 1,000,000 on average. A block of *K* < *K′* symbols is padded
to *K′* with symbols the decoder knows (§5.3.1), so it may need fewer than *K′* + *h*
received; this text asks for the count §5.8 states its bound at, and no fewer.
- **`raptor`**: when it has received *K* + *a* distinct encoding symbols of the block, where the
margin *a* is 8, 16 or 28 at *h* 0, 1 or 2. RFC 5053 pads no block to a table of sizes, since §5.7
indexes every *K* it admits, and it states no bound on a decoder's failure: its decoder recovers a
block from "any set of encoding symbols only slightly more in number than the number of source
symbols" (§1), and the example it gives can fail (§5.5.2.2, §5.5.2.3). 3GPP TS 26.346 §7.2.2
requires a receiver to recover a block from a "mathematically sufficient set", which no count
establishes. The margins are this text's, set by measurement. With distinct encoding symbols
chosen uniformly at random, an independent implementation of RFC 5053 failed on 14 to 31 blocks in
100 at *K* + 2, at most 81 in 10,000 at *K* + 8, at most 2 in 18,000 at *K* + 16 and at most 4 in
6,000,000 at *K* + 28, at block sizes from 4 to 8,192 symbols (`raptor_margin_measurement` in
`tests/carousel.rs`, whose samples shrink as blocks grow, to 1,800 blocks of 8,192). That is a
measurement of one implementation, not a bound, and no bound comes with the count.
An **object** is held when every one of its source blocks is.
@ -220,8 +273,8 @@ symbols:
- `repeat`: ESIs 0 to *m* − 1 in every pass;
- `advance`: in pass *j*, ESIs (*j* − 1) · *m* to *j* · *m* − 1. The first pass holds the source
symbols (ESIs 0 to *K* − 1, RFC 6330 §4.4.2) and every later pass only repair symbols, none
sent before.
symbols (ESIs 0 to *K* − 1, RFC 6330 §4.4.2, RFC 5053 §5.3.2) and every later pass only repair
symbols, none sent before.
An entry's symbols are ones its code can name (§4.2 rule 3.2) when:
@ -229,10 +282,32 @@ An entry's symbols are ones its code can name (§4.2 rule 3.2) when:
past a block's last source symbol names nothing;
- **`raptorq`**: the greatest ESI sent fits the 24-bit field (RFC 6330 §3.2): with *KL* the
larger block's *K*, *KL* + `repair` is at most 2<sup>24</sup> for `repeat`, and `count` ·
(*KL* + `repair`) at most 2<sup>24</sup> for `advance`, computed without overflow.
(*KL* + `repair`) at most 2<sup>24</sup> for `advance`, computed without overflow;
- **`raptor`**: the greatest ESI sent fits the 16-bit field (RFC 5053 §3.1): *KL* + `repair` is at
most 2<sup>16</sup> for `repeat`, and `count` · (*KL* + `repair`) at most 2<sup>16</sup> for
`advance`, computed without overflow.
The octets an entry's passes send, `bytes_sent`, are `count` · Σ *m* · T over its source blocks
for `raptorq`, and `count` · L for `no-code`, whose last symbol is short.
for `raptorq` and `raptor`, and `count` · L for `no-code`, whose last symbol is short.
### 5.5 Raptor
The `oti` of a `raptor` entry, F, T, Z, N and Al, is one RFC 5053 admits when:
- F is from 1 to 2<sup>45</sup> − 1 (§3.2.2);
- T is from 1 to 65,535 and a multiple of Al, and Al from 1 to 255 (§3.2.2, §3.2.3, §4.1);
- Z is from 1 to 65,535, the range of its 16-bit field (§3.2.3);
- N is from 1 to 255, the range of its 8-bit field, and at most T / Al, so each sub-block has a
sub-symbol of at least Al octets (§3.2.3, §5.3.1.2);
- with *Kt* = ⌈F / T⌉, ⌊*Kt* / Z⌋ is at least 4 and ⌈*Kt* / Z⌉ at most 8,192: §5.7 gives a systematic
index for those block sizes and no others, and *K*<sub>max</sub> is 8,192 (§5.1.2, §5.3.1.2).
The object is then divided as RFC 5053 §5.3.1.2 divides it, by the Partition[] of §5.1: (*KL*, *KS*,
*ZL*, *ZS*) = Partition[*Kt*, Z]. The first *ZL* source blocks have *KL* source symbols each and the
remaining *ZS* have *KS*. Each symbol is T octets. The last source symbol of the object is padded to T
for encoding; RFC 5053 §5.3.2 lets a sender leave that padding out of a packet, and `bytes_sent`
counts it. A limit a bearer sets on these values, as TS 26.346 §7.2.3 does on a sub-block's size, is
the bearer's.
## 6. When an object is held
@ -252,15 +327,20 @@ The loss bound and overhead are checked first: *L* above 999,999 is `loss_bound_
transmission claim by bearer-receipts §4.3 and a schedule by §4.3 here. A claim of another
class is `context_mismatch` at step 6. A claim refused at any step takes no further part.
3. **The declarations**: bearer-receipts §6.2 step 4.
4. **The schedules**: the accepted schedule claims by `bearer_id`. Two with one `bearer_id` and
different bodies set aside every schedule of that instance (`schedule_contradicted`): a
publisher that changes its plan for one transmission has stated neither, and a relying
party holds no promise that can change under it. Two with the same body are one. A schedule
is then set aside at the first of these that holds: step 3 set aside the declarations of its
`bearer_id` (`bearer_contradicted`); no declaration has its `bearer_id`
4. **The schedules**: the accepted schedule claims by `bearer_id`, each instance's in the order
of their signer's chain (§6.1), which §4.3 step 2 makes one chain. When one of them does not
extend the one before it (§4.5), every schedule of that instance is set aside
(`schedule_contradicted`): a publisher that drops an object from its plan for one
transmission, or withdraws, moves or changes a pass, has stated neither plan, and a relying
party holds no promise that can change under it. Otherwise the last of them is the instance's
schedule, and each before it with another body is **extended**: its passes are the first of
the last one's, and it takes no further part. Two with the same body are one. The instance's
schedules are then set aside at the first of these that holds: step 3 set aside the
declarations of its `bearer_id` (`bearer_contradicted`); no declaration has its `bearer_id`
(`bearer_not_declared`); its declaration's `bearer` is `bidirectional`
(`schedule_on_bidirectional`), since a receiver with a path back asks for what it lacks and
is not served by a plan. Otherwise it **stands**.
is not served by a plan. Otherwise the instance's schedule **stands**, and so does each before
it with its body.
5. **Availability**, for each entry of each standing schedule, by §6.3.
### 6.3 Availability
@ -270,18 +350,18 @@ For an entry, pass *j* ends at the instant `first_ms` + (*j* − 1) · `period_m
⌊*L* · *m* / 1,000,000⌋ the most a receiver within *L* misses of it in one pass, the first pass
after which a receiver within *L* holds the block is, under §5.3:
- **`raptorq`, `advance`**: the least *j* from 1 to `count` with *j* · (*m* − *x*) ≥ *K′* + *h*.
Every pass adds *m* − *x* symbols none before it sent.
- **`raptorq`, `repeat`**: 1 if *m* − *x* ≥ *K′* + *h*, and none otherwise. A later pass sends
the same ESIs, and the ones a receiver missed may be missed again: within *L*, nothing it adds
is certain.
- **`raptorq` or `raptor`, `advance`**: the least *j* from 1 to `count` with *j* · (*m* − *x*) at
least what §5.3 asks of the block. Every pass adds *m* − *x* symbols none before it sent.
- **`raptorq` or `raptor`, `repeat`**: 1 if *m* − *x* is at least what §5.3 asks, and none otherwise.
A later pass sends the same ESIs, and the ones a receiver missed may be missed again: within *L*,
nothing it adds is certain.
- **`no-code`**: 1 if *x* is 0, and none otherwise, for the same reason.
The object is **available** at the end of the greatest of its blocks' passes, or is **not
available within the schedule** if any block has none. A verifier reports for each entry its
`available_ms` (or `null`), the pass, `bytes_sent` (§5.4), and, for `raptorq`, the RFC 6330
§5.8 bound at *h* as `failure_one_in` (100, 10,000 or 1,000,000); for `no-code` it is `null`, a
block held being recovered exactly.
§5.8 bound at *h* as `failure_one_in` (100, 10,000 or 1,000,000). For `raptor` it is `null`, since
RFC 5053 states no bound (§5.3); for `no-code` it is `null`, a block held being recovered exactly.
### 6.4 Statements offered about an object or a group
@ -298,8 +378,8 @@ an overhead *h*:
A statement that holds says what the publisher planned: a receiver within *L*, hearing every
pass the schedule names, holds the object by *t*, but for a decoding failure no more likely than
`failure_one_in` states. It does not say the passes were sent (the declaration's) or heard (a
reconciliation's).
`failure_one_in` states, and under `raptor` no bound is stated on that failure. It does not say the
passes were sent (the declaration's) or heard (a reconciliation's).
A publisher's own deadline for an object is its timeliness claim's `by_ms`
([timeliness](timeliness.md) §4.1); held by it, the statement is the publisher's plan against its
@ -316,9 +396,9 @@ them, or that holds more source symbols of a `no-code` block than it has, is `ho
Only passes that start at or after *t* count: a pass under way at *t* has sent symbols the
receiver did not hear. A pass adds, to a block not held, with *m* and *x* as §6.3:
- **`raptorq`, `advance`**: *m* − *x*, none of them sent before;
- **`raptorq`, `repeat`** and **`no-code`**: max(0, *m* − *n* − *x*): of the *m* − *n* ESIs it
lacks, *x* may be the ones it misses again.
- **`raptorq` or `raptor`, `advance`**: *m* − *x*, none of them sent before;
- **`raptorq` or `raptor`, `repeat`**, and **`no-code`**: max(0, *m* − *n* − *x*): of the *m* − *n*
ESIs it lacks, *x* may be the ones it misses again.
The passes are taken in order of their start. While some block is not held, the next pass is
listened to if it adds to a block not held, each such block's *n* growing by what it adds; the
@ -329,7 +409,8 @@ of their durations: the time the receiver's radio must be on. Outside those pass
off. Under `repeat` and `no-code` at most one pass adds anything certain within *L*, since after
it a block lacks only symbols the receiver may miss again: a receiver that one pass does not
complete is not certain to be held after any number, and keeps listening, if it does, on its own
judgement of its channel.
judgement of its channel. A receiver that acquires a schedule extending the one it plans with plans
with the later from then on: what it holds still counts, since each pass sends what it did (§4.5).
## 8. Mappings
@ -340,7 +421,66 @@ signed by the publisher, beside the group's receipts, so a relying party with no
bearer can check a promise (§6.4) and a receiver can plan before it tunes (§7). A sink that holds
both compares them; a difference is the bearer's, and the sink reports it rather than choosing
one. The bearer's own repetition (its carousel of descriptions and files) is the schedule's
passes: a pass is one transmission of the object's symbols, whatever the bearer calls it.
passes: a pass is one transmission of the object's symbols, whatever the bearer calls it. A receiver
of 3GPP MBMS download supports the Compact No-Code and Raptor schemes (TS 26.346 §7.2.2), so a
schedule of such an instance names `no-code` or `raptor`.
### 8.1 In-band repetition
A receipt held only off the bearer reaches only a receiver that had another path to it, and a
receiver on a bearer with no return path cannot ask. On an instance whose schedule stands, the
publisher **MUST** send its **in-band set** on the instance itself, each part as bearer-receipts
§9.1 carries it:
- the group receipt;
- its claims from the head of their chain (`seq` 0, `parent_receipt_hash` `null`) to the
instance's last schedule, in the order of the chain, among them the instance's declaration and
every schedule of it. A receiver that holds no claim of the signer then accepts each at
jwp-receipts §3.1 step 5, which refuses a claim whose parent it does not hold
(`parent_not_held`), a parent that on this bearer it cannot fetch;
- the head revision of the key log of the origin the schedule's `signer_id` names. It anchors
nothing (bearer-receipts §8.2): it carries a rotation or a revocation to a receiver that has the
origin anchored.
A chain holds every claim its key has signed, so a publisher **SHOULD** sign the groups it schedules
with a key that signs little else, which its origin lists with `delivery`, `transmission` and
`schedule` (jwp-receipts §7). A claim of the chain about another group is refused at step 6 against
this group and breaks nothing: it is held, so the claims after it chain to it (jwp-receipts §3.1
step 5).
**How often.** A schedule's **interval** is the least `period_ms` of its entries of two passes or
more, or, where every entry has one pass, the least `duration_ms`. From the start of the schedule's
first pass to the end of its last, the publisher sends the whole set at least once in every span of
that length. A receiver that joins in that time, and misses nothing of the set, holds it no later
than one interval after it joins. The set's files are outside every entry: their octets are not in
`bytes_sent`, and the loss bound of §2 says nothing of them. A receiver that misses a copy takes the
next.
**An extension.** A later schedule (§4.5) is in the set from when it is signed, chained after the
one it extends. A receiver that held the earlier and acquires the later reads one schedule that
stands, and one that joins afterwards reads both in the chain.
### 8.2 What a receiver acquired
A receiver that joined an instance, or a verifier given what it acquired, **MAY** check that it
suffices. The inputs are the instance's `bearer_id`, the group receipt, the claims in the order of
their chain, and the key logs acquired, each keyed by the origin it names. §6.2 steps 1 to 4 run over
the group receipt and the claims alone, with no object held and no claim held before the first; a
group that fails is refused with its reason, and nothing is derived. Then, at the first of these
that fails, the set is refused:
1. no claim of it is refused at jwp-receipts §3.1 step 5 (`inband_chain_incomplete`), so it holds
each signer's chain from its head;
2. a schedule of the instance stands (`inband_not_scheduled`);
3. a key log acquired is held for the origin that schedule's `signer_id` names, under jwp-receipts
§7.2 rules 1 to 3, and lists the schedule's key (`inband_key_log_missing`). A `signer_id` that
names no origin has no log to carry, and is refused here.
Otherwise the set **suffices**, and a verifier reports each claim's label and the interval of the
schedule that stands as `inband_ms`. A set that suffices holds what a receiver needs to plan (§7) and
to confirm. Confirming is bearer-receipts §8.3's, under anchors the receiver already holds, so a log
that lists the key as revoked still suffices here, and confirmation reports it. It does not say the
publisher sent the set as §8.1 asks.
## 9. Reasons
@ -367,6 +507,9 @@ The codes this extension adds, each with where it is raised:
| `object_not_scheduled` | §6.4 |
| `availability_not_within_schedule` | §6.4 |
| `availability_after_deadline` | §6.4 |
| `inband_chain_incomplete` | §8.2 check 1 |
| `inband_not_scheduled` | §8.2 check 2 |
| `inband_key_log_missing` | §8.2 check 3 |
A schedule also raises `bearer_contradicted` and `bearer_not_declared` (bearer-receipts §10) at
§6.2 step 4; a group or object the codes of jwp-receipts §3.3; and a claim every code of
@ -379,15 +522,28 @@ fixed seeds by `wai_carousel_vectors gen` as `vectors.json`. It has:
- **Availability**: RaptorQ objects of one and of several source blocks of two sizes, under
`advance` and `repeat`, at loss bounds of zero and above, reaching a later pass, the last
pass, and none; at each overhead; a Compact No-Code object available at the first pass with
pass, and none; at each overhead; Raptor objects the same, among them one not held where the same
parameters under RaptorQ would be; a Compact No-Code object available at the first pass with
no loss and at none with any; a group of several objects; the same object simulcast on two
instances with different schedules.
- **Statements**: an object and a group held by a deadline, after it, not within the schedule,
and not scheduled.
- **Plans**: a receiver tuning in before the first pass, during one, between passes, holding
part of an object, holding all of it, and one under `repeat` that one pass does not complete.
- **Extensions** (§4.5): passes added until an object not within the schedule is held; an object
added, and the later schedule repeated; one pass, its period not read, given a period; `repeat`,
to which more passes add nothing certain; and each thing that does not extend: a pass withdrawn,
the first pass, the period, the duration, the repair, the ESIs or the code changed, an object
dropped, and a schedule that extends the first and withdraws a pass of the second. A plan under an
extended entry, and a chain on an instance no declaration names.
- **In-band sets** (§8.2): a set that suffices, with the interval of a repeated schedule, of one of
single passes and of a mix of the two; with an extension; with a chain that holds a claim about
another group, without its head and out of order; with no schedule, no declaration, a schedule of
another instance and a contradicted one; a key log absent, of another origin, naming another
origin, with a broken signature, unreadable, not listing the key and revoking it; and a group that
fails.
- **Negative controls**, one for each reason of §9 that a claim or a verdict raises, and the
bounds of §5.1, §5.2 and §5.4 on both sides.
bounds of §5.1, §5.2, §5.4 and §5.5 on both sides.
A conforming verifier gives every case the outcome, the reason and everything else the corpus
states. `wai_carousel_vectors verify` and the test `tests/carousel.rs` check that the file
@ -397,12 +553,16 @@ table. Against an independent implementation of RFC 6330, the test also requires
*K′* and Partition[] to agree for every block size, and decodes each RaptorQ object a case holds
available at a pass: it encodes the object under the entry's parameters, drops in each pass the
first ⌊*L* · *m* / 1,000,000⌋ symbols of each block (in the first pass, source symbols, so the
decoder must decode), and requires the object back from what is left by that pass's end.
decoder must decode), and requires the object back from what is left by that pass's end. It does the
same for each Raptor object against an independent implementation of RFC 5053, and on a fixed sample
requires that decoder to fail on at least 1 block in 10 at *K* + 2, on at most 1 in 100 at the margin of
*h* = 0, at most 1 in 10,000 at that of *h* = 1, and on none of 10,000 at that of *h* = 2.
`live-conformance/carousel_verify.py` is the second implementation, written from this text: it
shares no code with `wai-rs`; it verifies groups and objects with the JWP corpus's second
implementation (`jwp-conformance/jwp_verify.py`), and claims through steps 1–5 with the claim
corpus's (`claim-conformance/claim_verify.py`), whose step 4 holds §4.2 with §5.1, §5.2 and
§5.4; it reads Appendix A and §9 from this text, writes §4.3, §5.3, §6 and §7 itself, mutates
implementation (`jwp-conformance/jwp_verify.py`), claims through steps 1–5 with the claim
corpus's (`claim-conformance/claim_verify.py`), whose step 4 holds §4.2 with §5.1, §5.2, §5.4 and
§5.5, and key logs with the signer corpus's (`signer-conformance/signer_verify.py`); it reads
Appendix A and §9 from this text, writes §4.3, §4.5, §5.3, §6, §7 and §8.2 itself, mutates
copies of the corpus and requires every copy to fail, and drops each of its own checks in turn
and requires the corpus to catch the weakened verifier.
@ -426,7 +586,7 @@ product that implements the code:
product or part that does not fully implement the specification. The class is **royalty-free
non-assertion**, conditional on full implementation and suspended for a party that asserts.
Raptor (RFC 5053), which this revision does not name (§4.1), has disclosures from the same
Raptor (RFC 5053), which §4.1 names, has disclosures from the same
holder and an earlier one: #615, #618, #702 and #734 on its drafts; #1185 (8 September 2009) on
RFC 5053, which declared a reasonable and non-discriminatory licence with possible royalty or
fee; and #1289, #1508, #1744 and #1957 (6 February 2013), each updating the one before. #1957 has
@ -443,11 +603,12 @@ Therefore:
- **`no-code` is the default.** Neither RaptorQ nor Raptor is royalty-free for every
implementer: whether a royalty is due turns on the product, which a schedule does not state.
Under SPEC §5 a capability that is not royalty-free is never a default, and the same holds
here: a tool that writes schedules writes `no-code` unless it is set to write `raptorq`.
here: a tool that writes schedules writes `no-code` unless it is set to write `raptorq` or
`raptor`.
- **The reference implementation encodes and decodes no symbol** (§1). Its `carousel` module
computes, from the parameters an entry states, the division of §5 and the counts of §6 and §7.
Its test decodes RaptorQ with an independent implementation of RFC 6330 that is a development
dependency of `wai-rs`, which no build of the library carries.
Its test decodes RaptorQ with an independent implementation of RFC 6330, and Raptor with one of
RFC 5053, each a development dependency of `wai-rs`, which no build of the library carries.
- Whether a product that encodes or decodes RaptorQ or Raptor symbols needs a licence under
these disclosures is a decision for its distributor.
@ -457,6 +618,12 @@ Therefore:
a declaration attests transmission and no more, and a relying party that needs to know what
receivers held counts reconciliations (bearer-receipts §6). A schedule that promises more
than was sent is a false statement signed by the group's key, and is evidence of it.
- **An extension is the publisher's word too.** It adds to what the publisher said it would send,
and is evidence as the schedule it extends is. It changes no statement that held (§4.5), and a
later schedule that would is set aside with the rest of its instance's (§6.2 step 4).
- **What rides on the bearer is verified as anything is.** Whoever controls the bearer can withhold
the in-band set, and a receiver that does not acquire it holds nothing to plan with; it cannot
alter a signed claim, and a key log it carries anchors nothing (bearer-receipts §8.2).
- **A loss bound is the relying party's.** Every statement holds for every pattern of loss
within it (§2). A channel that exceeds it, in a fade or under interference, voids nothing in
the schedule and nothing here predicts how often that happens.
@ -464,9 +631,10 @@ Therefore:
parameters, which the bearer's own file descriptions already carry to every receiver. A
receiver that plans with it (§7) sends nothing, and a unidirectional bearer's property that
the source learns nothing of its receivers (unidirectional-delivery §2) is kept.
- **Bounded work.** An entry's arithmetic is a few products per source block, and an object has
at most 255 RaptorQ blocks of two sizes or Compact No-Code blocks of two sizes; a schedule's
cost is linear in its entries, whatever the passes' count.
- **Bounded work.** An entry's arithmetic is a few products per block size, and an object has
blocks of at most two sizes under every code; a schedule's
cost is linear in its entries, whatever the passes' count, and so is comparing it with the one
before it (§4.5).
## Appendix A. The values of K′

18
wai/extensions/jwp-receipts.md
View file

@ -418,7 +418,7 @@ key.
| `consumption` | `claim` | `object` (the first object it names) | §2.4.13 | the agent that consumed the objects: its key, named by its `did:key` |
| `transmission` | `claim` | `receipt` (the group transmitted) | §2.4.14 | the group's publisher: the key that signed the group |
| `reconciliation` | `claim` | `receipt` (the group reconciled) | §2.4.15 | the sink that received the objects, never the publisher |
| `schedule` | `claim` | `receipt` (the group scheduled) | §2.4.16 | the group's publisher: the key that signed the group |
| `schedule` | `claim` | `receipt` (the group scheduled) | §2.4.16 | the group's publisher: the key that signed the group; or a node, of an instance of its own ([cache-authority](cache-authority.md) §5.6) |
| `timeliness` | `claim` | `receipt` (the group whose objects it times) | §2.4.17 | the group's publisher: the key that signed the group |
| `cache-transmission` | `claim` | `receipt` (the group sent again) | §2.4.18 | a node that sent the objects from what it held, never the group's publisher |
@ -1376,15 +1376,17 @@ object receipts at the positions it names.
#### 2.4.16. `class: "schedule"`
A publisher's plan of when each object of a group passes on a bearer instance with no return
path, and with which forward error correction: for each object, its code (RaptorQ or Compact
No-Code) and that code's object transmission information, the repair symbols each pass adds,
path, and with which forward error correction: for each object, its code (RaptorQ, Raptor or
Compact No-Code) and that code's object transmission information, the repair symbols each pass adds,
and the times and count of its passes. It attests a plan, not a transmission or a delivery, and
it sits beside the group receipt and the instance's transmission claim, whose bytes it leaves
unchanged. [carousel](carousel.md) defines it: its body's members are §4.1 there
(`bearer_id`, `objects` and `attests`), read at §3.1 step 2; its class rules are §4.2 there, with
§5's bounds on each code's parameters, at step 4; and its context is §4.3 there, at step 6: the
group receipt, whose link hash is its `subject` and whose key signs it, and the object receipts
and envelopes at the positions it names.
and envelopes at the positions it names. A node that declares an instance of its own states its
plan for it in the same class, signed by its own key: [cache-authority](cache-authority.md) §5.6
replaces the check that the group's key signs it, in the verdict that holds that instance.
#### 2.4.17. `class: "timeliness"`
@ -2363,7 +2365,7 @@ and never folds it in: confirmation changes the outcome of no step here.
| 4 | `reconciliation_bitmap_malformed` | its `leaf_count` is 0, its `received` is not ⌈`leaf_count` / 8⌉ bytes, sets a bit at or past `leaf_count`, or sets none |
| 4 | `schedule_not_scheduled` | a `schedule` claim's `attests` is not `scheduled` ([carousel](carousel.md) §4.2) |
| 4 | `schedule_objects_malformed` | it has no entry, or its entries' `leaf_index` values do not strictly increase |
| 4 | `schedule_oti_invalid` | an entry's code parameters are not ones its code admits (carousel §5.1, §5.2) |
| 4 | `schedule_oti_invalid` | an entry's code parameters are not ones its code admits (carousel §5.1, §5.2, §5.5) |
| 4 | `schedule_symbols_invalid` | an entry's symbols are not ones its code can name (carousel §5.4) |
| 4 | `schedule_passes_invalid` | an entry's passes are not well formed: none, of no duration, overlapping, ending past 9999, or sending more than 2<sup>53</sup> − 1 octets |
| 4 | `timeliness_not_valued` | a `timeliness` claim's `attests` is not `valued` ([timeliness](timeliness.md) §4.2) |
@ -3777,8 +3779,10 @@ its keys is that party is the origin's decision, and its log records it:
class, and the declaration stands without it. A sink with no return path confirms under
anchors provisioned before it was cut off, and a key log that reaches it over the bearer
anchors nothing ([bearer-receipts](bearer-receipts.md) §8). `schedule` lets a key state the
passes of a group it signed over an instance (§2.4.16); an origin gives it to its publishing
keys, as it gives `transmission`, and it is neither pinned-only nor `delivery`. `timeliness`
passes of a group it signed over an instance (§2.4.16), or of a group another key signed over
an instance it declares as its own ([cache-authority](cache-authority.md) §5.6); an origin gives
it to its publishing keys, as it gives `transmission`, a node's operator to the nodes that
schedule their instances, and it is neither pinned-only nor `delivery`. `timeliness`
lets a key state when the objects of a group it signed are of use (§2.4.17); an origin gives
it to its publishing keys, as it gives `schedule`, and it is neither pinned-only nor
`delivery`.

2
wai/extensions/rung-price.md
View file

@ -582,7 +582,7 @@ define. Until it does, a comparer compares transmission prices (§6), and a ledg
### 7.2 The pass rate
For an entry of a carousel schedule, *P* = `bytes_sent` / `count` is the octets one pass sends: Σ
*m* · T over its source blocks for `raptorq`, and L for `no-code` (carousel §5.4). The division is
*m* · T over its source blocks for `raptorq` and `raptor`, and L for `no-code` (carousel §5.4). The division is
exact. The entry's **pass rate** is ⌈8000 × *P* / `duration_ms`⌉ bits per second, computed in 128
bits.

14
wai/extensions/timeliness.md
View file

@ -464,6 +464,11 @@ wanted objects; a group that fails refuses the run as it refuses a plan. The fir
force** from its *t*, and each later one **replaces** the one before it at its own *t*, wholly: an
object only an earlier plan wants is no longer run.
A run does not date its claims: every plan of it reads them all, and so reads the last schedule of an
instance (carousel §4.5), though that plan's *t* be before the schedule was signed. A sink that
acquires a later schedule during a run decided its earlier plans without it, and the run that
reproduces those decisions is the one given the claims it held then.
#### 7.5.2 State
For each object, a group's link hash and a position, the run keeps whether it is **settled**
@ -703,13 +708,16 @@ fixed seeds by `wai_timeliness_vectors gen` as `vectors.json`. It has:
refused on value, chosen at a value equal to the late primary's, from the first source in the
asker's order before an earlier one, and arriving at the deadline, late sources chosen by value
and order, an arrival only after the end, no source on a path with no return, the origin alone, a
schedule with no declaration and one on a two-way instance, every pass heard from the first, the
schedule with no declaration, one on a two-way instance, one a later schedule extends and one a
later schedule withdraws a pass of, every pass heard from the first, the
publisher's check from `from_ms` passing and failing, an object with no deadline, an instant at
and past the last instant, and each rule of §7.1. With a ladder: the first rung, both rungs in
time, a lower rung when the first is late, no rung when the late primary is worth more, the
first rung over a late primary worth less, a lower rung refused on value, the first source in
order, the primary on time over rungs, and no rung in time.
- **Runs** (§7.5): on time on the plan's own trace, over the origin and over a bearer; held at the
order, the primary on time over rungs, no rung in time, and a rung read over a schedule a later one
extends, over the same two withdrawing a pass, and over the earlier alone.
- **Runs** (§7.5): on time on the plan's own trace, over the origin and over a bearer; a rung over a
schedule a later one extends; held at the
deadline and a millisecond after; a late object, presented at its value then; a hard deadline
missed; every rung passed, with the primary later and with none; a rung chosen and presented, a
lower rung held in its place, both rungs held, a rung at the deadline and a millisecond after

13
wai/live-conformance/cache-authority/README.md
View file

@ -15,17 +15,20 @@ leaves), `GX` (a `group_start` that is not a date-time) and `GF` (four, its obje
from its object 1). `heads` holds the two test logs' head revisions, of `https://operator.example`
and `https://other.example`; `C2` is in neither.
- `claims` (§5) are each a `claim`, the signer's `earlier` claims in chain order, the `context`
- `claims` (§5, a cache-transmission claim or a schedule) are each a `claim`, the signer's `earlier` claims in chain order, the `context`
it is checked against (a group receipt, another receipt, or `null`) and the `objects` held under
it; `outcome` is `accept`, `unchecked` (no group: it stops at step 5) or
`reject step=N reason=…`.
- `placements` (§4.4) are each a `key`, an instant `at_ms` and the `heads` named, keyed by origin;
`outcome` is the placement as §4.4 writes it.
- `verdicts` (§7) are each the `group`, the `objects` (receipt, and envelope or `null`), the
`claims`, a `sink`'s own instance, a statement offered (`assertion`), `count_only` and the
`heads`; `outcome` gives each object's and each claim's label, the status, `sinks`, `whole`,
`received_by`, the publisher's `bearers` and the `caches` that stand, the sink's outcome and the
statement's (`holds` or `reject reason=…`), or `refuse reason=…` for a group that fails.
`claims`, a `sink`'s own instance, a statement offered (`assertion`), `count_only`, the
`heads`, and `held`: `null`, or carousel §6.4's statement with its bound and overhead,
`{"kind": "object", "bearer_id", "index", "by_ms", "loss_ppm", "overhead"}` or the same of
`"kind": "group"` with no `index`; `outcome` gives each object's and each claim's label, the
status, `sinks`, `whole`, `received_by`, the publisher's `bearers` and the `caches` that stand, the
sink's outcome, and each statement's (`holds` or `reject reason=…`), or `refuse reason=…` for a
group that fails.
- `tallies` (§8) are a verdict's inputs, the `acknowledgements` (each a record and its item lists,
in hexadecimal), the holdings `records`, the object's `index`, a `deadline_ms` or a timeliness
`deadline_claim`, the `region`, the `population` and `miss_target_ppm`; `outcome` is `met` or

3949
wai/live-conformance/cache-authority/vectors.json
View file

File diff suppressed because it is too large Load diff

122
wai/live-conformance/cache_authority_verify.py
View file

@ -56,6 +56,7 @@ import timeliness_verify as TV # noqa: E402
import rung_price_verify as RP # noqa: E402
HV = TV.HV # the holdings corpus's reader, which the timeliness verifier loads
KV = TV.KV # the carousel corpus's reader, which the timeliness verifier loads
EXTENSION = os.path.join(HERE, "..", "extensions", "cache-authority.md")
JWP_RECEIPTS = os.path.join(HERE, "..", "extensions", "jwp-receipts.md")
@ -223,6 +224,12 @@ def check_cache_transmission(c, g, objects):
need(drops("ct_root") or root.hex() == b["sent_root"], "cache_transmission_root_mismatch")
def check_any_schedule(c, g, pairs):
"""§5.6, §7.2 step 2: a schedule by the group's key is the publisher's, under carousel §4.3 whole;
by another key, a node's, under carousel §4.3 but for its step 2."""
KV.check_schedule(c, g, pairs, node=c["signer_pubkey"] != g["signer_pubkey"] and not drops("schedule_step2"))
def read_objects(receipts):
"""The object receipts given that read (jwp-receipts §2.4.2)."""
out = []
@ -261,6 +268,9 @@ def claim_outcome(case):
CV.check_chain(c, held.get(c["signer_pubkey"]))
if case["context"] is None and not drops("unchecked"):
return "unchecked"
if c["class"] == "schedule":
check_any_schedule(c, context_group(case["context"]), [(o, None) for o in read_objects(case["objects"])])
return "accept"
if c["class"] != "cache-transmission":
raise CV.Reject(6, "context_mismatch")
check_cache_transmission(c, context_group(case["context"]), read_objects(case["objects"]))
@ -297,6 +307,8 @@ class Verdict:
BV.check_reconciliation(c, gr, objs)
elif c["class"] == "cache-transmission":
check_cache_transmission(c, gr, objs)
elif c["class"] == "schedule":
check_any_schedule(c, gr, self.g.pairs)
else:
raise CV.Reject(6, "context_mismatch")
except CV.Reject as r:
@ -343,6 +355,55 @@ class Verdict:
"sent": positions(cs[0]["body"]["sent"])}
for i in idx:
labels[i] = "cache-declared"
# Step 4, the schedules: the publisher's by carousel §6.2 step 4, each node's by §5.6.
self.schedules = {}
def settle(bid, idx):
"""The schedules of one signer on one instance, each extending the one before it, or
set aside. The last stands and each before it with another body is extended."""
bodies = [accepted[i]["body"] for i in idx]
if not KV.monotone(bodies) and not drops("schedule_monotone"):
for i in idx:
labels[i] = "set-aside reason=schedule_contradicted"
return False
last = bodies[0] if drops("schedule_last") else bodies[-1]
self.schedules[bid] = last
for i in idx:
labels[i] = "stands" if accepted[i]["body"] == last else "extended"
return True
for bid, idx in by_instance("schedule").items():
publisher = [i for i in idx if accepted[i]["signer_pubkey"] == gr["signer_pubkey"]]
nodes = [i for i in idx if accepted[i]["signer_pubkey"] != gr["signer_pubkey"]]
def aside(group, reason):
for i in group:
labels[i] = "set-aside reason=" + reason
if publisher:
if not KV.monotone([accepted[i]["body"] for i in publisher]):
aside(publisher, "schedule_contradicted")
elif bid in self.pub_aside:
aside(publisher, "bearer_contradicted")
elif bid not in self.pub and not drops("schedule_publisher_declared"):
aside(publisher, "bearer_not_declared")
elif bid in self.pub and self.pub[bid]["bearer"] == "bidirectional":
aside(publisher, "schedule_on_bidirectional")
else:
settle(bid, publisher)
if nodes:
if (bid in self.pub or bid in self.pub_aside or bid in self.cache_aside) and not drops("schedule_taken"):
aside(nodes, "bearer_contradicted")
elif bid not in self.caches and not drops("schedule_undeclared"):
aside(nodes, "bearer_not_declared")
elif bid in self.caches:
server = self.caches[bid]["server"]
mine = [i for i in nodes if accepted[i]["signer_pubkey"] == server or drops("schedule_signer")]
aside([i for i in nodes if i not in mine], "cache_schedule_signer_mismatch")
if mine:
settle(bid, mine)
else:
settle(bid, nodes)
# Step 5: the reconciliations.
counted, self.bits, self.instance_bits, self.counted = set(), {}, {}, []
for i, c in enumerate(accepted):
@ -442,7 +503,30 @@ class Verdict:
assertion = "holds"
return {"objects": g.verdicts, "claims": self.labels, "status": status, "sinks": sinks, "whole": whole,
"received_by": received_by, "bearers": bearers, "caches": caches, "sink": sink,
"assertion": assertion}
"assertion": assertion, "held": self.held(case.get("held"), n)}
def held(self, h, n):
"""§7.3: carousel §6.4's statement against the schedule that stands on its instance."""
if h is None:
return None
bad = None if drops("held_bounds") else KV.bounds(h["loss_ppm"], h["overhead"])
if bad:
return "reject reason=" + bad
body = self.schedules.get(h["bearer_id"])
entries = {} if body is None else {e["leaf_index"]: e for e in body["objects"]}
def object_held(i):
e = entries.get(i)
if e is None:
return "object_not_scheduled"
at = KV.availability(e, h["loss_ppm"], h["overhead"])["available_ms"]
if at is None:
return "availability_not_within_schedule"
return "availability_after_deadline" if at > h["by_ms"] and not drops("held_deadline") else None
wanted = [h["index"]] if h["kind"] == "object" else range(n)
reason = next((r for r in (object_held(i) for i in wanted) if r), None)
return "holds" if reason is None else "reject reason=" + reason
def count_only_of(case):
@ -724,7 +808,7 @@ def lines(case):
if got is None:
return "reject reason=deadline_not_stated"
entry = got[0]
rungs = TV.rungs(entry)
rungs = TV.rungs(entry)[:1] if drops("line_every_rung") else TV.rungs(entry)
acks = acknowledgements(case["acknowledgements"], g)
rlabels, kept = TV.records_step(case["records"], [g])
standing = [kept[x] for x in sorted(kept) if rlabels[x] == "stands"]
@ -847,8 +931,36 @@ def mutations(doc):
def m_reason_gone(d):
d["claims"].remove(case(d, "leaf_count_wrong"))
def m_line_ladder_form(d):
# A ladder's line reports its list.
c = case(d, "timeliness_line_ladder")
c["outcome"][0]["fallback_ms"] = c["outcome"][0]["fallback_ms"][0]
def m_line_ladder_rung(d):
# The lower rung reported at the higher's value.
c = case(d, "timeliness_line_ladder")
for line in c["outcome"]:
if line["value_ppm"] == 400_000:
line["value_ppm"] = 700_000
def m_schedule_label(d):
case(d, "node_schedule_stands")["outcome"]["claims"][1] = "extended"
def m_schedule_held(d):
# A millisecond earlier than the second pass's end.
case(d, "node_schedule_stands")["held"]["by_ms"] -= 1
def m_schedule_signer(d):
# Dropped from the stated claims: C1's instance then has no other signer to set aside.
c = case(d, "node_schedule_other_signer")
c["claims"].pop()
def m_schedule_reason_gone(d):
d["verdicts"] = [c for c in d["verdicts"] if "cache_schedule_signer_mismatch" not in json.dumps(c["outcome"])]
return [m_until, m_least, m_receipt, m_credit, m_status, m_head, m_count_only, m_bound, m_population, m_region,
m_deadline, m_ppm, m_sole, m_items, m_claim_link, m_line, m_reason_gone]
m_deadline, m_ppm, m_sole, m_items, m_claim_link, m_line, m_reason_gone, m_schedule_label, m_schedule_held,
m_schedule_signer, m_schedule_reason_gone, m_line_ladder_form, m_line_ladder_rung]
WEAKENINGS = ["rec_second", "rec_round", "ack_last_us", "ack_round", "until_inclusive", "serving", "least_origin",
@ -861,7 +973,9 @@ WEAKENINGS = ["rec_second", "rec_round", "ack_last_us", "ack_round", "until_incl
"target_bound", "count_only", "group_start", "out_of_range", "verified", "records", "hit_inclusive",
"miss_ceil", "server_instant", "publisher", "target_inclusive", "line_keys", "deadline_envelopes",
"ack_order", "ack_digest", "server_earliest", "cache_earliest", "rec_before_ms", "line_fallback_acks",
"line_records_verified", "line_fallback_holders"]
"line_records_verified", "line_fallback_holders", "schedule_step2", "schedule_monotone", "schedule_last",
"schedule_publisher_declared", "schedule_taken", "schedule_undeclared", "schedule_signer", "held_bounds",
"held_deadline", "line_every_rung"]
def direct_checks():

21
wai/live-conformance/carousel/README.md
View file

@ -9,8 +9,9 @@ payloads are opaque test bytes written by the generator; no media and no input f
repository.
`keys` names every key by its role, as 64 lowercase hexadecimal digits: the publisher's
(`publisher`, `did:web:publisher.example#d1`), a second key of its origin (`publisher_d2`), and a
sink's (`sink`, signing under its `did:key`).
(`publisher`, `did:web:publisher.example#d1`), a second key of its origin (`publisher_d2`), a
sink's (`sink`, signing under its `did:key`), and the roots of the test key logs (`log_root`,
`other_log_root`).
`verdicts` (§6) are each one group of four objects, of 3,200, 896, 288 and 200 octets, as a
verifier holds it:
@ -23,27 +24,33 @@ verifier holds it:
statement offered (§6.4): `{"kind": "object", "bearer_id", "index", "by_ms"}` or
`{"kind": "group", "bearer_id", "by_ms"}`;
- `outcome`: `refuse` and its `reason`; or `verdict` with each object's verdict, each claim's
(`declared`, `stands`, `set-aside reason=…`, or `reject step=… reason=…`), a line for each entry
(`declared`, `stands`, `extended`, `set-aside reason=…`, or `reject step=… reason=…`), a line for each entry
of each schedule that stands — its `bearer_id`, `leaf_index`, `pass`, `available_ms`,
`bytes_sent` and `failure_one_in` — and the statement's result.
`claims` are each one schedule claim and its `outcome` through jwp-receipts §3.1 steps 1–4:
`accept`, or `reject step=… reason=…`. Among them every bound of §5.1, §5.2, §5.4 and §4.2 rule
3.3 that a schedule can reach, met and passed.
`accept`, or `reject step=… reason=…`. Among them every bound of §5.1, §5.2, §5.4, §5.5 and §4.2
rule 3.3 that a schedule can reach, met and passed.
`plans` (§7) are each an `entry` of a schedule, the instant `at_ms` a receiver is at, its
`loss_ppm` and `overhead`, and `held`, the encoding symbols it holds of each source block.
`outcome`: `plan`, with the `passes` it listens to (each `pass`, `start_ms` and `end_ms`),
`complete_ms` and `listen_ms`; or `refuse` and its `reason`.
`inband` (§8.2) are each what a receiver acquired on an instance: its `bearer_id`, the `group`
receipt, the `claims` in chain order, and the key `logs`, each a revision's text keyed by the origin
it was acquired for. `outcome`: `refuse` and its `reason`, for a group that fails; or `inband` with
each claim's label, `set` (`suffices`, or the reason it does not) and `inband_ms`, the interval of
the schedule that stands where the set suffices, `null` otherwise.
Everything is synthetic. Keys attest nothing; bearer instances use the example URN namespace
(`urn:example:`) and origins a reserved example domain. Each case states the outcome it was
written to have, declared in the generator, never copied from a verifier.
`wai_carousel_vectors verify` and `tests/carousel.rs` check that the file regenerates byte for
byte and that `wai-rs` reaches every stated outcome; the test also decodes each RaptorQ object a
case holds available with an independent implementation of RFC 6330, from what a receiver within
the case's loss bound holds. `../carousel_verify.py` is the second implementation, written from
case holds available with an independent implementation of RFC 6330, and each Raptor object with one
of RFC 5053, from what a receiver within the case's loss bound holds. `../carousel_verify.py` is the second implementation, written from
the extension's text, which also mutates copies of the corpus and drops each of its own checks
in turn and requires each to be caught. `../../claim-conformance/claim_verify.py carousel
vectors.json` reads every claim through jwp-receipts §3.1 steps 1–4.

3957
wai/live-conformance/carousel/vectors.json
View file

File diff suppressed because one or more lines are too long

245
wai/live-conformance/carousel_verify.py
View file

@ -7,13 +7,16 @@ on what they define: group and object receipts from the JWP corpus's
(`../jwp-conformance/jwp_verify.py`, jwp-receipts §2.2, §3), and a claim's steps 1 to 5 from the
claim corpus's (`../claim-conformance/claim_verify.py`, jwp-receipts §3.1), whose step 4 holds
the schedule class's rules with §5.1 and §5.2's division of an object into source blocks.
Everything else this extension adds is written here:
A key log's reading comes from the signer corpus's (`../signer-conformance/signer_verify.py`,
jwp-receipts §7.2). Everything else this extension adds is written here:
- §4.3: a schedule against the group, its object receipts and their envelopes;
- §4.5: when a later schedule extends an earlier one;
- §5.3: what a receiver needs of a block, K′ read from Appendix A of the extension's text;
- §6: the declarations and schedules that stand, each entry's availability within a loss bound,
and statements offered;
- §7: the passes a receiver listens to.
- §7: the passes a receiver listens to;
- §8.1 and §8.2: a schedule's interval, and whether what a receiver acquired suffices.
For every case it requires the outcome the corpus states, field for field; that every reason
of §9's table appears in the corpus; then it mutates copies of the corpus, one defect each, and
@ -33,10 +36,11 @@ import re
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
for d in ("claim-conformance", "jwp-conformance"):
for d in ("claim-conformance", "jwp-conformance", "signer-conformance"):
sys.path.insert(0, os.path.join(HERE, "..", d))
import claim_verify as CV # noqa: E402
import jwp_verify as JV # noqa: E402
import signer_verify as SG # noqa: E402
EXTENSION = os.path.join(HERE, "..", "extensions", "carousel.md")
@ -69,6 +73,7 @@ def k_primes():
K_PRIME = k_primes()
RAPTOR_MARGIN = (8, 16, 28) # §5.3: the symbols beyond K a Raptor block is held at, at each overhead
def k_prime(k):
@ -105,14 +110,15 @@ def check_transmission(c, g):
need(g["signer_pubkey"] == c["signer_pubkey"], "transmission_signer_mismatch")
def check_schedule(c, g, objects):
def check_schedule(c, g, objects, node=False):
"""§4.3 against a group that passed jwp-receipts §3 steps 1 and 2, and the object receipts
held under it with their envelopes where held: [(object, envelope or None)]."""
held under it with their envelopes where held: [(object, envelope or None)]. A node's schedule
of its own instance (cache-authority §5.6) is checked in every step but step 2."""
start = JV.seconds(g["group_start"])
need(start is not None, "context_invalid")
entries = c["body"]["objects"]
need(drops("subject") or JV.link(g).hex() == c["subject"], "schedule_subject_mismatch")
need(drops("signer") or g["signer_pubkey"] == c["signer_pubkey"], "schedule_signer_mismatch")
need(node or drops("signer") or g["signer_pubkey"] == c["signer_pubkey"], "schedule_signer_mismatch")
need(drops("range") or all(e["leaf_index"] < g["leaf_count"] for e in entries), "schedule_leaf_out_of_range")
need(drops("before") or all(e["first_ms"] // 1000 >= start for e in entries), "schedule_before_group")
for e in entries:
@ -124,10 +130,44 @@ def check_schedule(c, g, objects):
need(drops("length") or len(envelope) == e["oti"]["transfer_length"], "schedule_length_mismatch")
# ------------------------------------------------------------------ §4.5
def entry_extends(b, a):
"""§4.5: entry b's passes begin with entry a's: its members are a's but `count`, at least a's,
and `period_ms`, which may differ only where a has one pass."""
if b["count"] < a["count"] and not drops("extension_count"):
return False
period_read = a["count"] >= 2 or drops("extension_one_pass")
if period_read and b["period_ms"] != a["period_ms"] and not drops("extension_period"):
return False
return drops("extension_members") or all(b[k] == v for k, v in a.items() if k not in ("count", "period_ms"))
def extends(b, a):
"""§4.5: schedule b extends schedule a of its instance: for every entry of a, an entry of that
object that extends it. It may have entries for objects a has none for."""
by_leaf = {e["leaf_index"]: e for e in b["objects"]}
def covered(e):
later = by_leaf.get(e["leaf_index"])
return drops("extension_drops") if later is None else entry_extends(later, e)
return all(covered(e) for e in a["objects"])
def monotone(bodies):
"""§6.2 step 4: the schedules of one instance, in chain order: each extends the one before."""
if drops("extension_chain"):
return all(extends(b, bodies[0]) for b in bodies[1:])
return all(extends(b, a) for a, b in zip(bodies, bodies[1:]))
# ------------------------------------------------------------------ §5.3, §6.3
def needed(e, k, h):
"""§5.3: every source symbol for no-code, K′ + h for RaptorQ."""
"""§5.3: every source symbol for no-code, K′ + h for RaptorQ, K and its margin for Raptor."""
if e["fec"] == "raptor" and not drops("raptor_k_prime"):
return k + (h if drops("raptor_margin") else RAPTOR_MARGIN[h])
return k if e["fec"] == "no-code" else k_prime(k) + h
@ -145,9 +185,9 @@ def block_pass(e, k, loss, h):
m = k + e["repair"]
x = missed(loss, m)
need_ = needed(e, k, h)
if e["fec"] == "raptorq" and e["esi"] == "advance":
if e["fec"] != "no-code" and e["esi"] == "advance":
j = CV.ceil_div(need_, m - x)
elif e["fec"] == "raptorq":
elif e["fec"] != "no-code":
j = 1 if m - x >= need_ else None
else:
j = 1 if x == 0 or drops("no_code_loss") else None
@ -162,7 +202,8 @@ def availability(e, loss, h):
"pass": j,
"available_ms": None if j is None else pass_end(e, j),
"bytes_sent": CV.schedule_bytes(e, blocks),
"failure_one_in": [100, 10_000, 1_000_000][h] if e["fec"] == "raptorq" else None,
# §6.3: RFC 6330 §5.8's bound; RFC 5053 states none, and no-code recovers exactly.
"failure_one_in": [100, 10_000, 1_000_000][h] if e["fec"] == "raptorq" or (e["fec"] == "raptor" and drops("raptor_bound")) else None,
}
@ -176,39 +217,22 @@ def bounds(loss, h):
return None
def verdict(case):
bad = bounds(case["loss_ppm"], case["overhead"])
if bad:
return {"outcome": "refuse", "reason": bad}
def read_group(wire):
"""§6.2 step 1's group, or the refusal the corpus states."""
try:
g = JV.read_group(case["group"])
g = JV.read_group(wire)
JV.verify_group(g)
except JV.Refuse as r:
return {"outcome": "refuse", "reason": r.reason}
return None, {"outcome": "refuse", "reason": r.reason}
return g, None
# Step 1: the objects, the same on every bearer.
objs, overdicts = [], []
for o in case["objects"]:
try:
ob = JV.read_object(o["receipt"])
except JV.Refuse as r:
overdicts.append("reject reason=" + r.reason)
continue
env = None if o["envelope"] is None else bytes.fromhex(o["envelope"])
objs.append((ob, env))
try:
if env is None:
JV.inclusion(ob, g)
overdicts.append("inclusion-verified")
else:
JV.accept(ob, g, env)
overdicts.append("delivered-verified")
except JV.Refuse as r:
overdicts.append("reject reason=" + r.reason)
def judge(g, objs, wires):
"""§6.2 steps 2 to 4 over a group and the object receipts held: the claims' labels, and the
claim that stands on each instance, by its `bearer_id`."""
# Step 2: each claim, §3.1 steps 1-5, then step 6.
held, claims, standing_claims = {}, [], []
for wire in case["claims"]:
for wire in wires:
try:
c = CV.verify_claim(wire)
except CV.Reject as r:
@ -250,11 +274,11 @@ def verdict(case):
for i in idx:
claims[i] = "set-aside reason=bearer_contradicted"
# Step 4: the schedules.
# Step 4: the schedules, each instance's in the order of their signer's chain.
schedules = {}
for bid, idx in by_instance("schedule").items():
bodies = [standing_claims[i]["body"] for i in idx]
if not drops("schedule_contradiction") and any(b != bodies[0] for b in bodies):
if not drops("schedule_contradiction") and not monotone(bodies):
aside = "schedule_contradicted"
elif bid in contradicted:
aside = "bearer_contradicted"
@ -268,13 +292,49 @@ def verdict(case):
for i in idx:
claims[i] = "set-aside reason=" + aside
else:
schedules[bid] = bodies[0]
last = idx[0] if drops("extension_last") else idx[-1]
schedules[bid] = standing_claims[last]
for i in idx:
if standing_claims[i]["body"] != standing_claims[last]["body"]:
claims[i] = "extended"
return claims, schedules
def verdict(case):
bad = bounds(case["loss_ppm"], case["overhead"])
if bad:
return {"outcome": "refuse", "reason": bad}
g, refused = read_group(case["group"])
if refused:
return refused
# Step 1: the objects, the same on every bearer.
objs, overdicts = [], []
for o in case["objects"]:
try:
ob = JV.read_object(o["receipt"])
except JV.Refuse as r:
overdicts.append("reject reason=" + r.reason)
continue
env = None if o["envelope"] is None else bytes.fromhex(o["envelope"])
objs.append((ob, env))
try:
if env is None:
JV.inclusion(ob, g)
overdicts.append("inclusion-verified")
else:
JV.accept(ob, g, env)
overdicts.append("delivered-verified")
except JV.Refuse as r:
overdicts.append("reject reason=" + r.reason)
claims, schedules = judge(g, objs, case["claims"])
# Step 5.
loss, h = case["loss_ppm"], case["overhead"]
lines = []
for bid in sorted(schedules):
for e in schedules[bid]["objects"]:
for e in schedules[bid]["body"]["objects"]:
lines.append({"bearer_id": bid, "leaf_index": e["leaf_index"], **availability(e, loss, h)})
# §6.4.
@ -335,7 +395,7 @@ def plan(case):
for b in blocks:
m = ks[b] + e["repair"]
x = missed(loss, m)
if e["fec"] == "raptorq" and e["esi"] == "advance":
if e["fec"] != "no-code" and e["esi"] == "advance":
gains[b] = m - x
else:
gains[b] = max(0, m - n[b] - (0 if drops("again") else x))
@ -353,6 +413,58 @@ def plan(case):
}
# ------------------------------------------------------------------ §8.1, §8.2: what rides in-band
def interval(body):
"""§8.1: the least `period_ms` of the entries of two passes or more, or, where every entry has
one pass, the least `duration_ms`."""
repeated = [e["period_ms"] for e in body["objects"] if e["count"] >= 2 or drops("interval_one_pass")]
if repeated:
return min(repeated)
return min(e["duration_ms"] for e in body["objects"])
def key_log_held(c, logs):
"""§8.2 check 3: a log acquired is held for the origin the claim's `signer_id` names
(jwp-receipts §7.2 rules 1 to 3) and lists the claim's key."""
named = SG.classify(c["signer_id"])
if named[0] != "origin":
return False
text = logs.get(named[1])
if text is None:
return False
try:
r = SG.parse_revision(text)
if not drops("inband_log_held"):
SG.check_held(r, named[1])
except SG.Stop:
return False
return drops("inband_log_lists") or any(e["key"] == c["signer_pubkey"] for e in r["keys"])
def inband(case):
"""§8.2: §6.2 over the group and the set's claims alone, then its three checks in order."""
g, refused = read_group(case["group"])
if refused:
return refused
claims, schedules = judge(g, [], case["claims"])
c = schedules.get(case["bearer_id"])
if any(label.startswith("reject step=5 ") for label in claims) and not drops("inband_chain"):
result = "inband_chain_incomplete"
elif c is None:
result = "inband_not_scheduled"
elif not drops("inband_log") and not key_log_held(c, case["logs"]):
result = "inband_key_log_missing"
else:
result = "suffices"
return {
"outcome": "inband",
"claims": claims,
"set": result,
"inband_ms": interval(c["body"]) if result == "suffices" else None,
}
# ------------------------------------------------------------------ the corpus
def strings(v, out):
@ -377,7 +489,7 @@ def claim_outcome(wire):
def check(doc, quiet=False):
"""The failures of a corpus document: each case's outcome, and every reason of §9."""
fails = []
for kind, run in (("verdicts", verdict), ("claims", lambda c: claim_outcome(c["claim"])), ("plans", plan)):
for kind, run in (("verdicts", verdict), ("claims", lambda c: claim_outcome(c["claim"])), ("plans", plan), ("inband", inband)):
for c in doc[kind]:
try:
got = run(c)
@ -390,7 +502,7 @@ def check(doc, quiet=False):
if not quiet:
print(f"FAIL {c['name']}\n stated {json.dumps(c['outcome'])}\n got {json.dumps(got)}")
stated = []
for kind in ("verdicts", "claims", "plans"):
for kind in ("verdicts", "claims", "plans", "inband"):
for c in doc[kind]:
strings(c["outcome"], stated)
for code in reasons_table():
@ -402,7 +514,7 @@ def check(doc, quiet=False):
def mutations(doc):
"""Copies of the corpus, one defect each, every one of which must fail."""
def case(d, name):
return next(c for k in ("verdicts", "claims", "plans") for c in d[k] if c["name"] == name)
return next(c for k in ("verdicts", "claims", "plans", "inband") for c in d[k] if c["name"] == name)
def entry(c, i):
w = json.loads(c["claims"][i])
@ -443,11 +555,48 @@ def mutations(doc):
def m_reason_gone(d):
d["verdicts"].remove(case(d, "schedule_on_bidirectional"))
return [m_pass, m_bytes, m_failure, m_loss, m_repair, m_statement, m_plan, m_held, m_accept, m_reason_gone]
def m_extended_label(d):
case(d, "extension_adds_passes")["outcome"]["claims"][1] = "stands"
def m_extended_pass(d):
# Under the first schedule's five passes the object is not within the schedule.
c = case(d, "extension_adds_passes")
c["claims"].pop()
def m_withdrawn(d):
# The same schedules in the order that extends: nothing is contradicted.
c = case(d, "extension_withdraws_passes")
c["claims"][1], c["claims"][2] = c["claims"][2], c["claims"][1]
def m_interval(d):
case(d, "suffices")["outcome"]["inband_ms"] = 20_000
def m_set(d):
case(d, "key_log_absent")["logs"] = case(d, "suffices")["logs"]
def m_raptor_fec(d):
# The same parameters under RaptorQ, which object 1 at 30% would reach.
c = case(d, "raptor_30_percent")
w = json.loads(c["claims"][1])
w["body"]["objects"][1]["fec"] = "raptorq"
c["claims"][1] = json.dumps(w, separators=(",", ":"), sort_keys=True)
def m_raptor_bound(d):
case(d, "raptor_lossless")["outcome"]["lines"][0]["failure_one_in"] = 1_000_000
def m_set_reason_gone(d):
d["inband"] = [c for c in d["inband"] if c["outcome"].get("set") != "inband_chain_incomplete"]
return [m_pass, m_bytes, m_failure, m_loss, m_repair, m_statement, m_plan, m_held, m_accept, m_reason_gone,
m_extended_label, m_extended_pass, m_withdrawn, m_interval, m_set, m_set_reason_gone, m_raptor_fec,
m_raptor_bound]
WEAKENINGS = ["subject", "signer", "range", "before", "content", "length", "no_code_loss", "schedule_contradiction",
"undeclared", "bidirectional", "deadline", "holding", "late_pass", "again"]
"undeclared", "bidirectional", "deadline", "holding", "late_pass", "again", "extension_count",
"extension_one_pass", "extension_period", "extension_members", "extension_drops", "extension_chain",
"extension_last", "interval_one_pass", "inband_chain", "inband_log", "inband_log_held",
"inband_log_lists", "raptor_k_prime", "raptor_margin", "raptor_bound"]
def main(argv):
@ -462,8 +611,8 @@ def main(argv):
fails.append(f"files the corpus does not hold: {extra}")
if len(K_PRIME) != 477 or any(a >= b for a, b in zip(K_PRIME, K_PRIME[1:])) or K_PRIME[-1] != CV.K_PRIME_MAX:
fails.append("Appendix A is not 477 increasing values ending at K'max")
print(f"{len(doc['verdicts'])} verdict cases, {len(doc['claims'])} claim cases, {len(doc['plans'])} plans: "
f"{len(fails)} failure(s)")
print(f"{len(doc['verdicts'])} verdict cases, {len(doc['claims'])} claim cases, {len(doc['plans'])} plans, "
f"{len(doc['inband'])} in-band sets: {len(fails)} failure(s)")
caught = 0
for m in mutations(doc):
d = copy.deepcopy(doc)

2
wai/live-conformance/timeliness/README.md
View file

@ -44,7 +44,7 @@ give entry 0 the ladder E's object 0 at 700,000, then F's object 0 at 400,000.
string `from_ms`; the `sources` in the asker's order, each exactly `{kind: "bearer", bearer_id}`,
`{kind: "holder", holder, fetch_ms}` or `{kind: "origin", fetch_ms}`; and the `wanted` objects,
each `{group, index}`. `outcome` is `refuse reason=R`, or gives `claims` (`accepted`,
`declared`, `stands`, `set-aside reason=…`, `reject step=N reason=R`), `records` as a verdict's,
`declared`, `stands`, `extended`, `set-aside reason=…`, `reject step=N reason=R`), `records` as a verdict's,
and `choices`, one for each wanted object: `{outcome, object, rung, source, arrival_ms,
value_ppm}`, with `object` `primary`, `fallback` or `null`, `rung` the rung's index in the ladder
or `null` unless `object` is `fallback`, and `source` an index into `sources` or `null`, or

1510
wai/live-conformance/timeliness/vectors.json
View file

File diff suppressed because it is too large Load diff

30
wai/live-conformance/timeliness_verify.py
View file

@ -573,7 +573,7 @@ def schedules_step(labels, accepted):
schedules = {}
for k, idx in by_instance("schedule").items():
bodies = [accepted[i]["body"] for i in idx]
if any(b != bodies[0] for b in bodies):
if not KV.monotone(bodies) and not drops("schedule_contradicted"):
aside = "schedule_contradicted"
elif k in contradicted:
aside = "bearer_contradicted"
@ -583,10 +583,14 @@ def schedules_step(labels, accepted):
aside = "schedule_on_bidirectional"
else:
aside = None
# The last of an instance's schedules is the one that stands (carousel §6.2 step 4).
last = bodies[0] if drops("schedule_last") else bodies[-1]
for i in idx:
labels[i] = "stands" if aside is None else "set-aside reason=" + aside
if aside is None:
schedules[(accepted[i]["subject"], accepted[i]["body"]["bearer_id"])] = bodies[0]
if aside is not None:
labels[i] = "set-aside reason=" + aside
else:
labels[i] = "stands" if accepted[i]["body"] == last else "extended"
schedules[(accepted[i]["subject"], accepted[i]["body"]["bearer_id"])] = last
return schedules
@ -991,6 +995,18 @@ def mutations(doc):
def m_reason_gone(d):
d["verdicts"].remove(case(d, "record_before_group"))
def m_extended(d):
case(d, "plan_bearer_extended")["outcome"]["claims"][1] = "stands"
def m_withdrawn(d):
# The withdrawn pass's schedule given first and last alike: one body, which stands.
c = case(d, "plan_bearer_withdrawn")
c["claims"][2] = c["claims"][1]
def m_rung_extended(d):
# The rung above the one the extended schedule reaches.
case(d, "ladder_rung_on_extended_schedule")["outcome"]["choices"][0]["rung"] = 0
def m_rung_value(d):
# The lower rung presented at the higher's value.
case(d, "ladder_lower_rung_alone")["outcome"]["intents"][0]["lines"][0]["value_ppm"] = 700_000
@ -1036,7 +1052,8 @@ def mutations(doc):
d["runs"] = [c for c in d["runs"] if c["outcome"] != "refuse reason=arrivals_invalid"]
return [m_instant, m_end, m_floor, m_accept, m_line, m_earliest, m_statement, m_envelope, m_order, m_deadline,
m_fetch, m_loss, m_from_ms, m_object, m_reason_gone, m_rung_value, m_rung_instant, m_rung_form, m_plan_rung,
m_fetch, m_loss, m_from_ms, m_object, m_reason_gone, m_extended, m_withdrawn, m_rung_extended, m_rung_value,
m_rung_instant, m_rung_form, m_plan_rung,
m_arrival_late, m_arrival_gone, m_decision_value, m_decision_action, m_plan_dropped, m_plan_instant,
m_decision_order, m_reason_gone_run]
@ -1046,7 +1063,8 @@ WEAKENINGS = ["subject", "signer", "range", "before", "same_second", "content",
"least", "least_fallback", "own_statements", "key_order", "deadline_inclusive", "fallback_deadline",
"sink_rule", "floor", "no_intent", "loss_bound",
"overhead", "instant", "sources_empty", "sources_kind", "sources_uri", "sources_key", "sources_fetch",
"sources_twice", "sources_loss", "per_group", "undeclared", "bidirectional", "record_at_s",
"sources_twice", "sources_loss", "per_group", "schedule_contradicted", "schedule_last", "undeclared",
"bidirectional", "record_at_s",
"holder_record", "plan_order", "fallback_value", "fallback_tie", "fallback_order",
"plan_fallback_deadline", "late_zero", "late_tie",
"rung_layer", "rung_position", "rung_derived", "rung_holders", "rung_order",

7
wai/wai-rs/Cargo.lock generated
View file

@ -3393,6 +3393,12 @@ version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d20581732dd76fa913c7dff1a2412b714afe3573e94d41c34719de73337cc8ab"
[[package]]
name = "raptor-code"
version = "1.0.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d82d8b31384add1a3f5a235b17323f5b71cfc81bbbef5aeca9e6425f70b8f542"
[[package]]
name = "raptorq"
version = "2.0.1"
@ -4723,6 +4729,7 @@ dependencies = [
"opus",
"ort",
"pkg-config",
"raptor-code",
"raptorq",
"rav1e",
"regex",

7
wai/wai-rs/Cargo.toml
View file

@ -613,10 +613,10 @@ influence = ["contribution", "settlement", "ai_usage"]
bearer_receipts = ["key_discovery"]
# Carousel schedule (extensions/carousel.md): `wai.bearer.carousel` — when each object of a group
# passes on a bearer instance with no return path, and with which forward error correction
# (RaptorQ, RFC 6330; Compact No-Code, RFC 5445). The jwp-receipts `schedule` claim (its body is
# (RaptorQ, RFC 6330; Raptor, RFC 5053; Compact No-Code, RFC 5445). The jwp-receipts `schedule` claim (its body is
# claim_receipt's), when a receiver within a loss bound holds each object, and the passes a
# receiver listens to. Integer arithmetic over the codes' parameters only; no new dependency
# (its tests encode and decode with an RFC 6330 implementation, a dev-dependency).
# (its tests encode and decode with an RFC 6330 and an RFC 5053 implementation, dev-dependencies).
carousel = ["bearer_receipts"]
# Holdings (extensions/holdings.md): `wai.delivery.holdings` — a node's signed statement of which
# objects of which groups it holds now (`WHD1`: per group, whole or a bitmap), held to an instant
@ -1928,6 +1928,9 @@ wai-ops = { path = "../wai-ops", features = ["oracle"] }
# RaptorQ object of the carousel corpus with and decodes from what a receiver within its loss
# bound holds.
raptorq = "2.0.1"
# An independent implementation of RFC 5053 (MIT), which tests/carousel.rs encodes each Raptor object
# of the carousel corpus with and decodes from what a receiver within its loss bound holds.
raptor-code = "1.0.12"
# The project file's schema (wai/schema/) states its name rule as a regular expression; the
# reader's tests run it, as a validator of the schema does, beside the reader's own rule.
regex = "1"

125
wai/wai-rs/src/cache_authority.rs
View file

@ -5,8 +5,9 @@
//! `cache/serve` in its operator's key log, and a node that declares a bearer instance of its own
//! holds the class authority `cache-transmission`. This module places a server key by the heads a
//! relying party names, in jwp-receipts' three levels ([`place`], §4.4); derives the bearer
//! verdict with the caches' instances beside the publisher's ([`verdict`], §7), the
//! `cache-transmission` claim's body and context checks being `crate::claim_receipt::bearer`'s;
//! verdict with the caches' instances and their schedules beside the publisher's ([`verdict`],
//! §5.6, §7), the `cache-transmission` claim's body and context checks being
//! `crate::claim_receipt::bearer`'s and a schedule's `crate::claim_receipt::schedule`'s;
//! and counts, of a population of receiver keys stated for a region, how many held an object by a
//! deadline on their own signed word ([`tally`], §8), read as timeliness' held-at statements.
//!
@ -14,7 +15,9 @@
//!
//! - A placement labels a server and never changes a verdict or a hit.
//! - A cache's instance never makes a group `transmitted`, and a cache is credited per instance,
//! only for the objects it says it sent.
//! only for the objects it says it sent, whatever its schedule names.
//! - A node's schedule stands only beside the instance it declares, and another key's schedule of
//! that instance contradicts nothing.
//! - Every instant a held-at statement gives is the latest its signed word allows, rounded up to
//! the millisecond; a statement before its group is set aside, so a receiver whose clock was
//! never set is never on time.
@ -29,6 +32,7 @@ use crate::bearer_receipts::{
self, per_receiver_upper_bound, Assertion, BearerLine, ClaimVerdict as BearerVerdict, Instance, Status, BEARER_CONTRADICTED,
BEARER_NOT_DECLARED, DELIVERY_EXCEEDS_RECONCILED, DELIVERY_NOT_RECONCILED,
};
use crate::carousel::Statement;
use crate::claim_receipt::schedule::LAST_INSTANT_MS;
use crate::claim_receipt::{hex, Bearer, ClaimBody, ClaimReceipt, HeldGroup, Reason as ClaimReason, Rejection, TimelinessEntry};
use crate::holdings::{self, Holdings, RecordVerdict};
@ -50,6 +54,7 @@ pub const CODES: &[&str] = &[
"cache_transmission_leaf_count_mismatch",
"cache_transmission_objects_not_held",
"cache_transmission_root_mismatch",
CACHE_SCHEDULE_SIGNER_MISMATCH,
POPULATION_INVALID,
TARGET_INVALID,
OBJECT_NOT_VERIFIED,
@ -57,6 +62,7 @@ pub const CODES: &[&str] = &[
];
pub const SERVER_NOT_LISTED: &str = "server_not_listed";
pub const CACHE_SCHEDULE_SIGNER_MISMATCH: &str = "cache_schedule_signer_mismatch";
pub const POPULATION_INVALID: &str = "population_invalid";
pub const TARGET_INVALID: &str = "target_invalid";
pub const OBJECT_NOT_VERIFIED: &str = "object_not_verified";
@ -211,12 +217,22 @@ pub fn window_instant(end_us: u64) -> (u64, u64) {
// ───────────────────────────── §7: the verdict ─────────────────────────────
/// The inputs of §7.1: bearer-receipts §6.1's, with cache-transmission claims among the claims,
/// and the heads of §4.4, keyed by origin.
/// The inputs of §7.1: bearer-receipts §6.1's, with cache-transmission and schedule claims among
/// the claims, the heads of §4.4, keyed by origin, and optionally a statement of carousel §6.4.
#[derive(Clone, Debug, Default)]
pub struct Inputs {
pub bearer: bearer_receipts::Inputs,
pub heads: BTreeMap<String, String>,
pub held: Option<Held>,
}
/// A statement of carousel §6.4 offered about an instance, publisher's or node's, at a loss bound
/// and an overhead (§7.3).
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Held {
pub statement: Statement,
pub loss_ppm: u64,
pub overhead: u64,
}
/// What became of one claim (§7.2).
@ -226,6 +242,12 @@ pub enum ClaimVerdict {
Bearer(BearerVerdict),
/// A cache-transmission claim whose instance stands.
CacheDeclared,
/// A schedule that stands: the publisher's beside its declaration (carousel §6.2 step 4), a
/// node's beside its own instance (§5.6).
Stands,
/// A schedule a later schedule of its instance, by the same signer, extends (carousel §4.5)
/// with another body.
Extended,
}
impl ClaimVerdict {
@ -233,6 +255,8 @@ impl ClaimVerdict {
match self {
ClaimVerdict::Bearer(v) => v.label(),
ClaimVerdict::CacheDeclared => "cache-declared".into(),
ClaimVerdict::Stands => "stands".into(),
ClaimVerdict::Extended => "extended".into(),
}
}
}
@ -276,6 +300,9 @@ pub struct Verdict {
pub sink: Option<&'static str>,
/// bearer-receipts §6.4: `Ok` when the statement holds.
pub assertion: Option<Result<(), &'static str>>,
/// Carousel §6.4 against the schedule that stands on the statement's instance (§7.3): `Ok`
/// when it holds.
pub held: Option<Result<(), &'static str>>,
}
/// What [`judge`] keeps for the tally: the group, the accepted claims and the instances.
@ -320,11 +347,16 @@ fn judge(inp: &Inputs, count_only: Option<&[[u8; 32]]>) -> Result<Judged, Delive
// Step 1.
let (group, receipts, objects) = bearer_receipts::group_and_objects(&b.group, &b.objects)?;
let refs: Vec<&ObjectReceipt> = receipts.iter().collect();
// Step 2.
// Step 2. A schedule is checked against the object receipts and the envelopes held.
let parsed: Vec<(ObjectReceipt, Option<&[u8]>)> =
b.objects.iter().filter_map(|o| ObjectReceipt::from_json(&o.receipt).ok().map(|r| (r, o.envelope.as_deref()))).collect();
let with_bytes: Vec<(&ObjectReceipt, Option<&[u8]>)> = parsed.iter().map(|(r, e)| (r, *e)).collect();
let checked = check_claims(&b.claims, |c| match &c.body {
ClaimBody::Transmission(_) => c.check_transmission_against(&group),
ClaimBody::Reconciliation(_) => c.check_reconciliation_against(&group, &refs),
ClaimBody::CacheTransmission(_) => c.check_cache_transmission_against(&group, &refs),
ClaimBody::Schedule(_) if c.signer_pubkey == group.signer_pubkey => c.check_schedule_against(&group, &with_bytes),
ClaimBody::Schedule(_) => c.check_node_schedule_against(&group, &with_bytes),
_ => Err(Rejection { step: 6, reason: ClaimReason::ContextMismatch }),
});
let mut claims: Vec<ClaimVerdict> = Vec::with_capacity(checked.len());
@ -334,6 +366,7 @@ fn judge(inp: &Inputs, count_only: Option<&[[u8; 32]]>) -> Result<Judged, Delive
Ok(c) => {
claims.push(match &c.body {
ClaimBody::CacheTransmission(_) => ClaimVerdict::CacheDeclared,
ClaimBody::Schedule(_) => ClaimVerdict::Stands,
_ => ClaimVerdict::Bearer(BearerVerdict::Declared),
});
accepted.push(Some(c));
@ -394,6 +427,64 @@ fn judge(inp: &Inputs, count_only: Option<&[[u8; 32]]>) -> Result<Judged, Delive
}
}
// Step 4, the schedules: the publisher's by carousel §6.2 step 4 over its declarations, and
// each node's by §5.6 beside its own instance.
let publisher: Vec<Option<ClaimReceipt>> = accepted
.iter()
.map(|c| {
c.as_ref()
.filter(|c| c.transmission().is_some() || (c.schedule().is_some() && c.signer_pubkey == group.signer_pubkey))
.cloned()
})
.collect();
let of_publisher = crate::carousel::standing(&publisher);
let mut schedules: BTreeMap<String, usize> = of_publisher.schedules.clone();
for (&i, &reason) in &of_publisher.aside {
if publisher[i].as_ref().and_then(ClaimReceipt::schedule).is_some() {
claims[i] = ClaimVerdict::Bearer(BearerVerdict::SetAside(reason));
}
}
for &i in &of_publisher.extended {
claims[i] = ClaimVerdict::Extended;
}
let mut nodes: BTreeMap<String, Vec<usize>> = BTreeMap::new();
for (i, c) in accepted.iter().enumerate() {
if let Some(s) = c.as_ref().filter(|c| c.signer_pubkey != group.signer_pubkey).and_then(ClaimReceipt::schedule) {
nodes.entry(s.bearer_id.clone()).or_default().push(i);
}
}
for (id, idx) in &nodes {
let aside = |claims: &mut Vec<ClaimVerdict>, idx: &[usize], reason: &'static str| {
for &i in idx {
claims[i] = ClaimVerdict::Bearer(BearerVerdict::SetAside(reason));
}
};
if by_instance.contains_key(id) || contradicted.contains(id) {
aside(&mut claims, idx, BEARER_CONTRADICTED);
continue;
}
let Some(of) = cache_standing.get(id) else {
aside(&mut claims, idx, BEARER_NOT_DECLARED);
continue;
};
let server = accepted[of[0]].as_ref().expect("an accepted claim").signer_pubkey;
let (mine, others): (Vec<usize>, Vec<usize>) =
idx.iter().partition(|&&i| accepted[i].as_ref().is_some_and(|c| c.signer_pubkey == server));
aside(&mut claims, &others, CACHE_SCHEDULE_SIGNER_MISMATCH);
if mine.is_empty() {
continue;
}
match crate::carousel::monotone(&accepted, &mine) {
Err(reason) => aside(&mut claims, &mine, reason),
Ok((last, extended)) => {
schedules.insert(id.clone(), last);
for i in extended {
claims[i] = ClaimVerdict::Extended;
}
}
}
}
// Step 5: the reconciliations, over the instances that stand.
let mut instances: BTreeMap<String, (Instance, Option<Vec<u64>>)> = BTreeMap::new();
for (id, &i) in &standing {
@ -489,6 +580,12 @@ fn judge(inp: &Inputs, count_only: Option<&[[u8; 32]]>) -> Result<Judged, Delive
}
}
});
// §7.3: carousel §6.4 against the schedule that stands on the statement's instance.
let held = inp.held.as_ref().map(|h| {
let (Statement::Object { bearer_id, .. } | Statement::Group { bearer_id, .. }) = &h.statement;
let s = schedules.get(bearer_id).and_then(|&i| accepted[i].as_ref()).and_then(ClaimReceipt::schedule);
crate::carousel::statement_holds(s, group.leaf_count, &h.statement, h.loss_ppm, h.overhead)
});
let verdict = Verdict {
objects,
claims,
@ -500,6 +597,7 @@ fn judge(inp: &Inputs, count_only: Option<&[[u8; 32]]>) -> Result<Judged, Delive
caches,
sink,
assertion,
held,
};
Ok(Judged { group, accepted, instances, count, verdict })
}
@ -540,7 +638,20 @@ pub fn check_claim(earlier: &[String], wire: &str, context: Option<&Value>, obje
};
let receipts: Vec<ObjectReceipt> = objects.iter().filter_map(|o| ObjectReceipt::from_json(o).ok()).collect();
let refs: Vec<&ObjectReceipt> = receipts.iter().collect();
match c.check_cache_transmission_against(&group, &refs) {
let checked = match &c.body {
// A schedule by the group's key is the publisher's (carousel §4.3); by another, a node's
// (§5.6). No envelope is held here.
ClaimBody::Schedule(_) => {
let held: Vec<(&ObjectReceipt, Option<&[u8]>)> = refs.iter().map(|r| (*r, None)).collect();
if c.signer_pubkey == group.signer_pubkey {
c.check_schedule_against(&group, &held)
} else {
c.check_node_schedule_against(&group, &held)
}
}
_ => c.check_cache_transmission_against(&group, &refs),
};
match checked {
Ok(()) => "accept".into(),
Err(r) => reject(r),
}

414
wai/wai-rs/src/cache_authority/corpus.rs
View file

@ -12,10 +12,11 @@
use super::*;
use crate::bearer_receipts::HeldObject;
use crate::claim_receipt::bearer::{ATTESTS_RECEIVED, ATTESTS_TRANSMITTED};
use crate::claim_receipt::schedule::ATTESTS_SCHEDULED;
use crate::claim_receipt::timeliness::ATTESTS_VALUED;
use crate::claim_receipt::{
unhex, CacheTransmissionClaim, DecayPoint, Fallback, ReconciliationClaim, SessionEnergy, SubjectType, TimelinessClaim,
TransmissionClaim,
unhex, CacheTransmissionClaim, DecayPoint, Esi, Fallback, Ladder, Oti, ReconciliationClaim, ScheduleClaim, ScheduleEntry, SessionEnergy, SubjectType,
TimelinessClaim, TransmissionClaim,
};
use crate::container::{Conditioning, DerivedFrom, Manifest, ModelRequirement, Wai};
use crate::energy_class::{CounterFiltering, EnergyProvenance, Uncertainty};
@ -340,6 +341,41 @@ fn reconciliation(p: &Publication, bearer: Bearer, id: &str, idx: &[u64]) -> Cla
})
}
/// When a schedule's first pass starts here: an hour after the group's.
const PASSES_FROM: u64 = START_MS + HOUR;
/// An entry for object `i` of `p`: RaptorQ in one block of 16 source symbols of 128 octets (K′ 18)
/// and 8 repair symbols a pass, fresh symbols each pass, a pass of a second every ten seconds. A
/// receiver at overhead 2 needs 20 symbols, and within 30% gets 17 a pass (two passes), within 90%
/// gets 3 (seven passes).
fn schedule_entry(p: &Publication, i: usize, count: u64) -> ScheduleEntry {
ScheduleEntry {
leaf_index: i as u64,
content_hash: p.objects[i].1.content_hash,
oti: Oti::RaptorQ { transfer_length: LENGTH, symbol_size: 128, source_blocks: 1, sub_blocks: 1, alignment: 4 },
repair: 8,
esi: Esi::Advance,
first_ms: PASSES_FROM,
period_ms: 10_000,
duration_ms: 1_000,
count,
}
}
/// The end of pass `j` (from 1) of a [`schedule_entry`].
fn pass_end(j: u64) -> u64 {
PASSES_FROM + (j - 1) * 10_000 + 1_000
}
fn schedule_body(id: &str, objects: Vec<ScheduleEntry>) -> ClaimBody {
ClaimBody::Schedule(ScheduleClaim { bearer_id: id.into(), objects, attests: ATTESTS_SCHEDULED.into() })
}
/// A schedule on `id` of the given objects of `p`, with `count` passes each.
fn schedule(p: &Publication, id: &str, objects: &[usize], count: u64) -> ClaimBody {
schedule_body(id, objects.iter().map(|&i| schedule_entry(p, i, count)).collect())
}
/// P's timeliness claim about G: entries for `objects`, each of use from the group's start, by
/// `by_ms`, a hard deadline.
fn timeliness_claim(g: &Publication, objects: &[usize], by_ms: u64) -> String {
@ -379,7 +415,7 @@ impl Chains {
fn time(body: &ClaimBody) -> &'static str {
match body {
ClaimBody::Transmission(_) => TRANSMITTED,
ClaimBody::CacheTransmission(_) => CACHE_SENT,
ClaimBody::CacheTransmission(_) | ClaimBody::Schedule(_) => CACHE_SENT,
_ => RECONCILED,
}
}
@ -606,6 +642,89 @@ fn claim_cases(g: &Groups) -> Vec<ClaimCase> {
vec![],
"unchecked",
);
// §5.6: a node's schedule of its own instance.
push(
"schedule_by_node",
"C1 schedules objects 1 and 3 of G on its instance: its key is not the group's, and carousel §4.3 otherwise holds.",
vec![],
one(C1, &g.g, schedule(&g.g, MESH_C1, &[1, 3], 3)),
ctx(&g.g),
all(&g.g),
"accept",
);
let mut ch = Chains::default();
let first = ch.sign(C1, g.g.link(), cache(&g.g, mc, MESH_C1, &[1, 3], None));
let second = ch.sign(C1, g.g.link(), schedule(&g.g, MESH_C1, &[1, 3], 3));
push(
"schedule_chained_after_its_claim",
"The same schedule, chained after C1's cache-transmission claim.",
vec![first],
second,
ctx(&g.g),
all(&g.g),
"accept",
);
push(
"schedule_by_publisher",
"P's schedule of G is checked by carousel §4.3 whole.",
vec![],
one(P, &g.g, schedule(&g.g, SAT, &[1, 3], 3)),
ctx(&g.g),
all(&g.g),
"accept",
);
push(
"schedule_node_other_group",
"C1's schedule names G9.",
vec![],
one(C1, &g.g9, schedule(&g.g9, MESH_C1, &[1, 3], 3)),
ctx(&g.g),
all(&g.g),
"reject step=6 reason=schedule_subject_mismatch",
);
let mut past = schedule_entry(&g.g, 3, 3);
past.leaf_index = 4;
push(
"schedule_node_leaf_out_of_range",
"An entry for object 4 of a group of four.",
vec![],
one(C1, &g.g, schedule_body(MESH_C1, vec![past])),
ctx(&g.g),
all(&g.g),
"reject step=6 reason=schedule_leaf_out_of_range",
);
let mut early = schedule_entry(&g.g, 1, 3);
early.first_ms = START_MS - 1_000;
push(
"schedule_node_before_group",
"A first pass a second before the group starts.",
vec![],
one(C1, &g.g, schedule_body(MESH_C1, vec![early])),
ctx(&g.g),
all(&g.g),
"reject step=6 reason=schedule_before_group",
);
let mut wrong = schedule_entry(&g.g, 1, 3);
wrong.content_hash = g.g.objects[2].1.content_hash;
push(
"schedule_node_content",
"An entry for object 1 with object 2's content hash, its receipt held.",
vec![],
one(C1, &g.g, schedule_body(MESH_C1, vec![wrong])),
ctx(&g.g),
all(&g.g),
"reject step=6 reason=schedule_content_mismatch",
);
push(
"schedule_node_no_passes",
"An entry of no passes: carousel §4.2 rule 3.3 holds for a node's schedule too.",
vec![],
bad(C1, &g.g, schedule(&g.g, MESH_C1, &[1], 0)),
ctx(&g.g),
all(&g.g),
"reject step=4 reason=schedule_passes_invalid",
);
out
}
@ -697,10 +816,11 @@ struct Exp {
caches: Vec<CLine>,
sink: Option<&'static str>,
assertion: Option<&'static str>,
held: Option<&'static str>,
}
fn exp(claims: &[&'static str], status: &'static str) -> Exp {
Exp { claims: claims.to_vec(), status, sinks: 0, whole: 0, received_by: [0; 4], bearers: vec![], caches: vec![], sink: None, assertion: None }
Exp { claims: claims.to_vec(), status, sinks: 0, whole: 0, received_by: [0; 4], bearers: vec![], caches: vec![], sink: None, assertion: None, held: None }
}
impl Exp {
@ -718,6 +838,10 @@ impl Exp {
self.caches.push(l);
self
}
fn held(mut self, h: &'static str) -> Exp {
self.held = Some(h);
self
}
fn value(&self) -> Value {
json!({
"objects": [IV, IV, IV, IV],
@ -738,6 +862,7 @@ impl Exp {
})).collect::<Vec<_>>(),
"sink": self.sink,
"assertion": self.assertion,
"held": self.held,
})
}
}
@ -760,6 +885,7 @@ fn verdict_cases(g: &Groups) -> Vec<VerdictCase> {
let inputs = |claims: Vec<String>| Inputs {
bearer: bearer_receipts::Inputs { group: gg.group.to_json(), objects: gg.held(&[]), claims, ..Default::default() },
heads: both(),
held: None,
};
let mut push = |name, description, inputs: Inputs, outcome: Value| out.push(VerdictCase { name, description, inputs, outcome });
let (mc, uni) = (Bearer::Multicast, Bearer::Unidirectional);
@ -1012,6 +1138,207 @@ fn verdict_cases(g: &Groups) -> Vec<VerdictCase> {
inputs(vec![timeliness_claim(gg, &[1], D)]),
exp(&["reject step=6 reason=context_mismatch"], "published").value(),
);
// §5.6: a node's schedule of its own instance, and §7.3's statement against it.
let of_object = |id: &str, index: u64, by_ms: u64, loss_ppm: u64| {
Some(Held { statement: Statement::Object { bearer_id: id.into(), index, by_ms }, loss_ppm, overhead: 2 })
};
let with_held = |claims: Vec<String>, held: Option<Held>| {
let mut i = inputs(claims);
i.held = held;
i
};
let mesh_line = || cline(MESH_C1, mc, C1, c1_confirmed(), &[1, 3], 0, &[(1, 0), (3, 0)]);
let mesh_plan = |ch: &mut Chains, count: u64| ch.sign(C1, link, schedule(gg, MESH_C1, &[1, 3], count));
let scheduled = |by_ms: u64, loss_ppm: u64| {
let mut ch = Chains::default();
let claims = vec![mesh(&mut ch, &[1, 3]), mesh_plan(&mut ch, 3)];
with_held(claims, of_object(MESH_C1, 1, by_ms, loss_ppm))
};
push(
"node_schedule_stands",
"C1 declares mesh-c1 and schedules objects 1 and 3 on it: its schedule stands, and at 30% object 1 is held at the end of the second pass.",
scheduled(pass_end(2), 300_000),
exp(&["cache-declared", "stands"], "published").cache(mesh_line()).held("holds").value(),
);
push(
"node_schedule_after_deadline",
"The same statement a millisecond before that pass ends.",
scheduled(pass_end(2) - 1, 300_000),
exp(&["cache-declared", "stands"], "published").cache(mesh_line()).held("reject reason=availability_after_deadline").value(),
);
push(
"node_schedule_not_within",
"At 90% object 1 needs seven passes and the schedule has three.",
scheduled(pass_end(3), 900_000),
exp(&["cache-declared", "stands"], "published").cache(mesh_line()).held("reject reason=availability_not_within_schedule").value(),
);
push(
"node_schedule_loss_bound_invalid",
"A statement at a loss bound of a million: the statement is refused, and nothing else changes.",
scheduled(pass_end(3), 1_000_000),
exp(&["cache-declared", "stands"], "published").cache(mesh_line()).held("reject reason=loss_bound_invalid").value(),
);
{
let mut i = scheduled(pass_end(3), 0);
i.held.as_mut().expect("a statement").overhead = 3;
push(
"node_schedule_overhead_invalid",
"A statement at an overhead of 3.",
i,
exp(&["cache-declared", "stands"], "published").cache(mesh_line()).held("reject reason=overhead_invalid").value(),
);
}
{
let mut i = scheduled(pass_end(3), 0);
i.held = of_object(MESH_C1, 0, pass_end(3), 0);
push(
"node_schedule_object_not_scheduled",
"Object 0, which C1's schedule has no entry for.",
i,
exp(&["cache-declared", "stands"], "published").cache(mesh_line()).held("reject reason=object_not_scheduled").value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![mesh_plan(&mut ch, 3), mesh(&mut ch, &[1, 3])];
push(
"node_schedule_before_its_claim",
"A plan signed before the sending it plans, its cache-transmission claim after: one instance, and the schedule stands.",
with_held(claims, of_object(MESH_C1, 3, pass_end(1), 0)),
exp(&["stands", "cache-declared"], "published").cache(mesh_line()).held("holds").value(),
);
}
{
let mut ch = Chains::default();
let every = vec![
mesh(&mut ch, &[0, 1, 2, 3]),
ch.sign(C1, link, schedule(gg, MESH_C1, &[0, 1, 2, 3], 3)),
];
let mut i = inputs(every);
i.held = Some(Held { statement: Statement::Group { bearer_id: MESH_C1.into(), by_ms: pass_end(2) }, loss_ppm: 300_000, overhead: 2 });
push(
"node_schedule_group",
"Every object sent and scheduled: the group is held at the end of the second pass at 30%.",
i,
exp(&["cache-declared", "stands"], "published")
.cache(cline(MESH_C1, mc, C1, c1_confirmed(), &[0, 1, 2, 3], 0, &[(0, 0), (1, 0), (2, 0), (3, 0)]))
.held("holds")
.value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![mesh(&mut ch, &[1, 3]), mesh_plan(&mut ch, 3), mesh_plan(&mut ch, 7)];
push(
"node_schedule_extended",
"C1 adds passes: at 90% object 1 is held at the seventh's end, and the earlier schedule is extended (carousel §4.5).",
with_held(claims, of_object(MESH_C1, 1, pass_end(7), 900_000)),
exp(&["cache-declared", "extended", "stands"], "published").cache(mesh_line()).held("holds").value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![mesh(&mut ch, &[1, 3]), mesh_plan(&mut ch, 3), mesh_plan(&mut ch, 2)];
push(
"node_schedule_contradicted",
"C1 withdraws a pass: both schedules set aside, its instance stands, and no schedule is there to hold a statement.",
with_held(claims, of_object(MESH_C1, 1, pass_end(2), 0)),
exp(&["cache-declared", "set-aside reason=schedule_contradicted", "set-aside reason=schedule_contradicted"], "published")
.cache(mesh_line())
.held("reject reason=object_not_scheduled")
.value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![mesh(&mut ch, &[1, 3]), ch.sign(C2, link, schedule(gg, MESH_C1, &[1, 3], 3))];
push(
"node_schedule_other_signer",
"C2 schedules C1's instance: set aside, and no schedule stands there.",
with_held(claims, of_object(MESH_C1, 1, pass_end(2), 0)),
exp(&["cache-declared", "set-aside reason=cache_schedule_signer_mismatch"], "published")
.cache(mesh_line())
.held("reject reason=object_not_scheduled")
.value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![mesh(&mut ch, &[1, 3]), mesh_plan(&mut ch, 3), ch.sign(C2, link, schedule(gg, MESH_C1, &[1, 3], 2))];
push(
"node_schedule_stranger_contradicts_nothing",
"C2's schedule of C1's instance, with fewer passes, beside C1's: set aside alone, and C1's stands.",
with_held(claims, of_object(MESH_C1, 1, pass_end(1), 0)),
exp(&["cache-declared", "stands", "set-aside reason=cache_schedule_signer_mismatch"], "published").cache(mesh_line()).held("holds").value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![sat(&mut ch), ch.sign(C1, link, schedule(gg, SAT, &[1, 3], 3))];
push(
"node_schedule_on_publisher_instance",
"C1 schedules sat, P's instance: set aside, as a cache-transmission claim of it would be, and P's declaration stands.",
inputs(claims),
exp(&["declared", "set-aside reason=bearer_contradicted"], "transmitted").bearer((SAT, uni, 0, None, None)).value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![mesh(&mut ch, &[1]), mesh(&mut ch, &[1, 3]), mesh_plan(&mut ch, 3)];
push(
"node_schedule_on_contradicted_instance",
"C1's account of mesh-c1 contradicts itself: its schedule of it is set aside with its claims.",
inputs(claims),
exp(&["set-aside reason=bearer_contradicted", "set-aside reason=bearer_contradicted", "set-aside reason=bearer_contradicted"], "published").value(),
);
}
{
let claims = vec![Chains::default().sign(C1, link, schedule(gg, MESH_C1, &[1, 3], 3))];
push(
"node_schedule_not_declared",
"C1 schedules an instance it has not declared: set aside.",
with_held(claims, of_object(MESH_C1, 1, pass_end(2), 0)),
exp(&["set-aside reason=bearer_not_declared"], "published").held("reject reason=object_not_scheduled").value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![
mesh(&mut ch, &[1, 3]),
ch.sign(C1, link, schedule(gg, MESH_C1, &[0, 1, 2, 3], 3)),
rec(&mut ch, R1, mc, MESH_C1, &[1]),
];
push(
"node_schedule_credits_nothing",
"C1 schedules every object and says it sent 1 and 3: its instance is credited for those, whatever it scheduled.",
inputs(claims),
exp(&["cache-declared", "stands", "counted"], "reconciled")
.counts(1, 0, [0, 1, 0, 0])
.cache(cline(MESH_C1, mc, C1, c1_confirmed(), &[1, 3], 1, &[(1, 1), (3, 0)]))
.value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![sat(&mut ch), ch.sign(P, link, schedule(gg, SAT, &[1, 3], 3))];
push(
"publisher_schedule_stands",
"P's schedule of sat stands by carousel §6.2 step 4, and the statement is held to it as to a node's.",
with_held(claims, of_object(SAT, 1, pass_end(2), 300_000)),
exp(&["declared", "stands"], "transmitted").bearer((SAT, uni, 0, None, None)).held("holds").value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![mesh(&mut ch, &[1, 3]), ch.sign(P, link, schedule(gg, MESH_C1, &[1, 3], 3))];
push(
"publisher_schedule_on_cache_instance",
"P schedules C1's instance, which P has not declared: set aside, and C1's instance stands.",
inputs(claims),
exp(&["cache-declared", "set-aside reason=bearer_not_declared"], "published").cache(mesh_line()).value(),
);
}
let mut i = inputs(only());
i.bearer.group = broken_group(gg);
push("group_refused", "The group's signature is broken: nothing is derived.", i, json!("refuse reason=signature_verify_failed"));
@ -1148,6 +1475,7 @@ fn tally_cases(g: &Groups) -> Vec<TallyCase> {
let inputs = |claims: Vec<String>| Inputs {
bearer: bearer_receipts::Inputs { group: gg.group.to_json(), objects: gg.held(&[1]), claims, ..Default::default() },
heads: both(),
held: None,
};
let target = |pop: &[[u8; 32]], region: &str, deadline: u64, miss: u64| Target {
index: 1,
@ -1704,6 +2032,7 @@ fn lines_cases(g: &Groups) -> Vec<LinesCase> {
let inputs = |claims: Vec<String>| Inputs {
bearer: bearer_receipts::Inputs { group: gg.group.to_json(), objects: gg.held(&[1]), claims, ..Default::default() },
heads: both(),
held: None,
};
let on_time = |seed: u8, at: u64| json!({"holder": h(seed), "held_ms": at, "fallback_ms": null, "outcome": "on_time", "value_ppm": 1_000_000});
let replaced = |seed: u8, at: u64| json!({"holder": h(seed), "held_ms": null, "fallback_ms": at, "outcome": "replaced", "value_ppm": 400_000});
@ -1719,6 +2048,11 @@ fn lines_cases(g: &Groups) -> Vec<LinesCase> {
ch.sign(P, gf.link(), transmission(Bearer::Unidirectional, SAT)),
ch.sign(R1, gf.link(), reconciliation(gf, Bearer::Unidirectional, SAT, &[1, 2])),
];
let ladder_claims = gf_claims.clone();
// A line of a ladder: `fallback_ms` is the list of each rung's least instant.
let rungs_line = |seed: u8, held: Option<u64>, rungs: [Option<u64>; 2], outcome: &str, value: u64| {
json!({"holder": h(seed), "held_ms": held, "fallback_ms": rungs, "outcome": outcome, "value_ppm": value})
};
vec![
LinesCase {
name: "timeliness_line_from_ack",
@ -1741,7 +2075,7 @@ fn lines_cases(g: &Groups) -> Vec<LinesCase> {
LinesCase {
name: "timeliness_line_record_without_envelope",
description: "No envelope held: R4's holdings record of object 1, a second before the deadline, is on time, as timeliness §6 reads it; R3's acknowledgement of object 1 gives no statement, the object's length unknown.",
inputs: Inputs { bearer: bearer_receipts::Inputs { objects: gg.held(&[]), ..inputs(vec![]).bearer }, heads: both() },
inputs: Inputs { bearer: bearer_receipts::Inputs { objects: gg.held(&[]), ..inputs(vec![]).bearer }, heads: both(), held: None },
acks: vec![ack(gg, C1, R3, E)],
records: vec![record(gg, R4, D - 1_000, &[1])],
claim: timeliness_claim(gg, &[1], D),
@ -1762,6 +2096,7 @@ fn lines_cases(g: &Groups) -> Vec<LinesCase> {
inputs: Inputs {
bearer: bearer_receipts::Inputs { group: gf.group.to_json(), objects: gf.held(&[1, 2]), claims: gf_claims, ..Default::default() },
heads: both(),
held: None,
},
acks: vec![ack_of(gf, 2, C1, R3, E)],
records: vec![],
@ -1771,6 +2106,31 @@ fn lines_cases(g: &Groups) -> Vec<LinesCase> {
(R3, replaced(R3, D - 600_000)),
]),
},
LinesCase {
name: "timeliness_line_ladder",
description: "GF's object 1 has a ladder: its object 2, a floor of it, at 700,000, then G9's object 1 at 400,000. R1 reconciles the primary and the first rung over sat: on time. R3 acknowledges the first rung alone: replaced at 700,000. R4's only record holds the second rung: replaced at 400,000. R2 acknowledges the first rung after the deadline and its record holds the second before it: the first rung held by the deadline is the lower, 400,000.",
inputs: Inputs {
bearer: bearer_receipts::Inputs { group: gf.group.to_json(), objects: gf.held(&[1, 2]), claims: ladder_claims, ..Default::default() },
heads: both(),
held: None,
},
acks: vec![ack_of(gf, 2, C1, R3, E), ack_of(gf, 2, C1, R2, (D + 60_000) * 1000)],
records: vec![record(&g.g9, R4, D - 1_000, &[1]), record(&g.g9, R2, D - 2_000, &[1])],
claim: timeliness_claim_of(gf, vec![{
let mut e = entry(gf, 1, D, None);
e.fallback = Ladder::Rungs(vec![
Fallback { group: gf.link(), leaf_index: 2, content_hash: gf.objects[2].1.content_hash, value_ppm: 700_000 },
Fallback { group: g.g9.link(), leaf_index: 1, content_hash: g.g9.objects[1].1.content_hash, value_ppm: 400_000 },
]);
e
}]),
outcome: in_key_order(vec![
(R1, rungs_line(R1, Some(D - 1_800_000), [Some(D - 1_800_000), None], "on_time", 1_000_000)),
(R2, rungs_line(R2, None, [Some(D + 60_000), Some(D - 2_000)], "replaced", 400_000)),
(R3, rungs_line(R3, None, [Some(D - 600_000), None], "replaced", 700_000)),
(R4, rungs_line(R4, None, [None, Some(D - 1_000)], "replaced", 400_000)),
]),
},
]
}
@ -1799,6 +2159,18 @@ fn inputs_value(i: &Inputs) -> Map<String, Value> {
m
}
fn held_value(held: &Option<Held>) -> Value {
match held {
None => Value::Null,
Some(Held { statement: Statement::Object { bearer_id, index, by_ms }, loss_ppm, overhead }) => {
json!({"kind": "object", "bearer_id": bearer_id, "index": index, "by_ms": by_ms, "loss_ppm": loss_ppm, "overhead": overhead})
}
Some(Held { statement: Statement::Group { bearer_id, by_ms }, loss_ppm, overhead }) => {
json!({"kind": "group", "bearer_id": bearer_id, "by_ms": by_ms, "loss_ppm": loss_ppm, "overhead": overhead})
}
}
}
fn acks_value(acks: &[Given]) -> Value {
json!(acks.iter().map(|a| json!({"record": hex(&a.record), "items": a.items.iter().map(|l| hex(l)).collect::<Vec<_>>()})).collect::<Vec<_>>())
}
@ -1830,8 +2202,9 @@ pub fn generate() -> Vec<(String, String)> {
}
let doc = json!({
"about": "Conformance corpus of extensions/cache-authority.md (wai.delivery.cache): the cache-transmission claim \
(jwp-receipts §2.4.18) through jwp-receipts §3.1 steps 1 to 6, placing a server key by the heads named (§4.4), \
the bearer verdict with a cache's instances beside the publisher's (§7), the count of a stated population that \
(jwp-receipts §2.4.18) and a node's schedule of its own instance (§5.6) through jwp-receipts §3.1 steps 1 to 6, \
placing a server key by the heads named (§4.4), the bearer verdict with a cache's instances beside the \
publisher's (§7), the count of a stated population that \
held an object by a deadline (§8), and timeliness lines from acknowledgements and reconciliations (§9). Every \
key is Ed25519 from a published seed of one byte repeated 32 times and attests nothing, and both key logs are \
test logs; envelopes carry opaque test payloads no case decodes; bearer instances and origins use reserved \
@ -1862,6 +2235,7 @@ pub fn generate() -> Vec<(String, String)> {
m.insert("name".into(), c.name.into());
m.insert("description".into(), c.description.into());
m.extend(inputs_value(&c.inputs));
m.insert("held".into(), held_value(&c.inputs.held));
m.insert("outcome".into(), c.outcome.clone());
Value::Object(m)
}).collect::<Vec<_>>(),
@ -1956,9 +2330,24 @@ pub fn inputs_of(c: &Value) -> Result<Inputs, String> {
Value::Null => None,
v => Some(v.as_array().ok_or("count_only")?.iter().map(key_of).collect::<Result<_, _>>()?),
};
// Only a verdict case states `held`.
let held = match c.get("held") {
None | Some(Value::Null) => None,
Some(h) => {
let id = h["bearer_id"].as_str().ok_or("held bearer_id")?.to_owned();
let by_ms = h["by_ms"].as_u64().ok_or("held by_ms")?;
let statement = match h["kind"].as_str() {
Some("object") => Statement::Object { bearer_id: id, index: h["index"].as_u64().ok_or("held index")?, by_ms },
Some("group") => Statement::Group { bearer_id: id, by_ms },
_ => return Err("held kind".into()),
};
Some(Held { statement, loss_ppm: h["loss_ppm"].as_u64().ok_or("held loss_ppm")?, overhead: h["overhead"].as_u64().ok_or("held overhead")? })
}
};
Ok(Inputs {
bearer: bearer_receipts::Inputs { group: c["group"].clone(), objects, claims, sink, assertion, count_only },
heads: heads_of(&c["heads"])?,
held,
})
}
@ -2023,6 +2412,10 @@ fn verdict_outcome(v: &Result<Verdict, DeliveryError>) -> Value {
Ok(()) => "holds".to_string(),
Err(r) => format!("reject reason={r}"),
}),
"held": v.held.map(|a| match a {
Ok(()) => "holds".to_string(),
Err(r) => format!("reject reason={r}"),
}),
}),
}
}
@ -2060,10 +2453,9 @@ fn tally_outcome(t: &Result<Tally, Refusal>) -> Value {
fn lines_outcome(l: &Result<Vec<Line>, Refusal>) -> Value {
match l {
Err(r) => json!(format!("refuse reason={}", r.label())),
Ok(ls) => json!(ls
.iter()
.map(|l| json!({"holder": hex(&l.holder), "held_ms": l.held_ms, "fallback_ms": l.fallback_ms, "outcome": l.outcome.label(), "value_ppm": l.value_ppm}))
.collect::<Vec<_>>()),
// A line's form is timeliness' own: `fallback_ms` is the list of each rung's instant for a
// ladder.
Ok(ls) => json!(ls.iter().map(crate::timeliness::corpus::line_value).collect::<Vec<_>>()),
}
}

319
wai/wai-rs/src/carousel.rs
View file

@ -4,8 +4,9 @@
//! A publisher's `schedule` claim, whose body and context checks are
//! `crate::claim_receipt::schedule`'s, states a plan beside the group receipt and the
//! transmission claim of bearer-receipts. This module derives from it when each object is held
//! by a receiver whose losses stay within a bound ([`verdict`], §6) and the passes a receiver
//! listens to ([`plan`], §7).
//! by a receiver whose losses stay within a bound ([`verdict`], §6), the passes a receiver
//! listens to ([`plan`], §7), and whether what a receiver acquired on an instance suffices
//! ([`inband`], §8.2).
//!
//! # Rules enforced here rather than only documented
//!
@ -13,9 +14,11 @@
//! symbols a receiver needs most, and a `repeat` pass is taken to miss again the symbols the
//! receiver missed before.
//! - A RaptorQ block is held at K′ + h distinct encoding symbols, the count RFC 6330 §5.8
//! states its bound at, never fewer.
//! - A schedule stands only beside a declaration of its instance that is not `bidirectional`;
//! two schedules of one instance that differ set both aside.
//! states its bound at, never fewer. A Raptor block is held at K and a margin of this text's,
//! with no bound, since RFC 5053 states none.
//! - A schedule stands only beside a declaration of its instance that is not `bidirectional`.
//! - Along their chain, each schedule of an instance extends the one before it, or every
//! schedule of the instance is set aside: passes are added, never withdrawn, moved or changed.
//! - All arithmetic is on integers, and an entry's cost is a few products per block size,
//! whatever its count of passes.
@ -24,8 +27,10 @@ use std::collections::{BTreeMap, BTreeSet};
use serde_json::Value;
use crate::bearer_receipts::{HeldObject, BEARER_CONTRADICTED, BEARER_NOT_DECLARED};
use crate::claim_receipt::{Bearer, ClaimBody, ClaimReceipt, Esi, Fec, Reason as ClaimReason, Rejection, ScheduleEntry};
use crate::claim_receipt::{Bearer, ClaimBody, ClaimReceipt, Esi, Fec, Reason as ClaimReason, Rejection, ScheduleClaim, ScheduleEntry};
use crate::jwp_delivery::{DeliveryError, GroupReceipt, ObjectReceipt};
use crate::key_log::Revision;
use crate::signer_id::SignerIdForm;
pub mod corpus;
@ -50,6 +55,9 @@ pub const CODES: &[&str] = &[
OBJECT_NOT_SCHEDULED,
NOT_WITHIN_SCHEDULE,
AFTER_DEADLINE,
INBAND_CHAIN_INCOMPLETE,
INBAND_NOT_SCHEDULED,
INBAND_KEY_LOG_MISSING,
];
pub const SCHEDULE_CONTRADICTED: &str = "schedule_contradicted";
@ -60,6 +68,9 @@ pub const HOLDING_INVALID: &str = "holding_invalid";
pub const OBJECT_NOT_SCHEDULED: &str = "object_not_scheduled";
pub const NOT_WITHIN_SCHEDULE: &str = "availability_not_within_schedule";
pub const AFTER_DEADLINE: &str = "availability_after_deadline";
pub const INBAND_CHAIN_INCOMPLETE: &str = "inband_chain_incomplete";
pub const INBAND_NOT_SCHEDULED: &str = "inband_not_scheduled";
pub const INBAND_KEY_LOG_MISSING: &str = "inband_key_log_missing";
/// The greatest loss bound, in parts per million (carousel §2).
pub const MAX_LOSS_PPM: u64 = 999_999;
@ -109,17 +120,22 @@ pub fn k_prime(k: u64) -> Option<u64> {
K_PRIME.get(i).map(|&v| u64::from(v))
}
/// Carousel §5.3: the encoding symbols beyond K a Raptor block is held at, at overhead 0, 1 and 2.
pub const RAPTOR_MARGIN: [u64; 3] = [8, 16, 28];
/// RFC 6330 §5.8's bound on a decoder's failure at overhead `h` (0, 1 or 2): one in this many.
pub fn failure_one_in(h: u64) -> u64 {
[100, 10_000, 1_000_000][h as usize]
}
/// The encoding symbols a receiver needs of a block of `k` source symbols (carousel §5.3): every
/// source symbol for `no-code`, K′ + h for `raptorq`.
/// source symbol for `no-code`, K′ + h for `raptorq`, and K and a margin for `raptor`, which pads
/// no block and whose RFC states no bound at any count.
pub fn needed(fec: Fec, k: u64, h: u64) -> u64 {
match fec {
Fec::NoCode => k,
Fec::RaptorQ => k_prime(k).expect("an admitted block has at most K'max symbols") + h,
Fec::Raptor => k + RAPTOR_MARGIN[h as usize],
}
}
@ -145,8 +161,8 @@ pub fn block_pass(e: &ScheduleEntry, k: u64, loss_ppm: u64, h: u64) -> Option<u6
let x = missed(loss_ppm, m);
let need = needed(e.oti.fec(), k, h);
let j = match (e.oti.fec(), e.esi) {
(Fec::RaptorQ, Esi::Advance) => need.div_ceil(m - x),
(Fec::RaptorQ, Esi::Repeat) => {
(Fec::RaptorQ | Fec::Raptor, Esi::Advance) => need.div_ceil(m - x),
(Fec::RaptorQ | Fec::Raptor, Esi::Repeat) => {
if m - x >= need {
1
} else {
@ -173,7 +189,8 @@ pub struct Availability {
pub available_ms: Option<u64>,
/// §5.4.
pub bytes_sent: u64,
/// RFC 6330 §5.8's bound at the overhead, for `raptorq`; `None` for `no-code`.
/// RFC 6330 §5.8's bound at the overhead, for `raptorq`; `None` for `raptor`, whose RFC states
/// none, and for `no-code`, a block held being recovered exactly.
pub failure_one_in: Option<u64>,
}
@ -231,7 +248,7 @@ pub fn plan(e: &ScheduleEntry, at_ms: u64, loss_ppm: u64, h: u64, held: &[u64])
};
let left = (e.count + 1).saturating_sub(first);
let (take, complete) = match (fec, e.esi) {
(Fec::RaptorQ, Esi::Advance) => {
(Fec::RaptorQ | Fec::Raptor, Esi::Advance) => {
// Every pass adds m − x ≥ 1 symbols to every block.
let wanted = open.iter().map(|&(m, x, need, n)| (need - n).div_ceil(m - x)).max().unwrap_or(0);
(wanted.min(left), wanted <= left)
@ -284,6 +301,8 @@ pub enum ClaimVerdict {
Declared,
/// A schedule that stands.
Stands,
/// A schedule a later schedule of its instance extends (§4.5), with another body.
Extended,
/// Accepted, and set aside for the reason given.
SetAside(&'static str),
}
@ -294,6 +313,7 @@ impl ClaimVerdict {
ClaimVerdict::Refused(r) => format!("reject step={} reason={}", r.step, r.reason.label()),
ClaimVerdict::Declared => "declared".into(),
ClaimVerdict::Stands => "stands".into(),
ClaimVerdict::Extended => "extended".into(),
ClaimVerdict::SetAside(r) => format!("set-aside reason={r}"),
}
}
@ -336,13 +356,40 @@ impl Refusal {
}
}
/// What §6.2 steps 3 and 4 make of the claims accepted about one group.
#[derive(Clone, Debug, Default)]
pub(crate) struct Standing {
/// The reason each claim set aside is set aside for, by its position.
pub aside: BTreeMap<usize, &'static str>,
/// The schedules a later schedule of their instance extends with another body (§4.5), by
/// position: accepted, and taking no further part.
pub extended: BTreeSet<usize>,
/// The position of the schedule that stands on each instance, the last of its chain, by its
/// `bearer_id`.
pub schedules: BTreeMap<String, usize>,
}
/// §6.2 step 4 for the schedules of one instance, by their positions in the order of their
/// signer's chain: `Err(schedule_contradicted)` when one does not extend the one before it
/// (§4.5); otherwise the position of the last, which extends every one before it, and those
/// before it with another body, which it extends. Extension is transitive, so each extending
/// the one before it is each extending every one before it.
pub(crate) fn monotone(accepted: &[Option<ClaimReceipt>], idx: &[usize]) -> Result<(usize, Vec<usize>), &'static str> {
let body = |i: usize| accepted[i].as_ref().and_then(ClaimReceipt::schedule).expect("a schedule");
if idx.windows(2).any(|w| !body(w[1]).extends(body(w[0]))) {
return Err(SCHEDULE_CONTRADICTED);
}
let last = *idx.last().expect("an instance has a schedule");
Ok((last, idx.iter().copied().filter(|&i| body(i) != body(last)).collect()))
}
/// §6.2 steps 3 and 4 over the claims accepted about one group, each at its position (`None`
/// for one refused): the declarations by instance (bearer-receipts §6.2 step 4), then the
/// schedules by instance. Returns the reason each claim set aside is set aside for, by its
/// position, and the position of each standing schedule by its `bearer_id`. A timeliness plan
/// schedules by instance, each instance's in the order given, which is their signer's chain's
/// (§6.1; §4.3 step 2 makes the group's key the only signer). A timeliness plan
/// (`crate::timeliness`) reads the schedules that stand here.
pub(crate) fn standing(accepted: &[Option<ClaimReceipt>]) -> (BTreeMap<usize, &'static str>, BTreeMap<String, usize>) {
let mut aside: BTreeMap<usize, &'static str> = BTreeMap::new();
pub(crate) fn standing(accepted: &[Option<ClaimReceipt>]) -> Standing {
let mut out = Standing::default();
// Step 3: the declarations, by instance (bearer-receipts §6.2 step 4).
let mut declared: BTreeMap<String, Vec<usize>> = BTreeMap::new();
for (i, c) in accepted.iter().enumerate() {
@ -359,7 +406,7 @@ pub(crate) fn standing(accepted: &[Option<ClaimReceipt>]) -> (BTreeMap<usize, &'
} else {
contradicted.insert(id.clone());
for &i in idx {
aside.insert(i, BEARER_CONTRADICTED);
out.aside.insert(i, BEARER_CONTRADICTED);
}
}
}
@ -371,43 +418,49 @@ pub(crate) fn standing(accepted: &[Option<ClaimReceipt>]) -> (BTreeMap<usize, &'
scheduled.entry(s.bearer_id.clone()).or_default().push(i);
}
}
let mut standing: BTreeMap<String, usize> = BTreeMap::new();
for (id, idx) in &scheduled {
let first = accepted[idx[0]].as_ref().and_then(ClaimReceipt::schedule);
let reason = if idx.iter().any(|&i| accepted[i].as_ref().and_then(ClaimReceipt::schedule) != first) {
Some(SCHEDULE_CONTRADICTED)
} else if contradicted.contains(id) {
Some(BEARER_CONTRADICTED)
} else {
match declaration.get(id) {
None => Some(BEARER_NOT_DECLARED),
Some(Bearer::Bidirectional) => Some(SCHEDULE_ON_BIDIRECTIONAL),
Some(_) => None,
let reason = monotone(accepted, idx).and_then(|v| {
if contradicted.contains(id) {
return Err(BEARER_CONTRADICTED);
}
};
match declaration.get(id) {
None => Err(BEARER_NOT_DECLARED),
Some(Bearer::Bidirectional) => Err(SCHEDULE_ON_BIDIRECTIONAL),
Some(_) => Ok(v),
}
});
match reason {
Some(reason) => {
Err(reason) => {
for &i in idx {
aside.insert(i, reason);
out.aside.insert(i, reason);
}
}
None => {
standing.insert(id.clone(), idx[0]);
Ok((last, extended)) => {
out.schedules.insert(id.clone(), last);
out.extended.extend(extended);
}
}
}
(aside, standing)
out
}
/// §6.2: the verdict over one group.
pub fn verdict(inp: &Inputs) -> Result<Verdict, Refusal> {
check_bounds(inp.loss_ppm, inp.overhead).map_err(Refusal::Input)?;
/// What §6.2 steps 1 to 4 give: the objects' verdicts, the claims' and the claims accepted, and
/// the schedules that stand.
struct Judged {
group: GroupReceipt,
objects: Vec<String>,
claims: Vec<ClaimVerdict>,
accepted: Vec<Option<ClaimReceipt>>,
standing: Standing,
}
/// §6.2 steps 1 to 4, the loss bound and overhead aside.
fn judge(group: &Value, held_objects: &[HeldObject], wires: &[String]) -> Result<Judged, Refusal> {
// Step 1: the group, and its objects the same whatever the bearer.
let group = GroupReceipt::from_json(&inp.group).map_err(Refusal::Group)?;
let group = GroupReceipt::from_json(group).map_err(Refusal::Group)?;
group.verify().map_err(Refusal::Group)?;
let mut receipts: Vec<(ObjectReceipt, Option<&[u8]>)> = Vec::new();
let objects: Vec<String> = inp
.objects
let objects: Vec<String> = held_objects
.iter()
.map(|o| {
let r = match ObjectReceipt::from_json(&o.receipt) {
@ -426,9 +479,9 @@ pub fn verdict(inp: &Inputs) -> Result<Verdict, Refusal> {
// Step 2.
let mut held: BTreeMap<[u8; 32], ClaimReceipt> = BTreeMap::new();
let mut claims: Vec<ClaimVerdict> = Vec::with_capacity(inp.claims.len());
let mut accepted: Vec<Option<ClaimReceipt>> = Vec::with_capacity(inp.claims.len());
for wire in &inp.claims {
let mut claims: Vec<ClaimVerdict> = Vec::with_capacity(wires.len());
let mut accepted: Vec<Option<ClaimReceipt>> = Vec::with_capacity(wires.len());
for wire in wires {
let checked = (|| {
let c = ClaimReceipt::from_json(wire)?;
c.verify()?;
@ -454,14 +507,24 @@ pub fn verdict(inp: &Inputs) -> Result<Verdict, Refusal> {
}
// Steps 3 and 4.
let (aside, standing) = self::standing(&accepted);
for (&i, &reason) in &aside {
let standing = self::standing(&accepted);
for (&i, &reason) in &standing.aside {
claims[i] = ClaimVerdict::SetAside(reason);
}
for &i in &standing.extended {
claims[i] = ClaimVerdict::Extended;
}
Ok(Judged { group, objects, claims, accepted, standing })
}
/// §6.2: the verdict over one group.
pub fn verdict(inp: &Inputs) -> Result<Verdict, Refusal> {
check_bounds(inp.loss_ppm, inp.overhead).map_err(Refusal::Input)?;
let Judged { group, objects, claims, accepted, standing } = judge(&inp.group, &inp.objects, &inp.claims)?;
// Step 5.
let mut lines = Vec::new();
for (id, &i) in &standing {
for (id, &i) in &standing.schedules {
let s = accepted[i].as_ref().and_then(ClaimReceipt::schedule).expect("a schedule");
for e in &s.objects {
lines.push(Line {
@ -488,6 +551,91 @@ pub fn verdict(inp: &Inputs) -> Result<Verdict, Refusal> {
Ok(Verdict { objects, claims, lines, statement })
}
/// §6.4 on one schedule, for a verdict that holds the schedule standing on the statement's
/// instance (`None` when none stands there): `Ok` when the statement holds. A node's schedule of
/// its own instance (cache-authority §5.6) is held to it as a publisher's is.
pub fn statement_holds(schedule: Option<&ScheduleClaim>, leaf_count: u64, statement: &Statement, loss_ppm: u64, h: u64) -> Result<(), &'static str> {
check_bounds(loss_ppm, h)?;
let held_by = |index: u64, by: u64| -> Result<(), &'static str> {
let e = schedule.and_then(|s| s.entry(index)).ok_or(OBJECT_NOT_SCHEDULED)?;
match availability(e, loss_ppm, h).available_ms {
None => Err(NOT_WITHIN_SCHEDULE),
Some(t) if t > by => Err(AFTER_DEADLINE),
Some(_) => Ok(()),
}
};
match statement {
Statement::Object { index, by_ms, .. } => held_by(*index, *by_ms),
Statement::Group { by_ms, .. } => (0..leaf_count).try_for_each(|i| held_by(i, *by_ms)),
}
}
// ───────────────────────────── §8.1, §8.2: what rides in-band ─────────────────────────────
/// §8.1: the interval of a schedule, the least `period_ms` of its entries of two passes or
/// more, or, where every entry has one pass, the least `duration_ms`. The publisher sends the
/// in-band set at least once in every interval of it, from its first pass's start to its last
/// pass's end.
pub fn inband_interval(s: &ScheduleClaim) -> u64 {
let repeated = s.objects.iter().filter(|e| e.count >= 2).map(|e| e.period_ms).min();
repeated.or_else(|| s.objects.iter().map(|e| e.duration_ms).min()).expect("a schedule has an entry")
}
/// The inputs of §8.2: what a receiver acquired on an instance.
#[derive(Clone, Debug, Default)]
pub struct InbandInputs {
/// The instance the receiver joined.
pub bearer_id: String,
/// The group receipt, in its JSON form.
pub group: Value,
/// The set's claims, in the order of their signer's chain.
pub claims: Vec<String>,
/// The key logs acquired with them, each the text of a revision, keyed by the origin it names.
pub logs: BTreeMap<String, String>,
}
/// §8.2's verdict on what a receiver acquired.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Inband {
/// Each claim's label, as §6.2 gives it over the set alone.
pub claims: Vec<ClaimVerdict>,
/// `Ok` with the standing schedule's interval (§8.1) when the set suffices; the reason
/// otherwise.
pub set: Result<u64, &'static str>,
}
/// §8.2: whether what a receiver acquired on an instance suffices: §6.2 steps 1 to 4 over the
/// group receipt and the set's claims alone, with no claim held before them; then, at the first
/// of these that fails, refused: no claim of it is refused at jwp-receipts §3.1 step 5
/// (`inband_chain_incomplete`); a schedule of the instance stands (`inband_not_scheduled`); a
/// key log of the set is held for the origin that schedule's `signer_id` names (jwp-receipts §7.2
/// rules 1 to 3) and lists the schedule's key (`inband_key_log_missing`). Only the group refuses
/// it outright.
pub fn inband(inp: &InbandInputs) -> Result<Inband, Refusal> {
let j = judge(&inp.group, &[], &inp.claims)?;
let set = (|| {
if j.claims.iter().any(|c| matches!(c, ClaimVerdict::Refused(r) if r.step == 5)) {
return Err(INBAND_CHAIN_INCOMPLETE);
}
let &i = j.standing.schedules.get(&inp.bearer_id).ok_or(INBAND_NOT_SCHEDULED)?;
let c = j.accepted[i].as_ref().expect("a schedule that stands");
let s = c.schedule().expect("a schedule");
let SignerIdForm::OriginBound { origin, .. } = crate::signer_id::classify(&c.signer_id) else {
return Err(INBAND_KEY_LOG_MISSING);
};
let held = inp
.logs
.get(&origin)
.and_then(|t| Revision::from_json(t).ok())
.filter(|r| r.check_held(&origin).is_ok() && r.keys.iter().any(|e| e.key == c.signer_pubkey));
if held.is_none() {
return Err(INBAND_KEY_LOG_MISSING);
}
Ok(inband_interval(s))
})();
Ok(Inband { claims: j.claims, set })
}
#[cfg(test)]
mod tests {
use super::*;
@ -511,6 +659,10 @@ mod tests {
Oti::RaptorQ { transfer_length: f, symbol_size: t, source_blocks: z, sub_blocks: 1, alignment: 4 }
}
fn rp(f: u64, t: u64, z: u64) -> Oti {
Oti::Raptor { transfer_length: f, symbol_size: t, source_blocks: z, sub_blocks: 1, alignment: 4 }
}
#[test]
fn k_prime_is_the_least_table_value_at_least_k() {
assert_eq!(k_prime(1), Some(10));
@ -540,6 +692,22 @@ mod tests {
assert_eq!(block_pass(&e, 16, 850_000, 2), Some(5), "4 a pass: the last");
}
#[test]
fn raptor_needs_k_and_a_margin_with_no_bound() {
// 48 symbols in 3 blocks of 16; m = 50.
assert_eq!([needed(Fec::Raptor, 16, 0), needed(Fec::Raptor, 16, 1), needed(Fec::Raptor, 16, 2)], [24, 32, 44]);
let e = entry(rp(3_072, 64, 3), 34, Esi::Advance, 5);
assert_eq!(block_pass(&e, 16, 0, 2), Some(1));
assert_eq!(block_pass(&e, 16, 100_000, 2), Some(1), "45 a pass");
assert_eq!(block_pass(&e, 16, 300_000, 2), Some(2), "35 a pass");
assert_eq!(block_pass(&e, 16, 800_000, 2), Some(5), "10 a pass: the last");
assert_eq!(block_pass(&e, 16, 900_000, 2), None, "5 a pass: 9 passes, past 5");
assert_eq!(availability(&e, 0, 2).failure_one_in, None);
let r = entry(rp(3_072, 64, 3), 34, Esi::Repeat, 5);
assert_eq!(block_pass(&r, 16, 100_000, 2), Some(1), "50 - 5 = 45 >= 44");
assert_eq!(block_pass(&r, 16, 150_000, 2), None, "50 - 7 = 43: no later pass is certain to add");
}
#[test]
fn repeat_and_no_code_never_count_on_a_second_pass() {
let e = entry(rq(3_072, 64, 3), 8, Esi::Repeat, 5);
@ -569,6 +737,67 @@ mod tests {
assert_eq!(plan(&e, 0, 0, 3, &[0, 0, 0]), Err(OVERHEAD_INVALID));
}
fn claim(seq: u64, parent: Option<[u8; 32]>, objects: Vec<ScheduleEntry>) -> ClaimReceipt {
use crate::claim_receipt::schedule::ATTESTS_SCHEDULED;
use crate::claim_receipt::SubjectType;
let key = ed25519_dalek::SigningKey::from_bytes(&[0x11; 32]);
let body = ClaimBody::Schedule(ScheduleClaim { bearer_id: "urn:example:b".into(), objects, attests: ATTESTS_SCHEDULED.into() });
ClaimReceipt::seal(&key, "did:web:publisher.example#d1", SubjectType::Receipt, [0; 32], seq, "2026-11-02T12:00:03Z", body, parent).unwrap()
}
#[test]
fn schedules_are_monotone_along_their_chain_or_set_aside() {
let e = |count| entry(rq(3_072, 64, 3), 8, Esi::Advance, count);
let of = |counts: &[u64]| -> Vec<Option<ClaimReceipt>> {
let mut parent = None;
counts
.iter()
.enumerate()
.map(|(i, &n)| {
let c = claim(i as u64, parent, vec![e(n)]);
parent = Some(c.link_hash());
Some(c)
})
.collect()
};
let mono = |counts: &[u64]| {
let accepted = of(counts);
monotone(&accepted, &(0..counts.len()).collect::<Vec<_>>())
};
assert_eq!(mono(&[5]), Ok((0, vec![])));
assert_eq!(mono(&[5, 5]), Ok((1, vec![])), "one body, no schedule extended");
assert_eq!(mono(&[5, 7]), Ok((1, vec![0])));
assert_eq!(mono(&[5, 5, 7, 9, 9]), Ok((4, vec![0, 1, 2])), "the last's copy is not extended");
assert_eq!(mono(&[7, 5]), Err(SCHEDULE_CONTRADICTED), "a pass withdrawn");
assert_eq!(mono(&[5, 7, 6]), Err(SCHEDULE_CONTRADICTED), "the last extends the first, not the one before it");
}
#[test]
fn the_interval_is_the_least_period_read() {
let s = |objects| ScheduleClaim { bearer_id: "urn:example:b".into(), objects, attests: "scheduled".into() };
let mut one_pass = entry(rq(3_072, 64, 3), 8, Esi::Advance, 1);
one_pass.period_ms = 0;
one_pass.duration_ms = 500;
let repeated = entry(rq(3_072, 64, 3), 8, Esi::Advance, 5);
assert_eq!(inband_interval(&s(vec![repeated.clone()])), 10_000);
assert_eq!(inband_interval(&s(vec![one_pass.clone(), repeated])), 10_000, "a period of one pass is not read");
assert_eq!(inband_interval(&s(vec![one_pass])), 500, "with no period to read, the least duration");
}
#[test]
fn a_statement_is_held_to_the_schedule_it_is_given() {
let e = entry(rq(3_072, 64, 3), 8, Esi::Advance, 5);
let s = ScheduleClaim { bearer_id: "urn:example:b".into(), objects: vec![e.clone()], attests: "scheduled".into() };
let at = |by_ms| Statement::Object { bearer_id: "urn:example:b".into(), index: 0, by_ms };
assert_eq!(statement_holds(Some(&s), 1, &at(e.pass_end(2)), 300_000, 2), Ok(()));
assert_eq!(statement_holds(Some(&s), 1, &at(e.pass_end(2) - 1), 300_000, 2), Err(AFTER_DEADLINE));
assert_eq!(statement_holds(Some(&s), 1, &at(e.pass_end(5)), 900_000, 2), Err(NOT_WITHIN_SCHEDULE));
assert_eq!(statement_holds(None, 1, &at(e.pass_end(5)), 0, 2), Err(OBJECT_NOT_SCHEDULED));
assert_eq!(statement_holds(Some(&s), 2, &Statement::Group { bearer_id: "urn:example:b".into(), by_ms: e.pass_end(5) }, 0, 2), Err(OBJECT_NOT_SCHEDULED));
assert_eq!(statement_holds(Some(&s), 1, &at(0), 1_000_000, 2), Err(LOSS_BOUND_INVALID));
assert_eq!(statement_holds(Some(&s), 1, &at(0), 0, 3), Err(OVERHEAD_INVALID));
}
#[test]
fn a_repeat_plan_takes_at_most_one_pass() {
let e = entry(rq(3_072, 64, 3), 8, Esi::Repeat, 5);

622
wai/wai-rs/src/carousel/corpus.rs
View file

@ -15,12 +15,17 @@ use crate::claim_receipt::schedule::ATTESTS_SCHEDULED;
use crate::claim_receipt::{hex, unhex, Oti, ReconciliationClaim, ScheduleClaim, SubjectType, TransmissionClaim};
use crate::container::{Conditioning, Manifest, ModelRequirement, Wai};
use crate::jwp_delivery::GroupBuilder;
use crate::key_log::{commitment, EntryStatus, KeyEntry};
use ed25519_dalek::SigningKey;
use serde_json::{json, Map};
const PUBLISHER: u8 = 0x11;
const OTHER_KEY: u8 = 0x12;
const SINK: u8 = 0x41;
const LOG_ROOT: u8 = 0x71;
const LOG_NEXT: u8 = 0x72;
const OTHER_ROOT: u8 = 0x73;
const OTHER_NEXT: u8 = 0x74;
const D1: &str = "did:web:publisher.example#d1";
const D2: &str = "did:web:publisher.example#d2";
@ -33,6 +38,11 @@ const UNDECLARED: &str = "urn:example:wai-carousel:broadcast-9";
const GROUP_START: &str = "2026-11-02T12:00:00Z";
const GROUP_END: &str = "2026-11-02T12:00:02Z";
const SENT: &str = "2026-11-02T12:00:03Z";
/// The origin `D1` and `D2` name, and another.
const ORIGIN: &str = "https://publisher.example";
const OTHER_ORIGIN: &str = "https://other.example";
const LOG_TIME: &str = "2026-11-01T00:00:00Z";
const COMPROMISED: &str = "2026-11-01T06:00:00Z";
/// `GROUP_START` as an instant.
const START_MS: u64 = 1_793_620_800_000;
/// The first pass of the broadcast schedule: ten seconds after the group starts.
@ -101,6 +111,9 @@ fn other_group() -> GroupReceipt {
fn rq(f: u64, t: u64, z: u64) -> Oti {
Oti::RaptorQ { transfer_length: f, symbol_size: t, source_blocks: z, sub_blocks: 1, alignment: 4 }
}
fn rp(f: u64, t: u64, z: u64) -> Oti {
Oti::Raptor { transfer_length: f, symbol_size: t, source_blocks: z, sub_blocks: 1, alignment: 4 }
}
fn nc(l: u64, e: u64, b: u64) -> Oti {
Oti::NoCode { transfer_length: l, symbol_size: e, max_block_length: b }
}
@ -124,6 +137,19 @@ fn broadcast(p: &Publication) -> Vec<ScheduleEntry> {
]
}
/// A schedule under Raptor: object 0 in three blocks with fresh symbols each pass, object 1 in one
/// block repeated, object 2 in blocks of two sizes with fresh symbols each pass.
fn under_raptor(p: &Publication) -> Vec<ScheduleEntry> {
vec![
// F 3,200, T 64: 50 symbols in blocks of 17, 17, 16; m 51, 51, 50.
entry(p, 0, rp(3_200, 64, 3), 34, Esi::Advance, T0, 30_000, 4_000, 5),
// F 896: 14 symbols, one block; m 48.
entry(p, 1, rp(896, 64, 1), 34, Esi::Repeat, T0 + 5_000, 30_000, 2_000, 3),
// F 288, T 32: 9 symbols in blocks of 5 and 4; m 39, 38.
entry(p, 2, rp(288, 32, 2), 34, Esi::Advance, T0 + 8_000, 30_000, 1_000, 5),
]
}
/// The multicast schedule: every object, repeated with ample repair.
fn multicast(p: &Publication) -> Vec<ScheduleEntry> {
vec![
@ -143,7 +169,9 @@ fn end(e: &ScheduleEntry, j: u64) -> u64 {
/// The octets an entry sends, by count · (⌈F / T⌉ + Z · repair) · T and count · L.
fn sent(e: &ScheduleEntry) -> u64 {
match e.oti {
Oti::RaptorQ { transfer_length: f, symbol_size: t, source_blocks: z, .. } => e.count * (f.div_ceil(t) + z * e.repair) * t,
Oti::RaptorQ { transfer_length: f, symbol_size: t, source_blocks: z, .. } | Oti::Raptor { transfer_length: f, symbol_size: t, source_blocks: z, .. } => {
e.count * (f.div_ceil(t) + z * e.repair) * t
}
Oti::NoCode { transfer_length, .. } => e.count * transfer_length,
}
}
@ -558,6 +586,194 @@ fn verdict_cases(p: &Publication) -> Vec<VerdictCase> {
);
}
// §5.5: Raptor, whose blocks are held at K and a margin of 8, 16 or 28, with no bound.
{
let rs = under_raptor(p);
for (name, description, loss, h, passes) in [
("raptor_lossless", "No loss at overhead 2, where a block needs K + 28: every object is held after its first pass.", 0, 2, [Some(1), Some(1), Some(1)]),
("raptor_10_percent", "10%: object 0 gains 46 or 45 a pass and needs 45 or 44; object 1 holds 44 of the 42 it needs; object 2 gains 36 or 35 and needs 33 or 32.", 100_000, 2, [Some(1), Some(1), Some(1)]),
(
"raptor_30_percent",
"30%: object 0 gains 36 or 35 a pass and needs 45 or 44, after pass 2; object 1 holds 34 of 42, never, where K′ + 2 = 20 under RaptorQ would be met; object 2 gains 28 or 27 and needs 33 or 32, after pass 2.",
300_000,
2,
[Some(2), None, Some(2)],
),
("raptor_overhead_0", "50% at overhead 0, where a block needs K + 8: object 0 gains 26 or 25 and needs 25 or 24; object 1 holds 24 of 22; object 2 gains 20 or 19 and needs 13 or 12.", 500_000, 0, [Some(1), Some(1), Some(1)]),
("raptor_overhead_1", "50% at overhead 1, where a block needs K + 16: object 0 needs 33 or 32, after pass 2; object 1 holds 24 of 30, never; object 2 needs 21 or 20, after pass 2.", 500_000, 1, [Some(2), None, Some(2)]),
("raptor_80_percent_last_pass", "80%: object 0 gains 11 or 10 a pass and object 2 gains 8, each held after the fifth and last.", 800_000, 2, [Some(5), None, Some(5)]),
("raptor_90_percent", "90%: object 0 gains 6 or 5 a pass and needs 8 or 9 passes, and has 5; object 2 gains 4 and needs 9: never.", 900_000, 2, [None, None, None]),
] {
let mut ch = Chains::default();
let claims = vec![ch.sign(PUBLISHER, link, transmission(Bearer::Unidirectional, BCAST)), ch.sign(PUBLISHER, link, schedule(BCAST, rs.clone()))];
let mut exp = Exp::new(&ALL_D, &["declared", "stands"]);
for (e, j) in rs.iter().zip(passes) {
exp = exp.line(BCAST, e, j, h);
}
push(name, description, inputs(p, true, claims, loss, h), exp.to_value());
}
let mut ch = Chains::default();
let claims = vec![ch.sign(PUBLISHER, link, transmission(Bearer::Unidirectional, BCAST)), ch.sign(PUBLISHER, link, schedule(BCAST, rs.clone()))];
let mut inp = inputs(p, false, claims, 300_000, 2);
inp.statement = object_statement(BCAST, 0, end(&rs[0], 2) - 1);
push(
"raptor_after_deadline",
"Object 0 under Raptor a millisecond short of when it is held: refused, as under any code.",
inp,
Exp::new(&ALL_I, &["declared", "stands"]).line(BCAST, &rs[0], Some(2), 2).line(BCAST, &rs[1], None, 2).line(BCAST, &rs[2], Some(2), 2).statement(Err(AFTER_DEADLINE)).to_value(),
);
}
// §4.5: a later schedule that extends an earlier one.
let changed = |i: usize, f: &dyn Fn(&mut ScheduleEntry)| {
let mut v = bc.clone();
f(&mut v[i]);
v
};
let seven = changed(0, &|e| e.count = 7);
{
let mut ch = Chains::default();
let claims = vec![
ch.sign(PUBLISHER, link, transmission(Bearer::Unidirectional, BCAST)),
ch.sign(PUBLISHER, link, schedule(BCAST, bc.clone())),
ch.sign(PUBLISHER, link, schedule(BCAST, seven.clone())),
];
let mut inp = inputs(p, true, claims, 900_000, 2);
inp.statement = object_statement(BCAST, 0, end(&seven[0], 7));
push(
"extension_adds_passes",
"At 90% object 0 gains 3 a pass and needs 7 passes, and the first schedule has 5. A later one with 7 extends it: the first is \
extended, and the object is held at the end of the seventh.",
inp,
Exp::new(&ALL_D, &["declared", "extended", "stands"])
.line(BCAST, &seven[0], Some(7), 2)
.line(BCAST, &seven[1], None, 2)
.line(BCAST, &seven[2], None, 2)
.statement(Ok(()))
.to_value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![
ch.sign(PUBLISHER, link, transmission(Bearer::Unidirectional, BCAST)),
ch.sign(PUBLISHER, link, schedule(BCAST, bc[..2].to_vec())),
ch.sign(PUBLISHER, link, schedule(BCAST, bc.clone())),
ch.sign(PUBLISHER, link, schedule(BCAST, bc.clone())),
];
let mut inp = inputs(p, false, claims, 300_000, 2);
inp.statement = object_statement(BCAST, 2, end(&bc[2], 1));
push(
"extension_adds_an_object",
"A later schedule adds an entry for object 2, and is repeated: the first is extended, both copies of the last stand, and a \
statement about the added object holds.",
inp,
Exp::new(&ALL_I, &["declared", "extended", "stands", "stands"])
.line(BCAST, &bc[0], Some(2), 2)
.line(BCAST, &bc[1], None, 2)
.line(BCAST, &bc[2], Some(1), 2)
.statement(Ok(()))
.to_value(),
);
}
{
let mut single = bc[0].clone();
single.count = 1;
single.period_ms = 0;
let mut ch = Chains::default();
let claims = vec![
ch.sign(PUBLISHER, link, transmission(Bearer::Unidirectional, BCAST)),
ch.sign(PUBLISHER, link, schedule(BCAST, vec![single])),
ch.sign(PUBLISHER, link, schedule(BCAST, bc[..1].to_vec())),
];
push(
"extension_of_one_pass",
"One pass, its period 0 and not read, then five passes with a period: the pass stated stays the first, and object 0 is held \
after the second at 30%.",
inputs(p, true, claims, 300_000, 2),
Exp::new(&ALL_D, &["declared", "extended", "stands"]).line(BCAST, &bc[0], Some(2), 2).to_value(),
);
}
{
let more = changed(1, &|e| e.count = 6);
let mut ch = Chains::default();
let claims = vec![
ch.sign(PUBLISHER, link, transmission(Bearer::Unidirectional, BCAST)),
ch.sign(PUBLISHER, link, schedule(BCAST, bc[1..2].to_vec())),
ch.sign(PUBLISHER, link, schedule(BCAST, more[1..2].to_vec())),
];
push(
"extension_of_repeat",
"Object 1 under `repeat` at 30%: three more passes of the same ESIs add nothing certain, so it is still not within the schedule, \
and the octets sent are the later schedule's.",
inputs(p, true, claims, 300_000, 2),
Exp::new(&ALL_D, &["declared", "extended", "stands"]).line(BCAST, &more[1], None, 2).to_value(),
);
}
let contradicted = format!("set-aside reason={SCHEDULE_CONTRADICTED}");
for (name, description, earlier, later) in [
("extension_withdraws_passes", "The schedules of extension_adds_passes in the other order: the later withdraws two passes.", seven.clone(), bc.clone()),
("extension_moves_the_first_pass", "Six passes, each a second later.", bc.clone(), changed(0, &|e| {
e.count = 6;
e.first_ms += 1_000;
})),
("extension_changes_the_period", "Six passes, a second further apart: every pass after the first moves.", bc.clone(), changed(0, &|e| {
e.count = 6;
e.period_ms += 1_000;
})),
("extension_changes_the_duration", "Six passes, each a second longer.", bc.clone(), changed(0, &|e| {
e.count = 6;
e.duration_ms += 1_000;
})),
("extension_changes_the_repair", "Six passes with a repair symbol more in each: every pass sends other symbols.", bc.clone(), changed(0, &|e| {
e.count = 6;
e.repair += 1;
})),
("extension_changes_the_esi", "Object 1 moved from `repeat` to `advance`.", bc.clone(), changed(1, &|e| {
e.count = 4;
e.esi = Esi::Advance;
})),
("extension_changes_the_code", "Object 2's blocks of 3 stated as blocks of at most 2.", bc.clone(), changed(2, &|e| {
e.count = 5;
e.oti = nc(288, 32, 2);
})),
] {
let mut ch = Chains::default();
let claims = vec![
ch.sign(PUBLISHER, link, transmission(Bearer::Unidirectional, BCAST)),
ch.sign(PUBLISHER, link, schedule(BCAST, earlier)),
ch.sign(PUBLISHER, link, schedule(BCAST, later)),
];
push(name, description, inputs(p, false, claims, 0, 2), Exp::new(&ALL_I, &["declared", &contradicted, &contradicted]).to_value());
}
{
let six = changed(0, &|e| e.count = 6);
let mut ch = Chains::default();
let claims = vec![
ch.sign(PUBLISHER, link, transmission(Bearer::Unidirectional, BCAST)),
ch.sign(PUBLISHER, link, schedule(BCAST, bc.clone())),
ch.sign(PUBLISHER, link, schedule(BCAST, seven.clone())),
ch.sign(PUBLISHER, link, schedule(BCAST, six)),
];
push(
"extension_falls_back",
"Five passes, then seven, then six: the last extends the first and withdraws a pass of the second, and all three are set aside.",
inputs(p, false, claims, 0, 2),
Exp::new(&ALL_I, &["declared", &contradicted, &contradicted, &contradicted]).to_value(),
);
}
{
let mut ch = Chains::default();
let claims = vec![ch.sign(PUBLISHER, link, schedule(UNDECLARED, bc.clone())), ch.sign(PUBLISHER, link, schedule(UNDECLARED, seven.clone()))];
let aside = format!("set-aside reason={BEARER_NOT_DECLARED}");
push(
"extension_not_declared",
"A schedule and its extension on an instance no declaration names: both set aside.",
inputs(p, false, claims, 0, 2),
Exp::new(&ALL_I, &[&aside, &aside]).to_value(),
);
}
// Step 6.
let step6 = |reason: &str| format!("reject step=6 reason={reason}");
let mut context = |name: &'static str, description: &'static str, bytes: bool, signer: u8, subject: [u8; 32], objects: Vec<ScheduleEntry>, reason: &str| {
@ -712,6 +928,39 @@ fn claim_cases(p: &Publication) -> Vec<ClaimCase> {
e.period_ms = 4_000;
})));
let raptor = |f: u64, t: u64, z: u64, n: u64, al: u64| Oti::Raptor { transfer_length: f, symbol_size: t, source_blocks: z, sub_blocks: n, alignment: al };
accept("raptor_greatest_transfer", "F 35,183,298,355,200 at T 65,535 in 65,535 blocks of 8,192 symbols.", one(with(&|e| {
e.oti = raptor(RAPTOR_GREATEST, 65_535, 65_535, 1, 1);
e.repair = 0;
e.esi = Esi::Repeat;
e.count = 1;
})));
accept("raptor_greatest_alignment", "Al 255 and T 255, one sub-block of one sub-symbol.", one(with(&|e| e.oti = raptor(3_200, 255, 1, 1, 255))));
accept("raptor_greatest_sub_blocks", "N 255 at T 1,020, Al 4: the field's greatest, and T / Al.", one(with(&|e| e.oti = raptor(10_200, 1_020, 1, 255, 4))));
accept("raptor_greatest_blocks", "Z 65,535, its field's greatest: every block of 4 symbols.", one(with(&|e| {
e.oti = raptor(1_048_560, 4, 65_535, 1, 4);
e.repair = 0;
e.esi = Esi::Repeat;
e.count = 1;
})));
accept("raptor_block_of_the_fewest", "One block of 4 symbols, the fewest RFC 5053 §5.7 indexes.", one(with(&|e| e.oti = rp(256, 64, 1))));
accept("raptor_block_of_the_most", "One block of 8,192 symbols, Kmax.", one(with(&|e| {
e.oti = Oti::Raptor { transfer_length: 8_192 * 4, symbol_size: 4, source_blocks: 1, sub_blocks: 1, alignment: 4 };
e.repair = 0;
})));
accept("raptor_repeat_greatest_esi", "KL + repair = 2^16 under repeat.", one(with(&|e| {
e.oti = rp(3_200, 64, 3);
e.esi = Esi::Repeat;
e.repair = (1 << 16) - 17;
e.count = 1;
})));
accept("raptor_advance_greatest_esi", "count · (KL + repair) = 2^16 under advance: 16 passes of 2^12.", one(with(&|e| {
e.oti = rp(3_200, 64, 3);
e.repair = (1 << 12) - 17;
e.count = 16;
e.period_ms = 4_000;
})));
let mut refuse = |name: &'static str, description: &'static str, body: ClaimBody, step: u8, reason: &str| {
let claim = Chains::default().sign_unchecked(PUBLISHER, link, body);
out.push(ClaimCase { name, description, claim, outcome: format!("reject step={step} reason={reason}") });
@ -743,6 +992,50 @@ fn claim_cases(p: &Publication) -> Vec<ClaimCase> {
e.oti = Oti::RaptorQ { transfer_length: 56_404 * 4, symbol_size: 4, source_blocks: 1, sub_blocks: 1, alignment: 4 };
e.repair = 0;
})), 4, oti);
refuse("raptor_no_transfer", "F 0.", body(with(&|e| e.oti = rp(0, 64, 1))), 4, oti);
refuse("raptor_transfer_past", "F 2^45.", body(with(&|e| {
e.oti = raptor(1 << 45, 65_535, 65_535, 1, 1);
e.repair = 0;
e.count = 1;
})), 4, oti);
refuse("raptor_greatest_transfer_past", "F 35,183,298,355,201: a symbol more than 65,535 blocks of 8,192 hold.", body(with(&|e| {
e.oti = raptor(RAPTOR_GREATEST + 1, 65_535, 65_535, 1, 1);
e.repair = 0;
e.count = 1;
})), 4, oti);
refuse("raptor_symbol_past", "T 65,536.", body(with(&|e| e.oti = raptor(3_200, 65_536, 1, 1, 4))), 4, oti);
refuse("raptor_symbol_not_aligned", "T 66, Al 4.", body(with(&|e| e.oti = raptor(3_200, 66, 1, 1, 4))), 4, oti);
refuse("raptor_no_alignment", "Al 0.", body(with(&|e| e.oti = raptor(3_200, 64, 1, 1, 0))), 4, oti);
refuse("raptor_alignment_past", "Al 256, T 256.", body(with(&|e| e.oti = raptor(3_200, 256, 1, 1, 256))), 4, oti);
refuse("raptor_no_block", "Z 0.", body(with(&|e| e.oti = rp(3_200, 64, 0))), 4, oti);
refuse("raptor_blocks_past_field", "Z 65,536, every block of 4 symbols.", body(with(&|e| {
e.oti = raptor(1 << 20, 4, 65_536, 1, 4);
e.repair = 0;
e.esi = Esi::Repeat;
e.count = 1;
})), 4, oti);
refuse("raptor_no_sub_block", "N 0.", body(with(&|e| e.oti = raptor(3_200, 64, 3, 0, 4))), 4, oti);
refuse("raptor_sub_blocks_past_t_over_al", "N = T / Al + 1.", body(with(&|e| e.oti = raptor(3_200, 64, 3, 17, 4))), 4, oti);
refuse("raptor_sub_blocks_past_field", "N 256 at T 2,048, Al 4, where T / Al allows it.", body(with(&|e| e.oti = raptor(10_240, 2_048, 1, 256, 4))), 4, oti);
refuse("raptor_block_past_k_max", "One block of 8,193 symbols.", body(with(&|e| {
e.oti = Oti::Raptor { transfer_length: 8_193 * 4, symbol_size: 4, source_blocks: 1, sub_blocks: 1, alignment: 4 };
e.repair = 0;
})), 4, oti);
refuse("raptor_block_under_the_fewest", "One block of 3 symbols, which RFC 5053 §5.7 does not index.", body(with(&|e| e.oti = rp(192, 64, 1))), 4, oti);
refuse("raptor_smaller_block_under_the_fewest", "Blocks of 4 and 3 symbols: the smaller is held to it.", body(with(&|e| e.oti = rp(448, 64, 2))), 4, oti);
refuse("raptor_empty_block", "Z = ⌈F / T⌉ + 1.", body(with(&|e| e.oti = rp(3_200, 64, 51))), 4, oti);
refuse("raptor_repeat_esi_past", "KL + repair = 2^16 + 1 under repeat.", body(with(&|e| {
e.oti = rp(3_200, 64, 3);
e.esi = Esi::Repeat;
e.repair = (1 << 16) - 16;
e.count = 1;
})), 4, sym);
refuse("raptor_advance_esi_past", "count · (KL + repair) = 2^16 + 16 under advance.", body(with(&|e| {
e.oti = rp(3_200, 64, 3);
e.repair = (1 << 12) - 16;
e.count = 16;
e.period_ms = 4_000;
})), 4, sym);
refuse("no_code_no_transfer", "L 0.", body(with(&|e| {
e.oti = nc(0, 32, 4);
e.repair = 0;
@ -822,9 +1115,10 @@ fn claim_cases(p: &Publication) -> Vec<ClaimCase> {
// reads the signature.
let signed = Chains::default().sign(PUBLISHER, link, one(base.clone()));
for (name, description, reason) in [
("fec_unknown", "fec `raptor`.", "field_type"),
("fec_unknown", "fec `lt`, which names no code here.", "field_type"),
("esi_unknown", "esi `skip`.", "field_type"),
("oti_of_another_code", "A RaptorQ entry with Compact No-Code's members.", "missing_required_field"),
("raptor_oti_of_another_code", "A Raptor entry with Compact No-Code's members.", "missing_required_field"),
("oti_member_unknown", "An `oti` with a sixth member.", "unknown_member"),
("entry_member_missing", "An entry with no `count`.", "missing_required_field"),
("first_not_integer", "`first_ms` as a string.", "field_type"),
@ -833,7 +1127,7 @@ fn claim_cases(p: &Publication) -> Vec<ClaimCase> {
let m = v["body"]["objects"][0].as_object_mut().expect("an entry");
match name {
"fec_unknown" => {
m.insert("fec".into(), "raptor".into());
m.insert("fec".into(), "lt".into());
}
"esi_unknown" => {
m.insert("esi".into(), "skip".into());
@ -841,6 +1135,10 @@ fn claim_cases(p: &Publication) -> Vec<ClaimCase> {
"oti_of_another_code" => {
m.insert("oti".into(), json!({"transfer_length": 3_200, "symbol_size": 64, "max_block_length": 4}));
}
"raptor_oti_of_another_code" => {
m.insert("fec".into(), "raptor".into());
m.insert("oti".into(), json!({"transfer_length": 3_200, "symbol_size": 64, "max_block_length": 4}));
}
"oti_member_unknown" => {
m["oti"].as_object_mut().expect("oti").insert("sub_symbol".into(), 4.into());
}
@ -859,12 +1157,17 @@ fn claim_cases(p: &Publication) -> Vec<ClaimCase> {
/// 9999-12-31T23:59:59.999Z.
const LAST: u64 = crate::claim_receipt::schedule::LAST_INSTANT_MS;
/// 65,535 · 65,535 · 8,192: the greatest transfer length RFC 5053 §5.3.1.2 can divide.
const RAPTOR_GREATEST: u64 = 35_183_298_355_200;
/// 69,431 · 20,394,401: 6,361 passes of it send 2^53 − 1 octets.
const BIG_OBJECT: u64 = 1_416_003_655_831;
fn plan_cases(p: &Publication) -> Vec<PlanCase> {
let bc = broadcast(p);
let (e0, e1, e2) = (bc[0].clone(), bc[1].clone(), bc[2].clone());
let e0_seven = ScheduleEntry { count: 7, ..e0.clone() };
let rs = under_raptor(p);
let (r0, r1) = (rs[0].clone(), rs[1].clone());
let ok = |passes: &[u64], e: &ScheduleEntry, complete: Option<u64>| {
json!({
"outcome": "plan",
@ -891,7 +1194,13 @@ fn plan_cases(p: &Publication) -> Vec<PlanCase> {
case("holding_unevenly", "One block held, one a symbol short and one 19 short: two passes, for the second.", &e0, T0 + 29_000, 300_000, 2, &[20, 1, 19], ok(&[2, 3], &e0, Some(end(&e0, 3)))),
case("too_late", "After pass 4 starts: one pass left and two wanted; it listens to the one, and is not held.", &e0, T0 + 100_000, 300_000, 2, &[0, 0, 0], ok(&[5], &e0, None)),
case("after_the_last", "After the last pass starts: nothing left.", &e0, T0 + 120_001, 300_000, 2, &[0, 0, 0], ok(&[], &e0, None)),
case("too_late_extended", "The receiver of too_late under a schedule extended to seven passes: passes 5 and 6, and it is held.", &e0_seven, T0 + 100_000, 300_000, 2, &[0, 0, 0], ok(&[5, 6], &e0_seven, Some(end(&e0_seven, 6)))),
case("already_held", "Holding 20 of every block: held now, nothing to listen to.", &e0, T0 + 50_000, 300_000, 2, &[20, 20, 20], ok(&[], &e0, Some(T0 + 50_000))),
case("raptor_advance", "Raptor at 30%, tuned in before the first pass: each pass adds 36 or 35 of the 45 or 44 needed, so passes 1 and 2.", &r0, T0 - 1_000, 300_000, 2, &[0, 0, 0], ok(&[1, 2], &r0, Some(end(&r0, 2)))),
case("raptor_advance_holding_part", "Raptor at 30%, holding 9 of each of 45, 45 and 44: one pass adds what each lacks.", &r0, T0 - 1_000, 300_000, 2, &[9, 9, 9], ok(&[1], &r0, Some(end(&r0, 1)))),
case("raptor_repeat_one_pass", "Raptor under repeat at 10%: one pass leaves 44 of 48, and K + 28 is 42.", &r1, T0, 100_000, 2, &[0], ok(&[1], &r1, Some(end(&r1, 1)))),
case("raptor_repeat_short", "Raptor under repeat at 30%: one pass leaves 34, and no later pass is certain to add.", &r1, T0, 300_000, 2, &[0], ok(&[1], &r1, None)),
case("raptor_repeat_already_held", "Holding K + 28 of Raptor's one block: held now.", &r1, T0, 300_000, 2, &[42], ok(&[], &r1, Some(T0))),
case("repeat_one_pass", "Object 1 with no loss: the first pass sends 20, all it needs.", &e1, T0, 0, 2, &[0], ok(&[1], &e1, Some(end(&e1, 1)))),
case("repeat_short", "Object 1 at 10%: one pass leaves 18 of 20, and no later pass is certain to add.", &e1, T0, 100_000, 2, &[0], ok(&[1], &e1, None)),
case("repeat_adds_nothing", "Holding 18 at 10%: a pass adds max(0, 20 − 18 − 2) = 0.", &e1, T0, 100_000, 2, &[18], ok(&[], &e1, None)),
@ -903,6 +1212,275 @@ fn plan_cases(p: &Publication) -> Vec<PlanCase> {
]
}
/// A key log's head revision (jwp-receipts §7.2), for the corpus's origins.
fn log(origin: &str, root: u8, next: u8, keys: Vec<KeyEntry>) -> Revision {
let r = Revision {
origin: origin.into(),
seq: 0,
prev: None,
issued_at: LOG_TIME.into(),
root: [0; 32],
next_root: commitment(&pk(next)),
keys,
revoked_modules: vec![],
grant_max_s: 86_400,
status_max_age_s: 3_600,
sig: [0; 64],
hybrid: None,
};
Revision::seal(&key(root), r).expect("a well-formed revision")
}
/// The publisher's key log: both its keys, in force.
fn publisher_log() -> Revision {
log(
ORIGIN,
LOG_ROOT,
LOG_NEXT,
vec![KeyEntry::active("d1", pk(PUBLISHER), &["delivery", "transmission", "schedule"]), KeyEntry::active("d2", pk(OTHER_KEY), &["delivery"])],
)
}
fn logs(list: &[(&str, String)]) -> BTreeMap<String, String> {
list.iter().map(|(o, t)| ((*o).to_owned(), t.clone())).collect()
}
/// An in-band case: §8.2 over what a receiver acquired.
struct InbandCase {
name: &'static str,
description: &'static str,
inputs: InbandInputs,
outcome: Value,
}
fn inband_exp(claims: &[&str], set: Result<u64, &str>) -> Value {
json!({
"outcome": "inband",
"claims": claims,
"set": match set { Ok(_) => "suffices", Err(r) => r },
"inband_ms": set.ok(),
})
}
fn inband_outcome(r: &Result<Inband, Refusal>) -> Value {
match r {
Err(e) => refuse(e.label()),
Ok(i) => json!({
"outcome": "inband",
"claims": i.claims.iter().map(ClaimVerdict::label).collect::<Vec<_>>(),
"set": match i.set { Ok(_) => "suffices", Err(r) => r },
"inband_ms": i.set.ok(),
}),
}
}
fn inband_cases(p: &Publication) -> Vec<InbandCase> {
let bc = broadcast(p);
let link = p.link();
let head = || logs(&[(ORIGIN, publisher_log().to_json())]);
let case = |name, description, bearer_id: &str, claims: Vec<String>, logs: BTreeMap<String, String>, outcome| InbandCase {
name,
description,
inputs: InbandInputs { bearer_id: bearer_id.into(), group: p.group.to_json(), claims, logs },
outcome,
};
let declared = |ch: &mut Chains| ch.sign(PUBLISHER, link, transmission(Bearer::Unidirectional, BCAST));
let set = |objects: Vec<ScheduleEntry>| {
let mut ch = Chains::default();
vec![declared(&mut ch), ch.sign(PUBLISHER, link, schedule(BCAST, objects))]
};
let one_pass = |e: &ScheduleEntry| ScheduleEntry { count: 1, ..e.clone() };
// An earlier claim of the publisher's about another group, then the declaration and schedule.
let long_chain = || {
let mut ch = Chains::default();
vec![
ch.sign(PUBLISHER, other_group().link_hash(), transmission(Bearer::Unidirectional, BCAST2)),
declared(&mut ch),
ch.sign(PUBLISHER, link, schedule(BCAST, bc.clone())),
]
};
let stands = ["declared", "stands"];
let missing = inband_exp(&stands, Err(INBAND_KEY_LOG_MISSING));
let mut out = vec![
case(
"suffices",
"The declaration and the schedule, from the chain's head, with the origin's key log: the interval is the least period, 30 s.",
BCAST,
set(bc.clone()),
head(),
inband_exp(&stands, Ok(30_000)),
),
case(
"interval_every_entry_one_pass",
"Every entry has one pass, whose periods are not read: the interval is the least duration, 1 s.",
BCAST,
set(bc.iter().map(one_pass).collect()),
head(),
inband_exp(&stands, Ok(1_000)),
),
case(
"interval_skips_one_pass",
"Object 0 has one pass with a period of 0, object 2 repeats every 20 s and object 1 every 30 s: the least of the two read, 20 s.",
BCAST,
set(vec![
ScheduleEntry { count: 1, period_ms: 0, ..bc[0].clone() },
bc[1].clone(),
ScheduleEntry { period_ms: 20_000, ..bc[2].clone() },
]),
head(),
inband_exp(&stands, Ok(20_000)),
),
case(
"with_an_extension",
"An earlier copy and the schedule that extends it, which adds object 3 every 20 s: one schedule stands, and the interval is the later one's.",
BCAST,
{
let mut later = bc.clone();
later[0].count = 7;
later.push(entry(p, 3, rq(200, 40, 1), 12, Esi::Repeat, T0 + 30_000, 20_000, 1_000, 2));
let mut ch = Chains::default();
vec![declared(&mut ch), ch.sign(PUBLISHER, link, schedule(BCAST, bc.clone())), ch.sign(PUBLISHER, link, schedule(BCAST, later))]
},
head(),
inband_exp(&["declared", "extended", "stands"], Ok(20_000)),
),
case(
"chain_holds_another_group",
"The chain from its head holds a claim about another group: refused at step 6 here, it breaks nothing, and the set suffices.",
BCAST,
long_chain(),
head(),
inband_exp(&["reject step=6 reason=transmission_subject_mismatch", "declared", "stands"], Ok(30_000)),
),
case(
"chain_without_its_head",
"The same chain without its first claim: the declaration's parent is not held, so the set does not suffice.",
BCAST,
long_chain()[1..].to_vec(),
head(),
inband_exp(&["reject step=5 reason=parent_not_held", "set-aside reason=bearer_not_declared"], Err(INBAND_CHAIN_INCOMPLETE)),
),
case(
"chain_out_of_order",
"The schedule given before the declaration it follows in the chain.",
BCAST,
{
let mut c = set(bc.clone());
c.reverse();
c
},
head(),
inband_exp(&["reject step=5 reason=parent_not_held", "reject step=5 reason=chain_break"], Err(INBAND_CHAIN_INCOMPLETE)),
),
case(
"no_schedule",
"The declaration alone: there is nothing to plan with.",
BCAST,
vec![declared(&mut Chains::default())],
head(),
inband_exp(&["declared"], Err(INBAND_NOT_SCHEDULED)),
),
case(
"schedule_of_another_instance",
"The set of the broadcast, asked about the multicast the receiver joined.",
MCAST,
set(bc.clone()),
head(),
inband_exp(&stands, Err(INBAND_NOT_SCHEDULED)),
),
case(
"no_declaration",
"A schedule at its chain's head with no declaration: set aside, so none stands.",
BCAST,
vec![Chains::default().sign(PUBLISHER, link, schedule(BCAST, bc.clone()))],
head(),
inband_exp(&[&format!("set-aside reason={BEARER_NOT_DECLARED}")], Err(INBAND_NOT_SCHEDULED)),
),
case(
"contradicted",
"A later schedule that withdraws a pass: both set aside, so none stands.",
BCAST,
{
let mut ch = Chains::default();
let mut fewer = bc.clone();
fewer[0].count = 4;
vec![declared(&mut ch), ch.sign(PUBLISHER, link, schedule(BCAST, bc.clone())), ch.sign(PUBLISHER, link, schedule(BCAST, fewer))]
},
head(),
{
let c = format!("set-aside reason={SCHEDULE_CONTRADICTED}");
inband_exp(&["declared", &c, &c], Err(INBAND_NOT_SCHEDULED))
},
),
case("key_log_absent", "No key log acquired.", BCAST, set(bc.clone()), logs(&[]), missing.clone()),
case(
"key_log_of_another_origin",
"A key log of another origin only, under its own name.",
BCAST,
set(bc.clone()),
logs(&[(OTHER_ORIGIN, log(OTHER_ORIGIN, OTHER_ROOT, OTHER_NEXT, vec![KeyEntry::active("d1", pk(PUBLISHER), &["delivery"])]).to_json())]),
missing.clone(),
),
case(
"key_log_names_another_origin",
"Another origin's revision acquired at the publisher's location: not held.",
BCAST,
set(bc.clone()),
logs(&[(ORIGIN, log(OTHER_ORIGIN, OTHER_ROOT, OTHER_NEXT, vec![KeyEntry::active("d1", pk(PUBLISHER), &["delivery"])]).to_json())]),
missing.clone(),
),
case(
"key_log_signature_broken",
"The publisher's key log with its signature broken: not held.",
BCAST,
set(bc.clone()),
{
let mut r = publisher_log();
r.sig[0] ^= 1;
logs(&[(ORIGIN, r.to_json())])
},
missing.clone(),
),
case("key_log_not_read", "A file at the key log's location that is no revision.", BCAST, set(bc.clone()), logs(&[(ORIGIN, "{}".into())]), missing.clone()),
case(
"key_log_lacks_the_key",
"The publisher's key log, listing only its other key.",
BCAST,
set(bc.clone()),
logs(&[(ORIGIN, log(ORIGIN, LOG_ROOT, LOG_NEXT, vec![KeyEntry::active("d2", pk(OTHER_KEY), &["delivery"])]).to_json())]),
missing,
),
case(
"key_log_revokes_the_key",
"A held log that revokes the schedule's key lists it: the set suffices, and what the log says of the key is confirmation's (bearer-receipts §8.3).",
BCAST,
set(bc.clone()),
logs(&[(
ORIGIN,
log(
ORIGIN,
LOG_ROOT,
LOG_NEXT,
vec![KeyEntry { status: EntryStatus::Revoked, compromised_since: Some(COMPROMISED.into()), ..KeyEntry::active("d1", pk(PUBLISHER), &["delivery", "schedule"]) }],
)
.to_json(),
)]),
inband_exp(&stands, Ok(30_000)),
),
];
let mut broken = p.group.to_json();
let sig = broken["sig"].as_str().expect("a group signature");
let flipped = format!("{}{}", if sig.starts_with('0') { '1' } else { '0' }, &sig[1..]);
broken["sig"] = flipped.into();
out.push(InbandCase {
name: "group_refused",
description: "A group whose signature is broken: nothing is derived.",
inputs: InbandInputs { bearer_id: BCAST.into(), group: broken, claims: set(bc.clone()), logs: head() },
outcome: refuse("signature_verify_failed"),
});
out
}
fn statement_value(s: &Option<Statement>) -> Value {
match s {
None => Value::Null,
@ -930,12 +1508,13 @@ fn verdict_value(c: &VerdictCase) -> Value {
pub fn generate() -> Vec<(String, String)> {
let p = publish();
let mut keys = Map::new();
for (name, seed) in [("publisher", PUBLISHER), ("publisher_d2", OTHER_KEY), ("sink", SINK)] {
for (name, seed) in [("publisher", PUBLISHER), ("publisher_d2", OTHER_KEY), ("sink", SINK), ("log_root", LOG_ROOT), ("other_log_root", OTHER_ROOT)] {
keys.insert(name.into(), hex(&pk(seed)).into());
}
let doc = json!({
"about": "Conformance corpus of extensions/carousel.md (wai.bearer.carousel): the schedule claim (jwp-receipts §2.4.16), \
when a receiver within a loss bound holds each object (§6), and the passes a receiver listens to (§7). Every \
a later schedule that extends an earlier one (§4.5), when a receiver within a loss bound holds each object (§6), \
the passes a receiver listens to (§7), and whether what a receiver acquired on an instance suffices (§8.2). Every \
key is Ed25519 from a fixed 32-byte seed and attests nothing; envelopes carry opaque test payloads of exact \
lengths; bearer instances use the example URN namespace and origins reserved example domains. Each case \
states the outcome it was written to have, declared in the generator. Regenerate with `wai_carousel_vectors gen`.",
@ -952,6 +1531,15 @@ pub fn generate() -> Vec<(String, String)> {
"held": c.held,
"outcome": c.outcome,
})).collect::<Vec<_>>(),
"inband": inband_cases(&p).iter().map(|c| json!({
"name": c.name,
"description": c.description,
"bearer_id": c.inputs.bearer_id,
"group": c.inputs.group,
"claims": c.inputs.claims,
"logs": Value::Object(c.inputs.logs.iter().map(|(o, t)| (o.clone(), Value::String(t.clone()))).collect()),
"outcome": c.outcome,
})).collect::<Vec<_>>(),
});
let mut s = serde_json::to_string_pretty(&doc).expect("serialises");
s.push('\n');
@ -1007,6 +1595,13 @@ fn plan_of(c: &Value) -> Result<Result<Plan, &'static str>, String> {
Ok(plan(&e, n("at_ms")?, n("loss_ppm")?, n("overhead")?, &held))
}
/// An in-band case's inputs, as the corpus writes them.
fn inband_inputs_of(c: &Value) -> Result<InbandInputs, String> {
let claims = c["claims"].as_array().ok_or("claims")?.iter().map(|w| w.as_str().map(str::to_owned).ok_or("claim")).collect::<Result<_, _>>()?;
let logs = c["logs"].as_object().ok_or("logs")?.iter().map(|(o, t)| Ok((o.clone(), t.as_str().ok_or("a log")?.to_owned()))).collect::<Result<_, String>>()?;
Ok(InbandInputs { bearer_id: c["bearer_id"].as_str().ok_or("bearer_id")?.to_owned(), group: c["group"].clone(), claims, logs })
}
fn strings(v: &Value, out: &mut Vec<String>) {
match v {
Value::String(s) => out.push(s.clone()),
@ -1026,8 +1621,8 @@ pub fn verify(files: &BTreeMap<String, String>) -> Vec<String> {
Err(e) => return vec![format!("vectors.json: {e}")],
};
let empty = Vec::new();
let [verdicts, claims, plans] = ["verdicts", "claims", "plans"].map(|k| doc[k].as_array().unwrap_or(&empty));
if verdicts.is_empty() || claims.is_empty() || plans.is_empty() {
let [verdicts, claims, plans, sets] = ["verdicts", "claims", "plans", "inband"].map(|k| doc[k].as_array().unwrap_or(&empty));
if verdicts.is_empty() || claims.is_empty() || plans.is_empty() || sets.is_empty() {
fails.push("the corpus lacks a kind of case".into());
}
let mut stated = Vec::new();
@ -1065,6 +1660,19 @@ pub fn verify(files: &BTreeMap<String, String>) -> Vec<String> {
}
strings(&c["outcome"], &mut stated);
}
for c in sets {
let name = c["name"].as_str().unwrap_or("?");
match inband_inputs_of(c) {
Err(e) => fails.push(format!("{name}: {e}")),
Ok(i) => {
let got = inband_outcome(&inband(&i));
if got != c["outcome"] {
fails.push(format!("{name}: stated {} got {got}", c["outcome"]));
}
}
}
strings(&c["outcome"], &mut stated);
}
for code in CODES {
if !stated.iter().any(|s| s == code || s.ends_with(&format!("reason={code}"))) {
fails.push(format!("no case gives {code}"));

190
wai/wai-rs/src/claim_receipt/schedule.rs
View file

@ -4,7 +4,8 @@
//!
//! This file holds steps 2 and 4 of jwp-receipts §3.1 for the class — its body's members and
//! types, and carousel §4.2 with §5's admissibility of each code's parameters — and step 6,
//! its checks against the group receipt and the object receipts (§4.3). What a verifier
//! its checks against the group receipt and the object receipts (§4.3, and cache-authority §5.6
//! for a node's own schedule), and when one schedule extends another (§4.5). What a verifier
//! derives from a schedule, and what a receiver plans with one, is `crate::carousel`'s.
use super::*;
@ -19,6 +20,14 @@ pub const K_PRIME_MAX: u64 = 56_403;
pub const RAPTORQ_MAX_TRANSFER: u64 = 942_574_504_275;
/// RFC 6330 §3.2: ESIs are 24 bits.
pub const RAPTORQ_ESIS: u64 = 1 << 24;
/// RFC 5053 §5.1.2: the most source symbols one source block has.
pub const RAPTOR_K_MAX: u64 = 8_192;
/// RFC 5053 §5.7: the fewest source symbols a block has a systematic index for.
pub const RAPTOR_K_MIN: u64 = 4;
/// RFC 5053 §3.2.2: a transfer length is less than this.
pub const RAPTOR_TRANSFER_LIMIT: u64 = 1 << 45;
/// RFC 5053 §3.1: ESIs are 16 bits.
pub const RAPTOR_ESIS: u64 = 1 << 16;
/// RFC 5445 §3.2.1: source block numbers and ESIs are 16 bits.
pub const NO_CODE_FIELD: u64 = 1 << 16;
/// 9999-12-31T23:59:59.999Z, the last instant a pass may end at (carousel §4.2 rule 3.3).
@ -37,6 +46,8 @@ const NO_CODE_OTI: &[&str] = &["transfer_length", "symbol_size", "max_block_leng
pub enum Fec {
/// RFC 6330, FEC Encoding ID 6.
RaptorQ,
/// RFC 5053, FEC Encoding ID 1.
Raptor,
/// RFC 5445, FEC Encoding ID 0.
NoCode,
}
@ -45,12 +56,13 @@ impl Fec {
pub fn label(self) -> &'static str {
match self {
Fec::RaptorQ => "raptorq",
Fec::Raptor => "raptor",
Fec::NoCode => "no-code",
}
}
pub fn from_label(s: &str) -> Option<Fec> {
[Fec::RaptorQ, Fec::NoCode].into_iter().find(|f| f.label() == s)
[Fec::RaptorQ, Fec::Raptor, Fec::NoCode].into_iter().find(|f| f.label() == s)
}
}
@ -81,6 +93,8 @@ impl Esi {
pub enum Oti {
/// RFC 6330 §3.3.2, §3.3.3: F, T, Z, N, Al.
RaptorQ { transfer_length: u64, symbol_size: u64, source_blocks: u64, sub_blocks: u64, alignment: u64 },
/// RFC 5053 §3.2.2, §3.2.3: F, T, Z, N, Al.
Raptor { transfer_length: u64, symbol_size: u64, source_blocks: u64, sub_blocks: u64, alignment: u64 },
/// RFC 5445 §3.2.2.2: the transfer length, the encoding symbol length, and the maximum
/// source block length in symbols.
NoCode { transfer_length: u64, symbol_size: u64, max_block_length: u64 },
@ -104,23 +118,24 @@ impl Oti {
pub fn fec(&self) -> Fec {
match self {
Oti::RaptorQ { .. } => Fec::RaptorQ,
Oti::Raptor { .. } => Fec::Raptor,
Oti::NoCode { .. } => Fec::NoCode,
}
}
pub fn transfer_length(&self) -> u64 {
match *self {
Oti::RaptorQ { transfer_length, .. } | Oti::NoCode { transfer_length, .. } => transfer_length,
Oti::RaptorQ { transfer_length, .. } | Oti::Raptor { transfer_length, .. } | Oti::NoCode { transfer_length, .. } => transfer_length,
}
}
pub fn symbol_size(&self) -> u64 {
match *self {
Oti::RaptorQ { symbol_size, .. } | Oti::NoCode { symbol_size, .. } => symbol_size,
Oti::RaptorQ { symbol_size, .. } | Oti::Raptor { symbol_size, .. } | Oti::NoCode { symbol_size, .. } => symbol_size,
}
}
/// Carousel §5.1 and §5.2: whether the code admits these parameters.
/// Carousel §5.1, §5.2 and §5.5: whether the code admits these parameters.
pub fn admitted(&self) -> bool {
match *self {
Oti::RaptorQ { transfer_length: f, symbol_size: t, source_blocks: z, sub_blocks: n, alignment: al } => {
@ -134,6 +149,17 @@ impl Oti {
&& z <= f.div_ceil(t)
&& f.div_ceil(t).div_ceil(z) <= K_PRIME_MAX
}
Oti::Raptor { transfer_length: f, symbol_size: t, source_blocks: z, sub_blocks: n, alignment: al } => {
(1..RAPTOR_TRANSFER_LIMIT).contains(&f)
&& (1..=255).contains(&al)
&& (1..=65_535).contains(&t)
&& t.is_multiple_of(al)
&& (1..=65_535).contains(&z)
&& (1..=255).contains(&n)
&& n <= t / al
&& f.div_ceil(t) / z >= RAPTOR_K_MIN
&& f.div_ceil(t).div_ceil(z) <= RAPTOR_K_MAX
}
Oti::NoCode { transfer_length: l, symbol_size: e, max_block_length: b } => {
if !((1..1 << 48).contains(&l) && (1..=65_535).contains(&e) && (1..1 << 32).contains(&b)) {
return false;
@ -146,11 +172,12 @@ impl Oti {
}
/// The source blocks by size, in the order the object is divided into them: the larger
/// first, a size no block has left out (carousel §5.1, §5.2). Only for parameters
/// first, a size no block has left out (carousel §5.1, §5.2, §5.5). Only for parameters
/// [`Oti::admitted`] admits.
pub fn blocks(&self) -> Vec<BlockSize> {
let (large, small, n_large, n_small) = match *self {
Oti::RaptorQ { transfer_length: f, symbol_size: t, source_blocks: z, .. } => partition(f.div_ceil(t), z),
Oti::RaptorQ { transfer_length: f, symbol_size: t, source_blocks: z, .. }
| Oti::Raptor { transfer_length: f, symbol_size: t, source_blocks: z, .. } => partition(f.div_ceil(t), z),
Oti::NoCode { transfer_length: l, symbol_size: e, max_block_length: b } => {
let ts = l.div_ceil(e);
let nb = ts.div_ceil(b);
@ -168,7 +195,8 @@ impl Oti {
fn to_value(self) -> Value {
match self {
Oti::RaptorQ { transfer_length, symbol_size, source_blocks, sub_blocks, alignment } => json_obj(&[
Oti::RaptorQ { transfer_length, symbol_size, source_blocks, sub_blocks, alignment }
| Oti::Raptor { transfer_length, symbol_size, source_blocks, sub_blocks, alignment } => json_obj(&[
("transfer_length", transfer_length),
("symbol_size", symbol_size),
("source_blocks", source_blocks),
@ -185,7 +213,8 @@ impl Oti {
fn values(&self) -> Vec<u64> {
match *self {
Oti::RaptorQ { transfer_length, symbol_size, source_blocks, sub_blocks, alignment } => {
Oti::RaptorQ { transfer_length, symbol_size, source_blocks, sub_blocks, alignment }
| Oti::Raptor { transfer_length, symbol_size, source_blocks, sub_blocks, alignment } => {
vec![transfer_length, symbol_size, source_blocks, sub_blocks, alignment]
}
Oti::NoCode { transfer_length, symbol_size, max_block_length } => {
@ -236,14 +265,15 @@ impl ScheduleEntry {
fn symbols_named(&self) -> bool {
match self.oti {
Oti::NoCode { .. } => self.repair == 0 && self.esi == Esi::Repeat,
Oti::RaptorQ { .. } => {
Oti::RaptorQ { .. } | Oti::Raptor { .. } => {
let kl = self.oti.blocks()[0].k;
let Some(m) = kl.checked_add(self.repair) else { return false };
let greatest = match self.esi {
Esi::Repeat => Some(m),
Esi::Advance => self.count.checked_mul(m),
};
greatest.is_some_and(|g| g <= RAPTORQ_ESIS)
let esis = if self.oti.fec() == Fec::Raptor { RAPTOR_ESIS } else { RAPTORQ_ESIS };
greatest.is_some_and(|g| g <= esis)
}
}
}
@ -265,7 +295,7 @@ impl ScheduleEntry {
/// [`ScheduleEntry::bytes_sent`], `None` past `u64`.
fn checked_bytes_sent(&self) -> Option<u64> {
match self.oti {
Oti::RaptorQ { symbol_size, .. } => {
Oti::RaptorQ { symbol_size, .. } | Oti::Raptor { symbol_size, .. } => {
let per_pass = self.oti.blocks().iter().try_fold(0u64, |a, s| {
s.k.checked_add(self.repair).and_then(|m| m.checked_mul(s.blocks)).and_then(|n| a.checked_add(n))
})?;
@ -317,6 +347,16 @@ impl ScheduleEntry {
self.to_value()
}
/// Carousel §4.5: whether this entry's passes begin with `earlier`'s, at the same instants and
/// sending the same symbols. Its members are `earlier`'s but `count`, which is at least
/// `earlier`'s, and `period_ms`, which may differ only where `earlier` has one pass, whose
/// period is not read.
pub fn extends(&self, earlier: &ScheduleEntry) -> bool {
self.count >= earlier.count
&& (earlier.count == 1 || self.period_ms == earlier.period_ms)
&& ScheduleEntry { count: earlier.count, period_ms: earlier.period_ms, ..self.clone() } == *earlier
}
/// Step 2: the members, then each one's type in the table's order, `oti`'s members as the
/// code `fec` names reads them.
fn read(v: &Value) -> Result<ScheduleEntry, Reason> {
@ -337,6 +377,16 @@ impl ScheduleEntry {
alignment: get_int(o, "alignment")?,
}
}
Fec::Raptor => {
members(o, RAPTORQ_OTI)?;
Oti::Raptor {
transfer_length: get_int(o, "transfer_length")?,
symbol_size: get_int(o, "symbol_size")?,
source_blocks: get_int(o, "source_blocks")?,
sub_blocks: get_int(o, "sub_blocks")?,
alignment: get_int(o, "alignment")?,
}
}
Fec::NoCode => {
members(o, NO_CODE_OTI)?;
Oti::NoCode {
@ -422,6 +472,16 @@ impl ScheduleClaim {
pub fn entry(&self, i: u64) -> Option<&ScheduleEntry> {
self.objects.iter().find(|e| e.leaf_index == i)
}
/// Carousel §4.5: whether this schedule extends `earlier`, of the same instance: for every
/// entry of `earlier` it has an entry of that object that extends it
/// ([`ScheduleEntry::extends`]). It may have entries for objects `earlier` has none for.
/// Every schedule extends itself.
pub fn extends(&self, earlier: &ScheduleClaim) -> bool {
self.bearer_id == earlier.bearer_id
&& self.attests == earlier.attests
&& earlier.objects.iter().all(|a| self.entry(a.leaf_index).is_some_and(|b| b.extends(a)))
}
}
impl ClaimReceipt {
@ -437,6 +497,18 @@ impl ClaimReceipt {
/// entry's content hash (`schedule_content_mismatch`) and, where its bytes are held, the
/// entry's transfer length (`schedule_length_mismatch`).
pub fn check_schedule_against(&self, group: &GroupReceipt, objects: &[(&ObjectReceipt, Option<&[u8]>)]) -> Result<(), Rejection> {
self.check_schedule_context(group, objects, true)
}
/// Step 6 for a node's schedule of its own instance (cache-authority §5.6): carousel §4.3
/// but for its step 2, since a node's key, not the group's, signs it. Whose schedule it is,
/// and where it stands, is `crate::cache_authority`'s; this checks it against the group as
/// [`ClaimReceipt::check_schedule_against`] does in every other step.
pub fn check_node_schedule_against(&self, group: &GroupReceipt, objects: &[(&ObjectReceipt, Option<&[u8]>)]) -> Result<(), Rejection> {
self.check_schedule_context(group, objects, false)
}
fn check_schedule_context(&self, group: &GroupReceipt, objects: &[(&ObjectReceipt, Option<&[u8]>)], publisher: bool) -> Result<(), Rejection> {
let ClaimBody::Schedule(s) = &self.body else {
return Err(Rejection::at(6, Reason::ContextMismatch));
};
@ -449,7 +521,7 @@ impl ClaimReceipt {
if group.link_hash() != self.subject {
return at(Reason::ScheduleSubjectMismatch);
}
if group.signer_pubkey != self.signer_pubkey {
if publisher && group.signer_pubkey != self.signer_pubkey {
return at(Reason::ScheduleSignerMismatch);
}
if s.objects.iter().any(|e| e.leaf_index >= group.leaf_count) {
@ -494,6 +566,10 @@ mod tests {
Oti::RaptorQ { transfer_length: f, symbol_size: t, source_blocks: z, sub_blocks: n, alignment: al }
}
fn rp(f: u64, t: u64, z: u64, n: u64, al: u64) -> Oti {
Oti::Raptor { transfer_length: f, symbol_size: t, source_blocks: z, sub_blocks: n, alignment: al }
}
fn nc(l: u64, e: u64, b: u64) -> Oti {
Oti::NoCode { transfer_length: l, symbol_size: e, max_block_length: b }
}
@ -539,6 +615,47 @@ mod tests {
assert!(!rq(K_PRIME_MAX * 4 + 1, 4, 1, 1, 4).admitted(), "one block past K'max");
}
#[test]
fn raptor_blocks_follow_the_partition() {
// 15 symbols in two blocks: 8 and 7.
assert_eq!(rp(960, 64, 2, 1, 4).blocks(), vec![BlockSize { k: 8, blocks: 1 }, BlockSize { k: 7, blocks: 1 }]);
}
#[test]
fn raptor_parameters_are_held_to_rfc_5053() {
assert!(rp(1000, 64, 2, 1, 4).admitted());
assert!(!rp(0, 64, 1, 1, 4).admitted(), "no octets");
assert!(rp(65_535 * 65_535 * 8_192, 65_535, 65_535, 1, 1).admitted(), "the most blocks of the most symbols");
assert!(!rp(65_535 * 65_535 * 8_192 + 1, 65_535, 65_535, 1, 1).admitted(), "a block past Kmax");
assert!(!rp(1 << 45, 65_535, 65_535, 1, 1).admitted(), "past the greatest F");
assert!(!rp(1000, 66, 1, 1, 4).admitted(), "T not a multiple of Al");
assert!(!rp(1000, 64, 1, 1, 0).admitted(), "no alignment");
assert!(!rp(1000, 256, 1, 1, 256).admitted(), "past Al's field");
assert!(!rp(1000, 64, 0, 1, 4).admitted(), "no block");
assert!(rp(1 << 20, 4, 65_535, 1, 4).admitted(), "Z at its field's greatest");
assert!(!rp(1 << 20, 4, 65_536, 1, 4).admitted(), "past Z's field");
assert!(!rp(1000, 64, 1, 0, 4).admitted(), "no sub-block");
assert!(!rp(1000, 64, 1, 17, 4).admitted(), "a sub-symbol under Al");
assert!(rp(1000, 64, 1, 16, 4).admitted(), "sub-symbols of Al each");
assert!(rp(8_160, 1_020, 1, 255, 4).admitted(), "N at its field's greatest");
assert!(!rp(8_192, 2_048, 1, 256, 4).admitted(), "past N's field");
assert!(rp(256, 64, 1, 1, 4).admitted(), "a block of the fewest symbols");
assert!(!rp(192, 64, 1, 1, 4).admitted(), "a block under the fewest symbols");
assert!(!rp(448, 64, 2, 1, 4).admitted(), "blocks of 4 and 3: the smaller under the fewest");
assert!(rp(8_192 * 4, 4, 1, 1, 4).admitted(), "one block of Kmax");
assert!(!rp(8_192 * 4 + 1, 4, 1, 1, 4).admitted(), "one block past Kmax");
}
#[test]
fn raptor_symbols_fit_sixteen_bits() {
let e = |repair, esi, count| ScheduleEntry { oti: rp(1_000, 64, 1, 1, 4), repair, esi, count, ..entry(0, 1, 10_000) };
// 16 source symbols.
assert!(e((1 << 16) - 16, Esi::Repeat, 1).symbols_named());
assert!(!e((1 << 16) - 15, Esi::Repeat, 1).symbols_named());
assert!(e((1 << 12) - 16, Esi::Advance, 16).symbols_named());
assert!(!e((1 << 12) - 15, Esi::Advance, 16).symbols_named());
}
#[test]
fn no_code_parameters_are_held_to_rfc_5445() {
assert!(nc(100, 10, 4).admitted());
@ -553,4 +670,51 @@ mod tests {
assert!(nc(NO_CODE_FIELD, 1, 1).admitted(), "65,536 blocks");
assert!(!nc(NO_CODE_FIELD + 1, 1, 1).admitted(), "a block past the SBN field");
}
fn entry(leaf_index: u64, count: u64, period_ms: u64) -> ScheduleEntry {
ScheduleEntry {
leaf_index,
content_hash: [leaf_index as u8; 32],
oti: rq(1000, 64, 2, 1, 4),
repair: 4,
esi: Esi::Advance,
first_ms: 1_000_000,
period_ms,
duration_ms: 2_000,
count,
}
}
fn sched(objects: Vec<ScheduleEntry>) -> ScheduleClaim {
ScheduleClaim { bearer_id: "urn:example:b".into(), objects, attests: ATTESTS_SCHEDULED.into() }
}
#[test]
fn an_entry_extends_one_whose_passes_it_begins_with() {
let a = entry(0, 3, 10_000);
assert!(a.extends(&a), "itself");
assert!(entry(0, 5, 10_000).extends(&a), "more passes");
assert!(!a.extends(&entry(0, 5, 10_000)), "fewer passes");
assert!(!entry(0, 5, 12_000).extends(&a), "a later pass moved");
assert!(entry(0, 4, 12_000).extends(&entry(0, 1, 0)), "one pass: its period is not read");
assert!(!entry(0, 4, 12_000).extends(&entry(0, 2, 0)), "two passes: the period is read");
let moved = ScheduleEntry { first_ms: 1_000_001, ..entry(0, 5, 10_000) };
assert!(!moved.extends(&a), "the first pass moved");
let more_repair = ScheduleEntry { repair: 5, ..entry(0, 5, 10_000) };
assert!(!more_repair.extends(&a), "other symbols");
let longer = ScheduleEntry { duration_ms: 2_001, ..entry(0, 5, 10_000) };
assert!(!longer.extends(&a), "a longer pass");
}
#[test]
fn a_schedule_extends_one_whose_every_entry_it_extends() {
let a = sched(vec![entry(0, 3, 10_000)]);
assert!(a.extends(&a));
assert!(sched(vec![entry(0, 3, 10_000), entry(2, 1, 0)]).extends(&a), "an object added");
assert!(sched(vec![entry(0, 4, 10_000), entry(2, 1, 0)]).extends(&a), "an object added and a pass");
assert!(!sched(vec![entry(2, 1, 0)]).extends(&a), "an object dropped");
assert!(!sched(vec![entry(0, 3, 10_000), entry(2, 1, 0)]).extends(&sched(vec![entry(0, 4, 10_000)])), "a pass withdrawn");
let other = ScheduleClaim { bearer_id: "urn:example:c".into(), ..a.clone() };
assert!(!other.extends(&a), "another instance");
}
}

13
wai/wai-rs/src/timeliness.rs
View file

@ -217,6 +217,9 @@ pub enum ClaimVerdict {
Declared,
/// A schedule that stands (a plan's).
Stands,
/// A schedule a later schedule of its instance extends (carousel §4.5), with another body (a
/// plan's).
Extended,
/// A declaration or a schedule accepted and set aside for the reason given (a plan's).
SetAside(&'static str),
}
@ -228,6 +231,7 @@ impl ClaimVerdict {
ClaimVerdict::Accepted => "accepted".into(),
ClaimVerdict::Declared => "declared".into(),
ClaimVerdict::Stands => "stands".into(),
ClaimVerdict::Extended => "extended".into(),
ClaimVerdict::SetAside(r) => format!("set-aside reason={r}"),
}
}
@ -624,11 +628,14 @@ pub(crate) fn planned(inp: &PlanInputs) -> Result<(PlanVerdict, Vec<Option<Timel
.iter()
.map(|c| c.as_ref().filter(|c| c.subject == link && (c.transmission().is_some() || c.schedule().is_some())).cloned())
.collect();
let (aside, standing) = crate::carousel::standing(&of_group);
for (i, reason) in aside {
let standing = crate::carousel::standing(&of_group);
for (i, reason) in standing.aside {
claims[i] = ClaimVerdict::SetAside(reason);
}
schedules.insert(link, standing);
for i in standing.extended {
claims[i] = ClaimVerdict::Extended;
}
schedules.insert(link, standing.schedules);
}
let arrival = |source: &Source, group: &[u8; 32], index: u64, s: u64| -> Option<u64> {

107
wai/wai-rs/src/timeliness/corpus.rs
View file

@ -222,6 +222,11 @@ fn transmission(bearer: Bearer, id: &str) -> ClaimBody {
/// every ten seconds from T0. Within the plans' loss bound a receiver gets 5 symbols a pass and
/// needs K′ + 2 = 12, so it holds the object after three passes.
fn schedule(p: &Publication, id: &str, objects: &[usize]) -> ClaimBody {
schedule_of(p, id, objects, 10)
}
/// The same schedule with `count` passes.
fn schedule_of(p: &Publication, id: &str, objects: &[usize], count: u64) -> ClaimBody {
let entries = objects
.iter()
.map(|&i| ScheduleEntry {
@ -233,7 +238,7 @@ fn schedule(p: &Publication, id: &str, objects: &[usize]) -> ClaimBody {
first_ms: T0,
period_ms: 10_000,
duration_ms: 1_000,
count: 10,
count,
})
.collect();
ClaimBody::Schedule(ScheduleClaim { bearer_id: id.into(), objects: entries, attests: ATTESTS_SCHEDULED.into() })
@ -1363,6 +1368,39 @@ fn plan_cases(g: &Groups) -> Vec<PlanCase> {
json!({ "claims": ["declared", format!("set-aside reason={}", crate::carousel::SCHEDULE_ON_BIDIRECTIONAL), "accepted"], "records": ["stands"], "choices": [no_source()] }),
);
}
{
let mut ch = Chains::default();
let mut i = base(at(T0), vec![bearer(B2)], &[0]);
i.claims = vec![
ch.sign(PUBLISHER, gl, transmission(Bearer::Unidirectional, B2)),
ch.sign(PUBLISHER, gl, schedule_of(&g.g, B2, &[0, 1, 2], 2)),
ch.sign(PUBLISHER, gl, schedule_of(&g.g, B2, &[0, 1, 2], 3)),
ch.sign(PUBLISHER, gl, main()),
];
push(
"plan_bearer_extended",
"Two passes on b2, too few for the 12 symbols a receiver needs, then a schedule that extends them to three (carousel §4.5): the plan reads the later, and its third pass delivers.",
i,
json!({ "claims": ["declared", "extended", "stands", "accepted"], "records": ["stands"], "choices": [choice("on_time", "primary", 0, pass_end(3), 1_000_000)] }),
);
}
{
let mut ch = Chains::default();
let mut i = base(at(T0), vec![bearer(B2)], &[0]);
i.claims = vec![
ch.sign(PUBLISHER, gl, transmission(Bearer::Unidirectional, B2)),
ch.sign(PUBLISHER, gl, schedule_of(&g.g, B2, &[0, 1, 2], 3)),
ch.sign(PUBLISHER, gl, schedule_of(&g.g, B2, &[0, 1, 2], 2)),
ch.sign(PUBLISHER, gl, main()),
];
let c = format!("set-aside reason={}", crate::carousel::SCHEDULE_CONTRADICTED);
push(
"plan_bearer_withdrawn",
"The same two schedules in the other order: the later withdraws a pass, both are set aside, and no source.",
i,
json!({ "claims": ["declared", c, c, "accepted"], "records": ["stands"], "choices": [no_source()] }),
);
}
push(
"wanted_without_intent",
"Object 3, which no entry states.",
@ -1441,6 +1479,51 @@ fn plan_cases(g: &Groups) -> Vec<PlanCase> {
planned(vec![choice("on_time", "primary", 0, T0 + 5_000, 1_000_000)]),
);
// A ladder read over a schedule a later one extends (carousel §4.5): F's two passes on b2 are
// too few for the 12 symbols a receiver needs, and the third is what reaches rung 1.
let ladder_on_b2 = |counts: [u64; 2]| {
let mut ch = Chains::default();
let mut i = base(at(T0), vec![origin(66_000), bearer(B2)], &[0]);
i.claims = vec![
ch.sign(PUBLISHER, fl, transmission(Bearer::Unidirectional, B2)),
ch.sign(PUBLISHER, fl, schedule_of(&g.f, B2, &[0, 1, 2, 3], counts[0])),
ch.sign(PUBLISHER, fl, schedule_of(&g.f, B2, &[0, 1, 2, 3], counts[1])),
ch.sign(PUBLISHER, gl, timeliness(vec![e0_ladder(g)])),
];
i.records = vec![];
i
};
push(
"ladder_rung_on_extended_schedule",
"The primary six seconds late (300,000), no route to rung 0, and rung 1 on b2 under a schedule extended from two passes to three: the plan reads the later, and rung 1 arrives at T0 + 21 s.",
ladder_on_b2([2, 3]),
json!({ "claims": ["declared", "extended", "stands", "accepted"], "records": [], "choices": [rung_choice(1, 1, pass_end(3), 400_000)] }),
);
{
let c = format!("set-aside reason={}", crate::carousel::SCHEDULE_CONTRADICTED);
push(
"ladder_rung_on_withdrawn_schedule",
"The same two schedules in the other order: the later withdraws a pass, both are set aside, b2 delivers no rung, and the late primary is taken.",
ladder_on_b2([3, 2]),
json!({ "claims": ["declared", c, c, "accepted"], "records": [], "choices": [choice("after_deadline", "primary", 0, D + 6_000, 300_000)] }),
);
}
{
let mut i = ladder_on_b2([2, 3]);
let mut ch = Chains::default();
i.claims = vec![
ch.sign(PUBLISHER, fl, transmission(Bearer::Unidirectional, B2)),
ch.sign(PUBLISHER, fl, schedule_of(&g.f, B2, &[0, 1, 2, 3], 2)),
ch.sign(PUBLISHER, gl, timeliness(vec![e0_ladder(g)])),
];
push(
"ladder_rung_before_extension",
"The earlier schedule alone, its two passes too few: no rung arrives, so the extension above is what brings rung 1 in reach.",
i,
json!({ "claims": ["declared", "stands", "accepted"], "records": [], "choices": [choice("after_deadline", "primary", 0, D + 6_000, 300_000)] }),
);
}
let sources_case = |sources: Vec<Value>| base(at(T0), sources, &[0]);
for (name, description, sources) in [
("sources_empty", "No source.", vec![]),
@ -1809,6 +1892,26 @@ fn run_cases(g: &Groups) -> Vec<RunCase> {
);
}
{
// A run does not date its claims (§7.5.1): every plan reads the last schedule of b2.
let fl = g.f.link();
let mut ch = Chains::default();
let claims = vec![
ch.sign(PUBLISHER, fl, transmission(Bearer::Unidirectional, B2)),
ch.sign(PUBLISHER, fl, schedule_of(&g.f, B2, &[0, 1, 2, 3], 2)),
ch.sign(PUBLISHER, fl, schedule_of(&g.f, B2, &[0, 1, 2, 3], 3)),
ch.sign(PUBLISHER, gl, timeliness(vec![e0_ladder(g), e1(g), e2(g)])),
];
let plans = vec![plan_at(T0, vec![origin(66_000), bearer(B2)], &[0])];
let i = RunInputs { groups: vec![g.g.receipts(), g.f.receipts()], claims, records: vec![], loss: Some((LOSS, OVERHEAD)), plans, arrivals: vec![got(&g.f, 0, pass_end(3))] };
push(
"run_ladder_rung_on_extended_schedule",
"Rung 1 on b2 under a schedule extended from two passes to three: fetched from T0, held at the end of the third, presented at the deadline at 400,000.",
i,
ran(vec![vec![rung_choice(1, 1, pass_end(3), 400_000)]], vec![fetch_rung(T0, 0, 0, 1, 1), show_rung(D, 0, 0, 1, 400_000)]),
);
}
// §7.5.1, each rule, in its order.
let (plans_invalid, arrivals_invalid) = (scheduler::PLANS_INVALID, scheduler::ARRIVALS_INVALID);
let bad_arrivals = || vec![got(&g.g, 0, T0 + 5), got(&g.g, 1, T0 + 1)];
@ -2053,7 +2156,7 @@ pub fn plan_inputs_of(c: &Value) -> Result<PlanInputs, String> {
}
/// A line (§6.3): *f* is one instant for a ladder of one rung, and the list of them for more.
fn line_value(l: &Line) -> Value {
pub(crate) fn line_value(l: &Line) -> Value {
let f = if l.rungs_ms.len() > 1 { json!(l.rungs_ms) } else { json!(l.fallback_ms) };
json!({ "holder": hex(&l.holder), "held_ms": l.held_ms, "fallback_ms": f, "outcome": l.outcome.label(), "value_ppm": l.value_ppm })
}

152
wai/wai-rs/tests/carousel.rs
View file

@ -4,7 +4,8 @@
//! the class codes at their steps, and Appendix A is the reference's table of K′. Against an
//! independent implementation of RFC 6330: the table of K′ and the partition agree for every block
//! size, and each RaptorQ object of the corpus that a case holds available at a pass decodes from
//! what a receiver within its loss bound holds by that pass's end.
//! what a receiver within its loss bound holds by that pass's end. Against an independent
//! implementation of RFC 5053, each Raptor object does the same.
#![cfg(feature = "carousel")]
use std::collections::BTreeMap;
@ -139,6 +140,152 @@ fn receive(e: &ScheduleEntry, data: &[u8], loss_ppm: u64, pass: u64) -> Option<V
None
}
/// [`receive`] under an independent RFC 5053 decoder, block by block (RFC 5053 §5.3.1.2): `None`
/// where a block does not decode from what the receiver holds.
fn receive_raptor(e: &ScheduleEntry, data: &[u8], loss_ppm: u64, pass: u64) -> Option<Vec<u8>> {
let Oti::Raptor { transfer_length, symbol_size: t, source_blocks: z, .. } = e.oti else { return None };
let kt = transfer_length.div_ceil(t);
let (kl, ks, zl, zs) = partition(kt, z);
let mut padded = data.to_vec();
padded.resize((kt * t) as usize, 0);
let advance = e.esi == Esi::Advance;
let mut out = Vec::with_capacity(padded.len());
let mut at = 0usize;
for k in std::iter::repeat_n(kl, zl as usize).chain(std::iter::repeat_n(ks, zs as usize)) {
let length = (k * t) as usize;
let block = &padded[at..at + length];
at += length;
let mut encoder = raptor_code::SourceBlockEncoder::new(block, k as usize).expect("a block RFC 5053 admits");
assert_eq!(u64::from(encoder.nb_source_symbols()), k);
let mut decoder = raptor_code::SourceBlockDecoder::new(k as usize);
let m = k + e.repair;
let x = missed(loss_ppm, m);
for j in 1..=(if advance { pass } else { 1 }) {
let start = if advance { (j - 1) * m } else { 0 };
for esi in start + x..start + m {
decoder.push_encoding_symbol(&encoder.fountain(esi as u32), esi as u32);
}
}
if !decoder.fully_specified() {
return None;
}
out.extend(decoder.decode(length)?);
}
out.truncate(transfer_length as usize);
Some(out)
}
/// Of `trials` sets of `k` + `margin` distinct encoding symbols of a block of `k`, chosen uniformly
/// at random from a fixed seed, how many an independent RFC 5053 decoder cannot solve.
fn raptor_failures(k: usize, margin: usize, trials: usize, seed: u64) -> usize {
let block: Vec<u8> = (0..k * 4).map(|i| (i * 131 + 7) as u8).collect();
let mut encoder = raptor_code::SourceBlockEncoder::new(&block, k).expect("a block RFC 5053 admits");
let n = k + 3 * margin + 40;
let symbols: Vec<Vec<u8>> = (0..n as u32).map(|esi| encoder.fountain(esi)).collect();
let mut state = seed;
let mut next = move || {
state ^= state << 13;
state ^= state >> 7;
state ^= state << 17;
state as usize
};
(0..trials)
.filter(|_| {
let mut esis: Vec<usize> = (0..n).collect();
for i in 0..k + margin {
let j = i + next() % (n - i);
esis.swap(i, j);
}
let mut decoder = raptor_code::SourceBlockDecoder::new(k);
for &esi in &esis[..k + margin] {
decoder.push_encoding_symbol(&symbols[esi], esi as u32);
}
!decoder.fully_specified()
})
.count()
}
/// Carousel §5.3 holds a Raptor block at K and a margin of this text's, RFC 5053 stating no bound.
/// An independent decoder, on a fixed sample, fails on many blocks at K + 2 and on few at each
/// margin. This is a measurement of that sample, as §5.3 says, and no bound.
#[test]
fn raptor_margins_keep_decoding_failures_rare() {
use wai::carousel::RAPTOR_MARGIN;
for k in [16, 64] {
let seed = 0x9E37_79B9_7F4A_7C15 ^ k as u64;
let at_two = raptor_failures(k, 2, 400, seed);
assert!(at_two >= 40, "K = {k}: {at_two} of 400 at K + 2");
let first = raptor_failures(k, RAPTOR_MARGIN[0] as usize, 2_000, seed);
assert!(first <= 20, "K = {k}: {first} of 2,000 at K + {}", RAPTOR_MARGIN[0]);
let second = raptor_failures(k, RAPTOR_MARGIN[1] as usize, 10_000, seed);
assert!(second <= 1, "K = {k}: {second} of 10,000 at K + {}", RAPTOR_MARGIN[1]);
let third = raptor_failures(k, RAPTOR_MARGIN[2] as usize, 10_000, seed);
assert_eq!(third, 0, "K = {k}: of 10,000 at K + {}", RAPTOR_MARGIN[2]);
}
}
/// The measurement carousel §5.3 quotes, by block size and margin, from fixed seeds. It takes some
/// minutes in a release build: `cargo test --release --test carousel -- --ignored --nocapture`.
#[test]
#[ignore = "a measurement of some minutes, run by hand in a release build; carousel §5.3 quotes it"]
fn raptor_margin_measurement() {
let sample: &[(usize, &[(usize, usize)])] = &[
(2, &[(4, 6_000), (16, 6_000), (64, 6_000), (1_024, 600)]),
(8, &[(4, 1_200_000), (8, 1_200_000), (16, 1_200_000), (64, 1_200_000), (256, 240_000), (1_024, 42_000), (2_048, 18_000), (8_192, 1_800)]),
(16, &[(4, 1_800_000), (8, 1_800_000), (16, 1_800_000), (64, 1_800_000), (256, 360_000), (1_024, 60_000), (2_048, 18_000), (8_192, 1_800)]),
(28, &[(4, 6_000_000), (8, 6_000_000), (16, 6_000_000), (32, 6_000_000), (64, 6_000_000), (128, 3_000_000), (256, 1_200_000), (1_024, 120_000), (2_048, 24_000), (8_192, 1_800)]),
];
for (margin, sizes) in sample {
for (k, trials) in *sizes {
let (k, margin, trials) = (*k, *margin, *trials);
let failures: usize = std::thread::scope(|s| {
let runs: Vec<_> = (0..6u64)
.map(|t| s.spawn(move || raptor_failures(k, margin, trials / 6, 0x9E37_79B9_7F4A_7C15 ^ (((k as u64) << 24) | ((margin as u64) << 8) | (t + 1)))))
.collect();
runs.into_iter().map(|r| r.join().expect("a run")).sum()
});
println!("K {k}, K + {margin}: {failures} of {trials}");
}
}
}
#[test]
fn each_raptor_object_decodes_from_what_a_receiver_within_its_bound_holds() {
let files = committed();
let doc: Value = serde_json::from_str(&files["vectors.json"]).expect("the corpus");
let mut decoded = 0;
for c in doc["verdicts"].as_array().expect("verdict cases") {
let Some(lines) = c["outcome"]["lines"].as_array() else { continue };
let loss = c["loss_ppm"].as_u64().expect("a loss bound");
let schedules: Vec<Value> = c["claims"]
.as_array()
.expect("claims")
.iter()
.map(|w| serde_json::from_str::<Value>(w.as_str().expect("a claim")).expect("json"))
.filter(|v| v["class"] == "schedule")
.collect();
for line in lines {
let Some(pass) = line["pass"].as_u64() else { continue };
let leaf = line["leaf_index"].as_u64().expect("a leaf") as usize;
let Some(data) = c["objects"][leaf]["envelope"].as_str().map(hex) else { continue };
let entry = schedules
.iter()
.filter(|s| s["body"]["bearer_id"] == line["bearer_id"])
.flat_map(|s| s["body"]["objects"].as_array().expect("entries").iter())
.rfind(|o| o["leaf_index"] == line["leaf_index"])
.map(|o| ScheduleEntry::from_value(o).expect("an entry that stands"))
.expect("the entry of a line");
if !matches!(entry.oti, Oti::Raptor { .. }) {
continue;
}
let got = receive_raptor(&entry, &data, loss, pass);
assert_eq!(got.as_deref(), Some(&data[..]), "{} object {leaf} on {} by pass {pass}", c["name"], line["bearer_id"]);
decoded += 1;
}
}
assert!(decoded >= 10, "only {decoded} objects decoded");
}
#[test]
fn each_raptorq_object_decodes_from_what_a_receiver_within_its_bound_holds() {
let files = committed();
@ -162,7 +309,8 @@ fn each_raptorq_object_decodes_from_what_a_receiver_within_its_bound_holds() {
.iter()
.filter(|s| s["body"]["bearer_id"] == line["bearer_id"])
.flat_map(|s| s["body"]["objects"].as_array().expect("entries").iter())
.find(|o| o["leaf_index"] == line["leaf_index"])
// The schedule that stands is the last of its instance (§6.2 step 4).
.rfind(|o| o["leaf_index"] == line["leaf_index"])
.map(|o| ScheduleEntry::from_value(o).expect("an entry that stands"))
.expect("the entry of a line");
if !matches!(entry.oti, Oti::RaptorQ { .. }) {

11
wai/wai-rs/tests/fixtures/dev-stack/fake-stack.py vendored
View file

@ -15,6 +15,7 @@ import json
import os
import re
import socket
import socketserver
import sys
import threading
import time
@ -55,6 +56,14 @@ class Room(http.server.BaseHTTPRequestHandler):
pass
class Server(http.server.ThreadingHTTPServer):
"""HTTPServer looks up a name for its address before it listens, and nothing here reads the
name. On some hosts that lookup takes 35 s, past what `wai dev` waits for a service."""
def server_bind(self):
socketserver.TCPServer.server_bind(self)
def main(argv):
if argv[:1] in (["keygen"], ["relay-lines"]):
open(argv[1], "a").close()
@ -73,7 +82,7 @@ def main(argv):
while True:
c, _ = s.accept()
c.close()
http.server.ThreadingHTTPServer(listen_addr(config, "listen"), Room).serve_forever()
Server(listen_addr(config, "listen"), Room).serve_forever()
return 0