wai: a ladder of timeliness fallbacks, and the reference sink scheduler #655
18 changed files with 21212 additions and 3318 deletions
31
.github/wai-conformance/verifiers/timeliness.sh
vendored
31
.github/wai-conformance/verifiers/timeliness.sh
vendored
|
|
@ -5,23 +5,24 @@
|
|||
#
|
||||
# wai.delivery.timeliness (extensions/timeliness.md): the reference module's unit tests (the
|
||||
# value function against an exact reference, the collinearity rule, each rule refusing, the
|
||||
# judgement, the held-at statement's before-group test and the plan's sources, and the
|
||||
# corpus reaching every stated outcome and failing a changed one) and tests/timeliness.rs
|
||||
# (vectors.json regenerates byte for byte from its fixed seeds, wai-rs gives every case its
|
||||
# stated outcome, §9 lists exactly the reference's codes and §6.3 its outcomes, jwp-receipts
|
||||
# §3.1 lists the class codes at their steps, and delivery-telemetry §5.1 lists the switch
|
||||
# reason); then the corpus's second implementation, timeliness_verify.py, written from the
|
||||
# extension's text and sharing no code with wai-rs, which verifies groups and objects with
|
||||
# jwp_verify.py, claims through steps 1-5 with claim_verify.py, schedules and passes with
|
||||
# carousel_verify.py and records with holdings_verify.py, writes the context checks, the
|
||||
# value function, the judgement and the plans itself, reads §9 and §6.3, requires every
|
||||
# mutated copy of the corpus to fail, and drops each of its own checks in turn; run isolated
|
||||
# (-I). Last, claim-conformance's own verifier reads every claim of the corpus through
|
||||
# jwp-receipts §3.1 steps 1 to 4 and requires the stated outcome.
|
||||
# ladder's bounds and forms, the judgement, the held-at statement's before-group test, the plan's
|
||||
# sources and the scheduler's input checks, and the corpus reaching every stated outcome and
|
||||
# failing a changed one) and tests/timeliness.rs (vectors.json regenerates byte for byte from its
|
||||
# fixed seeds, wai-rs gives every case its stated outcome, §9 lists exactly the reference's codes,
|
||||
# §6.3 its outcomes and §7.5.3 its run's actions, jwp-receipts §3.1 lists the class codes at their
|
||||
# steps, and delivery-telemetry §5.1 lists the switch reason); then the corpus's second
|
||||
# implementation, timeliness_verify.py, written from the extension's text and sharing no code with
|
||||
# wai-rs, which verifies groups and objects with jwp_verify.py, claims through steps 1-5 with
|
||||
# claim_verify.py, schedules and passes with carousel_verify.py and records with
|
||||
# holdings_verify.py, writes the context checks, the value function, the judgement, the plans and
|
||||
# a sink's runs of them itself, reads §9, §6.3 and §7.5.3, requires every mutated copy of the corpus
|
||||
# to fail, and drops each of its own checks in turn; run isolated (-I). Last, claim-conformance's
|
||||
# own verifier reads every claim of the corpus through jwp-receipts §3.1 steps 1 to 4 and requires
|
||||
# the stated outcome.
|
||||
#
|
||||
# The paths this step brings to the workflow's trigger, and why:
|
||||
# wai/extensions/timeliness.md: tests/timeliness.rs requires §9 to list the reference's codes and
|
||||
# §6.3 its outcomes, and timeliness_verify.py reads both.
|
||||
# wai/extensions/timeliness.md: tests/timeliness.rs requires §9 to list the reference's codes,
|
||||
# §6.3 its outcomes and §7.5.3 its run's actions, and timeliness_verify.py reads all three.
|
||||
set -euo pipefail
|
||||
(cd wai-rs && cargo test --no-default-features --features "$WAI_FEATURES" --lib -- timeliness:: claim_receipt::timeliness::)
|
||||
(cd wai-rs && cargo test --no-default-features --features "$WAI_FEATURES" --test timeliness)
|
||||
|
|
|
|||
|
|
@ -1828,13 +1828,15 @@ bytes: [`extensions/holdings.md`](extensions/holdings.md).
|
|||
And for **when** an object is still worth delivering — `wai.delivery.timeliness`: a publisher's
|
||||
signed statement, a jwp-receipts claim of class `timeliness` beside the group receipt and one for
|
||||
every bearer, of each object's instant of use, its deadline, how its value falls after it (a stated
|
||||
piecewise-linear function in parts per million, floored) and the rendition a receiver presents in
|
||||
its place, a separate object derived from it. A verifier holds the statement to its group and
|
||||
piecewise-linear function in parts per million, floored) and the renditions a receiver presents in
|
||||
its place, a ladder of separate objects derived from it in order of falling value. A verifier holds
|
||||
the statement to its group and
|
||||
objects, refuses one for a staged layer, sets aside two that differ, and judges each object for
|
||||
each holder from holdings records, as a sink that decides at the deadline would present: on time,
|
||||
replaced, held only after its deadline at a value it shows at least, or not shown
|
||||
(`deadline_not_stated` where nothing is stated). An asker plans which of its sources, in its own
|
||||
order, delivers each object in time; the deadline maps onto MoQ's delivery timeout, CMCD's `dl`,
|
||||
order, delivers each object in time, and a sink runs its plans against its own clock; the deadline
|
||||
maps onto MoQ's delivery timeout, CMCD's `dl`,
|
||||
DVB's target acquisition latency, ATSC's entry-package load time and a bundle's lifetime:
|
||||
[`extensions/timeliness.md`](extensions/timeliness.md).
|
||||
And for **what delivering a rung costs** — `wai.delivery.rung_price`: a seller's signed list
|
||||
|
|
|
|||
|
|
@ -0,0 +1,38 @@
|
|||
### Timeliness: a ladder of fallbacks, and the reference scheduler a sink runs its plans with
|
||||
|
||||
- **The ladder** ([extensions/timeliness.md](extensions/timeliness.md) §4.1, §4.2, §5.3, §6.3,
|
||||
§7.3). An entry's `fallback` is now an ordered ladder of renditions: `null`, one rung as an
|
||||
object, or two to eight rungs as an array in strictly falling `value_ppm`, no object named twice
|
||||
(`timeliness_ladder_invalid`, §4.2 rule 3.4). One rung has one form, the object, and an array of
|
||||
one is refused, so an entry with one fallback object is a ladder of one rung, read, judged and
|
||||
planned as it was, and two entries that state one ladder are written alike. Every rung is held to
|
||||
§4.3's steps 6 to 8 and derives from the primary itself.
|
||||
- **Which rung, when** (§5.3). At `by_ms` a sink presents the primary it holds, else the first rung
|
||||
of the ladder it holds, the highest valued. Holding neither, it has passed every rung, states
|
||||
`passed`, and presents no rung in the primary's place from then on; it presents the primary when
|
||||
it first holds it, at its value then, where that is at least 1, and at the primary's end, having
|
||||
presented nothing, it abandons any request and states `ended`. A line's *f* is one instant for a
|
||||
ladder of one rung and the list of them for more (§6.3), and a plan names the rung it chooses, by
|
||||
its index (§7.3).
|
||||
- **Running a plan** (§7.5), the reference `timeliness::scheduler`: from a sink's plans and the
|
||||
instants at which it first held each object, the sink's decisions (`fetch`, `abandon`, `present`,
|
||||
`passed`, `ended`) by exact steps, a function of those inputs alone. A later plan replaces the
|
||||
one in force wholly: a fetch it does not continue from the same source is abandoned, an object it
|
||||
no longer wants is no longer run, and what is settled stays settled. `plans_invalid` and
|
||||
`arrivals_invalid` refuse a run's inputs (§7.5.1).
|
||||
- **The corpus** [`live-conformance/timeliness/`](live-conformance/timeliness/) gains the group E,
|
||||
ladder claims, verdicts and plans, and `runs`: the traces and edges of §7.5.3, among them a late
|
||||
object, every rung passed and a plan replaced mid-run. `timeliness_verify.py` writes the ladder
|
||||
and the run from the text and is required to give every case its stated outcome, and
|
||||
`claim_verify.py` holds rule 3.4 and reads every run's claims. A plan's choices in the corpus gain
|
||||
the member `rung`.
|
||||
- **Breaking:** `api:rust:wai::claim_receipt::Reason::TimelinessLadderInvalid`: the reason rule 3.4
|
||||
raises, which an exhaustive match over `Reason` needs.
|
||||
- **Breaking:** `api:rust:wai::claim_receipt::TimelinessEntry::fallback` and
|
||||
`api:rust:wai::claim_receipt::timeliness::TimelinessEntry::fallback`: the field is a `Ladder`
|
||||
rather than an `Option<Fallback>`, since a fallback may hold several rungs, and each form is kept
|
||||
as read so a claim writes back to its own bytes. An `Option<Fallback>` converts with `.into()`,
|
||||
and `fallback.rungs()` reads the rungs of any form.
|
||||
- **Breaking:** `api:rust:wai::timeliness::Choice::rung` and
|
||||
`api:rust:wai::timeliness::Line::rungs_ms`: a plan's choice names its rung, and a line holds each rung's instant, which a struct literal of
|
||||
either needs. `Line::fallback_ms` keeps its meaning for one rung and is `None` for an array.
|
||||
|
|
@ -1633,16 +1633,19 @@ def schedule_entry_rules(e):
|
|||
return blocks
|
||||
|
||||
|
||||
# timeliness §4.1: an entry's members, a decay point's and a fallback's; §2's full value.
|
||||
# timeliness §4.1: an entry's members, a decay point's and a rung's; §2's full value; §4.2 rule
|
||||
# 3.4's most rungs.
|
||||
TIMELINESS_ENTRY = ["leaf_index", "content_hash", "from_ms", "by_ms", "decay", "fallback"]
|
||||
TIMELINESS_POINT = ["at_ms", "value_ppm"]
|
||||
TIMELINESS_FALLBACK = ["group", "leaf_index", "content_hash", "value_ppm"]
|
||||
FULL_VALUE = 1_000_000
|
||||
MOST_RUNGS = 8
|
||||
|
||||
|
||||
def timeliness_entry_body(e):
|
||||
"""timeliness §4.1 at §3.1 step 2: one entry of `objects`, its members and then their types in
|
||||
the table's order, each point's and the fallback's members where the table reaches them."""
|
||||
the table's order, each point's and each rung's members where the table reaches them. A fallback
|
||||
is `null`, one rung as an object, or an array of rungs."""
|
||||
nested(e, TIMELINESS_ENTRY)
|
||||
need(is_int(e["leaf_index"]) and is_hex(e["content_hash"], 32), 2)
|
||||
need(is_int(e["from_ms"]) and is_int(e["by_ms"]), 2)
|
||||
|
|
@ -1651,15 +1654,21 @@ def timeliness_entry_body(e):
|
|||
nested(p, TIMELINESS_POINT)
|
||||
need(is_int(p["at_ms"]) and is_int(p["value_ppm"]), 2)
|
||||
f = e["fallback"]
|
||||
if f is not None:
|
||||
nested(f, TIMELINESS_FALLBACK)
|
||||
need(is_hex(f["group"], 32) and is_int(f["leaf_index"]), 2)
|
||||
need(is_hex(f["content_hash"], 32) and is_int(f["value_ppm"]), 2)
|
||||
for r in timeliness_rungs(f):
|
||||
nested(r, TIMELINESS_FALLBACK)
|
||||
need(is_hex(r["group"], 32) and is_int(r["leaf_index"]), 2)
|
||||
need(is_hex(r["content_hash"], 32) and is_int(r["value_ppm"]), 2)
|
||||
|
||||
|
||||
def timeliness_rungs(f):
|
||||
"""timeliness §4.1: a fallback's rungs in the ladder's order: none for `null`, the array's, or
|
||||
the one it is. Anything else `nested` refuses as a rung."""
|
||||
return [] if f is None else f if isinstance(f, list) else [f]
|
||||
|
||||
|
||||
def timeliness_entry_rules(e, subject):
|
||||
"""timeliness §4.2 rule 3 for one entry of a claim whose `subject` is given, the first of its
|
||||
three to fail: the window, the decay, the fallback."""
|
||||
four to fail: the window, the decay, each rung, the ladder."""
|
||||
need(e["from_ms"] <= e["by_ms"] < LAST_INSTANT_MS, 4, "timeliness_window_invalid")
|
||||
# Rule 3.2, with (t0, v0) = (by_ms, full value) before the points. Python's integers are
|
||||
# exact, so the products, which reach 2^68, need no wider type.
|
||||
|
|
@ -1671,10 +1680,19 @@ def timeliness_entry_rules(e, subject):
|
|||
and all((pts[k - 1][1] - pts[k][1]) * (pts[k + 1][0] - pts[k][0])
|
||||
!= (pts[k][1] - pts[k + 1][1]) * (pts[k][0] - pts[k - 1][0]) for k in range(1, n)))
|
||||
need(formed, 4, "timeliness_decay_invalid")
|
||||
f = e["fallback"]
|
||||
need(f is None or (1 <= f["value_ppm"] <= FULL_VALUE and f["content_hash"] != e["content_hash"]
|
||||
and (f["group"] != subject or f["leaf_index"] != e["leaf_index"])),
|
||||
4, "timeliness_fallback_invalid")
|
||||
rungs = timeliness_rungs(e["fallback"])
|
||||
for f in rungs:
|
||||
need(1 <= f["value_ppm"] <= FULL_VALUE and f["content_hash"] != e["content_hash"]
|
||||
and (f["group"] != subject or f["leaf_index"] != e["leaf_index"]),
|
||||
4, "timeliness_fallback_invalid")
|
||||
# Rule 3.4: an array holds 2 to 8 rungs, strictly falling in value, naming no object twice. One
|
||||
# rung's one form is the object, so an array of one is refused.
|
||||
if isinstance(e["fallback"], list):
|
||||
hashes = {f["content_hash"] for f in rungs}
|
||||
places = {(f["group"], f["leaf_index"]) for f in rungs}
|
||||
need(2 <= len(rungs) <= MOST_RUNGS and all(b["value_ppm"] < a["value_ppm"] for a, b in zip(rungs, rungs[1:]))
|
||||
and len(hashes) == len(rungs) and len(places) == len(rungs),
|
||||
4, "timeliness_ladder_invalid")
|
||||
|
||||
|
||||
def class_rules(c):
|
||||
|
|
@ -2489,8 +2507,8 @@ def timeliness_claims(path):
|
|||
"""§3.1 steps 1-4 over the claims of the timeliness corpus (live-conformance/timeliness/vectors.json,
|
||||
timeliness §10): a claim a verdict or a plan states refused at one of those steps is refused there
|
||||
with the stated reason, and every other `timeliness`, `transmission` or `schedule` claim passes
|
||||
them; and each claim case gets the outcome it states. Steps 5 and 6 are the timeliness
|
||||
verifier's."""
|
||||
them, as does every claim of a run; and each claim case gets the outcome it states. Steps 5 and 6
|
||||
are the timeliness verifier's."""
|
||||
with open(path, encoding="utf-8") as f:
|
||||
doc = json.load(f)
|
||||
fail = n = 0
|
||||
|
|
@ -2501,6 +2519,9 @@ def timeliness_claims(path):
|
|||
for wire, stated in zip(case["claims"], case["outcome"]["claims"]):
|
||||
m = re.fullmatch(r"reject step=(\d) reason=(\S+)", stated)
|
||||
checks.append((case["name"], wire, (int(m[1]), m[2]) if m and int(m[1]) <= 4 else None, True))
|
||||
for case in doc["runs"]:
|
||||
# §7.5: a run's claims are its plans', and every one the corpus gives passes steps 1 to 4.
|
||||
checks.extend((case["name"], wire, None, True) for wire in case["claims"])
|
||||
for case in doc["claims"]:
|
||||
m = re.fullmatch(r"reject step=(\d) reason=(\S+)", case["outcome"])
|
||||
checks.append((case["name"], case["claim"], (int(m[1]), m[2]) if m else None, False))
|
||||
|
|
|
|||
|
|
@ -1390,13 +1390,13 @@ and envelopes at the positions it names.
|
|||
|
||||
A publisher's statement of when each object of a group is of use: the instant it is useful from,
|
||||
the deadline by which a receiver must hold it for its full value, how that value falls after the
|
||||
deadline, and the rendition a receiver presents in its place when it does not hold it in time. It
|
||||
attests a value over time, not a transmission or a delivery, and sits beside the group receipt,
|
||||
deadline, and the renditions a receiver presents in its place when it does not hold it in time, a
|
||||
ladder in order of falling value. It attests a value over time, not a transmission or a delivery, and sits beside the group receipt,
|
||||
whose bytes it leaves unchanged, one statement for every bearer. [timeliness](timeliness.md)
|
||||
defines it: its body's members are §4.1 there (`objects` and `attests`), read at §3.1 step 2; its
|
||||
class rules are §4.2 there, at step 4; and its context is §4.3 there, at step 6: the group
|
||||
receipt, whose link hash is its `subject` and whose key signs it, the object receipts and
|
||||
envelopes at the positions it names, and each fallback's group receipt, object receipt and
|
||||
envelopes at the positions it names, and each rung's group receipt, object receipt and
|
||||
envelope.
|
||||
|
||||
#### 2.4.18. `class: "cache-transmission"`
|
||||
|
|
@ -2196,7 +2196,7 @@ order listed.
|
|||
`attests`, a transmission's energy figure and a reconciliation's bitmap; in a
|
||||
`schedule` claim, its `attests`, the order of its entries, and each entry's code
|
||||
parameters, symbols and passes; in a `timeliness` claim, its `attests`, the order
|
||||
of its entries, and each entry's window, decay and fallback; in a
|
||||
of its entries, and each entry's window, decay, fallback and ladder; in a
|
||||
`cache-transmission` claim, its `attests`, its `bearer`, its bitmap and its energy
|
||||
figure;
|
||||
and, in a `consumption` claim, the order of its
|
||||
|
|
@ -2257,7 +2257,7 @@ order listed.
|
|||
the object receipts and envelopes at the positions it names, as
|
||||
[carousel](carousel.md) §4.3 orders them (§2.4.16); and a `timeliness` claim against the
|
||||
group receipt its subject names, the object receipts and envelopes at the positions it
|
||||
names, and each fallback's group receipt, object receipt and envelope that the verifier
|
||||
names, and each rung's group receipt, object receipt and envelope that the verifier
|
||||
holds, as [timeliness](timeliness.md) §4.3 orders them (§2.4.17); and a `cache-transmission`
|
||||
claim against the group receipt its subject names and the object receipts at the positions it
|
||||
names, each verified by inclusion at least, as [cache-authority](cache-authority.md) §5.3
|
||||
|
|
@ -2370,7 +2370,8 @@ and never folds it in: confirmation changes the outcome of no step here.
|
|||
| 4 | `timeliness_objects_malformed` | it has no entry, or its entries' `leaf_index` values do not strictly increase |
|
||||
| 4 | `timeliness_window_invalid` | an entry's `from_ms` is after its `by_ms`, or its `by_ms` is not below the last instant, 9999-12-31T23:59:59.999Z |
|
||||
| 4 | `timeliness_decay_invalid` | an entry's decay is not well formed: not 1 to 16 points; instants not strictly increasing from after `by_ms` to at most the last instant; a value at or above 1,000,000, above the one before it, a last one other than 0 or another one of 0; or a point but the last on the line through its neighbours |
|
||||
| 4 | `timeliness_fallback_invalid` | an entry's fallback is valued at 0 or above 1,000,000, has the entry's content hash, or names the entry's own position |
|
||||
| 4 | `timeliness_fallback_invalid` | a rung of an entry's fallback is valued at 0 or above 1,000,000, has the entry's content hash, or names the entry's own position |
|
||||
| 4 | `timeliness_ladder_invalid` | an entry's fallback is an array of other than 2 to 8 rungs, whose values do not strictly decrease, or two of whose rungs have one content hash or one position |
|
||||
| 4 | `cache_transmission_not_transmitted` | a `cache-transmission` claim's `attests` is not `transmitted` ([cache-authority](cache-authority.md) §5.2) |
|
||||
| 4 | `cache_transmission_on_bidirectional` | its `bearer` is `bidirectional` |
|
||||
| 4 | `cache_transmission_bitmap_malformed` | its `leaf_count` is 0, its `sent` is not ⌈`leaf_count` / 8⌉ bytes, sets a bit at or past `leaf_count`, or sets none |
|
||||
|
|
@ -2431,9 +2432,9 @@ and never folds it in: confirmation changes the outcome of no step here.
|
|||
| 6 | `timeliness_leaf_out_of_range` | an entry's `leaf_index` is not less than the group's `leaf_count` |
|
||||
| 6 | `timeliness_before_group` | an entry's `from_ms` is before the group's `group_start`, whole seconds compared |
|
||||
| 6 | `timeliness_content_mismatch` | an entry's `content_hash` is not that of the object receipt the verifier holds at its position |
|
||||
| 6 | `timeliness_on_layer` | an envelope the verifier holds whose content hash is an entry's or a fallback's is a `WAI1` of a refinement-only capability |
|
||||
| 6 | `timeliness_fallback_mismatch` | a fallback's `leaf_index` is not less than the leaf count of its group, where the verifier holds that group receipt, or its `content_hash` is not that of the object receipt the verifier holds at its position |
|
||||
| 6 | `timeliness_fallback_not_derived` | an envelope the verifier holds whose content hash is a fallback's is not a `WAI1` whose `derived_from` names the entry's `content_hash` as its `motif` |
|
||||
| 6 | `timeliness_on_layer` | an envelope the verifier holds whose content hash is an entry's or a rung's is a `WAI1` of a refinement-only capability |
|
||||
| 6 | `timeliness_fallback_mismatch` | a rung's `leaf_index` is not less than the leaf count of its group, where the verifier holds that group receipt, or its `content_hash` is not that of the object receipt the verifier holds at its position |
|
||||
| 6 | `timeliness_fallback_not_derived` | an envelope the verifier holds whose content hash is a rung's is not a `WAI1` whose `derived_from` names the entry's `content_hash` as its `motif` |
|
||||
| 6 | `cache_transmission_subject_mismatch` | a `cache-transmission` claim's `subject` is not the link hash of the group receipt it is checked against ([cache-authority](cache-authority.md) §5.3) |
|
||||
| 6 | `cache_transmission_by_publisher` | it is signed by the group's key |
|
||||
| 6 | `cache_transmission_leaf_count_mismatch` | its `leaf_count` is not the group's |
|
||||
|
|
|
|||
|
|
@ -3,18 +3,21 @@
|
|||
> Status: Draft. Registers the record `wai.delivery.timeliness`: a publisher's signed statement, for
|
||||
> each object of a group, of when the object is of use: the instant it is useful from, the deadline
|
||||
> by which a receiver must hold it for its full value, how that value falls after the deadline, as a
|
||||
> stated function in integers, and the rendition a receiver presents in its place when it does not
|
||||
> hold it by the deadline. It is a jwp-receipts claim of class **timeliness** (jwp-receipts §2.4.17,
|
||||
> §4 here), signed by the group's key beside the group receipt, one statement for every bearer. From
|
||||
> it and from a holder's signed [holdings](holdings.md) record, the signed evidence WAI has that a
|
||||
> party held an object at an instant, a verifier judges each object for each holder: on time,
|
||||
> replaced by its fallback, held only after its deadline and at what value at least, or not shown
|
||||
> (§6). An asker plans, from an instant it states, which of its sources, in its own order, delivers
|
||||
> each object by its deadline: a pass of a [carousel](carousel.md), a neighbour, or the origin (§7).
|
||||
> stated function in integers, and the renditions a receiver presents in its place when it does not
|
||||
> hold it by the deadline, a ladder in order of falling value. It is a jwp-receipts claim of class
|
||||
> **timeliness** (jwp-receipts §2.4.17, §4 here), signed by the group's key beside the group
|
||||
> receipt, one statement for every bearer. From it and from a holder's signed
|
||||
> [holdings](holdings.md) record, the signed evidence WAI has that a party held an object at an
|
||||
> instant, a verifier judges each object for each holder: on time, replaced by a rung of its
|
||||
> fallback, held only after its deadline and at what value at least, or not shown (§6). An asker
|
||||
> plans, from an instant it states, which of its sources, in its own order, delivers each object by
|
||||
> its deadline: a pass of a [carousel](carousel.md), a neighbour, or the origin (§7); and a sink runs
|
||||
> plans against its own clock, deciding what it fetches, presents and states, and when (§7.5).
|
||||
> WAI defines no transport: the deadline is mapped onto the delivery timeouts, request keys,
|
||||
> carousel and window parameters and bundle lifetimes the bearers already define (§8). The reference
|
||||
> implementation is the `timeliness` module of `wai-rs` (feature `timeliness`), with the class's
|
||||
> body in `claim_receipt::timeliness`; the corpus is
|
||||
> body in `claim_receipt::timeliness` and the sink's run of a plan in `timeliness::scheduler`; the
|
||||
> corpus is
|
||||
> [`live-conformance/timeliness/`](../live-conformance/timeliness/), and its second implementation
|
||||
> is `live-conformance/timeliness_verify.py`.
|
||||
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
|
||||
|
|
@ -31,8 +34,8 @@
|
|||
|
||||
In scope: the timeliness claim (§4); an object's value at an instant, and the rule by which a sink
|
||||
that does not hold it by its deadline is graded (§5); judging, from holdings records, whether each
|
||||
object was held by its deadline (§6); planning which source delivers it by its deadline (§7); and
|
||||
the values a carriage derives from it (§8).
|
||||
object was held by its deadline (§6); planning which source delivers it by its deadline, and a
|
||||
sink's run of its plans against its own clock (§7); and the values a carriage derives from it (§8).
|
||||
|
||||
Out of scope: a transport; how a sink in fact presents, which is its own and is recorded in its
|
||||
`reconstruct-session` claim (jwp-receipts §2.4.12), but for the switch reason of §8.6; who may sign
|
||||
|
|
@ -58,7 +61,9 @@ It names no payload format and is not dispatchable.
|
|||
(9999-12-31T23:59:59.999Z), as carousel §4.2 bounds a pass.
|
||||
- **Value**: an integer from 0 to 1,000,000, parts per million of an object's full value.
|
||||
- **Entry**, **primary**: an entry of a timeliness claim (§4.1), and the object it names.
|
||||
**Fallback**: the object an entry names to present in the primary's place. **End**: §5.2.
|
||||
**Fallback**, **ladder**, **rung**: an entry's fallback is its ladder, the objects it names to
|
||||
present in the primary's place, each a rung, in order of falling value (§4.1); a fallback of one
|
||||
rung is that rung. **End**: §5.2.
|
||||
- **Holder**: a node named by its Ed25519 key that signs holdings records
|
||||
([holdings](holdings.md) §2).
|
||||
- **Held-at statement**: a key, a group's link hash, a position and an instant: the key's signed
|
||||
|
|
@ -114,10 +119,16 @@ Each entry of `objects` has exactly these members:
|
|||
| `from_ms` | integer | The instant from which the object is of use. Holding it earlier is no fault and no gain. |
|
||||
| `by_ms` | integer | The deadline: the last instant at which a receiver that first holds the object has it at its full value. |
|
||||
| `decay` | array | 1 to 16 points, each exactly `{ "at_ms": integer, "value_ppm": integer }`: the object's value at instants after `by_ms` (§5). |
|
||||
| `fallback` | object or `null` | The object to present in the primary's place when the primary is not held by `by_ms` (§5.3): exactly `{ "group": 32 B hex, "leaf_index": integer, "content_hash": 32 B hex, "value_ppm": integer }`, the link hash of its group receipt, its position in that group, its content hash, and the value of presenting it in the primary's place. `null` when there is none. |
|
||||
| `fallback` | object, array or `null` | The ladder: the objects to present in the primary's place when the primary is not held by `by_ms` (§5.3), each a **rung**, exactly `{ "group": 32 B hex, "leaf_index": integer, "content_hash": 32 B hex, "value_ppm": integer }`, the link hash of its group receipt, its position in that group, its content hash, and the value of presenting it in the primary's place. One rung is written as that object; two to eight as an array of them, in order of falling `value_ppm` (§4.2 rule 3.4). `null` when there is none. |
|
||||
|
||||
Every integer is one of jwp-receipts §2.4.3, read at §3.1 step 2 with the members of each point and
|
||||
of `fallback`, in the tables' order (`field_type`).
|
||||
of each rung, in the tables' order (`field_type`); an element of the array that is not an object is
|
||||
`field_type` too.
|
||||
|
||||
A ladder of one rung has one form, the object: an array of one is refused (§4.2 rule 3.4). So an
|
||||
entry with one fallback object, the form before ladders, is a ladder of one rung, read, judged and
|
||||
planned as it was, and two entries that state one ladder are written alike (§6.2 step 3). The rung
|
||||
a ladder's position gives is its **index**, from 0; a ladder of one rung has rung 0.
|
||||
|
||||
An example body (wire form has no whitespace; jwp-receipts §2.4.3):
|
||||
|
||||
|
|
@ -127,6 +138,11 @@ An example body (wire form has no whitespace; jwp-receipts §2.4.3):
|
|||
"group":"<64 hex>","leaf_index":0,"value_ppm":400000},"from_ms":1793664000000,
|
||||
"leaf_index":0}]}
|
||||
|
||||
The same entry with a ladder of two rungs, the higher valued first:
|
||||
|
||||
"fallback":[{"content_hash":"<64 hex>","group":"<64 hex>","leaf_index":0,"value_ppm":700000},
|
||||
{"content_hash":"<64 hex>","group":"<64 hex>","leaf_index":0,"value_ppm":400000}]
|
||||
|
||||
### 4.2 Rules
|
||||
|
||||
Checked at jwp-receipts §3.1 step 4, in this order:
|
||||
|
|
@ -146,16 +162,21 @@ Checked at jwp-receipts §3.1 step 4, in this order:
|
|||
to *n* − 1, (*v*<sub>k−1</sub> − *v*<sub>k</sub>) · (*t*<sub>k+1</sub> − *t*<sub>k</sub>)
|
||||
differs from (*v*<sub>k</sub> − *v*<sub>k+1</sub>) · (*t*<sub>k</sub> − *t*<sub>k−1</sub>),
|
||||
computed without overflow (the products reach 2<sup>68</sup>);
|
||||
3. its fallback is well formed (`timeliness_fallback_invalid`): `null`, or a `value_ppm` from 1
|
||||
to 1,000,000, a `content_hash` other than the entry's, and a position other than the
|
||||
entry's own (a `group` other than the claim's `subject`, or a `leaf_index` other than the
|
||||
entry's).
|
||||
3. each rung of its fallback is well formed (`timeliness_fallback_invalid`), in the ladder's
|
||||
order: a `value_ppm` from 1 to 1,000,000, a `content_hash` other than the entry's, and a
|
||||
position other than the entry's own (a `group` other than the claim's `subject`, or a
|
||||
`leaf_index` other than the entry's);
|
||||
4. its ladder is well formed (`timeliness_ladder_invalid`): where `fallback` is an array, it has
|
||||
2 to 8 rungs, their `value_ppm` strictly decrease, and no two of them have one
|
||||
`content_hash`, or one `group` and one `leaf_index`.
|
||||
|
||||
So a stated line has one form. `by_ms` is the last instant of full value, since no point after it
|
||||
holds full value; nothing is implied, since a hard deadline is the one point (`by_ms` + 1, 0); a
|
||||
step is a fall over one millisecond; and every point is a change of slope, since none but the last
|
||||
lies on the line its neighbours give. Two entries that state different lines are two statements,
|
||||
even where their floored values (§5.1) agree at every instant.
|
||||
even where their floored values (§5.1) agree at every instant. A ladder states one order too: each
|
||||
rung is worth less than the one before it, so the first rung a sink holds is the most it can
|
||||
present in the primary's place (§5.3), and no object is valued twice.
|
||||
|
||||
### 4.3 Against the group
|
||||
|
||||
|
|
@ -175,23 +196,24 @@ holds, each known by its content hash. In order:
|
|||
carousel §4.3 step 4 reads it. Nothing is of use before it exists;
|
||||
5. for each entry whose object receipt the verifier holds, the receipt's `content_hash` is the
|
||||
entry's (`timeliness_content_mismatch`);
|
||||
6. no entry and no fallback is a layer (`timeliness_on_layer`): an envelope the verifier holds
|
||||
whose content hash is an entry's or a fallback's is not a `WAI1` envelope whose capability is
|
||||
6. no entry and no rung is a layer (`timeliness_on_layer`): an envelope the verifier holds whose
|
||||
content hash is an entry's or a rung's is not a `WAI1` envelope whose capability is
|
||||
refinement-only (SPEC §5, *Refinement capabilities*; staged-delivery §3);
|
||||
7. each fallback is where it says (`timeliness_fallback_mismatch`): where the verifier holds the
|
||||
group receipt whose link hash is the fallback's `group`, the fallback's `leaf_index` is less
|
||||
than that group's `leaf_count`; and where it holds the object receipt at that position, the
|
||||
receipt's `content_hash` is the fallback's;
|
||||
8. each fallback is a rendition of its primary (`timeliness_fallback_not_derived`): an envelope the
|
||||
verifier holds whose content hash is a fallback's is a `WAI1` envelope whose manifest carries
|
||||
7. each rung is where it says (`timeliness_fallback_mismatch`): where the verifier holds the group
|
||||
receipt whose link hash is the rung's `group`, the rung's `leaf_index` is less than that
|
||||
group's `leaf_count`; and where it holds the object receipt at that position, the receipt's
|
||||
`content_hash` is the rung's;
|
||||
8. each rung is a rendition of its primary (`timeliness_fallback_not_derived`): an envelope the
|
||||
verifier holds whose content hash is a rung's is a `WAI1` envelope whose manifest carries
|
||||
`derived_from` (SPEC §7.1, *Derived renditions*) with `motif` the entry's `content_hash`, as
|
||||
packaging writes on every floor object ([packaging](packaging.md) §7, its paragraph 7.2). A
|
||||
`WAI2` envelope is refused here: it offers renditions a sink selects among (SPEC §7.1), so it
|
||||
names no one rendition the publisher has valued.
|
||||
names no one rendition the publisher has valued. Every rung of a ladder derives from the
|
||||
primary itself, not from the rung before it.
|
||||
|
||||
What the verifier does not hold is not checked: a claim checked without its group stops at step 5
|
||||
and is not counted by §6; an entry whose receipt, envelope or fallback group it does not hold
|
||||
passes the steps that need them, and the publisher's word stands there.
|
||||
and is not counted by §6; an entry whose receipt or envelope it does not hold, or a rung whose
|
||||
group it does not hold, passes the steps that need them, and the publisher's word stands there.
|
||||
|
||||
### 4.4 Authority
|
||||
|
||||
|
|
@ -225,17 +247,24 @@ end, never from an earlier zero.
|
|||
|
||||
### 5.3 The sink's rule
|
||||
|
||||
§6 judges, and §7 plans, for a sink that follows this rule; a sink **MAY** present otherwise, and
|
||||
its session claim records what it did. At `by_ms`:
|
||||
§6 judges, and §7 plans and runs, for a sink that follows this rule; a sink **MAY** present
|
||||
otherwise, and its session claim records what it did. At `by_ms`:
|
||||
|
||||
- a sink that holds the primary presents it, at full value;
|
||||
- one that does not, and holds the fallback, presents the fallback in the primary's place, at the
|
||||
fallback's `value_ppm`;
|
||||
- one that holds neither presents the primary when it holds it, at its value then (§5.1). A
|
||||
fallback first held after `by_ms` is not presented in the primary's place.
|
||||
- one that does not, and holds a rung, presents in the primary's place the first rung of the
|
||||
ladder that it holds, at that rung's `value_ppm`: the highest valued it holds (§4.2 rule 3.4). A
|
||||
lower rung it also holds is not presented;
|
||||
- one that holds neither the primary nor any rung has **passed** every rung, and states so at
|
||||
`by_ms` (`passed`, §7.5). No rung is presented in the primary's place from then on. It presents
|
||||
the primary when it first holds it, at its value then (§5.1), where that value is at least 1. At
|
||||
the primary's end (§5.2), where it has presented nothing, it abandons any request for the
|
||||
primary (§8.2) and states that the object ended unpresented (`ended`, §7.5). Its line is
|
||||
`after_deadline` or `not_shown` (§6.3), and it records no switch (§8.6).
|
||||
|
||||
The rule needs nothing the sink does not hold at `by_ms`. No sink knows at its deadline when a
|
||||
primary it lacks will arrive, so nothing is graded by what arrived after a fallback was due.
|
||||
A rung first held after `by_ms` is not presented in the primary's place, and a primary first held
|
||||
after a rung was presented does not replace it. The rule needs nothing the sink does not hold at
|
||||
`by_ms`. No sink knows at its deadline when a primary it lacks will arrive, so nothing is graded by
|
||||
what arrived after a rung was due.
|
||||
|
||||
### 5.4 Example
|
||||
|
||||
|
|
@ -244,6 +273,12 @@ With `by_ms` *D* and points (*D* + 2000, 600,000) and (*D* + 10,000, 0): the val
|
|||
at *D* + 9999; and 0 from *D* + 10,000, its end. A hard deadline is the one point (*D* + 1, 0):
|
||||
0 from *D* + 1, its end.
|
||||
|
||||
With that entry's ladder a rung at 700,000 and a rung at 400,000: a sink that holds both at *D*,
|
||||
and not the primary, presents the first, at 700,000; one that holds only the second presents it,
|
||||
at 400,000; one that holds the first only from *D* + 1 and the second only from *D* + 500 has
|
||||
passed both at *D*, and presents a primary it first holds at *D* + 1000 at 800,000, or nothing
|
||||
when no primary comes before *D* + 10,000.
|
||||
|
||||
## 6. Judging delivery
|
||||
|
||||
### 6.1 Inputs
|
||||
|
|
@ -298,26 +333,27 @@ which add acknowledgements and the reconciliations a bearer verdict counts.
|
|||
|
||||
For each intent that stands, in the order of the groups given and then of `leaf_index`, a verifier
|
||||
derives a **line** for each holder with a record that stands and has an entry for the intent's
|
||||
group or its fallback's group, in increasing order of key bytes. The lines cache-authority §9
|
||||
derives by this rule also have one for each key with a held-at statement of cache-authority §8.3's
|
||||
kinds for an object of the group. For a holder, *p* is the least instant of its held-at statements
|
||||
for the primary, and *f* the least for the fallback; either may have none. Its **outcome** and
|
||||
**value** are the first of these that holds:
|
||||
group or a rung's group, in increasing order of key bytes. The lines cache-authority §9 derives by
|
||||
this rule also have one for each key with a held-at statement of cache-authority §8.3's kinds for
|
||||
an object of the group. For a holder, *p* is the least instant of its held-at statements for the
|
||||
primary, and *f*<sub>k</sub> the least for rung *k* of the ladder; any may have none. Its
|
||||
**outcome** and **value** are the first of these that holds:
|
||||
|
||||
| outcome | when | value |
|
||||
|---|---|---|
|
||||
| `on_time` | *p* is at most `by_ms` | 1,000,000 |
|
||||
| `replaced` | *f* is at most `by_ms` | the fallback's `value_ppm` |
|
||||
| `replaced` | some *f*<sub>k</sub> is at most `by_ms` | the `value_ppm` of the first such rung in the ladder's order |
|
||||
| `after_deadline` | *p* is after `by_ms` | the primary's value at *p* (§5.1) |
|
||||
| `not_shown` | §6 only: none of the above | 0 |
|
||||
| `no_source` | §7 only: §7.3 rule 4 | 0 |
|
||||
|
||||
A line reports its holder, *p* and *f* (each `null` where none), its outcome and its value. Each
|
||||
instant is the latest the holder's word allows for the object's arrival, and §5.3 grades what a
|
||||
sink holding exactly what the evidence shows would present, so every value is the least the
|
||||
evidence shows. `after_deadline` says the holder held the object by *p*, not that it arrived after
|
||||
`by_ms`; `not_shown` says only that this evidence shows neither object held in time, nor the
|
||||
primary at all.
|
||||
A line reports its holder, *p* and *f* (each `null` where none), its outcome and its value: *f* is
|
||||
*f*<sub>0</sub> for a ladder of one rung, and for a ladder of two or more the list of every
|
||||
*f*<sub>k</sub>, in the ladder's order. Each instant is the latest the holder's word allows for
|
||||
the object's arrival, and §5.3 grades what a sink holding exactly what the evidence shows would
|
||||
present, so every value is the least the evidence shows. `after_deadline` says the holder held the
|
||||
object by *p*, not that it arrived after `by_ms`; `not_shown` says only that this evidence shows
|
||||
neither the primary nor a rung held in time, nor the primary at all.
|
||||
|
||||
A publisher's promise to a receiver within a loss bound that hears every pass is carousel §6.4's
|
||||
statement with *t* the entry's `by_ms`; to one that tunes in at an instant, a plan (§7). Neither
|
||||
|
|
@ -360,7 +396,7 @@ plan says what follows from it, not that it will hold.
|
|||
bearer-receipts §4.3 and a schedule by carousel §4.3; then the declarations (bearer-receipts §6.2
|
||||
step 4) and the schedules (carousel §6.2 step 4). A schedule that stands there stands here. For a
|
||||
wanted object whose intent stands, from the instant *s* (*t*, or its entry's `from_ms`), each
|
||||
source's **arrival** of the primary, and of the fallback where there is one, is:
|
||||
source's **arrival** of the primary, and of each rung of the ladder, is:
|
||||
|
||||
- **bearer instance**: where a schedule of the object's group stands on it with an entry for the
|
||||
object, the `complete_ms` of carousel §7's plan from *s*, at *L* and *h*, holding nothing of it;
|
||||
|
|
@ -376,17 +412,20 @@ otherwise the first of these that holds, with the outcomes of §6.3:
|
|||
|
||||
1. `on_time`: the first source, in the asker's order, whose arrival of the primary is at most
|
||||
`by_ms`;
|
||||
2. `replaced`: when the fallback's `value_ppm` is at least the greatest value (§5.1) of any
|
||||
arrival of the primary, the first source whose arrival of the fallback is at most `by_ms`;
|
||||
2. `replaced`: the first rung, in the ladder's order, whose `value_ppm` is at least the greatest
|
||||
value (§5.1) of any arrival of the primary and whose arrival from some source is at most
|
||||
`by_ms`: that rung, from the first such source in the asker's order;
|
||||
3. `after_deadline`: the source whose arrival of the primary has the greatest value, the first in
|
||||
the asker's order among equals, when that value is at least 1;
|
||||
4. `no_source`.
|
||||
|
||||
A plan reports for each object its outcome, which object (`primary`, `fallback` or `null`), the
|
||||
source by its index in the asker's list, the arrival and the value. A plan whose choice is a late
|
||||
primary fetches no fallback: a sink that holds the fallback by `by_ms` presents it (§5.3). An asker
|
||||
that has no path back lists no holder and no origin; one whose plan for an object is `no_source`
|
||||
has no source that reaches it with any value.
|
||||
rung by its index (`null` unless the object is `fallback`), the source by its index in the asker's
|
||||
list, the arrival and the value. A plan whose choice is a late primary fetches no rung, and one
|
||||
whose choice is a rung fetches no other: no higher rung arrives from any source by `by_ms`, a lower
|
||||
one is worth less, and a sink that holds any rung by `by_ms` presents the first it holds (§5.3). An
|
||||
asker that has no path back lists no holder and no origin; one whose plan for an object is
|
||||
`no_source` has no source that reaches it with any value.
|
||||
|
||||
### 7.4 The publisher's check
|
||||
|
||||
|
|
@ -396,6 +435,95 @@ for every object is the promise. `after_deadline` names the objects for which su
|
|||
not served in time, though one that heard every pass may be (carousel §6.4, at *t* the entry's
|
||||
`by_ms`).
|
||||
|
||||
### 7.5 Running a plan
|
||||
|
||||
A sink **runs** its plans: from them and from its own clock it decides what it fetches, what it
|
||||
presents in each object's place and when, and what it states. A run is a function of its inputs
|
||||
alone, so a verifier given a sink's inputs derives its decisions, and two sinks given the same
|
||||
inputs decide the same things at the same instants.
|
||||
|
||||
#### 7.5.1 Inputs
|
||||
|
||||
§7.1's groups, claims and records, and *L* and *h* where given, shared by every plan of the run;
|
||||
the run's **plans**, in order, each an instant *t* with sources and wanted objects as §7.1 gives
|
||||
them; and the sink's **arrivals**, each a group's link hash, a position and the instant from which
|
||||
the sink first holds that object, on its own clock. An arrival is the sink's own observation of
|
||||
whatever reached it: an object it fetched, or one a bearer it listens to brought.
|
||||
|
||||
These are checked first, in this order, and refuse the run:
|
||||
|
||||
1. the plans (`plans_invalid`): there is none; one is planned from `from_ms` rather than from an
|
||||
instant, since a sink runs from its clock; their instants do not strictly increase; or one
|
||||
wants an object twice;
|
||||
2. each plan in order, by §7.1, refused with its reason;
|
||||
3. the arrivals (`arrivals_invalid`): an instant is above the last instant; the instants, in the
|
||||
order given, decrease; or two arrivals name one object, since an arrival is the first holding.
|
||||
|
||||
Each plan is then derived by §7.2 and §7.3 from the shared inputs with its own *t*, sources and
|
||||
wanted objects; a group that fails refuses the run as it refuses a plan. The first plan is **in
|
||||
force** from its *t*, and each later one **replaces** the one before it at its own *t*, wholly: an
|
||||
object only an earlier plan wants is no longer run.
|
||||
|
||||
#### 7.5.2 State
|
||||
|
||||
For each object, a group's link hash and a position, the run keeps whether it is **settled**
|
||||
(presented, or ended), whether it has **passed** (§5.3), and its **fetch**, or none: the object it
|
||||
takes (the primary or a rung), its source, and the plan it serves, by that plan's index and the
|
||||
source's index in that plan's list. A fetch is outstanding from its decision until the sink holds
|
||||
the object it takes, or until it is abandoned. Two sources are **the same** where both name one
|
||||
bearer instance or one holder's key, or both are the origin, whatever their `fetch_ms`. The objects
|
||||
**run** are those the plan in force wants whose choice is not refused (`deadline_not_stated`), in
|
||||
its wanted order. The sink **holds** an object from its arrival's instant, and holds it **by** that
|
||||
instant and any later one.
|
||||
|
||||
#### 7.5.3 Decisions
|
||||
|
||||
The run reads, in increasing order, every instant that is a plan's *t*, an arrival's, or the
|
||||
`by_ms` or the end (§5.2) of an object a plan wants with a choice; at each, *n*, it takes these
|
||||
steps in order, each over the objects run in their wanted order. At no other instant does it decide
|
||||
anything.
|
||||
|
||||
1. **Arrivals.** The sink holds from *n* each object that arrives at *n*, and a fetch of it is no
|
||||
longer outstanding.
|
||||
2. **Plans.** Where a plan's *t* is *n*, it replaces the one in force. First, for each object run
|
||||
under the earlier plan that is not settled and has a fetch outstanding: `abandon` it, unless
|
||||
the new plan **continues** it, its choice for the object taking the same object from the same
|
||||
source; a fetch continued serves the new plan from then on. Then, for each object now run that
|
||||
is not settled and whose choice takes an object from a source: `fetch` it, unless the sink
|
||||
holds that object or the new plan continues the fetch. A choice of `no_source` fetches
|
||||
nothing.
|
||||
3. **Deadlines.** For each object run, neither settled nor passed, whose `by_ms` is at most *n*,
|
||||
by §5.3 with what the sink holds by `by_ms`: where it holds the primary, `present` it at
|
||||
1,000,000; else, where it holds a rung, `present` the first of the ladder that it holds, at
|
||||
that rung's `value_ppm`; else `passed`. A `present` settles the object, then abandons its fetch
|
||||
where one is outstanding (`abandon`); `passed` abandons its fetch where one is outstanding and
|
||||
takes a rung, since no rung can be presented in the primary's place from then on.
|
||||
4. **Late primaries.** For each object run that has passed and is not settled, where the sink holds
|
||||
the primary and the primary's value (§5.1) at its arrival's instant is at least 1: `present`
|
||||
the primary at that value, which settles the object.
|
||||
5. **Ends.** For each object run that has passed and is not settled, whose end is at most *n*:
|
||||
`abandon` its fetch where one is outstanding, then `ended`, which settles the object.
|
||||
|
||||
A plan given after an object's `by_ms` meets that object at step 3 at the plan's own *t*, by what
|
||||
the sink held by `by_ms`, as §6 would grade it; one given at or after its end meets steps 3 to 5 at
|
||||
that instant.
|
||||
|
||||
Each decision reports its instant *n*; a plan's index: for `abandon` the plan the fetch served,
|
||||
for every other action the plan in force; its action; the object's group and position; for `fetch`
|
||||
and `abandon`, the object the fetch takes (`primary` or `fallback`, with the rung's index or
|
||||
`null`) and its source's index in that plan's list; and for `present`, the object presented, with
|
||||
the rung's index or `null`, and its value. A member that does not apply is `null`. A run reports
|
||||
each plan's choices (§7.3), then its decisions in the order it takes them.
|
||||
|
||||
#### 7.5.4 What a run states
|
||||
|
||||
Each decision is the sink's own statement of its session. None is signed here, and none is graded:
|
||||
§6 grades from signed held-at statements alone. A carriage reports them as it reports any request:
|
||||
`fetch` is delivery-telemetry's `request`, `abandon` its `abandon` (delivery-telemetry §2; §8.2
|
||||
here, at the end), and `present` its `present`; a rung presented on another path is also §8.6's
|
||||
switch. `passed` states, at `by_ms`, that no rung will be presented in the primary's place;
|
||||
`ended` states, at the end, that nothing was presented for the object. Neither is a switch.
|
||||
|
||||
## 8. Mappings
|
||||
|
||||
Each value below is derived from an entry that stands, for an object that is not a layer (§1). A
|
||||
|
|
@ -438,8 +566,8 @@ it holds.
|
|||
end, on its own clock as http-delivery §10.2 runs a deadline (a clock that steps back has not
|
||||
passed it): past its end the object is of no value. http-delivery §10.1 admits this abandonment
|
||||
of a base and no other; the fetch then ends as one never answered (`base_unanswered`). The
|
||||
client fetches the fallback where its plan (§7) chooses it. Request priority (RFC 9218 §4.1,
|
||||
§4.2; http-delivery §10.1) is unchanged.
|
||||
client fetches the rung its plan (§7) chooses, where it chooses one. Request priority (RFC 9218
|
||||
§4.1, §4.2; http-delivery §10.1) is unchanged.
|
||||
- **CMCD.** For an object no buffer level governs (a file, a creative, a still, an application
|
||||
package), a sink's `object.dl_ms`, which delivery-telemetry §3 maps to CMCD `dl` (CTA-5004-B
|
||||
§4.3, Table 1: "Deadline from the request time until the first sample of this Segment/Object
|
||||
|
|
@ -498,11 +626,11 @@ time is 0, an unknown clock (§4.2.6), derives no lifetime from the entry.
|
|||
|
||||
### 8.6 Presentation
|
||||
|
||||
A sink that presents a fallback in a primary's place under §5.3, and whose path (`path`,
|
||||
A sink that presents a rung in a primary's place under §5.3, and whose path (`path`,
|
||||
[delivery-telemetry](delivery-telemetry.md) §2, its paragraph 2.2) changes with it from `primary`
|
||||
to `fallback`, records the switch with the reason `not-held-by-deadline` (delivery-telemetry §5,
|
||||
its paragraph 5.1; jwp-receipts §2.4.12 `switches`). A fallback presented on the primary's path is
|
||||
no switch.
|
||||
its paragraph 5.1; jwp-receipts §2.4.12 `switches`). A rung presented on the primary's path is no
|
||||
switch, and neither is a primary presented after the sink passed every rung.
|
||||
|
||||
## 9. Reasons
|
||||
|
||||
|
|
@ -515,6 +643,7 @@ The codes this extension adds, each with where it is raised:
|
|||
| `timeliness_window_invalid` | §4.2 rule 3.1 |
|
||||
| `timeliness_decay_invalid` | §4.2 rule 3.2 |
|
||||
| `timeliness_fallback_invalid` | §4.2 rule 3.3 |
|
||||
| `timeliness_ladder_invalid` | §4.2 rule 3.4 |
|
||||
| `timeliness_subject_mismatch` | §4.3 step 1 |
|
||||
| `timeliness_signer_mismatch` | §4.3 step 2 |
|
||||
| `timeliness_leaf_out_of_range` | §4.3 step 3 |
|
||||
|
|
@ -528,13 +657,15 @@ The codes this extension adds, each with where it is raised:
|
|||
| `deadline_not_stated` | §6.4, §7.3 |
|
||||
| `instant_invalid` | §7.1 |
|
||||
| `sources_invalid` | §7.1 |
|
||||
| `plans_invalid` | §7.5.1 |
|
||||
| `arrivals_invalid` | §7.5.1 |
|
||||
|
||||
A verdict or plan also raises a group's or object's codes of jwp-receipts §3.3; a claim's codes of
|
||||
jwp-receipts §3.1 steps 1 to 5 and those of step 6 named in §4.3, and in a plan in bearer-receipts
|
||||
§4.3 and carousel §4.3; a record's codes of holdings §7; and a plan `loss_bound_invalid` and
|
||||
`overhead_invalid` (carousel §9) at §7.1, and `bearer_contradicted`, `bearer_not_declared`
|
||||
(bearer-receipts §10), `schedule_contradicted` and `schedule_on_bidirectional` (carousel §9) at
|
||||
§7.2.
|
||||
§7.2. A run raises a plan's codes at §7.5.1.
|
||||
|
||||
## 10. Conformance
|
||||
|
||||
|
|
@ -545,8 +676,10 @@ fixed seeds by `wai_timeliness_vectors gen` as `vectors.json`. It has:
|
|||
inverted one, a deadline one millisecond before the last instant and at it, a decay of one, of
|
||||
sixteen and of seventeen points and an empty one, a point at full value, a step, a plateau and a
|
||||
plateau with an interior point, three points on one line and three just off it, a last point at
|
||||
the last instant and past it, a fallback valued at 1 and at full value — and one negative control
|
||||
for each rule, for the order of the rules within an entry and between entries, and for a body's
|
||||
the last instant and past it, a fallback valued at 1 and at full value, a ladder of two and of
|
||||
eight rungs and one of nine, of one rung written as an array and an empty one, rungs a part per
|
||||
million apart, equal and rising — and one negative control for each rule, for each rung, for
|
||||
the order of the rules within an entry and between entries, and for a body's and a rung's
|
||||
members at step 2.
|
||||
- **Values**: §5.1 at the deadline, a millisecond after, on and between points, at and past the
|
||||
end, a value floored to 0 before the end, a hard deadline, and a span whose products pass
|
||||
|
|
@ -559,7 +692,11 @@ fixed seeds by `wai_timeliness_vectors gen` as `vectors.json`. It has:
|
|||
other, a record at the start of its group's second and one before it, a record misstating a leaf
|
||||
count and one whose signature fails; intents contradicting and repeating; a schedule given as
|
||||
evidence and refused; and one negative control for each step of §4.3, each beside the case that
|
||||
shows the step passing at its edge.
|
||||
shows the step passing at its edge. With a ladder: the first rung held, a lower rung alone, a
|
||||
lower rung in time before a higher one late, every rung late with the primary later and with no
|
||||
primary, the primary on time over held rungs, a holder of a lower rung's group alone, a ladder
|
||||
and a single fallback stating one object, and steps 6 to 8 refused at a rung other than the
|
||||
first.
|
||||
- **Plans**: on time in the asker's order before an earlier source, at the deadline and a
|
||||
millisecond after, a neighbour when the carousel is late, a holder's record expired and not yet
|
||||
standing, a holder asked when only another holder's record holds the object, a fallback chosen and
|
||||
|
|
@ -568,7 +705,21 @@ fixed seeds by `wai_timeliness_vectors gen` as `vectors.json`. It has:
|
|||
and order, an arrival only after the end, no source on a path with no return, the origin alone, a
|
||||
schedule with no declaration and one on a two-way instance, every pass heard from the first, the
|
||||
publisher's check from `from_ms` passing and failing, an object with no deadline, an instant at
|
||||
and past the last instant, and each rule of §7.1.
|
||||
and past the last instant, and each rule of §7.1. With a ladder: the first rung, both rungs in
|
||||
time, a lower rung when the first is late, no rung when the late primary is worth more, the
|
||||
first rung over a late primary worth less, a lower rung refused on value, the first source in
|
||||
order, the primary on time over rungs, and no rung in time.
|
||||
- **Runs** (§7.5): on time on the plan's own trace, over the origin and over a bearer; held at the
|
||||
deadline and a millisecond after; a late object, presented at its value then; a hard deadline
|
||||
missed; every rung passed, with the primary later and with none; a rung chosen and presented, a
|
||||
lower rung held in its place, both rungs held, a rung at the deadline and a millisecond after
|
||||
it; a primary in time over a rung chosen, and one after a rung presented; no source; an object
|
||||
with no deadline; several objects in their wanted order; a plan replaced mid-run that continues
|
||||
a fetch, changes its source, changes it to a rung, finds that rung held, finds no source,
|
||||
re-plans after the deadline, drops an object, wants a dropped one back, finds an object already
|
||||
presented or a rung already presented, and arrives at the deadline itself; a first plan after a
|
||||
deadline, with the primary held before it and after it and with a rung held before it, and one
|
||||
after an end; and each rule of §7.5.1, in its order.
|
||||
|
||||
A conforming verifier gives every case the outcome, the reason and everything else the corpus
|
||||
states. `wai_timeliness_vectors verify` and the test `tests/timeliness.rs` check that the file
|
||||
|
|
@ -604,5 +755,7 @@ its own checks in turn and requires the corpus to catch the weakened verifier.
|
|||
bearer sends nothing, and a unidirectional bearer's property that the source learns nothing of
|
||||
its receivers ([unidirectional-delivery](unidirectional-delivery.md) §2) is kept. A record used
|
||||
as evidence discloses what holdings §9 says it does.
|
||||
- **Bounded work.** An entry has at most 16 points; a value or a rule is a few 128-bit products; a
|
||||
verdict is linear in entries times holders, and a plan in wanted objects times sources.
|
||||
- **Bounded work.** An entry has at most 16 points and 8 rungs; a value or a rule is a few 128-bit
|
||||
products; a verdict is linear in entries times holders times rungs, a plan in wanted objects
|
||||
times sources times rungs, and a run in its plans times their plans' work, with its instants
|
||||
times the objects run.
|
||||
|
|
|
|||
|
|
@ -710,10 +710,10 @@ def tally(case):
|
|||
|
||||
def lines(case):
|
||||
"""§9: timeliness §6.3's lines for object `index`. Records give held-at statements as
|
||||
timeliness §6 reads them, for the primary and its fallback whatever envelopes are held, with a
|
||||
line for each holder with a record that stands and has an entry for either group;
|
||||
acknowledgements and counted reconciliations give them for each object of the group held
|
||||
`delivered-verified`, the primary and a fallback in its group among them, with a line for each
|
||||
timeliness §6 reads them, for the primary and each rung of its fallback whatever envelopes are
|
||||
held, with a line for each holder with a record that stands and has an entry for any of those
|
||||
groups; acknowledgements and counted reconciliations give them for each object of the group
|
||||
held `delivered-verified`, the primary and a rung in its group among them, with a line for each
|
||||
key with one."""
|
||||
try:
|
||||
v = Verdict(case, count_only_of(case))
|
||||
|
|
@ -724,7 +724,7 @@ def lines(case):
|
|||
if got is None:
|
||||
return "reject reason=deadline_not_stated"
|
||||
entry = got[0]
|
||||
fb = entry["fallback"]
|
||||
rungs = TV.rungs(entry)
|
||||
acks = acknowledgements(case["acknowledgements"], g)
|
||||
rlabels, kept = TV.records_step(case["records"], [g])
|
||||
standing = [kept[x] for x in sorted(kept) if rlabels[x] == "stands"]
|
||||
|
|
@ -734,7 +734,7 @@ def lines(case):
|
|||
primary = TV.held_from_records(standing, g.link, i)
|
||||
if drops("line_records_verified") and verified(g, i) is None:
|
||||
primary = []
|
||||
fallback = [] if fb is None else TV.held_from_records(standing, fb["group"], fb["leaf_index"])
|
||||
fallback = [s for r in rungs for s in TV.held_from_records(standing, r["group"], r["leaf_index"])]
|
||||
for j in range(g.count):
|
||||
obj = verified(g, j)
|
||||
if obj is None:
|
||||
|
|
@ -745,7 +745,7 @@ def lines(case):
|
|||
holders.append(k)
|
||||
if j == i:
|
||||
primary += [(k, g.link, i, t) for t, _ in st]
|
||||
if fb is not None and fb["group"] == g.link and fb["leaf_index"] == j and not drops("line_fallback_acks"):
|
||||
if any(r["group"] == g.link and r["leaf_index"] == j for r in rungs) and not drops("line_fallback_acks"):
|
||||
fallback += [(k, g.link, j, t) for t, _ in st]
|
||||
return TV.lines(entry, holders, primary, fallback)
|
||||
|
||||
|
|
|
|||
|
|
@ -14,11 +14,13 @@ opaque test bytes written by the generator; no media and no input from outside t
|
|||
2026-11-03T00:00:00.500Z: **G**, three bases and a layer of `wai.image.int_refine` (object 3);
|
||||
**F**, a floor of each of G's objects, its manifest's `derived_from.motif` the content hash of G's
|
||||
object at the same position; **F_other**, F with object 2's floor derived from G's object 1; and
|
||||
**H**, two objects, the second a `WAI2`. G's entries, used throughout, have `from_ms`
|
||||
**H**, two objects, the second a `WAI2`; and **E**, another floor of each of G's objects, derived
|
||||
as F's are, the ladder's higher rung. G's entries, used throughout, have `from_ms`
|
||||
1,793,664,000,000 (2026-11-03T00:00:00Z) and `by_ms` a minute later: entry 0 falls to 600,000 two
|
||||
seconds after its deadline and to 0 ten seconds after, with F's object 0 at 400,000 as its
|
||||
fallback; entry 1 is a hard deadline with no fallback; entry 2 holds 700,000 from one to five
|
||||
seconds after, and 0 a second later, with F's object 2 at 1,000,000.
|
||||
seconds after, and 0 a second later, with F's object 2 at 1,000,000. The ladder cases and the runs
|
||||
give entry 0 the ladder E's object 0 at 700,000, then F's object 0 at 400,000.
|
||||
|
||||
- `claims` (§4.1, §4.2; jwp-receipts §3.1 steps 1 to 4) are each a claim's wire form, `claim`,
|
||||
and its `outcome`: `accept`, or `reject step=N reason=R`.
|
||||
|
|
@ -34,17 +36,26 @@ seconds after, and 0 a second later, with F's object 2 at 1,000,000.
|
|||
in the order of the groups given and then of `leaf_index`, each with its `group`, `leaf_index`,
|
||||
`intent` (`stands`, or `set-aside reason=timeliness_contradicted`), `end_ms` (`null` for one set
|
||||
aside) and `lines`, in increasing order of the holder's key bytes, each its `holder`, `held_ms`
|
||||
(*p*), `fallback_ms` (*f*), `outcome` and `value_ppm`; and `statement`, `{outcome, value_ppm}`,
|
||||
`reject reason=deadline_not_stated`, or `null`.
|
||||
(*p*), `fallback_ms` (*f*: one instant or `null`, and for a fallback written as an array the list
|
||||
of each rung's, in the ladder's order), `outcome` and `value_ppm`; and `statement`,
|
||||
`{outcome, value_ppm}`, `reject reason=deadline_not_stated`, or `null`.
|
||||
- `plans` (§7) are each a verdict's `groups`, `claims` (timeliness, transmission and schedule) and
|
||||
`records`; `loss_ppm` and `overhead`, both integers or both `null`; `at_ms`, an instant or the
|
||||
string `from_ms`; the `sources` in the asker's order, each exactly `{kind: "bearer", bearer_id}`,
|
||||
`{kind: "holder", holder, fetch_ms}` or `{kind: "origin", fetch_ms}`; and the `wanted` objects,
|
||||
each `{group, index}`. `outcome` is `refuse reason=R`, or gives `claims` (`accepted`,
|
||||
`declared`, `stands`, `set-aside reason=…`, `reject step=N reason=R`), `records` as a verdict's,
|
||||
and `choices`, one for each wanted object: `{outcome, object, source, arrival_ms, value_ppm}`,
|
||||
with `object` `primary`, `fallback` or `null` and `source` an index into `sources` or `null`, or
|
||||
and `choices`, one for each wanted object: `{outcome, object, rung, source, arrival_ms,
|
||||
value_ppm}`, with `object` `primary`, `fallback` or `null`, `rung` the rung's index in the ladder
|
||||
or `null` unless `object` is `fallback`, and `source` an index into `sources` or `null`, or
|
||||
`reject reason=deadline_not_stated`.
|
||||
- `runs` (§7.5) are each a plan's `groups`, `claims`, `records`, `loss_ppm` and `overhead`, shared
|
||||
by the run; its `plans`, each `{at_ms, sources, wanted}` as a plan case writes them; and the
|
||||
sink's `arrivals`, each `{group, index, at_ms}`, the trace of what it first held and when.
|
||||
`outcome` is `refuse reason=R`, or gives `plans`, each plan's `choices` as above, and `decisions`
|
||||
in the order taken, each `{at_ms, plan, action, group, index, object, rung, source, value_ppm}`:
|
||||
`action` is `fetch`, `abandon`, `present`, `passed` or `ended`, and a member that does not apply
|
||||
is `null`.
|
||||
|
||||
The plans' schedules are G's objects 0 to 2 on `urn:example:bearer:b1` and F's objects on `b1` and
|
||||
on `urn:example:bearer:b3`, each declared `unidirectional`: RaptorQ, one source block of eight
|
||||
|
|
@ -58,6 +69,7 @@ and bearer instances use the example URN namespace. Each case states the outcome
|
|||
have, declared in the generator, never copied from a verifier.
|
||||
|
||||
`wai_timeliness_vectors verify` and `tests/timeliness.rs` check that the file regenerates byte for
|
||||
byte and that `wai-rs` reaches every stated outcome. `../timeliness_verify.py` is the second
|
||||
implementation, written from the extension's text, which also mutates copies of the corpus and
|
||||
drops each of its own checks in turn and requires each to be caught.
|
||||
byte and that `wai-rs` reaches every stated outcome, the runs' through `timeliness::scheduler`.
|
||||
`../timeliness_verify.py` is the second implementation, written from the extension's text, which
|
||||
also mutates copies of the corpus and drops each of its own checks in turn and requires each to be
|
||||
caught.
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -13,15 +13,17 @@ record's form, signature and holdings from the holdings corpus's (`holdings_veri
|
|||
Everything else this extension adds is written here:
|
||||
|
||||
- §4.3: a timeliness claim against its group, the object receipts and envelopes held, and each
|
||||
fallback's group and object receipt;
|
||||
rung's group and object receipt;
|
||||
- §5: an entry's value at an instant, and its end;
|
||||
- §6: the intents that stand, the records that stand and the held-at statements they give
|
||||
(`held_from_records`, with §2's `before_group`), each intent's lines (`lines`), and a
|
||||
statement offered;
|
||||
- §7: the plan's inputs, each source's arrival, and the choice.
|
||||
- §7: the plan's inputs, each source's arrival, and the choice, the ladder's rung included;
|
||||
- §7.5: a sink's run of its plans against its clock (`run`).
|
||||
|
||||
For every case it requires the outcome the corpus states, field for field; that every reason
|
||||
of §9's table and every outcome of §6.3's appears in the corpus; then it mutates copies of the
|
||||
of §9's table, every outcome of §6.3's and every action §7.5.3 states appears in the corpus; then
|
||||
it mutates copies of the
|
||||
corpus, one defect each, and requires every copy to fail; and it drops each of its own checks in
|
||||
turn and requires the corpus to catch the weakened verifier. A bound no vector carries is checked
|
||||
directly.
|
||||
|
|
@ -56,6 +58,7 @@ TELEMETRY = os.path.join(HERE, "..", "extensions", "delivery-telemetry.md")
|
|||
FULL = 1_000_000 # §2: a value in parts per million of the full value
|
||||
LAST_INSTANT = 253_402_300_799_999 # §2: 9999-12-31T23:59:59.999Z
|
||||
MAX_INT = (1 << 53) - 1
|
||||
ACTIONS = ("fetch", "abandon", "present", "passed", "ended") # §7.5.3
|
||||
|
||||
# A weakened run drops one of these checks (see `WEAKENINGS`); None checks everything.
|
||||
DROPPED = None
|
||||
|
|
@ -209,6 +212,17 @@ def manifest(env):
|
|||
return m
|
||||
|
||||
|
||||
def rungs(entry):
|
||||
"""§4.1: an entry's ladder, in its order: none for `null`, the one object, or the array's."""
|
||||
fb = entry["fallback"]
|
||||
return [] if fb is None else fb if isinstance(fb, list) else [fb]
|
||||
|
||||
|
||||
def checked_rungs(entry, check):
|
||||
"""The rungs §4.3 reads, every one; a weakened run reads only the first."""
|
||||
return rungs(entry)[:1] if drops(check) else rungs(entry)
|
||||
|
||||
|
||||
def is_layer(env):
|
||||
"""§4.3 step 6: a WAI1 whose capability is refinement-only (SPEC §5)."""
|
||||
m = manifest(env)
|
||||
|
|
@ -245,7 +259,6 @@ def check_timeliness(c, g, groups):
|
|||
held under it."""
|
||||
need(g.start is not None, "context_invalid")
|
||||
entries = c["body"]["objects"]
|
||||
fallbacks = [e["fallback"] for e in entries if e["fallback"] is not None]
|
||||
envelopes = {}
|
||||
for x in groups.values():
|
||||
envelopes.update(x.envelopes)
|
||||
|
|
@ -256,17 +269,17 @@ def check_timeliness(c, g, groups):
|
|||
"timeliness_before_group")
|
||||
need(drops("content") or all(g.included.get(e["leaf_index"], e["content_hash"]) == e["content_hash"]
|
||||
for e in entries), "timeliness_content_mismatch")
|
||||
named = [e["content_hash"] for e in entries] + [f["content_hash"] for f in fallbacks]
|
||||
named = [e["content_hash"] for e in entries] + [f["content_hash"] for e in entries for f in checked_rungs(e, "rung_layer")]
|
||||
need(drops("layer") or not any(h in envelopes and is_layer(envelopes[h]) for h in named), "timeliness_on_layer")
|
||||
for f in fallbacks:
|
||||
for f in (f for e in entries for f in checked_rungs(e, "rung_position")):
|
||||
fg = groups.get(f["group"])
|
||||
if fg is not None and not drops("fallback_position"):
|
||||
need(f["leaf_index"] < fg.count and fg.included.get(f["leaf_index"], f["content_hash"]) == f["content_hash"],
|
||||
"timeliness_fallback_mismatch")
|
||||
for e in entries:
|
||||
f = e["fallback"]
|
||||
if f is not None and f["content_hash"] in envelopes and not drops("derived"):
|
||||
need(derived_from(envelopes[f["content_hash"]], e["content_hash"]), "timeliness_fallback_not_derived")
|
||||
for f in checked_rungs(e, "rung_derived"):
|
||||
if f["content_hash"] in envelopes and not drops("derived"):
|
||||
need(derived_from(envelopes[f["content_hash"]], e["content_hash"]), "timeliness_fallback_not_derived")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ §5: value over time
|
||||
|
|
@ -393,29 +406,32 @@ def has_entry(rec, group):
|
|||
|
||||
|
||||
def line_holders(group, entry, standing):
|
||||
"""§6.3: the holders with a record that stands and has an entry for the intent's group or its
|
||||
fallback's group, in the order of their first such record."""
|
||||
fb = entry["fallback"]
|
||||
"""§6.3: the holders with a record that stands and has an entry for the intent's group or a
|
||||
rung's group, in the order of their first such record."""
|
||||
rs = checked_rungs(entry, "rung_holders")
|
||||
out = []
|
||||
for rec in standing:
|
||||
h = rec[0].hex()
|
||||
if h not in out and (drops("line_holders") or has_entry(rec, group)
|
||||
or (fb is not None and has_entry(rec, fb["group"]))):
|
||||
or any(has_entry(rec, r["group"]) for r in rs)):
|
||||
out.append(h)
|
||||
return out
|
||||
|
||||
|
||||
def grade(entry, p, f):
|
||||
def grade(entry, p, fs):
|
||||
"""§6.3's table, the first outcome that holds, for a holder whose least instants for the
|
||||
primary and the fallback are p and f (None where none)."""
|
||||
by, fb = entry["by_ms"], entry["fallback"]
|
||||
primary and for each rung of the ladder are p and fs (None where none)."""
|
||||
by = entry["by_ms"]
|
||||
if p is not None and (p < by if drops("deadline_inclusive") else p <= by):
|
||||
return "on_time", FULL
|
||||
# §5.3: what arrived after the deadline grades nothing, so a later primary of more value
|
||||
# does not displace a fallback held by then.
|
||||
graded_late = drops("sink_rule") and None not in (p, fb) and value(entry, p) > fb["value_ppm"]
|
||||
if f is not None and (f <= by or drops("fallback_deadline")) and not graded_late:
|
||||
return "replaced", fb["value_ppm"]
|
||||
# The first rung, in the ladder's order, held by the deadline.
|
||||
in_time = [r for r, f in zip(rungs(entry), fs) if f is not None and (f <= by or drops("fallback_deadline"))]
|
||||
if in_time:
|
||||
r = in_time[-1] if drops("rung_order") else in_time[0]
|
||||
# §5.3: what arrived after the deadline grades nothing, so a later primary of more value
|
||||
# does not displace a rung held by then.
|
||||
if not (drops("sink_rule") and p is not None and value(entry, p) > r["value_ppm"]):
|
||||
return "replaced", r["value_ppm"]
|
||||
if p is not None:
|
||||
return "after_deadline", value(entry, p)
|
||||
return "not_shown", 0
|
||||
|
|
@ -423,7 +439,9 @@ def grade(entry, p, f):
|
|||
|
||||
def lines(entry, holders, primary, fallback):
|
||||
"""§6.3: an intent's line for each of `holders` (keys in hexadecimal), in increasing order of
|
||||
key bytes, from that holder's own held-at statements for its primary and its fallback."""
|
||||
key bytes, from that holder's own held-at statements for its primary and for the rungs of its
|
||||
fallback, each statement naming its object. *f* is one instant for a ladder of one rung, and
|
||||
the list of them for an array."""
|
||||
def least(statements, holder, latest=False):
|
||||
instants = [s[3] for s in statements if drops("own_statements") or s[0] == holder]
|
||||
if not instants:
|
||||
|
|
@ -432,16 +450,18 @@ def lines(entry, holders, primary, fallback):
|
|||
|
||||
out = []
|
||||
for h in (holders if drops("key_order") else sorted(holders, key=bytes.fromhex)):
|
||||
p, f = least(primary, h), least(fallback, h, drops("least_fallback"))
|
||||
outcome, v = grade(entry, p, f)
|
||||
p = least(primary, h)
|
||||
fs = [least([s for s in fallback if (s[1], s[2]) == (r["group"], r["leaf_index"])], h, drops("least_fallback"))
|
||||
for r in rungs(entry)]
|
||||
outcome, v = grade(entry, p, fs)
|
||||
f = fs if isinstance(entry["fallback"], list) else (fs[0] if fs else None)
|
||||
out.append({"holder": h, "held_ms": p, "fallback_ms": f, "outcome": outcome, "value_ppm": v})
|
||||
return out
|
||||
|
||||
|
||||
def intent_lines(link, entry, standing):
|
||||
fb = entry["fallback"]
|
||||
primary = held_from_records(standing, link, entry["leaf_index"])
|
||||
fallback = [] if fb is None else held_from_records(standing, fb["group"], fb["leaf_index"])
|
||||
fallback = [s for r in rungs(entry) for s in held_from_records(standing, r["group"], r["leaf_index"])]
|
||||
return lines(entry, line_holders(link, entry, standing), primary, fallback)
|
||||
|
||||
|
||||
|
|
@ -606,34 +626,43 @@ def arrival(src, group, index, s, ctx):
|
|||
return p["complete_ms"]
|
||||
|
||||
|
||||
def choose(entry, primary, fallback):
|
||||
"""§7.3 for an object whose intent stands, from each source's arrivals."""
|
||||
by, fb = entry["by_ms"], entry["fallback"]
|
||||
def choose(entry, primary, ladder):
|
||||
"""§7.3 for an object whose intent stands, from each source's arrivals of the primary and,
|
||||
for each rung of the ladder in its order, of that rung."""
|
||||
by = entry["by_ms"]
|
||||
|
||||
def choice(outcome, obj, i, arrived, v):
|
||||
return {"outcome": outcome, "object": obj, "source": i, "arrival_ms": arrived, "value_ppm": v}
|
||||
def choice(outcome, obj, r, i, arrived, v):
|
||||
return {"outcome": outcome, "object": obj, "rung": r, "source": i, "arrival_ms": arrived, "value_ppm": v}
|
||||
|
||||
timely = [i for i, a in enumerate(primary) if a is not None and a <= by]
|
||||
if timely:
|
||||
i = min(timely, key=lambda j: (primary[j], j)) if drops("plan_order") else timely[0]
|
||||
return choice("on_time", "primary", i, primary[i], FULL)
|
||||
return choice("on_time", "primary", None, i, primary[i], FULL)
|
||||
late = [(value(entry, a), i) for i, a in enumerate(primary) if a is not None]
|
||||
best = max((v for v, _ in late), default=0)
|
||||
# Rule 2: a fallback of at least the best late value, by the deadline, first in the asker's order.
|
||||
at_least = fb is not None and (fb["value_ppm"] > best if drops("fallback_tie") else fb["value_ppm"] >= best)
|
||||
if fb is not None and (at_least or drops("fallback_value")):
|
||||
in_time = [i for i, a in enumerate(fallback)
|
||||
# Rule 2: the first rung of at least the best late value that arrives by the deadline, from the
|
||||
# first such source in the asker's order.
|
||||
order = list(enumerate(rungs(entry)))
|
||||
if drops("plan_rung_first"):
|
||||
order = order[:1]
|
||||
if drops("plan_rung_order"):
|
||||
order.reverse()
|
||||
for r, fb in order:
|
||||
at_least = fb["value_ppm"] > best if drops("fallback_tie") else fb["value_ppm"] >= best
|
||||
if not (at_least or drops("fallback_value")):
|
||||
continue
|
||||
arrivals = ladder[r]
|
||||
in_time = [i for i, a in enumerate(arrivals)
|
||||
if a is not None and (a < by if drops("plan_fallback_deadline") else a <= by)]
|
||||
if in_time and drops("fallback_order"):
|
||||
in_time.sort(key=lambda j: (fallback[j], j))
|
||||
i = in_time[0] if in_time else None
|
||||
if i is not None:
|
||||
return choice("replaced", "fallback", i, fallback[i], fb["value_ppm"])
|
||||
in_time.sort(key=lambda j: (arrivals[j], j))
|
||||
if in_time:
|
||||
return choice("replaced", "fallback", r, in_time[0], arrivals[in_time[0]], fb["value_ppm"])
|
||||
if late and (best >= 1 or drops("late_zero")):
|
||||
ties = [i for v, i in late if v == best]
|
||||
i = ties[-1] if drops("late_tie") else ties[0]
|
||||
return choice("after_deadline", "primary", i, primary[i], best)
|
||||
return choice("no_source", None, None, None, 0)
|
||||
return choice("after_deadline", "primary", None, i, primary[i], best)
|
||||
return choice("no_source", None, None, None, None, 0)
|
||||
|
||||
|
||||
def plan(case):
|
||||
|
|
@ -657,13 +686,189 @@ def plan(case):
|
|||
choices.append("reject reason=deadline_not_stated")
|
||||
continue
|
||||
s = entry["from_ms"] if t == "from_ms" else t
|
||||
fb = entry["fallback"]
|
||||
primary = [arrival(src, w["group"], w["index"], s, ctx) for src in sources]
|
||||
fallback = [None if fb is None else arrival(src, fb["group"], fb["leaf_index"], s, ctx) for src in sources]
|
||||
choices.append(choose(entry, primary, fallback))
|
||||
ladder = [[arrival(src, r["group"], r["leaf_index"], s, ctx) for src in sources] for r in rungs(entry)]
|
||||
choices.append(choose(entry, primary, ladder))
|
||||
return {"claims": labels, "records": rlabels, "choices": choices}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ §7.5: running a plan
|
||||
|
||||
def run_inputs(case):
|
||||
"""§7.5.1's checks, in their order: the reason the run is refused, or None."""
|
||||
plans = case["plans"]
|
||||
if not plans and not drops("run_plans_none"):
|
||||
return "plans_invalid"
|
||||
if any(p["at_ms"] == "from_ms" for p in plans) and not drops("run_plans_from_ms"):
|
||||
return "plans_invalid"
|
||||
instants = [p["at_ms"] for p in plans]
|
||||
if not drops("run_plans_order") and any(a >= b for a, b in zip(instants, instants[1:])):
|
||||
return "plans_invalid"
|
||||
if not drops("run_plans_twice"):
|
||||
for p in plans:
|
||||
objects = [(w["group"], w["index"]) for w in p["wanted"]]
|
||||
if len(set(objects)) != len(objects):
|
||||
return "plans_invalid"
|
||||
for p in plans:
|
||||
bad = plan_inputs(case["loss_ppm"], case["overhead"], p["at_ms"], p["sources"])
|
||||
if bad:
|
||||
return bad
|
||||
arrivals = case["arrivals"]
|
||||
if not drops("run_arrivals_last") and any(not CV.is_int(a["at_ms"]) or a["at_ms"] > LAST_INSTANT for a in arrivals):
|
||||
return "arrivals_invalid"
|
||||
if not drops("run_arrivals_order") and any(a["at_ms"] > b["at_ms"] for a, b in zip(arrivals, arrivals[1:])):
|
||||
return "arrivals_invalid"
|
||||
if not drops("run_arrivals_twice") and len({(a["group"], a["index"]) for a in arrivals}) != len(arrivals):
|
||||
return "arrivals_invalid"
|
||||
return None
|
||||
|
||||
|
||||
def same_source(a, b):
|
||||
"""§7.5.2: both name one bearer instance or one holder's key, or both are the origin, whatever
|
||||
their `fetch_ms`."""
|
||||
if a["kind"] != b["kind"]:
|
||||
return False
|
||||
if drops("run_same_fetch_ms") and a.get("fetch_ms") != b.get("fetch_ms"):
|
||||
return False
|
||||
return a.get("bearer_id") == b.get("bearer_id") and a.get("holder") == b.get("holder")
|
||||
|
||||
|
||||
class Fetch:
|
||||
"""§7.5.2: the object a fetch takes, its source, and the plan it serves."""
|
||||
|
||||
def __init__(self, obj, rung, takes, plan, index, source):
|
||||
self.obj, self.rung, self.takes, self.plan, self.index, self.source = obj, rung, takes, plan, index, source
|
||||
|
||||
|
||||
def takes(key, entry, choice):
|
||||
"""The object a choice takes: the primary, or the rung it names."""
|
||||
if choice["object"] == "primary":
|
||||
return key
|
||||
r = rungs(entry)[choice["rung"]]
|
||||
return (r["group"], r["leaf_index"])
|
||||
|
||||
|
||||
def run(case):
|
||||
"""§7.5: the run of a sink's plans against its arrivals, or `refuse reason=R`."""
|
||||
bad = run_inputs(case)
|
||||
if bad:
|
||||
return "refuse reason=" + bad
|
||||
# Each plan derived by §7.2 and §7.3 from the shared inputs with its own instant, sources and
|
||||
# wanted objects; a group that fails refuses the run as it refuses a plan.
|
||||
plans = []
|
||||
for p in case["plans"]:
|
||||
derived = plan(dict(case, at_ms=p["at_ms"], sources=p["sources"], wanted=p["wanted"]))
|
||||
if isinstance(derived, str):
|
||||
return derived
|
||||
plans.append(derived["choices"])
|
||||
groups = read_groups(case["groups"])
|
||||
_, accepted = claims_step(case["claims"], groups, plan=True)
|
||||
intents = {(link, i): entry for link, i, entry in intents_step(accepted, groups)}
|
||||
|
||||
def objects_run(k):
|
||||
"""§7.5.2: what plan k wants whose choice is not refused, in its wanted order, each with its
|
||||
choice and its entry."""
|
||||
return [((w["group"], w["index"]), c, intents[(w["group"], w["index"])])
|
||||
for w, c in zip(case["plans"][k]["wanted"], plans[k]) if isinstance(c, dict)]
|
||||
|
||||
instants = {p["at_ms"] for p in case["plans"]} | {a["at_ms"] for a in case["arrivals"]}
|
||||
for k in range(len(plans)):
|
||||
for _, _, e in objects_run(k):
|
||||
instants |= {e["by_ms"], end(e)}
|
||||
|
||||
held, settled, passed, fetches, force, out = {}, set(), set(), {}, None, []
|
||||
|
||||
def decide(n, k, action, key, obj=None, rung=None, source=None, v=None):
|
||||
out.append({"at_ms": n, "plan": k, "action": action, "group": key[0], "index": key[1],
|
||||
"object": obj, "rung": rung, "source": source, "value_ppm": v})
|
||||
|
||||
def abandon(n, key):
|
||||
f = fetches.pop(key)
|
||||
decide(n, f.plan, "abandon", key, f.obj, f.rung, f.index)
|
||||
|
||||
for n in sorted(instants):
|
||||
# Step 1.
|
||||
for a in case["arrivals"]:
|
||||
if a["at_ms"] == n:
|
||||
arrived = (a["group"], a["index"])
|
||||
held[arrived] = n
|
||||
if not drops("run_arrival_completes"):
|
||||
for key in [key for key, f in fetches.items() if f.takes == arrived]:
|
||||
del fetches[key]
|
||||
# Step 2.
|
||||
new = next((k for k, p in enumerate(case["plans"]) if p["at_ms"] == n), None)
|
||||
if new is not None:
|
||||
sources = case["plans"][new]["sources"]
|
||||
choices = {key: (c, e) for key, c, e in objects_run(new)}
|
||||
|
||||
def continues(key, f):
|
||||
c, e = choices.get(key, (None, None))
|
||||
if c is None or c["object"] is None or drops("run_continue"):
|
||||
return False
|
||||
return takes(key, e, c) == f.takes and same_source(sources[c["source"]], f.source)
|
||||
|
||||
if force is not None:
|
||||
for key, _, _ in objects_run(force):
|
||||
if key not in settled and key in fetches and not continues(key, fetches[key]):
|
||||
if drops("run_drop_abandon") and key not in choices:
|
||||
continue
|
||||
abandon(n, key)
|
||||
force = new
|
||||
for key, c, e in objects_run(new):
|
||||
if (key in settled and not drops("run_settled_refetch")) or c["object"] is None:
|
||||
continue
|
||||
if takes(key, e, c) in held and not drops("run_held_fetch"):
|
||||
continue
|
||||
f = fetches.get(key)
|
||||
if f is not None and continues(key, f):
|
||||
if not drops("run_continued_plan"):
|
||||
f.plan, f.index = new, c["source"]
|
||||
continue
|
||||
fetches[key] = Fetch(c["object"], c["rung"], takes(key, e, c), new, c["source"], sources[c["source"]])
|
||||
decide(n, new, "fetch", key, c["object"], c["rung"], c["source"])
|
||||
if force is None:
|
||||
continue
|
||||
current = objects_run(force)
|
||||
# Step 3, by what the sink holds by `by_ms`.
|
||||
for key, _, e in current:
|
||||
if key in settled or key in passed or e["by_ms"] > n:
|
||||
continue
|
||||
limit = n if drops("run_by_deadline") else e["by_ms"]
|
||||
holds = [(r, f) for r, f in enumerate(rungs(e))
|
||||
if held.get((f["group"], f["leaf_index"]), limit + 1) <= limit]
|
||||
if held.get(key, limit + 1) <= limit:
|
||||
settled.add(key)
|
||||
decide(n, force, "present", key, "primary", v=FULL)
|
||||
elif holds:
|
||||
r, f = holds[-1] if drops("run_rung_order") else holds[0]
|
||||
settled.add(key)
|
||||
decide(n, force, "present", key, "fallback", r, v=f["value_ppm"])
|
||||
else:
|
||||
passed.add(key)
|
||||
decide(n, force, "passed", key)
|
||||
if key in fetches:
|
||||
after_present = key in settled and not drops("run_present_abandon")
|
||||
after_passed = key in passed and fetches[key].obj == "fallback" and not drops("run_passed_abandon")
|
||||
if after_present or after_passed:
|
||||
abandon(n, key)
|
||||
# Step 4, at the primary's value at its arrival.
|
||||
for key, _, e in current:
|
||||
if key in passed and key not in settled and key in held:
|
||||
v = value(e, n if drops("run_late_value") else held[key])
|
||||
if v >= 1 or drops("run_late_zero"):
|
||||
settled.add(key)
|
||||
decide(n, force, "present", key, "primary", v=v)
|
||||
# Step 5.
|
||||
for key, _, e in current:
|
||||
if key in passed and key not in settled and end(e) <= n:
|
||||
if key in fetches and not drops("run_end_abandon"):
|
||||
abandon(n, key)
|
||||
settled.add(key)
|
||||
if not drops("run_ended"):
|
||||
decide(n, force, "ended", key)
|
||||
return {"plans": plans, "decisions": out}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ the corpus
|
||||
|
||||
def claim_outcome(c):
|
||||
|
|
@ -682,7 +887,7 @@ def value_outcome(c):
|
|||
return {"value_ppm": value(e, c["at_ms"]), "end_ms": end(e)}
|
||||
|
||||
|
||||
RUNS = (("claims", claim_outcome), ("values", value_outcome), ("verdicts", verdict), ("plans", plan))
|
||||
RUNS = (("claims", claim_outcome), ("values", value_outcome), ("verdicts", verdict), ("plans", plan), ("runs", run))
|
||||
KINDS = tuple(k for k, _ in RUNS)
|
||||
|
||||
|
||||
|
|
@ -698,8 +903,8 @@ def strings(v, out):
|
|||
|
||||
|
||||
def check(doc, quiet=False):
|
||||
"""The failures of a corpus document: each case's outcome, every reason of §9 and every
|
||||
outcome of §6.3."""
|
||||
"""The failures of a corpus document: each case's outcome, every reason of §9, every outcome of
|
||||
§6.3 and every action of §7.5.3."""
|
||||
fails = []
|
||||
for kind, run in RUNS:
|
||||
for c in doc[kind]:
|
||||
|
|
@ -724,6 +929,9 @@ def check(doc, quiet=False):
|
|||
for outcome in outcomes_table():
|
||||
if outcome not in stated:
|
||||
fails.append("no case gives the outcome " + outcome)
|
||||
for action in ACTIONS:
|
||||
if action not in stated:
|
||||
fails.append("no case gives the action " + action)
|
||||
return fails
|
||||
|
||||
|
||||
|
|
@ -783,8 +991,54 @@ def mutations(doc):
|
|||
def m_reason_gone(d):
|
||||
d["verdicts"].remove(case(d, "record_before_group"))
|
||||
|
||||
def m_rung_value(d):
|
||||
# The lower rung presented at the higher's value.
|
||||
case(d, "ladder_lower_rung_alone")["outcome"]["intents"][0]["lines"][0]["value_ppm"] = 700_000
|
||||
|
||||
def m_rung_instant(d):
|
||||
# The lower rung a millisecond late: nothing held by the deadline.
|
||||
r = case(d, "ladder_lower_rung_alone")
|
||||
r["outcome"]["intents"][0]["lines"][0]["fallback_ms"][1] += 1
|
||||
|
||||
def m_rung_form(d):
|
||||
# A ladder's line reports its list.
|
||||
r = case(d, "ladder_first_rung_replaces")
|
||||
r["outcome"]["intents"][0]["lines"][0]["fallback_ms"] = r["outcome"]["intents"][0]["lines"][0]["fallback_ms"][0]
|
||||
|
||||
def m_plan_rung(d):
|
||||
case(d, "ladder_lower_rung")["outcome"]["choices"][0]["rung"] = 0
|
||||
|
||||
def m_arrival_late(d):
|
||||
# The trace a millisecond later is a late object at another value.
|
||||
case(d, "run_late_object")["arrivals"][0]["at_ms"] += 1
|
||||
|
||||
def m_arrival_gone(d):
|
||||
case(d, "run_on_time")["arrivals"] = []
|
||||
|
||||
def m_decision_value(d):
|
||||
case(d, "run_late_object")["outcome"]["decisions"][-1]["value_ppm"] += 1
|
||||
|
||||
def m_decision_action(d):
|
||||
case(d, "run_every_rung_passed_no_primary")["outcome"]["decisions"][1]["action"] = "ended"
|
||||
|
||||
def m_plan_dropped(d):
|
||||
c = case(d, "run_replaced_changes_source")
|
||||
c["plans"] = c["plans"][:1]
|
||||
|
||||
def m_plan_instant(d):
|
||||
case(d, "run_replaced_at_deadline")["plans"][1]["at_ms"] += 1
|
||||
|
||||
def m_decision_order(d):
|
||||
c = case(d, "run_in_wanted_order")
|
||||
c["outcome"]["decisions"][0], c["outcome"]["decisions"][1] = c["outcome"]["decisions"][1], c["outcome"]["decisions"][0]
|
||||
|
||||
def m_reason_gone_run(d):
|
||||
d["runs"] = [c for c in d["runs"] if c["outcome"] != "refuse reason=arrivals_invalid"]
|
||||
|
||||
return [m_instant, m_end, m_floor, m_accept, m_line, m_earliest, m_statement, m_envelope, m_order, m_deadline,
|
||||
m_fetch, m_loss, m_from_ms, m_object, m_reason_gone]
|
||||
m_fetch, m_loss, m_from_ms, m_object, m_reason_gone, m_rung_value, m_rung_instant, m_rung_form, m_plan_rung,
|
||||
m_arrival_late, m_arrival_gone, m_decision_value, m_decision_action, m_plan_dropped, m_plan_instant,
|
||||
m_decision_order, m_reason_gone_run]
|
||||
|
||||
|
||||
WEAKENINGS = ["subject", "signer", "range", "before", "same_second", "content", "layer", "fallback_position", "derived",
|
||||
|
|
@ -794,7 +1048,14 @@ WEAKENINGS = ["subject", "signer", "range", "before", "same_second", "content",
|
|||
"overhead", "instant", "sources_empty", "sources_kind", "sources_uri", "sources_key", "sources_fetch",
|
||||
"sources_twice", "sources_loss", "per_group", "undeclared", "bidirectional", "record_at_s",
|
||||
"holder_record", "plan_order", "fallback_value", "fallback_tie", "fallback_order",
|
||||
"plan_fallback_deadline", "late_zero", "late_tie"]
|
||||
"plan_fallback_deadline", "late_zero", "late_tie",
|
||||
"rung_layer", "rung_position", "rung_derived", "rung_holders", "rung_order",
|
||||
"plan_rung_first", "plan_rung_order",
|
||||
"run_plans_none", "run_plans_from_ms", "run_plans_order", "run_plans_twice", "run_arrivals_last",
|
||||
"run_arrivals_order", "run_arrivals_twice", "run_same_fetch_ms", "run_arrival_completes", "run_continue",
|
||||
"run_drop_abandon", "run_settled_refetch", "run_held_fetch", "run_continued_plan", "run_by_deadline",
|
||||
"run_rung_order", "run_present_abandon", "run_passed_abandon", "run_late_value", "run_late_zero",
|
||||
"run_end_abandon", "run_ended"]
|
||||
|
||||
|
||||
def direct_checks():
|
||||
|
|
@ -816,6 +1077,12 @@ def direct_checks():
|
|||
got = False
|
||||
if got != ok:
|
||||
fails.append(f"a point of {v1} at {half} is {'refused' if ok else 'accepted'}")
|
||||
# §7.5.3 states each action a run decides.
|
||||
decisions = text_of(EXTENSION)
|
||||
decisions = decisions[decisions.index("\n#### 7.5.3 Decisions"):decisions.index("\n#### 7.5.4 ")]
|
||||
for action in ACTIONS:
|
||||
if f"`{action}`" not in decisions:
|
||||
fails.append(f"§7.5.3 does not state {action}")
|
||||
# jwp-receipts §3.1 lists §4.2's codes at step 4 and §4.3's at step 6; delivery-telemetry §5.1
|
||||
# lists §8.6's switch reason.
|
||||
jwp = text_of(JWP_RECEIPTS)
|
||||
|
|
|
|||
|
|
@ -989,9 +989,9 @@ pub fn timeliness_lines(inp: &Inputs, acks: &[Given], records: &[Vec<u8>], claim
|
|||
let link = j.group.link_hash();
|
||||
let read = read_evidence(&j, acks, records, &verified(inp, &j), None);
|
||||
let standing: Vec<&Holdings> = read.standing.iter().collect();
|
||||
let groups: Vec<[u8; 32]> = std::iter::once(link).chain(entry.fallback.as_ref().map(|f| f.group)).collect();
|
||||
let groups: Vec<[u8; 32]> = std::iter::once(link).chain(entry.fallback.rungs().iter().map(|f| f.group)).collect();
|
||||
let asked: Vec<([u8; 32], u64)> =
|
||||
std::iter::once((link, entry.leaf_index)).chain(entry.fallback.as_ref().map(|f| (f.group, f.leaf_index))).collect();
|
||||
std::iter::once((link, entry.leaf_index)).chain(entry.fallback.rungs().iter().map(|f| (f.group, f.leaf_index))).collect();
|
||||
let mut held: Vec<HeldAt> = timeliness::held_from_records(&standing, &asked);
|
||||
let mut holders: BTreeSet<[u8; 32]> =
|
||||
standing.iter().filter(|h| groups.iter().any(|g| h.entry(g).is_some())).map(|h| h.holder).collect();
|
||||
|
|
|
|||
|
|
@ -354,7 +354,7 @@ fn entry(g: &Publication, i: usize, by_ms: u64, fallback: Option<Fallback>) -> T
|
|||
from_ms: START_MS,
|
||||
by_ms,
|
||||
decay: vec![DecayPoint { at_ms: by_ms + 1, value_ppm: 0 }],
|
||||
fallback,
|
||||
fallback: fallback.into(),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -139,7 +139,7 @@ pub mod schedule;
|
|||
pub use schedule::{BlockSize, Esi, Fec, Oti, ScheduleClaim, ScheduleEntry};
|
||||
/// cbindgen:ignore
|
||||
pub mod timeliness;
|
||||
pub use timeliness::{DecayPoint, Fallback, HeldGroup, TimelinessClaim, TimelinessEntry};
|
||||
pub use timeliness::{DecayPoint, Fallback, HeldGroup, Ladder, TimelinessClaim, TimelinessEntry};
|
||||
/// cbindgen:ignore
|
||||
pub mod consumption;
|
||||
pub use consumption::{ConsumptionClaim, UsageEntry, Vocabulary};
|
||||
|
|
@ -506,8 +506,11 @@ pub enum Reason {
|
|||
TimelinessWindowInvalid,
|
||||
/// A `timeliness` entry whose decay is not well formed.
|
||||
TimelinessDecayInvalid,
|
||||
/// A `timeliness` entry whose fallback is not well formed.
|
||||
/// A `timeliness` entry whose fallback has a rung that is not well formed.
|
||||
TimelinessFallbackInvalid,
|
||||
/// A `timeliness` entry whose ladder is not well formed: an array of other than two to eight
|
||||
/// rungs, whose values do not strictly fall, or two of whose rungs name one object.
|
||||
TimelinessLadderInvalid,
|
||||
/// A `timeliness` claim whose subject is not the link hash of the group it is checked
|
||||
/// against.
|
||||
TimelinessSubjectMismatch,
|
||||
|
|
@ -520,11 +523,11 @@ pub enum Reason {
|
|||
/// A `timeliness` entry whose content hash is not that of the object receipt at its
|
||||
/// position.
|
||||
TimelinessContentMismatch,
|
||||
/// A `timeliness` entry, or its fallback, that is a staged layer.
|
||||
/// A `timeliness` entry, or a rung of its fallback, that is a staged layer.
|
||||
TimelinessOnLayer,
|
||||
/// A `timeliness` fallback whose position or content hash its group's receipts contradict.
|
||||
/// A `timeliness` rung whose position or content hash its group's receipts contradict.
|
||||
TimelinessFallbackMismatch,
|
||||
/// A `timeliness` fallback whose envelope does not derive from the entry's object.
|
||||
/// A `timeliness` rung whose envelope does not derive from the entry's object.
|
||||
TimelinessFallbackNotDerived,
|
||||
/// A `cache-transmission` claim whose `attests` is not `transmitted`.
|
||||
CacheTransmissionNotTransmitted,
|
||||
|
|
@ -696,6 +699,7 @@ impl Reason {
|
|||
Reason::TimelinessWindowInvalid => "timeliness_window_invalid",
|
||||
Reason::TimelinessDecayInvalid => "timeliness_decay_invalid",
|
||||
Reason::TimelinessFallbackInvalid => "timeliness_fallback_invalid",
|
||||
Reason::TimelinessLadderInvalid => "timeliness_ladder_invalid",
|
||||
Reason::TimelinessSubjectMismatch => "timeliness_subject_mismatch",
|
||||
Reason::TimelinessSignerMismatch => "timeliness_signer_mismatch",
|
||||
Reason::TimelinessLeafOutOfRange => "timeliness_leaf_out_of_range",
|
||||
|
|
|
|||
|
|
@ -1,7 +1,8 @@
|
|||
//! `timeliness` (jwp-receipts §2.4.17; `extensions/timeliness.md` §4, §5): a publisher's
|
||||
//! statement of when each object of a group is of use — the instant it is useful from, the
|
||||
//! deadline by which a receiver must hold it for its full value, how that value falls after the
|
||||
//! deadline, and the rendition a receiver presents in its place.
|
||||
//! deadline, and the renditions a receiver presents in its place, a ladder in order of falling
|
||||
//! value.
|
||||
//!
|
||||
//! This file holds steps 2 and 4 of jwp-receipts §3.1 for the class — its body's members and
|
||||
//! types, and timeliness §4.2 — an entry's value at an instant (§5.1), and step 6, its checks
|
||||
|
|
@ -19,6 +20,8 @@ pub const ATTESTS_VALUED: &str = "valued";
|
|||
pub const FULL_PPM: u64 = 1_000_000;
|
||||
/// The most points a decay has (timeliness §4.2 rule 3.2).
|
||||
pub const MAX_POINTS: usize = 16;
|
||||
/// The most rungs a ladder has (timeliness §4.2 rule 3.4).
|
||||
pub const MAX_RUNGS: usize = 8;
|
||||
|
||||
/// A timeliness body's members (timeliness §4.1), in the order step 2 reads them.
|
||||
const TIMELINESS_MEMBERS: &[&str] = &["objects", "attests"];
|
||||
|
|
@ -34,7 +37,8 @@ pub struct DecayPoint {
|
|||
pub value_ppm: u64,
|
||||
}
|
||||
|
||||
/// The object a receiver presents in the primary's place (timeliness §4.1 `fallback`).
|
||||
/// One rung of a ladder: an object a receiver presents in the primary's place (timeliness §4.1
|
||||
/// `fallback`).
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct Fallback {
|
||||
/// The link hash of its group receipt.
|
||||
|
|
@ -45,6 +49,78 @@ pub struct Fallback {
|
|||
pub value_ppm: u64,
|
||||
}
|
||||
|
||||
/// An entry's fallback (timeliness §4.1): its ladder, the objects a receiver presents in the
|
||||
/// primary's place, in order of falling value. Each form is kept as it was read, so a claim
|
||||
/// writes back to its own bytes: one rung is the object, two or more an array, and an array of
|
||||
/// one is refused (§4.2 rule 3.4), never read as the object.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Eq)]
|
||||
pub enum Ladder {
|
||||
/// `null`: no rung.
|
||||
#[default]
|
||||
None,
|
||||
/// One rung, written as the object: the form before ladders.
|
||||
One(Fallback),
|
||||
/// An array of rungs, two to eight where well formed.
|
||||
Rungs(Vec<Fallback>),
|
||||
}
|
||||
|
||||
impl Ladder {
|
||||
/// The rungs, in the ladder's order: none, the one, or the array's.
|
||||
pub fn rungs(&self) -> &[Fallback] {
|
||||
match self {
|
||||
Ladder::None => &[],
|
||||
Ladder::One(f) => std::slice::from_ref(f),
|
||||
Ladder::Rungs(v) => v,
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the fallback is written as an array.
|
||||
pub fn is_array(&self) -> bool {
|
||||
matches!(self, Ladder::Rungs(_))
|
||||
}
|
||||
|
||||
/// Rule 3.4: an array has 2 to 8 rungs, their values strictly falling, and no two name one
|
||||
/// content hash or one position.
|
||||
fn formed(&self) -> bool {
|
||||
let Ladder::Rungs(v) = self else { return true };
|
||||
(2..=MAX_RUNGS).contains(&v.len())
|
||||
&& v.windows(2).all(|w| w[1].value_ppm < w[0].value_ppm)
|
||||
&& v.iter().enumerate().all(|(i, a)| {
|
||||
v[i + 1..].iter().all(|b| a.content_hash != b.content_hash && (a.group, a.leaf_index) != (b.group, b.leaf_index))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl From<Option<Fallback>> for Ladder {
|
||||
/// One fallback or none: the form before ladders.
|
||||
fn from(f: Option<Fallback>) -> Ladder {
|
||||
f.map_or(Ladder::None, Ladder::One)
|
||||
}
|
||||
}
|
||||
|
||||
impl Fallback {
|
||||
fn to_value(self) -> Value {
|
||||
let mut q = Map::new();
|
||||
q.insert("group".into(), hex(&self.group).into());
|
||||
q.insert("leaf_index".into(), self.leaf_index.into());
|
||||
q.insert("content_hash".into(), hex(&self.content_hash).into());
|
||||
q.insert("value_ppm".into(), self.value_ppm.into());
|
||||
Value::Object(q)
|
||||
}
|
||||
|
||||
/// Step 2 for one rung: an object, its members, then their types in the table's order.
|
||||
fn read(v: &Value) -> Result<Fallback, Reason> {
|
||||
let f = v.as_object().ok_or(Reason::FieldType)?;
|
||||
members(f, FALLBACK_MEMBERS)?;
|
||||
Ok(Fallback {
|
||||
group: get_hash(f, "group")?,
|
||||
leaf_index: get_int(f, "leaf_index")?,
|
||||
content_hash: get_hash(f, "content_hash")?,
|
||||
value_ppm: get_int(f, "value_ppm")?,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// One object's statement (timeliness §4.1).
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct TimelinessEntry {
|
||||
|
|
@ -55,7 +131,8 @@ pub struct TimelinessEntry {
|
|||
/// The last instant at which a receiver that first holds the object has it at full value.
|
||||
pub by_ms: u64,
|
||||
pub decay: Vec<DecayPoint>,
|
||||
pub fallback: Option<Fallback>,
|
||||
/// The ladder: the objects presented in the primary's place, in order of falling value.
|
||||
pub fallback: Ladder,
|
||||
}
|
||||
|
||||
impl TimelinessEntry {
|
||||
|
|
@ -93,13 +170,14 @@ impl TimelinessEntry {
|
|||
if !self.decay_formed() {
|
||||
return Err(Reason::TimelinessDecayInvalid);
|
||||
}
|
||||
if let Some(f) = &self.fallback
|
||||
&& (!(1..=FULL_PPM).contains(&f.value_ppm)
|
||||
|| f.content_hash == self.content_hash
|
||||
|| (f.group == *subject && f.leaf_index == self.leaf_index))
|
||||
{
|
||||
if self.fallback.rungs().iter().any(|f| {
|
||||
!(1..=FULL_PPM).contains(&f.value_ppm) || f.content_hash == self.content_hash || (f.group == *subject && f.leaf_index == self.leaf_index)
|
||||
}) {
|
||||
return Err(Reason::TimelinessFallbackInvalid);
|
||||
}
|
||||
if !self.fallback.formed() {
|
||||
return Err(Reason::TimelinessLadderInvalid);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
|
@ -144,15 +222,9 @@ impl TimelinessEntry {
|
|||
});
|
||||
m.insert("decay".into(), Value::Array(points.collect()));
|
||||
let fallback = match &self.fallback {
|
||||
None => Value::Null,
|
||||
Some(f) => {
|
||||
let mut q = Map::new();
|
||||
q.insert("group".into(), hex(&f.group).into());
|
||||
q.insert("leaf_index".into(), f.leaf_index.into());
|
||||
q.insert("content_hash".into(), hex(&f.content_hash).into());
|
||||
q.insert("value_ppm".into(), f.value_ppm.into());
|
||||
Value::Object(q)
|
||||
}
|
||||
Ladder::None => Value::Null,
|
||||
Ladder::One(f) => f.to_value(),
|
||||
Ladder::Rungs(v) => Value::Array(v.iter().map(|f| f.to_value()).collect()),
|
||||
};
|
||||
m.insert("fallback".into(), fallback);
|
||||
Value::Object(m)
|
||||
|
|
@ -172,8 +244,8 @@ impl TimelinessEntry {
|
|||
Ok(e)
|
||||
}
|
||||
|
||||
/// Step 2: the members, then each one's type in the table's order, each point's and the
|
||||
/// fallback's members and types where the table reaches them.
|
||||
/// Step 2: the members, then each one's type in the table's order, each point's and each
|
||||
/// rung's members and types where the table reaches them.
|
||||
fn read(v: &Value) -> Result<TimelinessEntry, Reason> {
|
||||
let m = v.as_object().ok_or(Reason::FieldType)?;
|
||||
members(m, ENTRY_MEMBERS)?;
|
||||
|
|
@ -192,16 +264,9 @@ impl TimelinessEntry {
|
|||
})
|
||||
.collect::<Result<_, Reason>>()?;
|
||||
let fallback = match &m["fallback"] {
|
||||
Value::Null => None,
|
||||
Value::Object(f) => {
|
||||
members(f, FALLBACK_MEMBERS)?;
|
||||
Some(Fallback {
|
||||
group: get_hash(f, "group")?,
|
||||
leaf_index: get_int(f, "leaf_index")?,
|
||||
content_hash: get_hash(f, "content_hash")?,
|
||||
value_ppm: get_int(f, "value_ppm")?,
|
||||
})
|
||||
}
|
||||
Value::Null => Ladder::None,
|
||||
f @ Value::Object(_) => Ladder::One(Fallback::read(f)?),
|
||||
Value::Array(v) => Ladder::Rungs(v.iter().map(Fallback::read).collect::<Result<_, _>>()?),
|
||||
_ => return Err(Reason::FieldType),
|
||||
};
|
||||
Ok(TimelinessEntry { leaf_index, content_hash, from_ms, by_ms, decay, fallback })
|
||||
|
|
@ -215,7 +280,7 @@ impl TimelinessEntry {
|
|||
int_ok(p.at_ms)?;
|
||||
int_ok(p.value_ppm)?;
|
||||
}
|
||||
if let Some(f) = &self.fallback {
|
||||
for f in self.fallback.rungs() {
|
||||
int_ok(f.leaf_index)?;
|
||||
int_ok(f.value_ppm)?;
|
||||
}
|
||||
|
|
@ -291,9 +356,9 @@ impl ClaimReceipt {
|
|||
/// every entry's object is in it (`timeliness_leaf_out_of_range`); none is of use before
|
||||
/// it begins (`timeliness_before_group`); each entry's object receipt, where held, has the
|
||||
/// entry's content hash (`timeliness_content_mismatch`); no envelope held of an entry or a
|
||||
/// fallback is a layer (`timeliness_on_layer`); each fallback is where it says, where its
|
||||
/// group and object receipts are held (`timeliness_fallback_mismatch`); and each fallback's
|
||||
/// envelope held derives from its primary (`timeliness_fallback_not_derived`).
|
||||
/// rung is a layer (`timeliness_on_layer`); each rung is where it says, where its group and
|
||||
/// object receipts are held (`timeliness_fallback_mismatch`); and each rung's envelope held
|
||||
/// derives from its primary (`timeliness_fallback_not_derived`).
|
||||
pub fn check_timeliness_against(&self, group: &GroupReceipt, held: &[HeldGroup<'_>]) -> Result<(), Rejection> {
|
||||
let ClaimBody::Timeliness(t) = &self.body else {
|
||||
return Err(Rejection::at(6, Reason::ContextMismatch));
|
||||
|
|
@ -339,25 +404,22 @@ impl ClaimReceipt {
|
|||
let bytes_of = |hash: &[u8; 32]| -> Vec<&[u8]> { envelopes.iter().filter(|(h, _)| h == hash).map(|(_, b)| *b).collect() };
|
||||
let is_layer = |b: &&[u8]| Wai::from_bytes(b).is_ok_and(|w| crate::codecs::is_refinement_only(w.capability()));
|
||||
for e in &t.objects {
|
||||
if bytes_of(&e.content_hash).iter().any(is_layer)
|
||||
|| e.fallback.as_ref().is_some_and(|f| bytes_of(&f.content_hash).iter().any(is_layer))
|
||||
{
|
||||
if bytes_of(&e.content_hash).iter().any(is_layer) || e.fallback.rungs().iter().any(|f| bytes_of(&f.content_hash).iter().any(is_layer)) {
|
||||
return at(Reason::TimelinessOnLayer);
|
||||
}
|
||||
}
|
||||
for f in t.objects.iter().filter_map(|e| e.fallback.as_ref()) {
|
||||
for f in t.objects.iter().flat_map(|e| e.fallback.rungs()) {
|
||||
let Some(g) = held.iter().map(|h| h.group).find(|g| g.link_hash() == f.group && g.verify().is_ok()) else { continue };
|
||||
if f.leaf_index >= g.leaf_count || receipts(&f.group, g, f.leaf_index).iter().any(|h| *h != f.content_hash) {
|
||||
return at(Reason::TimelinessFallbackMismatch);
|
||||
}
|
||||
}
|
||||
for e in &t.objects {
|
||||
let Some(f) = &e.fallback else { continue };
|
||||
let motif = hex(&e.content_hash);
|
||||
let derived = |b: &&[u8]| {
|
||||
Wai::from_bytes(b).ok().and_then(|w| w.manifest.derived_from.as_ref().and_then(|v| DerivedFrom::from_value(v).ok())).is_some_and(|d| d.motif == motif)
|
||||
};
|
||||
if bytes_of(&f.content_hash).iter().any(|b| !derived(b)) {
|
||||
if e.fallback.rungs().iter().any(|f| bytes_of(&f.content_hash).iter().any(|b| !derived(b))) {
|
||||
return at(Reason::TimelinessFallbackNotDerived);
|
||||
}
|
||||
}
|
||||
|
|
@ -386,7 +448,7 @@ mod tests {
|
|||
from_ms: by_ms.min(1_000),
|
||||
by_ms,
|
||||
decay: decay.iter().map(|&(at_ms, value_ppm)| DecayPoint { at_ms, value_ppm }).collect(),
|
||||
fallback: None,
|
||||
fallback: Ladder::None,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -492,11 +554,56 @@ mod tests {
|
|||
assert_eq!(with(&|e| e.decay[0].value_ppm = 1), Err(Reason::TimelinessDecayInvalid));
|
||||
assert_eq!(with(&|e| e.decay[0].at_ms = LAST_INSTANT_MS + 1), Err(Reason::TimelinessDecayInvalid));
|
||||
let fallback = Fallback { group: [7; 32], leaf_index: 0, content_hash: [2; 32], value_ppm: 1 };
|
||||
assert_eq!(with(&|e| e.fallback = Some(fallback)), Ok(()));
|
||||
assert_eq!(with(&|e| e.fallback = Some(Fallback { value_ppm: 0, ..fallback })), Err(Reason::TimelinessFallbackInvalid));
|
||||
assert_eq!(with(&|e| e.fallback = Some(Fallback { value_ppm: FULL_PPM + 1, ..fallback })), Err(Reason::TimelinessFallbackInvalid));
|
||||
assert_eq!(with(&|e| e.fallback = Some(Fallback { content_hash: [1; 32], ..fallback })), Err(Reason::TimelinessFallbackInvalid));
|
||||
assert_eq!(with(&|e| e.fallback = Some(Fallback { group: subject, ..fallback })), Err(Reason::TimelinessFallbackInvalid));
|
||||
assert_eq!(with(&|e| e.fallback = Some(Fallback { group: subject, leaf_index: 1, ..fallback })), Ok(()));
|
||||
let one = |f: Fallback| Ladder::One(f);
|
||||
assert_eq!(with(&|e| e.fallback = one(fallback)), Ok(()));
|
||||
assert_eq!(with(&|e| e.fallback = one(Fallback { value_ppm: 0, ..fallback })), Err(Reason::TimelinessFallbackInvalid));
|
||||
assert_eq!(with(&|e| e.fallback = one(Fallback { value_ppm: FULL_PPM + 1, ..fallback })), Err(Reason::TimelinessFallbackInvalid));
|
||||
assert_eq!(with(&|e| e.fallback = one(Fallback { content_hash: [1; 32], ..fallback })), Err(Reason::TimelinessFallbackInvalid));
|
||||
assert_eq!(with(&|e| e.fallback = one(Fallback { group: subject, ..fallback })), Err(Reason::TimelinessFallbackInvalid));
|
||||
assert_eq!(with(&|e| e.fallback = one(Fallback { group: subject, leaf_index: 1, ..fallback })), Ok(()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_ladder_is_two_to_eight_rungs_falling_each_named_once() {
|
||||
let subject = [9; 32];
|
||||
let check = |rungs: Vec<Fallback>| {
|
||||
let mut e = entry(D, &[(D + 1, 0)]);
|
||||
e.fallback = Ladder::Rungs(rungs);
|
||||
e.check(&subject)
|
||||
};
|
||||
// Rung k: group [7], position k, content hash [10 + k], at value v.
|
||||
let rung = |k: u8, v: u64| Fallback { group: [7; 32], leaf_index: u64::from(k), content_hash: [10 + k; 32], value_ppm: v };
|
||||
let falling = |n: u8| (0..n).map(|k| rung(k, 900_000 - 100_000 * u64::from(k))).collect::<Vec<_>>();
|
||||
assert_eq!(check(falling(2)), Ok(()));
|
||||
assert_eq!(check(falling(8)), Ok(()));
|
||||
assert_eq!(check(vec![rung(0, 2), rung(1, 1)]), Ok(()), "a part per million apart");
|
||||
assert_eq!(check(vec![rung(0, FULL_PPM), rung(1, 1)]), Ok(()), "the first at full value");
|
||||
assert_eq!(check(falling(9)), Err(Reason::TimelinessLadderInvalid));
|
||||
assert_eq!(check(falling(1)), Err(Reason::TimelinessLadderInvalid), "an array of one: its form is the object");
|
||||
assert_eq!(check(vec![]), Err(Reason::TimelinessLadderInvalid));
|
||||
assert_eq!(check(vec![rung(0, 400_000), rung(1, 400_000)]), Err(Reason::TimelinessLadderInvalid), "equal");
|
||||
assert_eq!(check(vec![rung(0, 400_000), rung(1, 700_000)]), Err(Reason::TimelinessLadderInvalid), "rising");
|
||||
assert_eq!(check(vec![rung(0, 700_000), Fallback { leaf_index: 1, ..rung(0, 400_000) }]), Err(Reason::TimelinessLadderInvalid), "one content hash");
|
||||
assert_eq!(check(vec![rung(0, 700_000), Fallback { content_hash: [99; 32], ..rung(0, 400_000) }]), Err(Reason::TimelinessLadderInvalid), "one position");
|
||||
// Rule 3.3 is each rung's, before rule 3.4.
|
||||
assert_eq!(check(vec![rung(0, 700_000), rung(1, 0)]), Err(Reason::TimelinessFallbackInvalid), "a rung at 0");
|
||||
assert_eq!(check(vec![rung(0, 0)]), Err(Reason::TimelinessFallbackInvalid), "rule 3.3 before 3.4");
|
||||
assert_eq!(check(vec![rung(0, 700_000), Fallback { content_hash: [1; 32], ..rung(1, 400_000) }]), Err(Reason::TimelinessFallbackInvalid), "a rung that is the primary");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn each_form_writes_back_to_itself() {
|
||||
let rung = Fallback { group: [7; 32], leaf_index: 0, content_hash: [2; 32], value_ppm: 400_000 };
|
||||
for ladder in [Ladder::None, Ladder::One(rung), Ladder::Rungs(vec![rung]), Ladder::Rungs(vec![Fallback { value_ppm: 700_000, content_hash: [3; 32], leaf_index: 1, ..rung }, rung])] {
|
||||
let mut e = entry(D, &[(D + 1, 0)]);
|
||||
e.fallback = ladder;
|
||||
let read = TimelinessEntry::read(&e.to_json()).expect("reads");
|
||||
assert_eq!(read, e, "the form is kept");
|
||||
}
|
||||
let mut v = entry(D, &[(D + 1, 0)]).to_json();
|
||||
v["fallback"] = serde_json::json!([5]);
|
||||
assert_eq!(TimelinessEntry::read(&v), Err(Reason::FieldType), "a rung that is not an object");
|
||||
v["fallback"] = serde_json::json!([{"group": hex(&[7; 32]), "leaf_index": 0, "content_hash": hex(&[2; 32])}]);
|
||||
assert_eq!(TimelinessEntry::read(&v), Err(Reason::MissingRequiredField));
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,17 +3,18 @@
|
|||
//!
|
||||
//! A publisher's `timeliness` claim, whose body, value function and context checks are
|
||||
//! `crate::claim_receipt::timeliness`'s, states for each object the instant it is of use from,
|
||||
//! its deadline, how its value falls after it and the rendition presented in its place. This
|
||||
//! module judges, from holdings records, whether each holder held each object in time
|
||||
//! ([`verdict`], §6), and plans which of an asker's sources delivers each object by its deadline
|
||||
//! ([`plan`], §7).
|
||||
//! its deadline, how its value falls after it and the ladder of renditions presented in its
|
||||
//! place. This module judges, from holdings records, whether each holder held each object in time
|
||||
//! ([`verdict`], §6), plans which of an asker's sources delivers each object by its deadline
|
||||
//! ([`plan`], §7), and runs a sink's plans against its clock ([`scheduler::run`], §7.5).
|
||||
//!
|
||||
//! # Rules enforced here rather than only documented
|
||||
//!
|
||||
//! - Every instant a held-at statement gives is the latest its word allows, so every value a line
|
||||
//! reports is the least the evidence shows; an object with no statement is never on time.
|
||||
//! - A sink is graded by what it held at the deadline (§5.3): a fallback held by then replaces a
|
||||
//! primary that arrives later, whatever that primary's value.
|
||||
//! - A sink is graded by what it held at the deadline (§5.3): the first rung of the ladder held by
|
||||
//! then replaces a primary that arrives later, whatever that primary's value, and no rung held
|
||||
//! only after it replaces anything.
|
||||
//! - Two statements of one object that differ set both aside, as do two records of one holder at
|
||||
//! one instant that differ; a record dated before its group began is refused whole.
|
||||
//! - The held-at statement ([`HeldAt`]) is the family's one form: an extension that derives one
|
||||
|
|
@ -32,6 +33,7 @@ use crate::holdings::{self, Holdings, RecordVerdict};
|
|||
use crate::jwp_delivery::{DeliveryError, GroupReceipt, ObjectReceipt};
|
||||
|
||||
pub mod corpus;
|
||||
pub mod scheduler;
|
||||
|
||||
/// The codes timeliness §9 lists, in its order.
|
||||
pub const CODES: &[&str] = &[
|
||||
|
|
@ -40,6 +42,7 @@ pub const CODES: &[&str] = &[
|
|||
"timeliness_window_invalid",
|
||||
"timeliness_decay_invalid",
|
||||
"timeliness_fallback_invalid",
|
||||
"timeliness_ladder_invalid",
|
||||
"timeliness_subject_mismatch",
|
||||
"timeliness_signer_mismatch",
|
||||
"timeliness_leaf_out_of_range",
|
||||
|
|
@ -53,6 +56,8 @@ pub const CODES: &[&str] = &[
|
|||
DEADLINE_NOT_STATED,
|
||||
INSTANT_INVALID,
|
||||
SOURCES_INVALID,
|
||||
scheduler::PLANS_INVALID,
|
||||
scheduler::ARRIVALS_INVALID,
|
||||
];
|
||||
|
||||
/// The outcomes timeliness §6.3 lists, in its order.
|
||||
|
|
@ -136,8 +141,11 @@ pub struct Line {
|
|||
pub holder: [u8; 32],
|
||||
/// *p*: the least instant of the holder's held-at statements for the primary.
|
||||
pub held_ms: Option<u64>,
|
||||
/// *f*: the least for the fallback.
|
||||
/// *f*: the least for the fallback where it is one rung; `None` for a ladder written as an
|
||||
/// array, whose instants are [`Line::rungs_ms`].
|
||||
pub fallback_ms: Option<u64>,
|
||||
/// *f*<sub>k</sub>: the least for each rung of the ladder, in its order.
|
||||
pub rungs_ms: Vec<Option<u64>>,
|
||||
pub outcome: Outcome,
|
||||
pub value_ppm: u64,
|
||||
}
|
||||
|
|
@ -148,18 +156,22 @@ pub(crate) fn lines(group: &[u8; 32], entry: &TimelinessEntry, holders: &BTreeSe
|
|||
let least = |key: &[u8; 32], g: &[u8; 32], i: u64| {
|
||||
held.iter().filter(|h| h.key == *key && h.group == *g && h.index == i).map(|h| h.instant_ms).min()
|
||||
};
|
||||
let rungs = entry.fallback.rungs();
|
||||
holders
|
||||
.iter()
|
||||
.map(|key| {
|
||||
let p = least(key, group, entry.leaf_index);
|
||||
let f = entry.fallback.as_ref().and_then(|fb| least(key, &fb.group, fb.leaf_index));
|
||||
let (outcome, value_ppm) = match (p, f, &entry.fallback) {
|
||||
(Some(p), _, _) if p <= entry.by_ms => (Outcome::OnTime, FULL_PPM),
|
||||
(_, Some(f), Some(fb)) if f <= entry.by_ms => (Outcome::Replaced, fb.value_ppm),
|
||||
(Some(p), _, _) => (Outcome::AfterDeadline, entry.value_at(p)),
|
||||
_ => (Outcome::NotShown, 0),
|
||||
let fs: Vec<Option<u64>> = rungs.iter().map(|r| least(key, &r.group, r.leaf_index)).collect();
|
||||
// The first rung, in the ladder's order, held by the deadline.
|
||||
let in_time = rungs.iter().zip(&fs).find(|(_, f)| f.is_some_and(|f| f <= entry.by_ms));
|
||||
let (outcome, value_ppm) = match (p, in_time) {
|
||||
(Some(p), _) if p <= entry.by_ms => (Outcome::OnTime, FULL_PPM),
|
||||
(_, Some((r, _))) => (Outcome::Replaced, r.value_ppm),
|
||||
(Some(p), None) => (Outcome::AfterDeadline, entry.value_at(p)),
|
||||
(None, None) => (Outcome::NotShown, 0),
|
||||
};
|
||||
Line { holder: *key, held_ms: p, fallback_ms: f, outcome, value_ppm }
|
||||
let fallback_ms = if entry.fallback.is_array() { None } else { fs.first().copied().flatten() };
|
||||
Line { holder: *key, held_ms: p, fallback_ms, rungs_ms: fs, outcome, value_ppm }
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
|
@ -417,17 +429,15 @@ pub fn verdict(inp: &Inputs) -> Result<Verdict, Refusal> {
|
|||
for i in &j.intents {
|
||||
if let Ok(e) = &i.standing {
|
||||
asked.push((i.group, i.leaf_index));
|
||||
if let Some(f) = &e.fallback {
|
||||
asked.push((f.group, f.leaf_index));
|
||||
}
|
||||
asked.extend(e.fallback.rungs().iter().map(|f| (f.group, f.leaf_index)));
|
||||
}
|
||||
}
|
||||
let held = held_from_records(&standing, &asked);
|
||||
// Step 6: the lines, for each holder with a record that stands and an entry for the intent's
|
||||
// group or its fallback's.
|
||||
// group or a rung's.
|
||||
for i in &mut j.intents {
|
||||
let Ok(e) = &i.standing else { continue };
|
||||
let groups: Vec<[u8; 32]> = std::iter::once(i.group).chain(e.fallback.map(|f| f.group)).collect();
|
||||
let groups: Vec<[u8; 32]> = std::iter::once(i.group).chain(e.fallback.rungs().iter().map(|f| f.group)).collect();
|
||||
let holders: BTreeSet<[u8; 32]> =
|
||||
standing.iter().filter(|h| groups.iter().any(|g| h.entry(g).is_some())).map(|h| h.holder).collect();
|
||||
i.lines = lines(&i.group, e, &holders, &held);
|
||||
|
|
@ -542,6 +552,8 @@ impl Which {
|
|||
pub struct Choice {
|
||||
pub outcome: Outcome,
|
||||
pub object: Option<Which>,
|
||||
/// The rung's index in the ladder, for a [`Which::Fallback`].
|
||||
pub rung: Option<usize>,
|
||||
/// The source's index in the asker's list.
|
||||
pub source: Option<usize>,
|
||||
pub arrival_ms: Option<u64>,
|
||||
|
|
@ -568,7 +580,15 @@ fn record_at<'a>(read: &'a BTreeMap<usize, Holdings>, key: &[u8; 32], s: u64) ->
|
|||
|
||||
/// §7.
|
||||
pub fn plan(inp: &PlanInputs) -> Result<PlanVerdict, Refusal> {
|
||||
if let Some((l, h)) = inp.loss {
|
||||
planned(inp).map(|(p, _)| p)
|
||||
}
|
||||
|
||||
/// The arrival of an object from a source (§7.2), or none.
|
||||
type ArrivalOf<'a> = dyn Fn(&Source, &[u8; 32], u64, u64) -> Option<u64> + 'a;
|
||||
|
||||
/// §7.1's checks, in its order: *L* and *h*, the instant, then the sources, read.
|
||||
pub(crate) fn checked(loss: Option<(u64, u64)>, at: At, v: &[Value]) -> Result<Vec<Source>, Refusal> {
|
||||
if let Some((l, h)) = loss {
|
||||
if l > MAX_LOSS_PPM {
|
||||
return Err(Refusal::Input(LOSS_BOUND_INVALID));
|
||||
}
|
||||
|
|
@ -576,12 +596,18 @@ pub fn plan(inp: &PlanInputs) -> Result<PlanVerdict, Refusal> {
|
|||
return Err(Refusal::Input(OVERHEAD_INVALID));
|
||||
}
|
||||
}
|
||||
if let At::Instant(t) = inp.at
|
||||
if let At::Instant(t) = at
|
||||
&& t > LAST_INSTANT_MS
|
||||
{
|
||||
return Err(Refusal::Input(INSTANT_INVALID));
|
||||
}
|
||||
let sources = sources(&inp.sources, inp.loss.is_some()).map_err(Refusal::Input)?;
|
||||
sources(v, loss.is_some()).map_err(Refusal::Input)
|
||||
}
|
||||
|
||||
/// §7, with the entry that stands for each wanted object whose choice is not refused, in the
|
||||
/// order of `wanted`.
|
||||
pub(crate) fn planned(inp: &PlanInputs) -> Result<(PlanVerdict, Vec<Option<TimelinessEntry>>), Refusal> {
|
||||
let sources = checked(inp.loss, inp.at, &inp.sources)?;
|
||||
let j = judge(&inp.groups, &inp.claims, &inp.records, true)?;
|
||||
|
||||
// §7.2: the declarations and schedules of each group given (bearer-receipts §6.2 step 4;
|
||||
|
|
@ -625,46 +651,54 @@ pub fn plan(inp: &PlanInputs) -> Result<PlanVerdict, Refusal> {
|
|||
|
||||
// §7.3.
|
||||
let mut choices = Vec::with_capacity(inp.wanted.len());
|
||||
let mut entries = Vec::with_capacity(inp.wanted.len());
|
||||
for (group, index) in &inp.wanted {
|
||||
let intent = j.intents.iter().find(|i| i.group == *group && i.leaf_index == *index);
|
||||
let Some(Ok(e)) = intent.map(|i| &i.standing) else {
|
||||
choices.push(Err(DEADLINE_NOT_STATED));
|
||||
entries.push(None);
|
||||
continue;
|
||||
};
|
||||
let s = match inp.at {
|
||||
At::Instant(t) => t,
|
||||
At::FromMs => e.from_ms,
|
||||
};
|
||||
let primary: Vec<Option<u64>> = sources.iter().map(|src| arrival(src, group, *index, s)).collect();
|
||||
let take = |outcome, object, source: usize, arrival: u64, value_ppm| Choice {
|
||||
outcome,
|
||||
object: Some(object),
|
||||
source: Some(source),
|
||||
arrival_ms: Some(arrival),
|
||||
value_ppm,
|
||||
};
|
||||
if let Some(k) = primary.iter().position(|a| a.is_some_and(|a| a <= e.by_ms)) {
|
||||
choices.push(Ok(take(Outcome::OnTime, Which::Primary, k, primary[k].expect("an arrival"), FULL_PPM)));
|
||||
continue;
|
||||
}
|
||||
let best = primary.iter().flatten().map(|&a| e.value_at(a)).max().unwrap_or(0);
|
||||
if let Some(f) = &e.fallback
|
||||
&& f.value_ppm >= best
|
||||
{
|
||||
let fallback: Vec<Option<u64>> = sources.iter().map(|src| arrival(src, &f.group, f.leaf_index, s)).collect();
|
||||
if let Some(k) = fallback.iter().position(|a| a.is_some_and(|a| a <= e.by_ms)) {
|
||||
choices.push(Ok(take(Outcome::Replaced, Which::Fallback, k, fallback[k].expect("an arrival"), f.value_ppm)));
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if best >= 1 {
|
||||
let k = primary.iter().position(|a| a.is_some_and(|a| e.value_at(a) == best)).expect("the best value's source");
|
||||
choices.push(Ok(take(Outcome::AfterDeadline, Which::Primary, k, primary[k].expect("an arrival"), best)));
|
||||
continue;
|
||||
}
|
||||
choices.push(Ok(Choice { outcome: Outcome::NoSource, object: None, source: None, arrival_ms: None, value_ppm: 0 }));
|
||||
choices.push(Ok(choose(e, &sources, &arrival, group, *index, s)));
|
||||
entries.push(Some(e.clone()));
|
||||
}
|
||||
Ok(PlanVerdict { claims, records: j.records, choices })
|
||||
Ok((PlanVerdict { claims, records: j.records, choices }, entries))
|
||||
}
|
||||
|
||||
/// §7.3 for one wanted object whose intent stands, planned from `s`.
|
||||
fn choose(e: &TimelinessEntry, sources: &[Source], arrival: &ArrivalOf<'_>, group: &[u8; 32], index: u64, s: u64) -> Choice {
|
||||
let primary: Vec<Option<u64>> = sources.iter().map(|src| arrival(src, group, index, s)).collect();
|
||||
let take = |outcome, object, rung, source: usize, arrival: u64, value_ppm| Choice {
|
||||
outcome,
|
||||
object: Some(object),
|
||||
rung,
|
||||
source: Some(source),
|
||||
arrival_ms: Some(arrival),
|
||||
value_ppm,
|
||||
};
|
||||
// Rule 1: the first source on time.
|
||||
if let Some(k) = primary.iter().position(|a| a.is_some_and(|a| a <= e.by_ms)) {
|
||||
return take(Outcome::OnTime, Which::Primary, None, k, primary[k].expect("an arrival"), FULL_PPM);
|
||||
}
|
||||
// Rule 2: the first rung, in the ladder's order, worth at least the best late primary and
|
||||
// delivered by the deadline; the rungs fall in value, so none after one worth less is.
|
||||
let best = primary.iter().flatten().map(|&a| e.value_at(a)).max().unwrap_or(0);
|
||||
for (r, f) in e.fallback.rungs().iter().enumerate().take_while(|(_, f)| f.value_ppm >= best) {
|
||||
let fallback: Vec<Option<u64>> = sources.iter().map(|src| arrival(src, &f.group, f.leaf_index, s)).collect();
|
||||
if let Some(k) = fallback.iter().position(|a| a.is_some_and(|a| a <= e.by_ms)) {
|
||||
return take(Outcome::Replaced, Which::Fallback, Some(r), k, fallback[k].expect("an arrival"), f.value_ppm);
|
||||
}
|
||||
}
|
||||
// Rule 3: the best late source, the first in order among equals.
|
||||
if best >= 1 {
|
||||
let k = primary.iter().position(|a| a.is_some_and(|a| e.value_at(a) == best)).expect("the best value's source");
|
||||
return take(Outcome::AfterDeadline, Which::Primary, None, k, primary[k].expect("an arrival"), best);
|
||||
}
|
||||
Choice { outcome: Outcome::NoSource, object: None, rung: None, source: None, arrival_ms: None, value_ppm: 0 }
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
@ -674,7 +708,7 @@ mod tests {
|
|||
|
||||
const D: u64 = 1_793_664_060_000;
|
||||
|
||||
fn entry(fallback: Option<crate::claim_receipt::Fallback>) -> TimelinessEntry {
|
||||
fn entry(fallback: crate::claim_receipt::Ladder) -> TimelinessEntry {
|
||||
TimelinessEntry {
|
||||
leaf_index: 0,
|
||||
content_hash: [1; 32],
|
||||
|
|
@ -692,7 +726,7 @@ mod tests {
|
|||
#[test]
|
||||
fn a_line_grades_what_was_held_at_the_deadline() {
|
||||
let (g, f) = ([7; 32], [8; 32]);
|
||||
let e = entry(Some(crate::claim_receipt::Fallback { group: f, leaf_index: 0, content_hash: [2; 32], value_ppm: 400_000 }));
|
||||
let e = entry(crate::claim_receipt::Ladder::One(crate::claim_receipt::Fallback { group: f, leaf_index: 0, content_hash: [2; 32], value_ppm: 400_000 }));
|
||||
let holders: BTreeSet<[u8; 32]> = [[1; 32], [2; 32], [3; 32], [4; 32], [5; 32]].into_iter().collect();
|
||||
let held = [
|
||||
// 1: on time, its later statement ignored.
|
||||
|
|
@ -722,6 +756,51 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_ladder_line_takes_the_first_rung_held_by_the_deadline() {
|
||||
use crate::claim_receipt::{Fallback, Ladder};
|
||||
let (g, hi, lo) = ([7; 32], [8; 32], [9; 32]);
|
||||
let rungs = vec![
|
||||
Fallback { group: hi, leaf_index: 0, content_hash: [2; 32], value_ppm: 700_000 },
|
||||
Fallback { group: lo, leaf_index: 0, content_hash: [3; 32], value_ppm: 400_000 },
|
||||
];
|
||||
let e = entry(Ladder::Rungs(rungs));
|
||||
let holders: BTreeSet<[u8; 32]> = (1..=6).map(|k| [k; 32]).collect();
|
||||
let held = [
|
||||
// 1: both rungs by the deadline: the first.
|
||||
at(1, hi, 0, D),
|
||||
at(1, lo, 0, D - 5),
|
||||
// 2: the lower rung alone.
|
||||
at(2, lo, 0, D),
|
||||
// 3: the higher rung late, the lower in time: the lower.
|
||||
at(3, hi, 0, D + 1),
|
||||
at(3, lo, 0, D - 1),
|
||||
// 4: every rung late, the primary later: after the deadline.
|
||||
at(4, hi, 0, D + 1),
|
||||
at(4, lo, 0, D + 2),
|
||||
at(4, g, 0, D + 1_000),
|
||||
// 5: every rung late, no primary.
|
||||
at(5, hi, 0, D + 1),
|
||||
// 6: the primary on time over both rungs.
|
||||
at(6, g, 0, D),
|
||||
at(6, hi, 0, D - 1),
|
||||
at(6, lo, 0, D - 1),
|
||||
];
|
||||
let got: Vec<(Outcome, u64, Vec<Option<u64>>)> =
|
||||
lines(&g, &e, &holders, &held).into_iter().map(|l| (l.outcome, l.value_ppm, l.rungs_ms)).collect();
|
||||
assert_eq!(
|
||||
got,
|
||||
vec![
|
||||
(Outcome::Replaced, 700_000, vec![Some(D), Some(D - 5)]),
|
||||
(Outcome::Replaced, 400_000, vec![None, Some(D)]),
|
||||
(Outcome::Replaced, 400_000, vec![Some(D + 1), Some(D - 1)]),
|
||||
(Outcome::AfterDeadline, 800_000, vec![Some(D + 1), Some(D + 2)]),
|
||||
(Outcome::NotShown, 0, vec![Some(D + 1), None]),
|
||||
(Outcome::OnTime, 1_000_000, vec![Some(D - 1), Some(D - 1)]),
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_held_at_statement_before_its_group_is_by_whole_seconds() {
|
||||
let start = 1_793_664_000;
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
418
wai/wai-rs/src/timeliness/scheduler.rs
Normal file
418
wai/wai-rs/src/timeliness/scheduler.rs
Normal file
|
|
@ -0,0 +1,418 @@
|
|||
//! The reference sink scheduler: a sink's run of its plans (`extensions/timeliness.md` §7.5).
|
||||
//!
|
||||
//! From a sink's plans (§7) and its own clock — the instants from which it first holds each
|
||||
//! object — [`run`] decides what the sink fetches, what it presents in each object's place and
|
||||
//! when, and what it states, by the rule a sink is graded by (§5.3), the ladder included. A run is
|
||||
//! a function of its inputs alone, so a verifier given a sink's inputs derives its decisions.
|
||||
//!
|
||||
//! # Rules enforced here rather than only documented
|
||||
//!
|
||||
//! - At an object's deadline the sink presents the primary it holds, else the first rung of the
|
||||
//! ladder it holds; holding neither, it has passed every rung and says so (`passed`), and no rung
|
||||
//! is presented in the primary's place from then on.
|
||||
//! - A late primary is presented at its value at its arrival, where that is at least 1; at the
|
||||
//! end, a sink that presented nothing abandons its fetch and says so (`ended`).
|
||||
//! - A later plan replaces the one in force wholly: a fetch it does not continue is abandoned, and
|
||||
//! an object only the earlier plan wanted is no longer run. What is settled stays settled.
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use super::{checked, planned, At, Choice, GroupInput, PlanInputs, Refusal, Source, Which};
|
||||
use crate::claim_receipt::schedule::LAST_INSTANT_MS;
|
||||
use crate::claim_receipt::timeliness::FULL_PPM;
|
||||
use crate::claim_receipt::TimelinessEntry;
|
||||
|
||||
/// §7.5.1: no plan, one planned from `from_ms`, instants that do not strictly increase, or an
|
||||
/// object wanted twice by one plan.
|
||||
pub const PLANS_INVALID: &str = "plans_invalid";
|
||||
/// §7.5.1: an arrival past the last instant, arrivals whose instants decrease, or two of one
|
||||
/// object.
|
||||
pub const ARRIVALS_INVALID: &str = "arrivals_invalid";
|
||||
|
||||
/// The decisions of §7.5.3, in their order there.
|
||||
pub const ACTIONS: &[&str] = &["fetch", "abandon", "present", "passed", "ended"];
|
||||
|
||||
/// An object: a group's link hash and a position.
|
||||
type Object = ([u8; 32], u64);
|
||||
|
||||
/// One plan of a run (§7.5.1): the instant it is given at and planned from, the sink's sources
|
||||
/// in its order, as [`super::sources`] reads them, and the wanted objects.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct RunPlan {
|
||||
pub at: At,
|
||||
pub sources: Vec<Value>,
|
||||
pub wanted: Vec<Object>,
|
||||
}
|
||||
|
||||
/// An arrival (§7.5.1): the instant from which the sink first holds an object, on its clock.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct Arrival {
|
||||
pub group: [u8; 32],
|
||||
pub index: u64,
|
||||
pub at_ms: u64,
|
||||
}
|
||||
|
||||
/// The inputs of §7.5.1.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct RunInputs {
|
||||
/// §7.1's groups, claims and records, and *L* and *h*, shared by every plan.
|
||||
pub groups: Vec<GroupInput>,
|
||||
pub claims: Vec<String>,
|
||||
pub records: Vec<Vec<u8>>,
|
||||
pub loss: Option<(u64, u64)>,
|
||||
/// The plans, the first in force from its instant and each later one replacing it.
|
||||
pub plans: Vec<RunPlan>,
|
||||
/// The sink's arrivals, in the order of its clock.
|
||||
pub arrivals: Vec<Arrival>,
|
||||
}
|
||||
|
||||
/// A decision's action (§7.5.3).
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum Action {
|
||||
Fetch,
|
||||
Abandon,
|
||||
Present,
|
||||
Passed,
|
||||
Ended,
|
||||
}
|
||||
|
||||
impl Action {
|
||||
pub fn label(self) -> &'static str {
|
||||
match self {
|
||||
Action::Fetch => "fetch",
|
||||
Action::Abandon => "abandon",
|
||||
Action::Present => "present",
|
||||
Action::Passed => "passed",
|
||||
Action::Ended => "ended",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One decision (§7.5.3).
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct Decision {
|
||||
pub at_ms: u64,
|
||||
/// The plan in force; for an `abandon`, the plan the fetch served.
|
||||
pub plan: usize,
|
||||
pub action: Action,
|
||||
pub group: [u8; 32],
|
||||
pub index: u64,
|
||||
/// For `fetch` and `abandon` the object the fetch takes, for `present` the one presented.
|
||||
pub object: Option<Which>,
|
||||
/// The rung's index in the ladder, where the object is a rung.
|
||||
pub rung: Option<usize>,
|
||||
/// For `fetch` and `abandon`, the source's index in that plan's list.
|
||||
pub source: Option<usize>,
|
||||
/// For `present`, the value presented.
|
||||
pub value_ppm: Option<u64>,
|
||||
}
|
||||
|
||||
/// §7.5's run: each plan's choices (§7.3), then the decisions in the order taken.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct Run {
|
||||
pub plans: Vec<Vec<Result<Choice, &'static str>>>,
|
||||
pub decisions: Vec<Decision>,
|
||||
}
|
||||
|
||||
/// A fetch outstanding (§7.5.2).
|
||||
#[derive(Clone, Debug)]
|
||||
struct Fetch {
|
||||
object: Which,
|
||||
rung: Option<usize>,
|
||||
/// The object it takes.
|
||||
target: Object,
|
||||
source: Source,
|
||||
/// The plan it serves and the source's index in that plan's list.
|
||||
plan: usize,
|
||||
index: usize,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct State {
|
||||
settled: bool,
|
||||
passed: bool,
|
||||
fetch: Option<Fetch>,
|
||||
}
|
||||
|
||||
/// One plan derived: its sources, and the objects it runs in its wanted order, each with its
|
||||
/// choice and the entry that stands for it.
|
||||
struct Derived {
|
||||
at: u64,
|
||||
sources: Vec<Source>,
|
||||
run: Vec<(Object, Choice, TimelinessEntry)>,
|
||||
}
|
||||
|
||||
/// §7.5.2: two sources are the same where both name one bearer instance or one holder's key, or
|
||||
/// both are the origin, whatever their `fetch_ms`.
|
||||
fn same(a: &Source, b: &Source) -> bool {
|
||||
match (a, b) {
|
||||
(Source::Bearer(x), Source::Bearer(y)) => x == y,
|
||||
(Source::Holder { key: x, .. }, Source::Holder { key: y, .. }) => x == y,
|
||||
(Source::Origin { .. }, Source::Origin { .. }) => true,
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// The object a choice takes: the primary, or the rung it names.
|
||||
fn target(object: &Object, entry: &TimelinessEntry, which: Which, rung: Option<usize>) -> Object {
|
||||
match (which, rung) {
|
||||
(Which::Fallback, Some(r)) => {
|
||||
let f = entry.fallback.rungs()[r];
|
||||
(f.group, f.leaf_index)
|
||||
}
|
||||
_ => *object,
|
||||
}
|
||||
}
|
||||
|
||||
/// §7.5.1: the checks, in their order.
|
||||
fn check(inp: &RunInputs) -> Result<(), Refusal> {
|
||||
let instant = |a: At| match a {
|
||||
At::Instant(t) => Some(t),
|
||||
At::FromMs => None,
|
||||
};
|
||||
let formed = !inp.plans.is_empty()
|
||||
&& inp.plans.iter().all(|p| instant(p.at).is_some())
|
||||
&& inp.plans.windows(2).all(|w| instant(w[0].at) < instant(w[1].at))
|
||||
&& inp.plans.iter().all(|p| p.wanted.iter().collect::<BTreeSet<_>>().len() == p.wanted.len());
|
||||
if !formed {
|
||||
return Err(Refusal::Input(PLANS_INVALID));
|
||||
}
|
||||
for p in &inp.plans {
|
||||
checked(inp.loss, p.at, &p.sources)?;
|
||||
}
|
||||
let arrivals_formed = inp.arrivals.iter().all(|a| a.at_ms <= LAST_INSTANT_MS)
|
||||
&& inp.arrivals.windows(2).all(|w| w[0].at_ms <= w[1].at_ms)
|
||||
&& inp.arrivals.iter().map(|a| (a.group, a.index)).collect::<BTreeSet<_>>().len() == inp.arrivals.len();
|
||||
if !arrivals_formed {
|
||||
return Err(Refusal::Input(ARRIVALS_INVALID));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// §7.5: the run of a sink's plans against its arrivals.
|
||||
pub fn run(inp: &RunInputs) -> Result<Run, Refusal> {
|
||||
check(inp)?;
|
||||
// Each plan derived by §7.2 and §7.3 from the shared inputs.
|
||||
let mut plans: Vec<Vec<Result<Choice, &'static str>>> = Vec::with_capacity(inp.plans.len());
|
||||
let mut derived: Vec<Derived> = Vec::with_capacity(inp.plans.len());
|
||||
for p in &inp.plans {
|
||||
let At::Instant(at) = p.at else { unreachable!("checked: every plan is at an instant") };
|
||||
let pi = PlanInputs {
|
||||
groups: inp.groups.clone(),
|
||||
claims: inp.claims.clone(),
|
||||
records: inp.records.clone(),
|
||||
loss: inp.loss,
|
||||
at: p.at,
|
||||
sources: p.sources.clone(),
|
||||
wanted: p.wanted.clone(),
|
||||
};
|
||||
let (verdict, entries) = planned(&pi)?;
|
||||
let sources = checked(inp.loss, p.at, &p.sources)?;
|
||||
let run = p
|
||||
.wanted
|
||||
.iter()
|
||||
.zip(&verdict.choices)
|
||||
.zip(entries)
|
||||
.filter_map(|((o, c), e)| Some((*o, *c.as_ref().ok()?, e?)))
|
||||
.collect();
|
||||
derived.push(Derived { at, sources, run });
|
||||
plans.push(verdict.choices);
|
||||
}
|
||||
|
||||
// The instants the run reads: each plan's, each arrival's, and each deadline and end.
|
||||
let mut instants: BTreeSet<u64> = derived.iter().map(|d| d.at).collect();
|
||||
instants.extend(inp.arrivals.iter().map(|a| a.at_ms));
|
||||
for d in &derived {
|
||||
for (_, _, e) in &d.run {
|
||||
instants.insert(e.by_ms);
|
||||
instants.insert(e.end_ms());
|
||||
}
|
||||
}
|
||||
|
||||
let mut held: BTreeMap<Object, u64> = BTreeMap::new();
|
||||
let mut state: BTreeMap<Object, State> = BTreeMap::new();
|
||||
let mut in_force: Option<usize> = None;
|
||||
let mut out: Vec<Decision> = Vec::new();
|
||||
let decide = |out: &mut Vec<Decision>, at_ms, plan, action, o: &Object, object, rung, source, value_ppm| {
|
||||
out.push(Decision { at_ms, plan, action, group: o.0, index: o.1, object, rung, source, value_ppm });
|
||||
};
|
||||
let abandon = |out: &mut Vec<Decision>, n: u64, o: &Object, f: &Fetch| {
|
||||
out.push(Decision {
|
||||
at_ms: n,
|
||||
plan: f.plan,
|
||||
action: Action::Abandon,
|
||||
group: o.0,
|
||||
index: o.1,
|
||||
object: Some(f.object),
|
||||
rung: f.rung,
|
||||
source: Some(f.index),
|
||||
value_ppm: None,
|
||||
});
|
||||
};
|
||||
|
||||
for n in instants {
|
||||
// Step 1: arrivals.
|
||||
for a in inp.arrivals.iter().filter(|a| a.at_ms == n) {
|
||||
let o = (a.group, a.index);
|
||||
held.insert(o, n);
|
||||
for st in state.values_mut() {
|
||||
if st.fetch.as_ref().is_some_and(|f| f.target == o) {
|
||||
st.fetch = None;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Step 2: a plan given at n replaces the one in force.
|
||||
if let Some(k) = derived.iter().position(|d| d.at == n) {
|
||||
let new = &derived[k];
|
||||
let continues = |o: &Object, f: &Fetch| {
|
||||
new.run.iter().any(|(w, c, e)| {
|
||||
w == o
|
||||
&& match (c.object, c.source) {
|
||||
(Some(which), Some(si)) => target(w, e, which, c.rung) == f.target && same(&new.sources[si], &f.source),
|
||||
_ => false,
|
||||
}
|
||||
})
|
||||
};
|
||||
if let Some(old) = in_force {
|
||||
for (o, _, _) in &derived[old].run {
|
||||
let st = state.entry(*o).or_default();
|
||||
if st.settled {
|
||||
continue;
|
||||
}
|
||||
if let Some(f) = st.fetch.take_if(|f| !continues(o, f)) {
|
||||
abandon(&mut out, n, o, &f);
|
||||
}
|
||||
}
|
||||
}
|
||||
in_force = Some(k);
|
||||
for (o, c, e) in &new.run {
|
||||
let st = state.entry(*o).or_default();
|
||||
let (Some(which), Some(si)) = (c.object, c.source) else { continue };
|
||||
if st.settled {
|
||||
continue;
|
||||
}
|
||||
let t = target(o, e, which, c.rung);
|
||||
if held.contains_key(&t) {
|
||||
continue;
|
||||
}
|
||||
if let Some(f) = st.fetch.as_mut()
|
||||
&& f.target == t
|
||||
&& same(&f.source, &new.sources[si])
|
||||
{
|
||||
// Continued: it serves the new plan from now on.
|
||||
(f.plan, f.index) = (k, si);
|
||||
continue;
|
||||
}
|
||||
st.fetch = Some(Fetch { object: which, rung: c.rung, target: t, source: new.sources[si].clone(), plan: k, index: si });
|
||||
decide(&mut out, n, k, Action::Fetch, o, Some(which), c.rung, Some(si), None);
|
||||
}
|
||||
}
|
||||
|
||||
let Some(k) = in_force else { continue };
|
||||
// Step 3: deadlines, by §5.3 with what the sink holds by `by_ms`.
|
||||
for (o, _, e) in &derived[k].run {
|
||||
let st = state.entry(*o).or_default();
|
||||
if st.settled || st.passed || e.by_ms > n {
|
||||
continue;
|
||||
}
|
||||
let by = |t: &Object| held.get(t).is_some_and(|&a| a <= e.by_ms);
|
||||
if by(o) {
|
||||
st.settled = true;
|
||||
decide(&mut out, n, k, Action::Present, o, Some(Which::Primary), None, None, Some(FULL_PPM));
|
||||
} else if let Some((r, f)) = e.fallback.rungs().iter().enumerate().find(|(_, f)| by(&(f.group, f.leaf_index))) {
|
||||
st.settled = true;
|
||||
decide(&mut out, n, k, Action::Present, o, Some(Which::Fallback), Some(r), None, Some(f.value_ppm));
|
||||
} else {
|
||||
st.passed = true;
|
||||
decide(&mut out, n, k, Action::Passed, o, None, None, None, None);
|
||||
}
|
||||
// After a `present`, any fetch; after `passed`, a rung's.
|
||||
let settled = st.settled;
|
||||
if let Some(f) = st.fetch.take_if(|f| settled || f.object == Which::Fallback) {
|
||||
abandon(&mut out, n, o, &f);
|
||||
}
|
||||
}
|
||||
// Step 4: late primaries.
|
||||
for (o, _, e) in &derived[k].run {
|
||||
let st = state.entry(*o).or_default();
|
||||
if !st.passed || st.settled {
|
||||
continue;
|
||||
}
|
||||
if let Some(&a) = held.get(o) {
|
||||
let v = e.value_at(a);
|
||||
if v >= 1 {
|
||||
st.settled = true;
|
||||
decide(&mut out, n, k, Action::Present, o, Some(Which::Primary), None, None, Some(v));
|
||||
}
|
||||
}
|
||||
}
|
||||
// Step 5: ends.
|
||||
for (o, _, e) in &derived[k].run {
|
||||
let st = state.entry(*o).or_default();
|
||||
if !st.passed || st.settled || e.end_ms() > n {
|
||||
continue;
|
||||
}
|
||||
if let Some(f) = st.fetch.take() {
|
||||
abandon(&mut out, n, o, &f);
|
||||
}
|
||||
st.settled = true;
|
||||
decide(&mut out, n, k, Action::Ended, o, None, None, None, None);
|
||||
}
|
||||
}
|
||||
Ok(Run { plans, decisions: out })
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn sources_are_the_same_by_what_they_name() {
|
||||
let key = [3; 32];
|
||||
assert!(same(&Source::Origin { fetch_ms: 1 }, &Source::Origin { fetch_ms: 9 }));
|
||||
assert!(same(&Source::Holder { key, fetch_ms: 1 }, &Source::Holder { key, fetch_ms: 2 }));
|
||||
assert!(!same(&Source::Holder { key, fetch_ms: 1 }, &Source::Holder { key: [4; 32], fetch_ms: 1 }));
|
||||
assert!(same(&Source::Bearer("urn:example:bearer:b1".into()), &Source::Bearer("urn:example:bearer:b1".into())));
|
||||
assert!(!same(&Source::Bearer("urn:example:bearer:b1".into()), &Source::Origin { fetch_ms: 1 }));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_inputs_are_checked_in_order() {
|
||||
let base = || RunInputs {
|
||||
groups: Vec::new(),
|
||||
claims: Vec::new(),
|
||||
records: Vec::new(),
|
||||
loss: None,
|
||||
plans: vec![RunPlan { at: At::Instant(5), sources: vec![serde_json::json!({"kind": "origin", "fetch_ms": 1})], wanted: vec![([1; 32], 0)] }],
|
||||
arrivals: vec![Arrival { group: [1; 32], index: 0, at_ms: 7 }],
|
||||
};
|
||||
assert_eq!(check(&base()), Ok(()));
|
||||
let with = |f: &dyn Fn(&mut RunInputs)| {
|
||||
let mut i = base();
|
||||
f(&mut i);
|
||||
check(&i)
|
||||
};
|
||||
let plans = Err(Refusal::Input(PLANS_INVALID));
|
||||
let arrivals = Err(Refusal::Input(ARRIVALS_INVALID));
|
||||
assert_eq!(with(&|i| i.plans.clear()), plans);
|
||||
assert_eq!(with(&|i| i.plans[0].at = At::FromMs), plans);
|
||||
assert_eq!(with(&|i| i.plans.push(i.plans[0].clone())), plans, "two plans at one instant");
|
||||
assert_eq!(with(&|i| i.plans[0].wanted.push(([1; 32], 0))), plans);
|
||||
assert_eq!(with(&|i| i.plans[0].sources.clear()), Err(Refusal::Input(super::super::SOURCES_INVALID)));
|
||||
assert_eq!(
|
||||
with(&|i| {
|
||||
i.plans[0].sources.clear();
|
||||
i.arrivals.push(i.arrivals[0]);
|
||||
}),
|
||||
Err(Refusal::Input(super::super::SOURCES_INVALID)),
|
||||
"a plan's reason before the arrivals'"
|
||||
);
|
||||
assert_eq!(with(&|i| i.arrivals[0].at_ms = LAST_INSTANT_MS + 1), arrivals);
|
||||
assert_eq!(with(&|i| i.arrivals.push(i.arrivals[0])), arrivals, "one object twice");
|
||||
assert_eq!(with(&|i| i.arrivals.insert(0, Arrival { group: [2; 32], index: 0, at_ms: 8 })), arrivals, "instants that decrease");
|
||||
assert_eq!(with(&|i| i.arrivals.insert(0, Arrival { group: [2; 32], index: 0, at_ms: 7 })), Ok(()), "one instant twice");
|
||||
}
|
||||
}
|
||||
|
|
@ -1,13 +1,13 @@
|
|||
//! `wai.delivery.timeliness` (extensions/timeliness.md): the committed corpus
|
||||
//! `live-conformance/timeliness/` regenerates byte for byte from its fixed seeds, holds no other
|
||||
//! file, every case in it gets the outcome it states, §9 lists exactly the reasons the reference
|
||||
//! reports and §6.3 exactly its outcomes, jwp-receipts §3.1 lists the class codes at their steps,
|
||||
//! and delivery-telemetry §5.1 lists the switch reason of §8.6.
|
||||
//! reports, §6.3 exactly its outcomes and §7.5.3 its run's actions, jwp-receipts §3.1 lists the
|
||||
//! class codes at their steps, and delivery-telemetry §5.1 lists the switch reason of §8.6.
|
||||
#![cfg(feature = "timeliness")]
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::PathBuf;
|
||||
use wai::timeliness::{corpus, CODES, OUTCOMES};
|
||||
use wai::timeliness::{corpus, scheduler, CODES, OUTCOMES};
|
||||
|
||||
fn wai_dir() -> PathBuf {
|
||||
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("..")
|
||||
|
|
@ -74,15 +74,25 @@ fn the_extension_lists_the_outcomes() {
|
|||
assert_eq!(listed(section(&text, "### 6.3 Lines")), want, "§6.3's table and OUTCOMES differ");
|
||||
}
|
||||
|
||||
/// §7.5.3 states each action a run decides, in backticks.
|
||||
#[test]
|
||||
fn the_extension_states_the_runs_actions() {
|
||||
let text = read("extensions/timeliness.md");
|
||||
let steps = section(&text, "#### 7.5.3 Decisions");
|
||||
for action in scheduler::ACTIONS {
|
||||
assert!(steps.contains(&format!("`{action}`")), "§7.5.3 does not state {action}");
|
||||
}
|
||||
}
|
||||
|
||||
/// jwp-receipts §3.1's table lists the class's codes: its rules at step 4, its checks against the
|
||||
/// group, its objects and their fallbacks at step 6.
|
||||
/// group, its objects and their rungs at step 6.
|
||||
#[test]
|
||||
fn jwp_receipts_lists_the_class_codes() {
|
||||
let text = read("extensions/jwp-receipts.md");
|
||||
let class: Vec<&str> = CODES.iter().copied().filter(|c| c.starts_with("timeliness_") && *c != "timeliness_contradicted").collect();
|
||||
assert_eq!(class.len(), 13, "the class's codes");
|
||||
assert_eq!(class.len(), 14, "the class's codes");
|
||||
for (k, code) in class.iter().enumerate() {
|
||||
let step = if k < 5 { 4 } else { 6 };
|
||||
let step = if k < 6 { 4 } else { 6 };
|
||||
assert!(text.contains(&format!("| {step} | `{code}` |")), "jwp-receipts §3.1 does not list {code} at step {step}");
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue