open-standards/wai/extensions/staged-delivery.md

WAI Extension: Staged Delivery

Status: Draft. A base first, refinements after, each its own object, bound to what it refines by content hash. Every stage is presentable, every prefix of a stage set states its determinism tier, and a refinement that never arrives leaves the stages below it intact. Reference impl: wai-rs — staged::{StageChain, StageSink}, staged::{wir1, wcp1, side, predict, wir1_arith, tier, schedule}, staged::claim::StageObserver, container::Refines, codecs::BASE_ADMISSION; the wai stage tool; the corpus staged-conformance/. The level form (§6.2) is int_field::FieldStages, with its corpus in volumetric-conformance/fields/. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

1. Scope and model

A stage set is a base and layers. The base is any envelope a sink decodes under SPEC §4 or §7.1. A layer is its own WAI1 envelope whose capability is refinement-only (SPEC §5, Refinement capabilities) and which names, by content hash, the base and the envelopes it refines.

A sink holds, for every unit of the base's output, a stage (0 after the base decodes) and a producer: the content hash of the envelope that produced the unit's current output, the base at stage 0. Layers apply per unit, and every stage is presentable. A layer of the conditioned form (§6.3) also reads side planes: pictures the sink holds exactly besides the base, named in the layer's refines (§3.1). A base never names its layers, so a layer MAY be produced after its base, by the base's publisher or by a party the sink accepts (§10.1).

2. Terms

• Content hash: jwp-receipts §2.1, BLAKE3("moq-jwp:object\x01" ‖ envelope_bytes) over the envelope bytes exactly as received, written as 64 lowercase hexadecimal digits.
• Unit: the smallest presentable piece of a base's output, fixed by the base's capability: one frame of video; the whole picture of an image; the whole clip of audio; the whole attribute tensor of a splat cloud; the whole field of an integer neural field.
• Span: a contiguous run of a base's units.
• Producer: for a unit, the content hash of the envelope that produced its current output.
• Refinement-only capability: a capability registered with that role (SPEC §5, §8).
• Side plane: an RGB8 picture a layer of the conditioned form reads besides the unit's current output (§3.1): the output of an object the sink decoded, or a frame the sink rendered itself.

3. The refines member

A WAI1 manifest whose model_requirement.capability is refinement-only MUST carry refines, and its model_requirement.fallback MUST be null. A manifest whose capability is not refinement-only MUST NOT carry it. A sink routes an envelope by its capability: one whose capability is refinement-only is a layer, which §4 processes; any other is a base, dispatched by SPEC §4, and a refines it carries is ignored. Likewise a WAI2 rendition is a layer only when its capability is refinement-only (SPEC §7.1).

"refines": { "base": "<64 hex>",
             "prev": "<64 hex>" | [ {"producer": "<64 hex>", "count": 4}, ... ],
             "layer": 2,
             "span": { "from": 0, "count": 4 } }

member	presence	meaning
base	REQUIRED	Content hash of the stage set's base.
prev	REQUIRED	The producer each unit the layer covers is expected to have. A string means every such unit. An array of runs gives, in unit order, count consecutive units whose producer is producer.
layer	REQUIRED	Integer 1–255: the stage this layer brings each unit to.
span	OPTIONAL	from (integer ≥ 0) and count (integer ≥ 1): the units the layer covers. Absent, or null (which is the same as absent, as SPEC §3.1 reads an OPTIONAL member): every unit of the base.
side	OPTIONAL	The side planes a layer of the conditioned form reads (§3.1). Absent, or null: none.

Rules:

• Hashes MUST be 64 lowercase hexadecimal digits (in a WAI2 entry they are component ids; SPEC §7.1, Refinement components).
• When layer is 1, every producer prev names MUST be base.
• A run array MUST NOT contain two adjacent runs with the same producer. A layer whose units have one producer MUST write prev as a string, so an array of one run is malformed.
• With a span, the run counts MUST sum to its count. The span MUST lie within the base's units, and with no span the run counts MUST sum to the base's unit count; a sink checks these once it holds the base (§4.1 step 2).
• from + count MUST fit in 32 bits, and so MUST each count.

Any other JSON type (an integer is written as SPEC §2 defines one), or a value outside these rules, makes refines malformed (refines-malformed); the envelope still parses. A member of refines this revision does not define is ignored (SPEC §8).

3.1 Side planes

side is read for a capability whose registration says its layers read side planes: in this revision wai.image.int_cond_refine (§6.3). For any other capability the member is ignored. Present and not null, it is an array of 1 to 4 entries, in the order the layer's payload reads them, each one of:

• a content hash (64 lowercase hexadecimal digits): the output of the object whose content hash it is, an envelope the sink decoded under SPEC §4 and holds. A G-buffer of rendered content (depth, normals, albedo, coded into the three channels as its producer chooses), or any other picture both sides hold, is carried this way as an exact component of its own;
• an object of exactly the members object (a content hash), renderer (the string wai-splat-render:i1) and pose (a pose of splat-exact §2): the frame that renderer gives at that pose (splat-exact §5) for the splat set the sink decoded from object through wai.splat.int_codec. The sink renders it itself, so the plane costs no bytes on the wire.

No entry appears twice: two content hashes are the same entry when they are equal, two render entries when their object is equal and their poses are the same pose (splat-exact §2). Any other value makes refines malformed (refines-malformed).

A sink holds an entry's plane when it holds the object an entry names, decoded, and, for a render entry, the object was decoded through wai.splat.int_codec and the sink has rendered the frame at the pose; a render the renderer refuses (splat-exact §10) is no plane. A plane is exact when the object it comes from was decoded through its manifest's model_requirement.capability, not a fallback, and the object's base step has the tier decode-equivalence (§5): the bytes of the plane are then the same on every conforming sink. A plane is read as one RGB8 picture: the object's output when it is one picture, the frame of a render.

A layer that reads side planes travels as a WAI1 object. A WAI2 refinement component carries no side (SPEC §7.1, Refinement components), and a packer refuses to pack a WAI1 layer that carries one: its side planes are objects of their own, not components of the base's envelope.

4. Applying layers

4.1 Algorithm

A sink that implements this extension MUST process each received envelope R whose capability is refinement-only as follows. A sink routes by capability (§3), so an envelope whose capability is not refinement-only reaches step 1 only where a caller offers it to this algorithm as a layer; step 1 refuses it, as it refuses a refinement-only envelope whose fallback is not null (capability-not-refinement). Steps 1–6 decide for the whole envelope; steps 7–10 decide for each unit u the layer covers. The first step that fails is the reason for the envelope (1–6) or for the unit (7–10).

1. refines is present and well formed (§3) (refines-malformed). R's capability is refinement-only, and its model_requirement.fallback is null (capability-not-refinement).
2. The sink has decoded a base whose content hash is refines.base. Otherwise it MAY hold R until its bound (§4.3), and then discards it (base-absent). With the base, refines is checked against the base's unit count (§3) (refines-malformed).
3. The base was decoded through its manifest's model_requirement.capability, or through the WAI2 component whose exploded envelope (SPEC §7.1) has the content hash refines.base, and not through a fallback (base-via-fallback).
4. The capability the base was decoded through is admitted by R's capability (SPEC §5, Base admission), at the tier that table requires of it (base-not-admitted). Then, where R's capability reads side planes, each entry of side in order (§3.1): the sink holds its plane (side-absent), and the plane is exact (side-not-exact).
5. Where the sink verified a delivery receipt for the base, it has verified one for R as §10.1 requires (refinement-unattested).
6. The payload is well formed for R's capability, in this order: it parses in its staged form (Appendix A; Appendix B for the conditioned form) (payload-malformed); its unit count is the number of units the layer covers, and, for the conditioned form, its n_side the number of entries side names (payload-malformed); its declared cost (Appendix A.5, B.5) is within the limit the sink applies, if it applies one (cost-over-limit); retaining R keeps the chain within its budget (§4.5) (chain-over-limit); and every unit's stream decodes to its last symbol and ends canonically (payload-malformed). A payload malformed for one unit is malformed for all. No unit's shape is read before step 9. A payload of the level form (§6.2) parses as neural-fields §7.2 states, is one unit, and has no stream and no declared cost.
7. Let P(u) be the producer prev gives for u. u is conflicted when it is frozen, or when an envelope with different content, the same base and the same layer, whose producer for u is also P(u), has passed step 8 for u, earlier (before the base's chain was dropped included, §4.4) or in the same batch (§4.4) (layer-conflict). A conflicted unit is frozen: the sink MUST apply no further layer to it, except the replay of its own record after its base's chain is dropped and the base decoded again (§4.4); it keeps the output it had (a layer applied to it before the conflict stays applied), and the sink MUST report the conflict.
8. u is ready when its stage is layer − 1 and its producer is P(u). A unit whose stage is below layer − 1 keeps R held for it, and is evaluated again when its stage advances; at the bound R is discarded for that unit (layer-gap). Any other unit is not ready (prev-mismatch).
9. R's payload reads u's current output, and each side plane step 4 found, in the shape its staged form needs (geometry-mismatch).
10. Under a live schedule, R arrived at least lead_ms before u's presentation time (§8) (deadline-missed).

The sink then computes, for every unit that passed, the next stage by R's staged form (§6), and sets the unit's stage to layer and its producer to R's content hash. Application to one unit is atomic. A sink MUST NOT present a unit's output from a layer that failed for that unit, and MUST NOT present a partly applied unit.

4.2 Reasons

step	reason	scope	effect
1	refines-malformed, capability-not-refinement	envelope	discarded
2	base-absent	envelope	held, then discarded
2	refines-malformed	envelope	discarded
3	base-via-fallback	envelope	discarded
4	base-not-admitted	envelope	discarded
4	side-absent, side-not-exact	envelope	discarded
5	refinement-unattested	envelope	discarded
6	payload-malformed, cost-over-limit, chain-over-limit	envelope	discarded
§4.5	chain-over-limit, for a base	the base's envelope	the base is not added; its layers stay held
7	layer-conflict	unit	not applied; the unit is frozen
8	prev-mismatch	unit	not applied to the unit
8	layer-gap	unit	held for the unit, then not applied
9	geometry-mismatch	unit	not applied to the unit
10	deadline-missed	unit	not applied to the unit

The list is closed. Two conforming sinks that receive the same envelopes in the same order, at the same times, with the same receipts, hold bound, cost limit, chain budget and schedule, report the same reasons per envelope and per unit and reach the same stages and producers. The order of arrival changes the outcome only through the chain budget at step 6 (§4.5), step 7, the hold bound at step 8 (§4.3: what is refused on arrival and what expires) and step 10.

4.3 Holding

A sink MUST declare the bound to which it holds envelopes it cannot yet apply, as a count of envelopes and a duration from arrival. An envelope held past the duration is discarded with its hold reason (base-absent, or layer-gap for each unit it is held for). An envelope that would exceed the count is discarded on arrival with the reason it would have been held for. A unit refined by a held envelope that is released within the bound, and that the chain budget admits, reaches the same output as one refined in order.

The bound holds envelopes for their base only. A layer whose side plane the sink does not hold is discarded (§4.1 step 4, side-absent), so a sink acquires a layer's side planes before it offers the layer: it decodes the object, or renders the frame, first, and a layer it receives again once it holds them is processed as any arrival.

An envelope held for its base and received again is held once. If it comes with stronger evidence of its delivery receipt (§10.1: a receipt verified over its bytes is stronger than one verified by inclusion alone, which is stronger than none), the sink keeps the stronger evidence for steps 5 and 10; its hold still runs from its first arrival.

4.4 Batches

When the base an envelope names is decoded, the envelopes held for it are released together; when a unit's stage advances, the envelopes held for that unit are released together. Each envelope released, or offered on arrival, passes steps 1–6, and for each unit step 7 against the envelopes that passed step 8 earlier and step 8, alone. The (envelope, unit) pairs that are ready then form one batch: step 7 compares every pair of the batch with the others, and only then do steps 9 and 10 and application run. So two conflicting layers released together are both refused and the unit frozen, whichever arrived first; a layer that conflicts with one that passed step 8 earlier is refused, and the unit frozen at the stage it has. When a unit is frozen, every envelope held for it is refused for it (layer-conflict). Applying a batch can release another, which is evaluated the same way.

A sink that decodes again a base whose chain it holds keeps the chain: re-decoding a base does not reset its units' stages, producers or frozen state, and releases and reports nothing. If the sink has verified the base's delivery receipt since, the base is attested from then on (§10.1). Attestation belongs to the base's content hash: a base once attested stays attested, held or dropped, and a decode that carries no receipt does not undo it; a receipt the sink verifies before it decodes the base makes the base attested when it is decoded. Only forgetting the base (§14) ends it. A sink MAY drop a base's chain when it no longer presents the base. Every envelope held for the chain's units is then discarded (base-absent, reported once for each), and an envelope for that base is held as for any base the sink has not decoded. The sink keeps each unit's record: the producers of the stages it reached, the envelope that passed step 8 at its last stage without being applied, and whether it is frozen. A record is empty when the unit reached no stage, nothing passed step 8 for it and it is not frozen; the sink keeps only the others, and nothing of a base all of whose records are empty and that is not attested. When it decodes the base again, each unit starts at stage 0 with its record. Step 7 compares a layer with the recorded producers, so a unit advances again only along them: a layer whose key a recorded stage holds with other content conflicts, and the unit freezes. A frozen unit stays frozen: it may replay its own recorded producers, in order, up to its recorded stage and never past it, and every other layer is refused for it (layer-conflict). So it presents its base's output or the output of one of its recorded producers, never any other.

A sink MUST declare a record budget, in bytes, for what it keeps of the bases it does not hold: their records, and the attestation of a base dropped while attested or attested before it was decoded (§10.1). A kept base counts 32 bytes, and each record it keeps 8 bytes, 32 for each recorded stage and 32 for an envelope that passed step 8 without being applied. When what it keeps passes the budget, the sink forgets whole bases, the one whose record or attestation it wrote longest ago first, until it is within it. A forgotten base is as one the sink never decoded and never attested (§14).

These counts are the interoperable measure: every conforming sink counts the same bytes, so two sinks with the same record budget keep and forget the same bases. What an implementation allocates for each counted byte is its own, and it states that ratio beside its record budget.

4.5 Chain budget

A sink MUST declare a budget, in bytes, for the chain of each base it decodes. The chain's state is the sum of:

• for every unit, the bytes of its current output (1 per u8 element, 2 per i16 sample), and 32 bytes for each stage its record holds (the producers step 7 compares: its stages above 0, or, for a base decoded again, the stages it reached before, §4.4);
• for every layer the chain retains, its payload's bytes and its declared working bytes (Appendix A.5), and, for each unit it covers and has not been applied to, one unit's output (its declared output over its units) and 32 bytes (the stage it would add).

A chain retains a layer from the moment it passes step 6 to the end of the offer, release or expiry after which no unit is held for it; the sink then drops it. At step 6, after the cost limit, a layer whose payload, declared output and working bytes and 32 bytes for each unit it covers, added to the state, pass the budget is refused (chain-over-limit) before any stream is decoded. An envelope the chain still retains, received again, adds nothing. Applying a layer to a unit replaces its output with one, and adds a stage, that the layer's retention counted, and that unit's share leaves the layer's: an application lowers the state by the unit's old output (and by 32 bytes more where it replays a recorded stage). The state rises only when a layer is admitted, so it never passes the budget, and the bytes a chain holds never pass the state. A side plane (§3.1) is held by the sink, not by a chain, and is no part of a chain's state; a sink bounds the objects it holds for side planes as it bounds any object it decodes.

A chain starts with its base. A base whose units' output, with 32 bytes for each stage the records the sink keeps of it hold (§4.4), passes the budget is not added: the sink reports chain-over-limit for the base's envelope, keeps no chain for it, leaves the records it keeps of it as they were, and holds the layers that name it as for a base it has not decoded (§4.3). A receipt verified with the base is recorded all the same (§10.1). The base's own decode is bounded before it starts: where its capability declares a cost before decoding (extensions/integer-payloads.md §7), a sink applies its cost limit to that declared cost first, and SPEC §4 treats a base past it as one this sink does not decode (cost_over_limit), so dispatch moves on to the fallback.

5. The tier of a stage, and the stage hash

• The base step: decode-equivalence when the capability the base was decoded through has the registry class StandardDefined, or NeuralIntegerExact with a pin that resolved (SPEC §3.1) and declares decode-equivalence, or is wai.video.av1 over a grain-free stream whose frames are the reference decode's or a platform decoder's whose probe matched (platform-decode §6.2); entropy-consistency when the class is NeuralEntropyExact; otherwise none.
• A layer step: the tier its registration gives (§6); decode-equivalence for the residual form and the level form, and for the conditioned form, whose side planes are exact (§4.1 step 4).

A unit's tier is the weakest step of its chain, composed as derived-tracks §3 rule 1. A layer acts on reconstructed output, so entropy-consistency does not pass through it (derived-tracks §3 rule 2): a unit over such a base holds no tier once a layer applies.

The stage hash of a unit is the SHA-256 of its output in the canonical buffer of SPEC §7 Digests: RGB8 for a picture or a frame; the samples as signed 16-bit little-endian integers, channels interleaved, for audio; the attribute tensor's bytes, plane after plane, for a splat; for a field, the RGB8 frame of its render at its own view (neural-fields §7.4). The stage hash of several units is the SHA-256 of their 32-byte stage hashes concatenated in unit order, as that section defines a clip digest. A stage hash MUST NOT be published, compared as a conformance value, or sealed in a receipt unless every unit it covers holds decode-equivalence.

6. Staged forms

A refinement-only capability's registration fixes its staged form, payload format, admitted bases (SPEC §5, Base admission), tier and truncation points.

6.1 Residual (WIR1)

wai.image.int_refine, wai.video.int_refine, wai.audio.int_refine and wai.splat.int_refine carry a WIR1 payload (Appendix A): one rANS stream per unit. For each element of a unit, the next stage is clamp(pred + r · q). pred is the payload's predictor applied to the unit's current output; r is the decoded residual; q is the payload's step; and the clamp is to the element type's range. Every operation is on integers of at most 64 bits.

capability	unit	element	predictors
wai.image.int_refine	the picture, RGB8	u8	identity, up2
wai.video.int_refine	one frame, RGB8 (over a wai.video.av1 base, the frame converted by platform-decode §6.3)	u8	identity, up2
wai.audio.int_refine	the clip, i16 PCM, channels interleaved	i16	identity, rate
wai.splat.int_refine	the attribute tensor	u8	identity

A residual layer reads no parameter set: everything its decode reads is in its payload and in the unit's current output. The base's parameter set was checked when the base was decoded (SPEC §3.1, check 5 among them), and nothing a layer reads needs the check again. Tier: decode-equivalence. Every layer boundary is a truncation point.

6.2 Level (WIL1)

wai.field.int_refine carries a WIL1 payload (neural-fields §7.2): one grid level of an integer neural field, over the base wai.field.int_grid. The unit is the whole field. The next stage is the field with that level applied as its next level (neural-fields §7.3): with the field holding levels 0 to h − 1, the payload's level is h, of the features, table size and side the field declares for it, or §4.1 step 9 fails (geometry-mismatch). Layer k of a field whose payload carries K levels carries level K + k − 1. A level reads no parameter set: everything its decode reads is in its payload and in the field below. Tier: decode-equivalence. Every layer boundary is a truncation point.

6.3 Conditioned residual (WCP1)

wai.image.int_cond_refine carries a WCP1 payload (Appendix B): an integer network and a residual. Its unit is the picture, RGB8, and the next stage is three steps: the base prediction, the unit's current output (identity) or its up2 at twice the resolution, as §6.1's up2; the network's prediction, the base prediction plus a correction an integer network computes from it and the side planes the layer names (§3.1), clamped to [0, 255]; and the residual, decoded over the network's prediction as §6.1 decodes a unit with the identity predictor. Every operation is on integers of at most 64 bits.

One mechanism carries two uses. Over rendered content, the side planes are the G-buffers both sides hold: the exact components the origin sends once, or the frame the sink renders itself from a splat set at a pinned pose, and the network turns them and the base into a prediction the residual corrects. Over a base at a lower resolution, the layer is the detail stream of a dual stream: the up2 base prediction is the reference, the network the detail conditioned on it, the residual what remains. A layer with no side reads no side plane.

The sink composes the stage from objects of their own: the base, each side object and the layer, each named by its content hash and covered by its own receipt (§10.1). It computes the stage itself, and the stage hash (§5) is the digest a stage claim states (§10.2) and a sink that presents the picture checks.

Its network is carried in its payload and it reads no parameter set; the network runs through one graph of the integer operator set (Appendix B.6), and every payload that parses meets acc64. Its admitted bases are wai.image.int_refine's (SPEC §5, Base admission). Tier: decode-equivalence, given exact side planes (§4.1 step 4). Every layer boundary is a truncation point.

6.4 Other staged forms

This revision registers the residual form (§6.1), the level form (§6.2) and the conditioned residual (§6.3). A refinement-only capability of another staged form (the continuation of a world's state, an enhancement layer carried as its own object, a byte prefix of a progressive codestream, the further codebooks of a residual-vector-quantised code) is registered by a revision that fixes its form, payload, admitted bases, tier and truncation points. Where the form decodes a prefix of a base codec's own codestream, that revision cites the clause of the codec's standard that defines the decode of such a prefix, and its reference sink decodes one.

6.5 Truncation points

A refinement capability declares truncation points only where its staged form defines the decode of a prefix. For the residual form, the level form and the conditioned residual that is every layer boundary.

7. Deferral

A layer MAY be published after its base, served only on request, or never. A sink MUST present the current stage of a unit without waiting for a layer it has not been told to expect. A later revision of a refinement is a new chain from the same base, distinguished by prev from layer 2 on. Every layer 1 names the base as its producer, so prev cannot distinguish two revisions of layer 1: they are two envelopes of the same base, layer and producer, and conflict (§4.1 step 7).

8. Live deadlines

Under a live schedule, every unit has a presentation time, given by the base's timing and the target latency the sink applies, and every layer has a lead, lead_ms: the time before a unit's presentation by which the layer must have arrived, and been verified under §10.1, to be applied to it. A carriage may declare both: on MoQ, the tracks' target latency and each layer track's lead_ms (moq-streaming-format §4.1). A sink MUST NOT delay presenting a unit's current stage to wait for any layer. A layer that arrived less than lead_ms before a unit's presentation time is not applied to that unit (deadline-missed). It MAY be applied to units of its span that are not yet due. A still image MAY be refined after it is presented. A sink MUST count every unit presented below the stage its expected layers would have reached on time. Verifying a layer's receipt (§10.1) adds the time to receive and verify that receipt to the layer's arrival, which sets the least lead_ms that can be met: over a base whose receipt the sink verified, a layer counts as arrived when the sink verified its receipt (§4.3).

A stage set split across bases (a base for each group of frames) has one clock: a base whose units are the set's units from position k on presents its unit i at the set's position k + i, and a sink gives each base its first position.

9. Carriage

A layer is a WAI1 envelope, carried and stored as any envelope is, or a WAI2 component with role: "refinement" (SPEC §7.1, Refinement components). A component and its exploded envelope (SPEC §7.1) are the same object for every rule that names an object by content hash, so a layer reaches the same stages whichever way it travelled. Two bindings carry each layer as an object of its own: moq-streaming-format §4.1, a track for each layer index, and http-delivery §10.4, a resource for each layer.

10. Receipts and claims

10.1 Attestation of layers

Each layer is an object under jwp-receipts. A sink that verified a delivery receipt for a base MUST NOT apply a layer unless it has verified a delivery receipt for the layer, through that extension's §3 step 6, over the layer's envelope bytes as received (delivered-verified). That receipt must be signed by the base's signer, or by a key the sink holds with the authority delivery under the same origin (jwp-receipts §7.5). A receipt verified without the layer's envelope bytes is inclusion-verified (jwp-receipts §3, Verified by inclusion): it shows that a publisher signed an object with that content hash, and it never authorizes an application. A layer on a track of its own has its receipts on a chain of their own, one per publisher key and track (jwp-receipts §3 step 3), so a sink verifies the layers it subscribes to without the receipts of the ones it does not. A content hash in refines binds a layer to a base, not to its publisher. A sink that verifies a base's delivery receipt after it decoded the base applies step 5 to every envelope that arrives after it; envelopes it admitted before keep the decision they had. The verification belongs to the base's content hash and holds across the base's chain being dropped and the base decoded again (§4.4).

10.2 Claims

A party that observed a stage set MAY sign a stage claim about its base (jwp-receipts §2.4.10): the stage it asked for, the stage it was promised, the stage the objects it verified over their bytes reach, the stages it presented on time, the tier and stage hash of what it presented, and every layer object with what became of it and what it verified of the object's receipt. A layer object it never received but whose receipt it verified by inclusion is listed as dropped with the evidence inclusion-verified; one that arrived after its units were due is late. A claim covers what its signer observed since it last forgot the base: a base the sink forgot (§4.4, §14) is, decoded again, one it never decoded and never attested, so its observation starts over, and nothing applied to it before is part of the claim. An envelope the sink still holds for the base when it forgets it stays observed, with no decision, and is stated as the sink then decides it. A claim never spans two such epochs. The claim records what its signer observed, not that content was delivered.

A relay that did not forward a layer object MAY sign a delivery-gap claim about it (jwp-receipts §2.4.11), beside the object's receipts, which it forwards all the same (jwp-receipts §4.1).

11. Counting bytes

A figure of delivered bytes for a stage set:

• MUST count the bytes of the WAI1 and WAI2 envelopes as delivered (SPEC §2: magic, length prefixes, manifest and payload);
• MUST state separately the receipt, index and transport framing bytes, and the parameter-set bytes a sink fetched;
• MUST NOT count a text re-encoding of a payload (JSON, base64) in place of the payload's registered binary format.

A figure over a payload in any other form is not a staged-delivery byte figure.

12. Conformance

A conforming sink of this extension implements §4, §5, §8 and §10.1, and every staged form whose capabilities it advertises. The corpus staged-conformance/ holds the cases: for each, a conforming sink reports the stated reason per envelope and unit, and reaches the stated stage, producer, tier and stage hash per unit. Its README.md states the cases, what each base's stage 0 is checked against, and the independent verifier staged_verify.py. The cases of the conditioned form, with the side objects a sink holds and the frames it renders, are its cond/, which a sink that advertises wai.image.int_cond_refine reproduces.

13. Earlier layered carriage

A prototype MoQ binding in this repository carries a layer index in an object header (mrl_layer, with mrl_total_layers and a subscriber's max_mrl_layer). For an object that is a WAI1 envelope with refines, a sink reads the layer from refines.layer, which sits inside the envelope and so inside its content hash, and ignores the header. On a track of packaging wai, a subscriber caps its stage by the layer tracks it subscribes to, and a relay reads no header or envelope to cap it (moq-streaming-format §4.1).

14. Security considerations

• A layer is not authenticated by its refines; §10.1 authenticates it.
• Deferral lets anyone mint a chain for a public base hash. §10.1 and §4.1 step 7 bound what a sink presents. A conflict between two attested chains is evidence that an authorized signer signed both, and the sink reports it; a stage claim (§10.2) records it as conflicts.
• A hold bound (§4.3), a cost limit (Appendix A.5) and a chain budget (§4.5) are resource limits. A layer's header declares its output, its symbols and its working bytes before any symbol is decoded, so a sink refuses a layer past its limit or its budget without decoding it. A base of an integer codec declares its cost the same way (extensions/integer-payloads.md §7), so a sink refuses a base past its limit before decoding it, and one whose output passes the chain budget before keeping it (§4.5): a few bytes of envelope can declare a large output, and only the declared cost bounds the work. A sink's own work for a layer is linear in the layer's payload bytes and declared symbols, up to a factor logarithmic in its units: it parses the payload once, at step 6; decides each unit the layer covers once for each offer or release of the layer; finds a unit's producer in prev by bisection over its runs; and compares a unit's key at step 7 with the producers of that unit's stages. Beyond the chain's state, it keeps a fixed amount per unit of the base and per unit held.
• A side plane is named by the content hash of the object it comes from, so a layer reads the bytes it was made against or none (§3.1). A plane that is not exact is refused (side-not-exact), so a sink never presents a stage whose prediction rests on an output another sink computes differently. A render entry costs the sink a render (splat-exact §5) at a pose of at most 2^20 pixels; a sink renders only for a layer it accepts to process. A layer's declared cost counts its network's working bytes and multiply-accumulates (Appendix B.5), so a sink refuses a network past its limit before it runs.
• An encrypted layer (confidentiality) binds to the content hash of its base's envelope as delivered, ciphertext included, and is decrypted before §4.1 step 6.
• A sink keeps the record of a base whose chain it dropped (§4.4), so a base decoded again cannot be refined along other producers than it had; the record counts in the chain's state when the base is decoded again (§4.5), and, while the base is not held, in the sink's record budget (§4.4). A sink that forgets a base, by StageSink::forget_base in the reference or because its record budget is full, treats it as never decoded and never attested: a layer that conflicted with the forgotten history can apply, and step 5 no longer requires a receipt for it. A sink sizes its record budget for the bases it may decode again.
• Where the base is attested (§10.1), a frozen unit presents only its base's output or the output of one of the producers it recorded, replayed in order (§4.4): a party that can sign conflicting layers can stop a unit's refinement, and cannot make it present anything else. Over a base whose receipt the sink did not verify, anyone who learns the base's hash can mint a layer, and the first one ready for a unit applies to it.

Appendix A. WIR1

A.1 Layout

Every multi-byte field is little-endian; offsets are in bytes.

off	size	field	rule
0	4	magic WIR1
4	1	version	1
5	1	elem	1 = u8, 2 = i16
6	1	predictor	0 identity, 1 up2, 2 rate
7	1	table_mode	0 one table for every unit, 1 one per unit
8	4	n_units	≥ 1
12	4	d0	≥ 1: rows (picture), samples per channel (audio), planes (splat)
16	4	d1	≥ 1: columns, 1, the planes' width
20	4	d2	≥ 1: channels (3 for RGB8), channels, the planes' height
24	4	rate_num	predictor 2: the output rate, ≥ 1; otherwise 0
28	4	rate_den	predictor 2: the rate of the stage below, ≥ 1; otherwise 0
32	4	q	1..=65535
36	4	offset	i32: the symbol offset of every table
40	2	cdf_len	3..=4097
42	2	reserved	0
44	T·cdf_len·4	the tables	T = 1 or n_units; each a table of integer-payloads §2
…	n_units·4	stream lengths	u32 each, a multiple of 4, ≥ 8
…	Σ	the streams	one per unit, in unit order; nothing follows

Each stream is decoded as integer-payloads §2 decodes a rANS stream (precision 16, the bypass escape of 4-bit groups), against the unit's table at offset.

A.2 The unit kind

A capability fixes what its units hold (§6.1), and so the fields a payload may carry:

• RGB8 (wai.image.int_refine, wai.video.int_refine): elem 1, predictor 0 or 1, d2 = 3;
• PCM (wai.audio.int_refine): elem 2, predictor 0 or 2, d1 = 1;
• attribute tensor (wai.splat.int_refine): elem 1, predictor 0.

A.3 Order of the checks

A parser checks, in this order, and refuses with the first that fails:

1. the magic (bad_magic);
2. the version byte: present (stream_length), and 1 (version);
3. the 44-byte header: present (stream_length);
4. the reserved field (reserved_nonzero);
5. the header's fields, in the order elem, predictor, table_mode, n_units, d0/d1/d2, the rate fields, q, cdf_len; then the unit kind's rules (A.2) (field_range);
6. the caps, in this order (too_large): a unit's d0·d1·d2 ≤ 2^26; n_units ≤ 2^20; n_units·d0·d1·d2 ≤ 2^30;
7. the tables, one at a time, each present (stream_length) then checked (cdf_malformed); the stream lengths, present (stream_length), each a multiple of 4 and at least 8 (stream_length); the streams, present (stream_length);
8. nothing after the last stream (trailing).

Every size is formed in 64-bit arithmetic with overflow checked, and a size past 64 bits is past the caps, so a 32-bit sink refuses exactly what a 64-bit one does. A parser reads tables and lengths from the bytes the payload holds and allocates nothing a payload's bytes do not justify.

A.4 Decode

For each unit u the layer is applied to, with the unit's current output of shape (e0, e1, e2):

1. The shape must be the one the predictor reads (geometry_mismatch), before any symbol is decoded: (d0, d1, d2) for identity; (⌈d0/2⌉, ⌈d1/2⌉, d2) for up2; (n′, 1, d2) with ⌊n′·rate_num/rate_den⌋ = d0 for rate.
2. For element i in raster order (i = (a·d1 + b)·d2 + c), decode the residual r from u's stream. A stream that runs out during a symbol's decode is refused at that symbol (stream_exhausted), and nothing after it is decoded. The element is clamp(pred(i) + r·q), to [0, 255] for u8 and [−32768, 32767] for i16. r lies in [−2^31, 2^31 − 1] and |r·q| < 2^47, so every step fits i64.
3. After the unit's last element, the stream ends canonically (integer-payloads §2) (stream_noncanonical).

Predictors, with prev the unit's current output:

• identity: pred(i) = prev(i).
• up2: prev is h′ × w′ × c with h′ = ⌈d0/2⌉, w′ = ⌈d1/2⌉ and c = d2, so odd sizes are allowed. For output (y, x, ch): iy = y >> 1, fy = y & 1, ix = x >> 1, fx = x & 1, and p(Y, X) = prev[min(Y, h′ − 1), min(X, w′ − 1), ch]. The prediction is p(iy, ix) when both are 0; (p(iy, ix) + p(iy, ix + 1) + 1) >> 1 when only fx is 1; (p(iy, ix) + p(iy + 1, ix) + 1) >> 1 when only fy is 1; and (p(iy, ix) + p(iy, ix + 1) + p(iy + 1, ix) + p(iy + 1, ix + 1) + 2) >> 2 when both are. This is the half-pel arithmetic of integer-payloads §3.
• rate: prev holds n′ samples of c = d2 interleaved channels. For output sample i and channel ch: s = i·rate_den, lo = s div rate_num, ρ = s mod rate_num, hi = min(lo + 1, n′ − 1), a = prev[lo·c + ch], b = prev[hi·c + ch], and the prediction is rdiv(a·rate_num + (b − a)·ρ, rate_num), where rdiv(n, d) = sign(n)·⌊(2|n| + d) / 2d⌋ rounds half away from zero. This is a + (b − a)·ρ/rate_num rounded half away from zero, the linear interpolation at source position i·rate_den/rate_num, in exact integer arithmetic.

This decode is the stage operators of integer-operators §5E, with no graph (§13.5 there): each unit's stream is E1 rans64_decode with t4097, lenient and canonical; the up2 prediction is E11 up2_predict and the rate prediction E12 rate_predict; and the element is E9 residual_step on the element type's range. On both ranges E9's 32-bit form holds. The reference decoder decodes through them.

A.5 Declared cost

A payload's header declares, before any symbol is decoded: its output, n_units·d0·d1·d2 elements of 1 byte (u8) or 2 bytes (i16); its symbols, n_units·d0·d1·d2, one entropy-decode step each; and the most bytes its decode holds at once beyond the payload and the stages below, its parsed tables (4·T·cdf_len), the index of its streams (8·n_units) and one unit's output while it is built. It runs no network. A sink that applies a limit compares this cost with it after the parse and before any stream is decoded (§4.1 step 6, cost-over-limit), and counts it in the chain's state while it retains the layer (§4.5).

A.6 Refusal codes

The list is closed, and its codes are those integer-payloads §8 uses for the same faults: bad_magic, version, reserved_nonzero, field_range, too_large, cdf_malformed, stream_length, trailing, stream_exhausted, stream_noncanonical and geometry_mismatch. A parser reports the first that applies, in the order of A.3. §4.1 runs A.4 in two parts: step 6 decodes every unit's stream to its end (A.4 steps 2 and 3, which read no prediction), so stream_exhausted and stream_noncanonical are the envelope's payload-malformed, reported before any unit's shape is read; step 9 then checks each unit's shape (A.4 step 1), so geometry_mismatch is that unit's geometry-mismatch. A payload whose n_units is not the number of units the layer covers is the envelope's payload-malformed with the code field_range. Every other code is the envelope's payload-malformed.

Appendix B. WCP1

The payload of wai.image.int_cond_refine (§6.2). Its refusal codes are Appendix A.6's, and §4.1 reads them as it reads a WIR1 payload's.

B.1 Layout

Every multi-byte field is little-endian; offsets are in bytes.

off	size	field	rule
0	4	magic WCP1
4	1	version	1
5	1	predictor	0 identity, 1 up2: the base prediction
6	1	n_side	0 to 4: the side planes the network reads
7	1	n_layers	1 to 8
8	2	act_hi	1 to 32767: the ceiling of every hidden activation
10	2	reserved	0
12	…	the layers	n_layers layers, one after another
…	…	the residual	a WIR1 payload (Appendix A); nothing follows it

A layer is the layer of exact-upscale §3.1:

size	field	rule
1	k	1, 3 or 5: the kernel's side
1	shift	0 to 31
2	cout	1 to 64; the last layer's is 3
cout · cin · k · k	weights	signed bytes, in the order [cout][cin][k][k]
4 · cout	biases	signed 32-bit integers
2 · cout	multipliers	unsigned 16-bit integers, each at least 1

cin is 3 · (1 + n_side) for the first layer, the base prediction's three channels and each side plane's, and the previous layer's cout for every other. The residual is of the RGB8 unit kind (A.2) with the identity predictor; its d0 and d1 are the output's rows and columns, and its n_units the units the layer covers.

B.2 Side planes

The network reads, after the base prediction, the planes of the entries of refines.side (§3.1), in their order, each an RGB8 picture of d0 rows and d1 columns.

B.3 Order of the checks

A parser checks, in this order, and refuses with the first that fails:

1. the magic (bad_magic);
2. the version byte: present (stream_length), and 1 (version);
3. the 12-byte header: present (stream_length);
4. the reserved field (reserved_nonzero);
5. the header's fields, in the order predictor, n_side, n_layers, act_hi (field_range);
6. each layer in turn: its first four bytes present (stream_length); its k, shift and cout (field_range); its weights, biases and multipliers present (stream_length); each multiplier at least 1 (field_range);
7. the residual: at least one byte present (stream_length); then Appendix A.3 over the bytes that remain, for the RGB8 unit kind, with its codes in its order; then its predictor identity (field_range);
8. the network's caps (too_large): its widest tensor, max(3 · (1 + n_side), the largest cout) · d0 · d1 elements, at most 2^26.

B.4 Decode

For each unit u the layer is applied to, with the unit's current output prev and the side planes S_1 … S_n (B.2):

1. The shapes must be the ones the payload reads (geometry_mismatch), before any symbol is decoded and before the network runs: prev is (d0, d1, 3) for identity and (⌈d0/2⌉, ⌈d1/2⌉, 3) for up2; each side plane is (d0, d1, 3).
2. The base prediction b: prev for identity; for up2, A.4's up2 of prev at d0 × d1.
3. Centring. x₀[c][y][x] = b(x, y)[c] − 128 for c < 3, and x₀[3j + c][y][x] = S_j(x, y)[c] − 128 for the j-th side plane (j from 1).
4. The layers, as exact-upscale §3.2 step 3 states them, at the output's size: for each layer i, a[o][y][x] = Σ W[o][ci][ky][kx] · x̃ᵢ[ci][y + ky − ⌊k/2⌋][x + kx − ⌊k/2⌋], with x̃ᵢ zero outside the picture; then v = R_half_up((a + bias[o]) · mult[o], shift) (integer-operators §2); xᵢ₊₁ = clamp(v, 0, act_hi) for every layer but the last, whose v, of 3 channels, is the correction d.
5. The network's prediction p(x, y)[c] = clamp(b(x, y)[c] + d[c][y][x], 0, 255).
6. The residual: A.4 steps 2 and 3 for unit u of the residual, with p as the unit's prediction: each element clamp(p(i) + r · q, 0, 255), then the stream's canonical end (stream_exhausted, stream_noncanonical).

A network whose last layer's weights and biases are all 0 gives p = b. Every value the network forms fits i64: an input lies in [−128, 127] or [0, act_hi], a weight in a byte, a sum has at most 64 · 25 terms, a bias 32 bits and a multiplier 16 bits.

B.5 Declared cost

A payload's header declares, before any symbol is decoded: its output and its symbols, the residual's (A.5); its working bytes, the residual's (A.5) and 3 · d0 · d1 for the base prediction and 8 · d0 · d1 · max(cin + cout) over its layers, eight bytes for each element of the widest pair of consecutive network tensors; and its multiply-accumulates, n_units · d0 · d1 · Σ cout · cin · k² over its layers. A sink that applies a limit compares this cost with it after the parse and before any stream is decoded (§4.1 step 6, cost-over-limit, naming the figure past it: output bytes, symbols, working bytes, then multiply-accumulates), and counts the output and working bytes in the chain's state while it retains the layer (§4.5).

B.6 Through the integer operator set

The network is one WHG1 graph (integer-operators §10). The lowering wcp1/1 writes, for a payload:

• 1 + n_side data inputs, sizes set per decode: tensor 0, the base prediction [3, d0, d1], then each side plane [3, d0, d1] in order, each with the policy check over [0, 255];
• then the operators, each one's output taking the next tensor index, every tensor derived with no exponent: with side planes, concat of the inputs in order; rescale with no multiplier, shift 0 and a pre bias of −128 on each channel; for each layer, conv with its cout, kernel k × k, stride 1, zero padding ⌊k/2⌋ on each side, one group, its weights at one byte, and an rq with one multiplier per channel at four bytes, its one shift, a pre bias per channel at four bytes and the rounding half_up; after each layer but the last, clamp(0, act_hi); then add of tensor 0 and the correction; clamp(0, 255), the graph's one output.

The driver forms the base prediction with E11 for up2, supplies the inputs, reads the output back as RGB8, and decodes the residual with E1 and E9 as a WIR1 layer's. A sink loads the graph from its canonical bytes under acc64 with its bounds certificate (integer-operators §8): every payload B.3 admits meets acc64 (B.4). The capability's registry ops member names the set, the profile acc64, the lowering wcp1/1 and E1's modes (integer-operators §11).