0e56b96963
The Internet-Draft (xml2rfc v3 source and its text rendering) for WAI as an MSF packaging: the "wai" and "wai-receipts" packaging values; the envelope, dispatch with fallback, the parameter-set pin by SHA-256 and the determinism tiers; catalog fields wai_capabilities, wai_determinism, wai_priors, the receipts fields and wai_staged; the receipts track (WJR1), its verification and the WAI_RECEIPT object property; the codecs parameter; security and IANA considerations (WAI_RECEIPT requested from the Specification Required range, 0x3A41 meanwhile; no action for the packaging values; application/wai registered). 36 pages; idnits 3.3.0 reports only the three normative references to Internet-Drafts. Not submitted, and the NMSF comment is not sent: both are the owner's decision. Author email is a placeholder until the owner confirms it. Co-Authored-By: Claude Opus 5.5 <[email protected]>
5.2 KiB
Comment on draft-herz-moq-nmsf-01: a model digest and a determinism field
Status: draft comment for the authors of
draft-herz-moq-nmsf-01and the moq list. Not sent. Sending it is the owner's decision. Section numbers below are those ofdraft-herz-moq-nmsf-01unless another document is named.
Thank you for the draft. Two catalog fields would let a subscriber check two things the current text leaves to agreement between the endpoints: which model weights a track needs, and whether a decoder on another device reproduces what the encoder's context assumes. Both proposals are small and OPTIONAL, and neither changes the object wire format of §3.2.
1. A model digest
§3.9's modelVersion is a version string. Two endpoints can hold different weights
under the same string, through a fine-tune, a re-export or a quantised variant. A
learned decoder run against weights other than the encoder's does not fail: it
produces a plausible picture that is wrong.
Proposal: an OPTIONAL modelDigest member in the nvc object, the SHA-256 of the
weights' bytes as 64 lowercase hexadecimal digits, with two rules:
- a subscriber that holds no weights with that digest SHOULD NOT subscribe, which extends §3.9's existing "SHOULD NOT subscribe" rule to a check a subscriber can make;
- a decoder MUST NOT decode a track against weights whose digest differs.
Where the weights span several files (networks, entropy tables), the digest can
cover a listing of per-file digests. The WAI specification defines one form for such
a listing: one line per file, giving its SHA-256 and its path, in byte order of the
paths. A checksum utility can verify it without WAI software (WAI SPEC.md §3.1,
"What the digest covers").
2. A determinism field
§4.6 has the encoder run the synthesis transform so that its context buffer "contains exactly the same data that a decoder would produce". That holds only where the decoder computes the same values as the encoder, on whatever device it runs:
- Entropy decode. §4.2 step 4 entropy-decodes the latent with CDF parameters derived from the decoded hyperprior. If that derivation runs in floating point, two devices can derive different parameters from one bitstream, select different distributions, and decode different symbols from then on.
- Context. §4.1 feeds each reconstructed frame back as the next frame's context. Where the reconstruction differs between devices, the difference carries forward until the next Intra frame. Where the entropy model also reads that context (a temporal prior), the entropy decode itself diverges.
Floating-point basic operations are correctly rounded, but runtimes choose accumulation order, fused multiply-add, vector width and reduced precision per device, so the same network does not in general give the same bits on two devices.
Proposal: an OPTIONAL entropyDeterminism member in the nvc object, with two
values:
integer-exact: the decoder derives every entropy-coding parameter in integer or fixed-point arithmetic, or reads it from the bitstream, and so decodes the same symbols on every device;unchecked: no such guarantee is made. This is the meaning when the member is absent.
A codec whose whole decode, synthesis and context included, is integer could also
state that, so that a subscriber knows the reconstructed frames themselves reproduce.
WAI names these two levels entropy-consistency and decode-equivalence, and
explains why parameter transmission reaches the first but not the second
(extensions/determinism-tiers.md §2).
Both levels are attainable in practice. WAI registers learned decoders whose
hyper-decoder, entropy-parameter derivation and synthesis are integer
(wai.neural.int_hyper, and wai.video.int_hyper for integer motion-compensated
P-frames after such a keyframe). The repository's CI decodes their committed test
vectors on Linux x86_64, Linux arm64 and macOS arm64, and requires the same output
hashes on each (extensions/determinism-tiers.md §6). Those vectors are small, and
the full-scale model is checked against its exporter's integer reference on one
machine per run.
3. Integrity, for completeness
§8 relies on hop-by-hop TLS. TLS protects each hop, but not a payload against a relay
that alters it. A publisher signature per group, over the hashes of the group's
objects, carried on a separate track, lets a subscriber detect alteration across
relays and account for objects a relay dropped. If that is of interest, the WAI
receipts track defines one way to do it (draft-charlot-moq-wai-packaging-00 §6;
extensions/jwp-receipts.md §2 and §3).
Pointers
- WAI
SPEC.md§3.1: the parameter-set pin, by SHA-256 over the whole set, its verification order and its refusal reasons. extensions/determinism-tiers.md§2 and §3: the two tiers and what each lets a receiver hash.extensions/moq-streaming-format.md§4 and §4.2: catalog fields that declare a track's capabilities, its weakest tier, and the parameter sets its objects pin, by digest.draft-charlot-moq-wai-packaging-00(in preparation): the same, as an MSF packaging.
All in the WAI repository: https://git.transaction.science/Transaction-Science/open-standards
(directory wai/).