open-standards/wai/ietf/nmsf-comment.md

# Comment on `draft-herz-moq-nmsf-01`: a model digest and a determinism field

> Status: draft comment for the authors of `draft-herz-moq-nmsf-01` and the moq list.
> **Not sent.** Sending it is the owner's decision. Section numbers below are those of
> `draft-herz-moq-nmsf-01` unless another document is named.

Thank you for the draft. Two catalog fields would let a subscriber check two things
the current text leaves to agreement between the endpoints: which model weights a
track needs, and whether a decoder on another device reproduces what the encoder's
context assumes. Both proposals are small and OPTIONAL, and neither changes the
object wire format of §3.2.

## 1. A model digest

§3.9's `modelVersion` is a version string. Two endpoints can hold different weights
under the same string, through a fine-tune, a re-export or a quantised variant. A
learned decoder run against weights other than the encoder's does not fail: it
produces a plausible picture that is wrong.

Proposal: an OPTIONAL `modelDigest` member in the `nvc` object, the SHA-256 of the
weights' bytes as 64 lowercase hexadecimal digits, with two rules:

- a subscriber that holds no weights with that digest SHOULD NOT subscribe, which
  extends §3.9's existing "SHOULD NOT subscribe" rule to a check a subscriber can
  make;
- a decoder MUST NOT decode a track against weights whose digest differs.

Where the weights span several files (networks, entropy tables), the digest can
cover a listing of per-file digests. The WAI specification defines one form for such
a listing: one line per file, giving its SHA-256 and its path, in byte order of the
paths. A checksum utility can verify it without WAI software (WAI `SPEC.md` §3.1,
"What the digest covers").

## 2. A determinism field

§4.6 has the encoder run the synthesis transform so that its context buffer "contains
exactly the same data that a decoder would produce". That holds only where the
decoder computes the same values as the encoder, on whatever device it runs:

- **Entropy decode.** §4.2 step 4 entropy-decodes the latent with CDF parameters
  derived from the decoded hyperprior. If that derivation runs in floating point, two
  devices can derive different parameters from one bitstream, select different
  distributions, and decode different symbols from then on.
- **Context.** §4.1 feeds each reconstructed frame back as the next frame's context.
  Where the reconstruction differs between devices, the difference carries forward
  until the next Intra frame. Where the entropy model also reads that context (a
  temporal prior), the entropy decode itself diverges.

Floating-point basic operations are correctly rounded, but runtimes choose
accumulation order, fused multiply-add, vector width and reduced precision per
device, so the same network does not in general give the same bits on two devices.

Proposal: an OPTIONAL `entropyDeterminism` member in the `nvc` object, with two
values:

- `integer-exact`: the decoder derives every entropy-coding parameter in integer or
  fixed-point arithmetic, or reads it from the bitstream, and so decodes the same
  symbols on every device;
- `unchecked`: no such guarantee is made. This is the meaning when the member is
  absent.

A codec whose whole decode, synthesis and context included, is integer could also
state that, so that a subscriber knows the reconstructed frames themselves reproduce.
WAI names these two levels `entropy-consistency` and `decode-equivalence`, and
explains why parameter transmission reaches the first but not the second
(`extensions/determinism-tiers.md` §2).

Both levels are attainable in practice. WAI registers learned decoders whose
hyper-decoder, entropy-parameter derivation and synthesis are integer
(`wai.neural.int_hyper`, and `wai.video.int_hyper` for integer motion-compensated
P-frames after such a keyframe). The repository's CI decodes their committed test
vectors on Linux x86_64, Linux arm64 and macOS arm64, and requires the same output
hashes on each (`extensions/determinism-tiers.md` §6). Those vectors are small, and
the full-scale model is checked against its exporter's integer reference on one
machine per run.

## 3. Integrity, for completeness

§8 relies on hop-by-hop TLS. TLS protects each hop, but not a payload against a relay
that alters it. A publisher signature per group, over the hashes of the group's
objects, carried on a separate track, lets a subscriber detect alteration across
relays and account for objects a relay dropped. If that is of interest, the WAI
receipts track defines one way to do it (`draft-charlot-moq-wai-packaging-00` §6;
`extensions/jwp-receipts.md` §2 and §3).

## Pointers

- WAI `SPEC.md` §3.1: the parameter-set pin, by SHA-256 over the whole set, its
  verification order and its refusal reasons.
- `extensions/determinism-tiers.md` §2 and §3: the two tiers and what each lets a
  receiver hash.
- `extensions/moq-streaming-format.md` §4 and §4.2: catalog fields that declare a
  track's capabilities, its weakest tier, and the parameter sets its objects pin, by
  digest.
- `draft-charlot-moq-wai-packaging-00` (in preparation): the same, as an MSF packaging.

All in the WAI repository: https://git.transaction.science/Transaction-Science/open-standards
(directory `wai/`).