open-standards/wai/extensions/prior-carriage.md
Transaction Science a031b9dee8 wai: SPEC §5 pins the integer codecs' payloads; extensions/integer-payloads.md
SPEC §5 gains *Payloads of the WAI-native integer codecs*. It pins WIV1
to wai.video.int_motion (keyframe 0) and to wai.video.int_hyper
(keyframe 1), WIH1 to wai.neural.int_hyper, WIA1 to wai.audio.int_codec
and WIS1 to wai.splat.int_codec, each with its parameter set. These rows
named a decoder and the tool that wrote a bitstream, and no byte layout,
so the pin completes their registration and redefines nothing. The JSON
the tools write is a converter input; an encoder must not carry it under
these strings. The subsection lists the envelopes the pin makes
non-conforming; they still parse.

The passages the pin contradicted now say what holds. The §5
introduction no longer says that every payload is a codec's own bytes.
The neural introduction no longer says that the standard specifies no
neural payload layout. The integer rows name their formats, and name
the tools only as writers of converter input.

The parameter-set paragraph names the members the integer decoders read
and registers wai.audio.int_codec (model.wam) and wai.splat.int_codec
(g_s.whm) as one file; that is the widening the code made. The
prior-carriage §7 table gives the same members. REGISTRY.md's
definition of "registered" says these strings got their formats in
v0.4.0. The README says the payloads are binary and the JSON is
converted.

extensions/integer-payloads.md defines each layout field by field,
the rANS stream and CDF table rules, the order of the checks, each
decode, the caps and the checks a decode makes, the closed list of
refusal codes, the parameter sets, the converter rules for the JSON
forms, and the reference implementation.

codecs::registry's spec_pins_the_integer_payloads_the_registry_lists
holds SPEC's table to PINNED_INT_PAYLOADS and to each set's registered
shape and members, and the extension to one section per format.
`wai registry export --check` passes with nothing written: no exported
field records a set's shape.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-01 13:12:44 -04:00

23 KiB
Raw Permalink Blame History

WAI Extension: Prior Carriage

Status: Draft. Registers the wire forms a pin's digest may cover — wai.prior.set, the listing of a parameter set (SPEC.md §3.1), and ISO/IEC 15938-17 (neural-network coding, "NNC") — registers a candidate general-tensor payload, and fixes how an in-band signature inside such an artifact relates to WAI's digest. The pin itself, its verification and its refusal reasons are SPEC.md §3.1. Reference impl: wai-rs — container::{Prior, PinRefusal}, pin::{verify_bytes, verify, parse_listing, write_listing, listing_of_dir}, codecs::prior_form, codecs::{CAP_PRIOR_SET, CAP_PRIOR_NNC, CAP_TENSOR_NNC}. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

1. A prior has a digest, and had no form

A neural capability decodes against a parameter set the envelope does not carry (SPEC.md §1). WAI pins that set's identity (SPEC.md §3.1): a pin names an id, a sha256 over the whole parameter set and a determinism description, and MAY state a tier and the set it supersedes. A sink verifies the pin before it decodes, refuses a pin its sets do not satisfy with a named reason, and replicate soundness is gated on the declared tier (SPEC §7).

The digest fixes which bytes. Until now nothing fixed what those bytes are: the sink hashes whatever single file it registered — a serialized graph, an integer table bundle — so two implementations can hold the same network under different digests, or agree on a digest while reading the bytes differently. A determinism tier is a claim about reproducing a computation; the computation's operand needs a form both ends can name.

2. The form field

"prior": {
  "id":          "int_hyper/q4",
  "sha256":      "<64 lowercase hexadecimal digits: the SHA-256 of the set, in its declared form>",
  "determinism": "int-rans/p16+int-synth",
  "form":        "wai.prior.set",
  "tier":        "decode-equivalence"
}
  • form is OPTIONAL. Absent, the form is undeclared and the digest covers the one file that holds the parameter set; SPEC §3.1 admits that only for a capability registered with a one-file set, and a sink refuses such a pin over any other capability (form_required). A manifest without form round-trips byte for byte.
  • Present, form MUST be a registered prior form (§3), and sha256 MUST be computed over the parameter set serialized in exactly that form: for a coded form, the coded bitstream, not an unpacking or a transcoding of it; for wai.prior.set, the listing SPEC §3.1 defines, never the files' bytes concatenated or an archive of them.
  • A sink that does not know the declared form as a prior form refuses the pin (form_not_registered), and one that cannot read it refuses it (form_unreadable, SPEC §3.1). It MUST NOT resolve the pin against an artifact in another form, even one whose unpacked parameters compare equal: equality after unpacking is a different fact from the digest the manifest pins.
  • A declared form names a self-contained parameter set. An NNC incremental-update bitstream decodes only against its base, so it MUST NOT be pinned on its own, and no form is registered for a base-plus-updates chain. A pin names the self-contained set an update yields, and MAY name the set it replaced in supersedes (SPEC §3.1).
  • A form is a capability string. When a prior artifact travels in a WAI envelope it does so under its form's capability, so the envelope's per-object content hash and the Prior digest cover the same bytes.

3. Registered form and candidate payload

capability payload status
wai.prior.set the listing of a parameter set of one or more files (SPEC §3.1) registered; WAI-native; established exact (§5)
wai.prior.nnc the parameters of a trained network as a bitstream conforming to ISO/IEC 15938-17:2024 (NNC, 2nd edition: parameters and their incremental updates, DeepCABAC entropy coding) registered; a coded form
wai.tensor.nnc general tensorial data — split-inference feature maps, 3D/4D scene parameters — coded with NNC candidate: payload format not fixed

The NNC entries are sink-supplied: wai-rs contains no NNC reader or writer. The standard's reference software is itself standardized, in ISO/IEC 15938-18, Conformance and reference software for compression of neural networks (1st edition 2023, 2nd edition 2025). That part also supplies the conformance bitstreams and the procedures for testing decoders and bitstreams. Its scope makes the software an integral part of it, gives the requirements of ISO/IEC 15938-17 precedence over the software's behaviour, and does not require an implementation to use it.

Registry classification (codecs::capability_*):

  • wai.prior.set: fidelity tier lossless and replicate class StandardDefined — a listing is its own output, and SPEC §3.1 fixes its bytes exactly; licence royalty-free (WAI-native, Apache-2.0 reference).

For the NNC entries:

  • Fidelity tier: lossy, by primary role. The registry's tier names a capability's primary role, as it does for AVIF, which also has a lossless mode; it is not a statement that every NNC artifact approximates its source. NNC's primary role is compressing a network by parameter reduction and quantisation. The published standard is not quantisation-only, though. The 2022 edition defines an NNR unit as carrying compressed or uncompressed network data. Its table of contents lists decoding methods for an integer payload type (NNR_PT_INT) and a raw-float payload type (NNR_PT_RAW_FLOAT), and a decoding process for an integer weight tensor. Those decoding clauses were not read, and neither was the 2024 edition. An open implementation that states compliance with the 2024 edition reads an NNR_PT_RAW_FLOAT payload as raw 32-bit floats and passes int32 tensors through with no quantisation step. So an integer network, or a float network carried raw, need not be approximated at all. Lossy is kept because a capability-level lossless would present every NNC artifact as reproducing its source, and whether a given artifact does depends on its payload types, and on the decoding process of §5.
  • Licence: patent-pool, conservatively. Royalty status is not established. The Introduction of ISO/IEC 15938-17:2022 records a patent claimed to be required for compliance, whose holder has given a reasonable-and-non-discriminatory licensing assurance; the corresponding statement in the 2024 edition was not read. No royalty-free commitment was found, and an openly available implementation that states compliance with the 2024 edition is released under a licence that expressly grants no patent rights (see the research note in §8). The registry places NNC where it places MPEG-I Haptics for the same reason, and never presents it as royalty-free.
  • Replicate class: the class the registry gives a standard whose decoder conformance it has not checked, as it gives MPEG-I Haptics. The tensors are reconstructed by the standard's decoding process, not by a model WAI runs, and NNC's decoder conformance (§5) was not read, so nothing here establishes that every conforming decoder reconstructs bit-identical tensors. A registry that has one class for every standard codec gives NNC that class, which then says only that the named standard specifies the reconstruction. This concerns an NNC payload. How a declared form bounds replicate over a neural capability is §5.

wai.tensor.nnc is a candidate. Coding of general tensorial data is part of an NNC amendment that has not been published (§4), so this payload's format is not fixed. Until the capability is pinned to a published edition, an encoder MUST NOT emit it and a sink MUST treat it as unsupported. SPEC §5 (Candidate capabilities) states the rule for every candidate, and codecs::is_candidate lists them. SPEC §8's rule that a registered payload format never changes applies from the edition it is pinned to. It is registered now because split-inference feature maps and scene tensors are a payload class WAI otherwise has no name for. For Gaussian-splat attributes specifically, wai.splat.int_codec is the path with a portable hash (splat-equivalence); wai.tensor.nnc would be an interchange form with no determinism guarantee.

4. What is pinned, what is tracked, and why form is optional

The standard is at two different stages, and WAI treats them differently.

Pinned — the published editions. ISO/IEC 15938-17 is a published International Standard: 1st edition 2022, 2nd edition 2024. wai.prior.nnc is defined against the 2nd edition. Incremental-update coding is in that edition; it is not new, and an implementation MUST NOT describe it as forthcoming.

Tracked, not pinned — the amendment. MPEG's press release for its 154th meeting (Santa Eulària, 27 April – 1 May 2026) states that a Committee Draft of an amendment to NNC was published. It adds lossless compression, including mixed lossy and lossless compression in one bitstream; source-data-type indication for models with limited bit depth; finer-grained configuration of the quantisation; configurable extensions for initialising, ordering and packing data; coding of other tensorial data, such as split-inference feature maps and the parameters of 3D and 4D scene representations; and signing and verification of data units, which the release says lets a verifier check the authenticity of a set of network parameters, part of a network, or an update. The amendment's text was not read, so what a data-unit signature covers and how its key is identified are not known here. A committee draft can still change. No rule in this extension depends on any of it. When a published edition carries these features:

  • it gets a new capability string — SPEC §8 forbids redefining wai.prior.nnc;
  • wai.tensor.nnc is pinned to that edition, or withdrawn;
  • nothing about exactness changes by that alone. Integer and raw-float payload types are already in the published text (§3). What decides whether a form can be listed as exact is the decoding process and its conformance requirement (§5), for any edition, lossless compression included.

Optional, not recommended. form is MAY, and NNC is not a SHOULD, for three reasons:

  1. WAI treats royalty-free implementation as a precondition for adoption (SPEC Appendix C). It does not recommend, for every prior, a form whose royalty status it cannot establish.
  2. WAI's reference implementation (wai-rs) cannot read or write NNC, so it cannot demonstrate the form end to end.
  3. Whether conforming NNC decoders reconstruct identical tensors is not established (§5), so the form cannot yet carry WAI's strongest claims.

5. Determinism through a form

  • Declaring a form does not change the prior's determinism contract, which still states how the network's own decode reproduces.
  • Reading an artifact out of a coded form is itself a decode. For a quantised payload, NNC's entropy decode produces integer quantisation indices, which the decoding process turns into parameter values. An integer or raw-float payload carries its values without quantisation (§3). Whether every conforming decoder reconstructs bit-identical parameter values, from any of these payload types, was not established here. So an integer-exact network can still reach two sinks as different weights.
  • Where that would be settled: ISO/IEC 15938-18 states that decoder conformance to ISO/IEC 15938-17 is specified in clause 7 of 15938-17 (headed Decoding process in the 2022 edition), and supplies reference bitstreams and a decoder test procedure (15938-18 §4.5). Neither clause 7 nor that procedure was read, so whether conformance requires identical reconstructed parameters, or permits a tolerance, is open. What was read, in the 2022 edition's public sample: its scope requires only a decoder's externally observable output to conform; its number conventions define float by ISO/IEC 60559 and make operator results mathematically exact unless a float result is explicitly specified; and its clause 7 includes decoding methods for integer, float and raw-float payload types. Clause 7 is where a form would be established exact, for the published editions and for any later one.
  • Therefore an envelope with intent = replicate MUST NOT rest on a prior whose declared form is not established exact. The registry lists the forms that are established (codecs::prior_form::form_reconstruction_is_exact). It lists one, wai.prior.set: reading a set through its listing compares digests and hands the decoder each file's bytes unchanged, so every sink that holds the set reads the same parameters. No coded form is listed. In the reference implementation the rule is part of Prior::is_cross_hardware_exact, which requires Prior::form_preserves_exactness as well as an integer-exact contract. Every replicate check reads that predicate, so container::replicate_is_sound and Wai::replicate_is_sound reject such a replicate, and decode_envelope_strict refuses it with a reason naming the form.
  • The same bound applies to the tiers of determinism-tiers: decode-equivalence needs identical tensors at every sink, so it cannot be claimed through a form not established exact.
  • The digest is unaffected. It is computed over bytes, so it is portable whatever the reconstruction does.

6. In-band data-unit signatures and WAI's digest

MPEG's press release for its 154th meeting states that the NNC amendment adds signing and verification of data units (§4). Its text was not read, and nothing here depends on its details. This section fixes the precedence now, so the two cannot diverge later. It takes the posture reconstruction-binding takes toward a C2PA manifest: WAI's binding sits beside the standard's, never in place of it.

  • Scope. Prior.sha256, and a JWP per-object content hash over a wai.prior.nnc or wai.tensor.nnc payload, cover every byte of the coded bitstream, including any unit that carries an in-band signature. Neither is computed over a subset chosen by an in-band signing scope.
  • An in-band signature is an attested input. A verifier MAY check it and MAY record the verdict in a receipt, naming the signature scheme and the key location as found. The verdict never substitutes for the WAI digest check. A matching WAI digest attests that these are the bytes the manifest pinned; an in-band signature attests that a key holder signed these units. Neither implies the other.
  • A failed in-band signature is recorded as failed. It does not unpin a prior whose WAI digest matches: the pinned artifact is the one that carries the bad signature.
  • Stripping is not repackaging. Removing signature units changes the bytes, so the result no longer matches its pin. It is a different artifact.
  • Keys. WAI's key discovery (jwp-receipts §7) covers the signers of WAI's own receipts and claims, and who may authorise a pinned parameter set (jwp-receipts §7.9). A key location found in-band in a prior artifact is outside it: it is recorded, not trusted, and never an anchor (jwp-receipts §7.3).

7. Reference implementation

  • container::Prior has the members of SPEC §3.1; form, tier and supersedes are serde-skipped when absent, so a pin written without them round-trips byte for byte, and a member this revision does not define is kept (extra). Prior is #[non_exhaustive]: it is built with Prior::new and the with_form, with_tier (a typed codecs::DeclaredTier, so the crate writes only a registered token) and with_supersedes builders, so a member a later revision adds does not break code that builds one. Prior::check_syntax applies SPEC §3.1 check 1, Prior::supersedes_finding the supersedes rule, and Prior::declared_tier reads tier or, without it, the reading SPEC §7 fixes (Prior::determinism_class). container::PinRefusal::code() is the reason SPEC §3.1 names. Wai::check_pin and WaiMulti::check_pins report what a verifier can decide without holding a parameter set.
  • pin::verify_bytes runs SPEC §3.1's checks in order over parameter sets held in memory (a browser sink calls it); pin::verify reads held files and listings from disk, keeps the bytes it hashed, and calls it. pin::parse_listing, pin::write_listing and pin::listing_of_dir read and write a listing. pin::READABLE_FORMS holds wai.prior.set: the reference sink reads no coded form.
  • pin::verify_bytes_with and pin::verify_with run SPEC §3.1's check 5 with a pin::PinAuthority, before any held file is read. key_log::SinkKeyLog (feature key_discovery) is the one in this crate: built from the key log a sink holds, it runs jwp-receipts §7.9 over the pinned sets of the capabilities the sink's relying party holds to it. Without one, check 5 is skipped.
  • neural::ModelRegistry holds, for each capability, its registered path and any further paths given to hold. For a form-less pin it hashes each held file — for wai.neural.int_synth, the int_synth.json beside it. For a wai.prior.set pin it takes every *.sha256 file directly inside each held path's directory (the path itself if it is a directory) as a listing rooted there. ModelRegistry::with_key_log gives it a key log for check 5.
  • container::Wai::dispatch_verified applies SPEC §4 steps 3–6 and container::WaiMulti::pick SPEC §7.1 steps 2 and 5, each with a pin verifier the caller supplies. neural::decode_envelope and neural::decode_multi decode through them; their strict variants apply SPEC §7 to the capability they decode through.
  • The decoders read parameters only through neural::ParamSource, which hands a pinned decode the verified bytes; ONNX sessions are built from those bytes, and a file the verified set lacks is refused with set_incomplete. The integer capabilities decode through int_payload::decode_int, which reads its set members from memory; int_payload::resolve verifies a pin over sets held in memory and runs check 5, pinned or unpinned (integer-payloads §12). wai.neural.bmshj2018 caches its CDF tables by content digest.
  • The C ABI (ffi) returns each manifest as its original bytes, carries a WAI2 rendition's identity and pin in the rendition table, and reports a refused pin as WAI_ERR_PRIOR with its reason first in wai_last_error; wai_neural_decode_ex decodes a WAI1 or WAI2 envelope against a registry of held sets. The browser crate (wai-web) wraps the same types and exposes pick, companionsOf and verifyPin to JS.
  • CI: the library unit tests cover container, codecs and pin; wai_pin_vectors verify ../pin-conformance checks the corpus, and pin-conformance/pin_verify.py, which shares no code with wai-rs, checks it independently; the model-free neural-sink and neural C ABI tests, and the browser crate's tests (natively and under wasm32), are configured to run in CI.
  • Not implemented: an NNC reader or writer, and any cross-decoder measurement of NNC reconstruction.

The files the reference decoders read from a set (informative). A set built for another decoder of the capability may name other files; a decode that needs a file the set lacks is refused with set_incomplete.

capability shape files the reference decoder reads from a set
wai.neural.int_mlicpp one file model.wmb
wai.neural.int_synth one file int_synth.json
wai.audio.int_codec one file model.wam
wai.splat.int_codec one file g_s.whm
wai.neural.int_hyper, wai.video.int_hyper several g_s.whm, h_s.whm, entropy.json
wai.neural.encodec32, .dac, .mimi, .wavtokenizer, .snac, .video_bmshj2018 one file decoder.onnx
wai.neural.bmshj2018 several decoder.onnx, cdfs_q<q>.json
wai.neural.mbt2018_mean several hyper_cdfs_q<q>.json, hyper_synthesis.onnx, gauss_cdfs_q<q>.json, synthesis.onnx
wai.neural.elic several the mbt2018_mean files, and context.onnx
wai.neural.mlicpp, .mlicv2, .cmic several the mbt2018_mean files, and entropy.onnx
wai.neural.glc several hyper_cdfs_q<q>.json, hyper_entropy.onnx, context_entropy.onnx, synthesis.onnx
wai.neural.dcvc_rt, .dcvc_fm several hyper_cdfs_q<q>.json, hyper_synthesis.onnx, gauss_cdfs_q<q>.json, temporal_decoder.onnx
wai.image.jpegai several exact path: norm.json, weights/{hs,hyp,synth}_{y,uv}.json, weights/mcm{0..3}_y.json, weights/icci.json. Float path: hyper_scale_decoder_int_weights.json, model_constants.json, jpegai_normative_tables.json, me-tANS/bounds.csv, jpegai_factorized_cdfs.json, jpegai_scaler_vec.json, jpegai_dequant_consts.json, model/CCS_SGMM/tools_0/model_{y,uv}/common_modules/hyper_decoder.onnx, model/CCS_SGMM/tools_0/model_y/common_modules/MCM/stage{0..3}.onnx, model16/CCS_SGMM/tools_0/model_{y,uv}/synthesis16.onnx, e2e/pfchain/icci_{y_m6,uv_m3}.onnx. A set holding only the exact path's files decodes through that path; where that path refuses (a non-default floating-point environment), the float path needs files the set lacks, and the decode is refused with set_incomplete instead of running on unverified files.
wai.generate.spectral several, until an envelope decode path registers its files — (no envelope decode path in the reference sink)

8. Sources

  • ISO/IEC 15938-17:2022 and ISO/IEC 15938-17:2024, Compression of neural networks for multimedia content description and analysis. Read: the public sample of the 2022 edition (Introduction, clauses 1–4 and the table of contents), which is the source for the scope, the definition of an NNR unit, the number conventions, the clause-7 headings (including the integer and raw-float payload types) and the patent statement. Not read: clause 7 itself, and the 2024 edition.
  • MPEG, press release of the 154th meeting, Compressing Tensor Data in AI-based Media Coding: MPEG publishes Committee Draft of Amendment of Neural Network Coding (NNC) (mpeg.org, 4 June 2026), and MPEG's meeting list (mpeg.org) for the meeting's place and dates: the source for the amendment's stage and features, including signing and verification of data units.
  • An open implementation that states compliance with ISO/IEC 15938-17:2024: its high-level-syntax source reads an NNR_PT_RAW_FLOAT payload as raw 32-bit floats, and its integer path gives int32 tensors a skip approximation with no quantisation. Named in the research note.
  • ISO/IEC 15938-18:2023 and ISO/IEC 15938-18:2025, Conformance and reference software for compression of neural networks: the reference software, the conformance bitstreams and the decoder and bitstream test procedures. Read: the public sample of its draft (DIS, 2022: Introduction, clauses 1–4.3, part of 4.4, and the table of contents), and the published 2023 edition's catalogue abstract. Not read: the decoder test procedure (§4.5) and either published edition's full text.
  • research/2030-transport-landscape.md §6.4 (the amendment's contents and stage, the reference software, and the open implementation whose licence grants no patent rights), §8 item 9, §9 item 5 (the signing flag, resolved by the press release).