prior_forms_verify.py implements prior-carriage §9 to §12 and Appendix A from the text (strict readers, canonical writers, the arithmetic coder both ways, both bindings) and agrees with wai-rs on every vector; it also flips bytes of every accepted file and requires each copy to be caught. §12.4 records the measurement: an open implementation of ISO/IEC 15938-17 reads our NNC files to the same integers, its encoder writes byte-identical data units, and our reader reads its bitstreams. tools/nnc_interop_check.py is the check; it runs that implementation's code and is not a CI step. Co-Authored-By: Claude Opus 5.5 <[email protected]>
58 KiB
WAI Extension: Prior Carriage
Status: Draft. Registers the wire forms a pin's digest may cover —
wai.prior.set, the listing of a parameter set (SPEC.md §3.1), the tensor fileswai.prior.safetensorsandwai.prior.gguf(§9), and ISO/IEC 15938-17 (neural-network coding, "NNC") — registers a candidate general-tensor payload, and fixes how an in-band signature inside such an artifact relates to WAI's digest. §10 defines the integer-quantised profilewai-int/1, through which a prior carried in a tensor file or in NNC loads, bit for bit, into the native artifact an integer kernel reads; §11 the digest rule; §12 the NNC reader and writer and where they stop. The pin itself, its verification and its refusal reasons are SPEC.md §3.1. Reference impl:wai-rs—container::{Prior, PinRefusal},pin::{verify_bytes, verify, parse_listing, write_listing, listing_of_dir},codecs::prior_form,codecs::{CAP_PRIOR_SET, CAP_PRIOR_SAFETENSORS, CAP_PRIOR_GGUF, CAP_PRIOR_NNC, CAP_TENSOR_NNC}, andprior_wire(featureprior_wire) for §9 to §14; corpuslive-conformance/prior-forms/, second implementationlive-conformance/prior_forms_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.
1. A prior has a digest, and had no form
A neural capability decodes against a parameter set the envelope does not carry
(SPEC.md §1). WAI pins that set's identity
(SPEC.md §3.1): a pin names an id, a sha256 over the whole
parameter set and a determinism description, and MAY state a tier and the set
it supersedes. A sink verifies the pin before it decodes, refuses a pin its sets
do not satisfy with a named reason, and replicate soundness is gated on the
declared tier (SPEC §7).
The digest fixes which bytes. Until now nothing fixed what those bytes are: the sink hashes whatever single file it registered — a serialized graph, an integer table bundle — so two implementations can hold the same network under different digests, or agree on a digest while reading the bytes differently. A determinism tier is a claim about reproducing a computation; the computation's operand needs a form both ends can name.
2. The form field
"prior": {
"id": "int_hyper/q4",
"sha256": "<64 lowercase hexadecimal digits: the SHA-256 of the set, in its declared form>",
"determinism": "int-rans/p16+int-synth",
"form": "wai.prior.set",
"tier": "decode-equivalence"
}
formis OPTIONAL. Absent, the form is undeclared and the digest covers the one file that holds the parameter set; SPEC §3.1 admits that only for a capability registered with a one-file set, and a sink refuses such a pin over any other capability (form_required). A manifest withoutformround-trips byte for byte.- Present,
formMUST be a registered prior form (§3), andsha256MUST be computed over the parameter set serialized in exactly that form: for a coded form, the coded bitstream, not an unpacking or a transcoding of it; forwai.prior.set, the listing SPEC §3.1 defines, never the files' bytes concatenated or an archive of them. - A sink that does not know the declared form as a prior form refuses the pin
(
form_not_registered), and one that cannot read it refuses it (form_unreadable, SPEC §3.1). It MUST NOT resolve the pin against an artifact in another form, even one whose unpacked parameters compare equal: equality after unpacking is a different fact from the digest the manifest pins. - A declared form names a self-contained parameter set. An NNC
incremental-update bitstream decodes only against its base, so it MUST NOT be
pinned on its own, and no form is registered for a base-plus-updates chain. A
pin names the self-contained set an update yields, and MAY name the set it
replaced in
supersedes(SPEC §3.1). - A form is a capability string. When a prior artifact travels in a WAI envelope
it does so under its form's capability, so the envelope's per-object content
hash and the
Priordigest cover the same bytes.
3. Registered form and candidate payload
| capability | payload | status |
|---|---|---|
wai.prior.set |
the listing of a parameter set of one or more files (SPEC §3.1) | registered; WAI-native; established exact (§5) |
wai.prior.safetensors |
a parameter set as one safetensors file: an 8-byte header length, a JSON header naming each tensor's type, shape and bytes, the tensors' bytes (§9.1) | registered; established exact (§5) |
wai.prior.gguf |
a parameter set as one GGUF version 3 file: key-value metadata, tensor descriptions, aligned tensor data (§9.2) | registered; established exact (§5) |
wai.prior.nnc |
the parameters of a trained network as a bitstream conforming to ISO/IEC 15938-17:2024 (NNC, 2nd edition: parameters and their incremental updates, DeepCABAC entropy coding) | registered; a coded form |
wai.tensor.nnc |
general tensorial data — split-inference feature maps, 3D/4D scene parameters — coded with NNC | candidate: payload format not fixed |
wai-rs reads and writes the NNC subset of §12 (integer tensors under the
standard's arithmetic coding, raw 32-bit floats) and refuses the rest, quantised
payloads among them, with a named reason; otherwise the NNC entries are
sink-supplied. The standard's reference software is itself standardized, in ISO/IEC 15938-18,
Conformance and reference software for compression of neural networks (1st
edition 2023, 2nd edition 2025). That part also supplies the conformance
bitstreams and the procedures for testing decoders and bitstreams. Its scope
makes the software an integral part of it, gives the requirements of
ISO/IEC 15938-17 precedence over the software's behaviour, and does not require
an implementation to use it.
Registry classification (codecs::capability_*):
wai.prior.set: fidelity tier lossless and replicate classStandardDefined— a listing is its own output, and SPEC §3.1 fixes its bytes exactly; licence royalty-free (WAI-native, Apache-2.0 reference).wai.prior.safetensors,wai.prior.gguf: fidelity tier lossless and replicate classStandardDefined— a reader that follows §9 hands over each tensor's bytes as the file holds them, and §9 leaves no choice of which bytes those are; licence royalty-free (the safetensors layout is published with its reference implementation under Apache-2.0, GGUF under MIT).
For the NNC entries:
- Fidelity tier: lossy, by primary role. The registry's tier names a
capability's primary role, as it does for AVIF, which also has a lossless mode;
it is not a statement that every NNC artifact approximates its source. NNC's
primary role is compressing a network by parameter reduction and quantisation.
The published standard is not quantisation-only, though. The 2022 edition
defines an NNR unit as carrying compressed or uncompressed network data. Its
table of contents lists decoding methods for an integer payload type
(
NNR_PT_INT) and a raw-float payload type (NNR_PT_RAW_FLOAT), and a decoding process for an integer weight tensor. Those decoding clauses were not read, and neither was the 2024 edition. An open implementation that states compliance with the 2024 edition reads anNNR_PT_RAW_FLOATpayload as raw 32-bit floats and passes int32 tensors through with no quantisation step. So an integer network, or a float network carried raw, need not be approximated at all. Lossy is kept because a capability-levellosslesswould present every NNC artifact as reproducing its source, and whether a given artifact does depends on its payload types, and on the decoding process of §5. - Licence: patent-pool, conservatively. Royalty status is not established. The Introduction of ISO/IEC 15938-17:2022 records a patent claimed to be required for compliance, whose holder has given a reasonable-and-non-discriminatory licensing assurance; the corresponding statement in the 2024 edition was not read. No royalty-free commitment was found, and an openly available implementation that states compliance with the 2024 edition is released under a licence that expressly grants no patent rights (see the research note in §8). The registry places NNC where it places MPEG-I Haptics for the same reason, and never presents it as royalty-free.
- Replicate class: the class the registry gives a standard whose decoder
conformance it has not checked, as it gives MPEG-I Haptics. The tensors are
reconstructed by the standard's decoding process, not by a model WAI runs, and
NNC's decoder conformance (§5) was not read, so nothing here establishes that
every conforming decoder reconstructs bit-identical tensors. A registry that has
one class for every standard codec gives NNC that class, which then says only
that the named standard specifies the reconstruction. This concerns an NNC
payload. How a declared form bounds
replicateover a neural capability is §5.
wai.tensor.nnc is a candidate. Coding of general tensorial data is part of
an NNC amendment that has not been published (§4), so this payload's format is
not fixed. Until the capability is pinned to a published edition, an encoder
MUST NOT emit it and a sink MUST treat it as unsupported. SPEC §5
(Candidate capabilities) states the rule for every candidate, and
codecs::is_candidate lists them. SPEC §8's rule that a
registered payload format never changes applies from the edition it is pinned to.
It is registered now because split-inference feature maps and scene tensors are a
payload class WAI otherwise has no name for. For Gaussian-splat attributes
specifically, wai.splat.int_codec is the path with a portable hash
(splat-equivalence); wai.tensor.nnc would be an interchange form with no
determinism guarantee.
4. What is pinned, what is tracked, and why form is optional
The standard is at two different stages, and WAI treats them differently.
Pinned — the published editions. ISO/IEC 15938-17 is a published
International Standard: 1st edition 2022, 2nd edition 2024. wai.prior.nnc is
defined against the 2nd edition. Incremental-update coding is in that edition;
it is not new, and an implementation MUST NOT describe it as forthcoming.
Tracked, not pinned — the amendment. MPEG's press release for its 154th meeting (Santa Eulària, 27 April – 1 May 2026) states that a Committee Draft of an amendment to NNC was published. It adds lossless compression, including mixed lossy and lossless compression in one bitstream; source-data-type indication for models with limited bit depth; finer-grained configuration of the quantisation; configurable extensions for initialising, ordering and packing data; coding of other tensorial data, such as split-inference feature maps and the parameters of 3D and 4D scene representations; and signing and verification of data units, which the release says lets a verifier check the authenticity of a set of network parameters, part of a network, or an update. The amendment's text was not read, so what a data-unit signature covers and how its key is identified are not known here. A committee draft can still change. No rule in this extension depends on any of it. When a published edition carries these features:
- it gets a new capability string — SPEC §8 forbids redefining
wai.prior.nnc; wai.tensor.nncis pinned to that edition, or withdrawn;- nothing about exactness changes by that alone. Integer and raw-float payload types are already in the published text (§3). What decides whether a form can be listed as exact is the decoding process and its conformance requirement (§5), for any edition, lossless compression included.
Optional, not recommended. form is MAY, and NNC is not a SHOULD, for three
reasons:
- WAI treats royalty-free implementation as a precondition for adoption (SPEC Appendix C). It does not recommend, for every prior, a form whose royalty status it cannot establish.
- WAI's reference implementation (
wai-rs) reads and writes only the subset of NNC that §12 states, written from an open implementation rather than from the standard's clauses, so it cannot demonstrate the form end to end. - Whether conforming NNC decoders reconstruct identical tensors is not established (§5), so the form cannot yet carry WAI's strongest claims.
5. Determinism through a form
- Declaring a form does not change the prior's
determinismcontract, which still states how the network's own decode reproduces. - Reading an artifact out of a coded form is itself a decode. For a quantised payload, NNC's entropy decode produces integer quantisation indices, which the decoding process turns into parameter values. An integer or raw-float payload carries its values without quantisation (§3). Whether every conforming decoder reconstructs bit-identical parameter values, from any of these payload types, was not established here. So an integer-exact network can still reach two sinks as different weights.
- Where that would be settled: ISO/IEC 15938-18 states that decoder conformance
to ISO/IEC 15938-17 is specified in clause 7 of 15938-17 (headed Decoding
process in the 2022 edition), and supplies reference bitstreams and a decoder
test procedure (15938-18 §4.5). Neither clause 7 nor that procedure was read,
so whether conformance requires identical reconstructed parameters, or permits
a tolerance, is open. What was read, in the 2022 edition's public sample:
its scope requires only a decoder's externally observable output to conform;
its number conventions define
floatby ISO/IEC 60559 and make operator results mathematically exact unless a float result is explicitly specified; and its clause 7 includes decoding methods for integer, float and raw-float payload types. Clause 7 is where a form would be established exact, for the published editions and for any later one. - Therefore an envelope with
intent = replicateMUST NOT rest on a prior whose declared form is not established exact. The registry lists the forms that are established (codecs::prior_form::form_reconstruction_is_exact). It lists three.wai.prior.set: reading a set through its listing compares digests and hands the decoder each file's bytes unchanged, so every sink that holds the set reads the same parameters.wai.prior.safetensorsandwai.prior.gguf: a reader that follows §9 finds each tensor's bytes where the file's header puts them and hands them over unchanged, and §9 refuses every file whose header could be read two ways. No coded form is listed: within the subset of §12 the integers an NNC reader obtains are a function of the bits (§12.4), but the form also covers quantised payloads, whose reconstruction is the open question above. In the reference implementation the rule is part ofPrior::is_cross_hardware_exact, which requiresPrior::form_preserves_exactnessas well as an integer-exact contract. Everyreplicatecheck reads that predicate, socontainer::replicate_is_soundandWai::replicate_is_soundreject such areplicate, anddecode_envelope_strictrefuses it with a reason naming the form. - The same bound applies to the tiers of
determinism-tiers:
decode-equivalenceneeds identical tensors at every sink, so it cannot be claimed through a form not established exact. - The digest is unaffected. It is computed over bytes, so it is portable whatever the reconstruction does.
6. In-band data-unit signatures and WAI's digest
MPEG's press release for its 154th meeting states that the NNC amendment adds signing and verification of data units (§4). Its text was not read, and nothing here depends on its details. This section fixes the precedence now, so the two cannot diverge later. It takes the posture reconstruction-binding takes toward a C2PA manifest: WAI's binding sits beside the standard's, never in place of it.
- Scope.
Prior.sha256, and a JWP per-object content hash over awai.prior.nncorwai.tensor.nncpayload, cover every byte of the coded bitstream, including any unit that carries an in-band signature. Neither is computed over a subset chosen by an in-band signing scope. - An in-band signature is an attested input. A verifier MAY check it and MAY record the verdict in a receipt, naming the signature scheme and the key location as found. The verdict never substitutes for the WAI digest check. A matching WAI digest attests that these are the bytes the manifest pinned; an in-band signature attests that a key holder signed these units. Neither implies the other.
- A failed in-band signature is recorded as failed. It does not unpin a prior whose WAI digest matches: the pinned artifact is the one that carries the bad signature.
- Stripping is not repackaging. Removing signature units changes the bytes, so the result no longer matches its pin. It is a different artifact.
- Keys. WAI's key discovery (jwp-receipts §7) covers the signers of WAI's own receipts and claims, and who may authorise a pinned parameter set (jwp-receipts §7.9). A key location found in-band in a prior artifact is outside it: it is recorded, not trusted, and never an anchor (jwp-receipts §7.3).
7. Reference implementation
container::Priorhas the members of SPEC §3.1;form,tierandsupersedesare serde-skipped when absent, so a pin written without them round-trips byte for byte, and a member this revision does not define is kept (extra).Prioris#[non_exhaustive]: it is built withPrior::newand thewith_form,with_tier(a typedcodecs::DeclaredTier, so the crate writes only a registered token) andwith_supersedesbuilders, so a member a later revision adds does not break code that builds one.Prior::check_syntaxapplies SPEC §3.1 check 1,Prior::supersedes_findingthesupersedesrule, andPrior::declared_tierreadstieror, without it, the reading SPEC §7 fixes (Prior::determinism_class).container::PinRefusal::code()is the reason SPEC §3.1 names.Wai::check_pinandWaiMulti::check_pinsreport what a verifier can decide without holding a parameter set.pin::verify_bytesruns SPEC §3.1's checks in order over parameter sets held in memory (a browser sink calls it);pin::verifyreads held files and listings from disk, keeps the bytes it hashed, and calls it.pin::parse_listing,pin::write_listingandpin::listing_of_dirread and write a listing.pin::READABLE_FORMSholdswai.prior.set: the reference sink reads no coded form.pin::verify_bytes_withandpin::verify_withrun SPEC §3.1's check 5 with apin::PinAuthority, before any held file is read.key_log::SinkKeyLog(featurekey_discovery) is the one in this crate: built from the key log a sink holds, it runs jwp-receipts §7.9 over the pinned sets of the capabilities the sink's relying party holds to it. Without one, check 5 is skipped.neural::ModelRegistryholds, for each capability, its registered path and any further paths given tohold. For a form-less pin it hashes each held file — forwai.neural.int_synth, theint_synth.jsonbeside it. For awai.prior.setpin it takes every*.sha256file directly inside each held path's directory (the path itself if it is a directory) as a listing rooted there.ModelRegistry::with_key_loggives it a key log for check 5.container::Wai::dispatch_verifiedapplies SPEC §4 steps 3–6 andcontainer::WaiMulti::pickSPEC §7.1 steps 2 and 5, each with a pin verifier the caller supplies.neural::decode_envelopeandneural::decode_multidecode through them; their strict variants apply SPEC §7 to the capability they decode through.- The decoders read parameters only through
neural::ParamSource, which hands a pinned decode the verified bytes; ONNX sessions are built from those bytes, and a file the verified set lacks is refused withset_incomplete. The integer capabilities decode throughint_payload::decode_int, which reads its set members from memory;int_payload::resolveverifies a pin over sets held in memory and runs check 5, pinned or unpinned (integer-payloads §12).wai.neural.bmshj2018caches its CDF tables by content digest. - The C ABI (
ffi) returns each manifest as its original bytes, carries aWAI2rendition's identity and pin in the rendition table, and reports a refused pin asWAI_ERR_PRIORwith its reason first inwai_last_error;wai_neural_decode_exdecodes aWAI1orWAI2envelope against a registry of held sets. The browser crate (wai-web) wraps the same types and exposespick,companionsOfandverifyPinto JS. - CI: the library unit tests cover
container,codecsandpin;wai_pin_vectors verify ../pin-conformancechecks the corpus, andpin-conformance/pin_verify.py, which shares no code withwai-rs, checks it independently; the model-free neural-sink and neural C ABI tests, and the browser crate's tests (natively and under wasm32), are configured to run in CI. prior_wire(featureprior_wire):safetensors,ggufandnncread a file into aTensorSetand write one canonically;cabacis §12.4's coding;bindingholdswsr1/1andwhm1/1;loadruns §10.3 andencodethe reverse. Its corpus islive-conformance/prior-forms/(wai_prior_forms_vectors gen|verify,tests/prior_wire.rs, which also loads every corpus prior in every form into the integer kernels and compares the native artifact and, forwsr1/1, the derivation's output).- Not implemented: the pin verifier (
pin::READABLE_FORMS) does not read the tensor-file forms or NNC, so the reference sink refuses a pin in them withform_unreadable, and its decoders read native files only; the NNC payload types and options §12.1 lists as not read.
The files the reference decoders read from a set (informative). A set built for
another decoder of the capability may name other files; a decode that needs a
file the set lacks is refused with set_incomplete.
| capability | shape | files the reference decoder reads from a set |
|---|---|---|
wai.neural.int_mlicpp |
one file | model.wmb |
wai.neural.int_synth |
one file | int_synth.json |
wai.audio.int_codec |
one file | model.wam |
wai.splat.int_codec |
one file | g_s.whm |
wai.neural.int_hyper, wai.video.int_hyper |
several | g_s.whm, h_s.whm, entropy.json |
wai.neural.encodec32, .dac, .mimi, .wavtokenizer, .snac, .video_bmshj2018 |
one file | decoder.onnx |
wai.neural.bmshj2018 |
several | decoder.onnx, cdfs_q<q>.json |
wai.neural.mbt2018_mean |
several | hyper_cdfs_q<q>.json, hyper_synthesis.onnx, gauss_cdfs_q<q>.json, synthesis.onnx |
wai.neural.elic |
several | the mbt2018_mean files, and context.onnx |
wai.neural.mlicpp, .mlicv2, .cmic |
several | the mbt2018_mean files, and entropy.onnx |
wai.neural.glc |
several | hyper_cdfs_q<q>.json, hyper_entropy.onnx, context_entropy.onnx, synthesis.onnx |
wai.neural.dcvc_rt, .dcvc_fm |
several | hyper_cdfs_q<q>.json, hyper_synthesis.onnx, gauss_cdfs_q<q>.json, temporal_decoder.onnx |
wai.image.jpegai |
several | exact path: norm.json, weights/{hs,hyp,synth}_{y,uv}.json, weights/mcm{0..3}_y.json, weights/icci.json. Float path: hyper_scale_decoder_int_weights.json, model_constants.json, jpegai_normative_tables.json, me-tANS/bounds.csv, jpegai_factorized_cdfs.json, jpegai_scaler_vec.json, jpegai_dequant_consts.json, model/CCS_SGMM/tools_0/model_{y,uv}/common_modules/hyper_decoder.onnx, model/CCS_SGMM/tools_0/model_y/common_modules/MCM/stage{0..3}.onnx, model16/CCS_SGMM/tools_0/model_{y,uv}/synthesis16.onnx, e2e/pfchain/icci_{y_m6,uv_m3}.onnx. A set holding only the exact path's files decodes through that path; where that path refuses (a non-default floating-point environment), the float path needs files the set lacks, and the decode is refused with set_incomplete instead of running on unverified files. |
wai.generate.spectral |
several (no one-file shape registered); never pinned: its payload names its one model file by SHA-256, and its envelope carries no pin (SPEC §5, Payload of wai.generate.spectral) |
the model file the payload's model names (generate_spectral_model.json in the conformance vector) |
8. Sources
- ISO/IEC 15938-17:2022 and ISO/IEC 15938-17:2024, Compression of neural networks for multimedia content description and analysis. Read: the public sample of the 2022 edition (Introduction, clauses 1–4 and the table of contents), which is the source for the scope, the definition of an NNR unit, the number conventions, the clause-7 headings (including the integer and raw-float payload types) and the patent statement. Not read: clause 7 itself, and the 2024 edition.
- MPEG, press release of the 154th meeting, Compressing Tensor Data in AI-based Media Coding: MPEG publishes Committee Draft of Amendment of Neural Network Coding (NNC) (mpeg.org, 4 June 2026), and MPEG's meeting list (mpeg.org) for the meeting's place and dates: the source for the amendment's stage and features, including signing and verification of data units.
- An open implementation that states compliance with ISO/IEC 15938-17:2024: its
high-level-syntax source reads an
NNR_PT_RAW_FLOATpayload as raw 32-bit floats, and its integer path gives int32 tensors askipapproximation with no quantisation. Named in the research note. - ISO/IEC 15938-18:2023 and ISO/IEC 15938-18:2025, Conformance and reference software for compression of neural networks: the reference software, the conformance bitstreams and the decoder and bitstream test procedures. Read: the public sample of its draft (DIS, 2022: Introduction, clauses 1–4.3, part of 4.4, and the table of contents), and the published 2023 edition's catalogue abstract. Not read: the decoder test procedure (§4.5) and either published edition's full text.
- research/2030-transport-landscape.md §6.4 (the amendment's contents and stage, the reference software, and the open implementation whose licence grants no patent rights), §8 item 9, §9 item 5 (the signing flag, resolved by the press release).
- The safetensors layout, as published with its reference implementation (format description and source, read October 2026; Apache-2.0): the source for §9.1's layout, its element types and its 100 000 000-byte header limit.
- GGUF, as published with its reference implementation (the format description, read October 2026; MIT): the source for §9.2's layout, its metadata value types, its key rule, its alignment default and rule, and the type numbers of §9.2's table.
- An open implementation of ISO/IEC 15938-17 that states compliance with the 2024 edition (its high-level syntax and its arithmetic coder, read October 2026, under a BSD licence that grants no patent rights): the source for §12's syntax and for Appendix A's procedure and tables. Named in the research note above.
9. Tensor files: wai.prior.safetensors and wai.prior.gguf
A parameter set that is one artifact of named tensors MAY be pinned as one tensor file.
Its pin's sha256 is the SHA-256 of the whole file (§11). Two layouts are registered,
each read as this section states; a reader refuses a file that breaks a rule with the
code of §13 named at the rule, and the first rule broken, in the order given, is the
one reported.
9.1 safetensors
- The first 8 bytes are
N, an unsigned 64-bit little-endian integer. A file shorter than 8 bytes isheader_truncated. Npast 100 000 000 isheader_too_large;8 + Npast the end of the file isheader_truncated.- The next
Nbytes are a JSON text (RFC 8259, UTF-8) whose value is an object, with no member named twice in any object and at most 64 levels of nesting (header_malformed). Whitespace around the value is admitted: the layout pads its header with spaces. - The member
__metadata__, where present, is an object whose every value is a string (header_malformed). - Every other member is a tensor, named by the member's name. Checked in this order:
the name is not empty; the value is an object of exactly the members
dtype,shapeanddata_offsets;dtypeis a string (eachtensor_malformed);dtypeis in the table below (dtype_unknown);shapeis a list of integers;data_offsetsis a list of two integers[begin, end]withbeginat mostend(eachtensor_malformed). An integer here is written in plain decimal (no sign, fraction, exponent or leading zero) and is at most 2^64 − 1. - A tensor's element count is the product of its
shape(1 for an emptyshape); a count, or a size in bytes, past 2^64 − 1 istensor_malformed.end − beginis that count times the element's size (offset_mismatch). - Offsets count from the byte after the header. Ordered by
begin, the tensors cover those bytes exactly: the first begins at 0, each begins where the one before ends, and the last ends at the end of the file (offset_mismatch).
Steps 4 to 6 apply to the header's members in the order it writes them, and step 7 once they all pass.
A tensor's bytes are its elements in row-major order (the outermost dimension first), each little-endian.
dtype |
element | bytes |
|---|---|---|
BOOL |
boolean | 1 |
U8, I8 |
unsigned, signed integer | 1 |
U16, I16 |
unsigned, signed integer | 2 |
F16, BF16 |
binary16, the 16-bit truncation of binary32 | 2 |
U32, I32 |
unsigned, signed integer | 4 |
F32 |
binary32 | 4 |
U64, I64 |
unsigned, signed integer | 8 |
F64 |
binary64 | 8 |
F8_E5M2, F8_E4M3 |
8-bit floating point | 1 |
9.2 GGUF
Version 3, little-endian. Every integer below is little-endian; a field due past the end
of the file is header_truncated.
- Bytes 0 to 3 are
GGUF(bad_magic; a file shorter than 4 bytes isheader_truncated). version, a u32, is 3 (version_unsupported). The big-endian variant of version 3 reads here as another number and is refused the same way.tensor_countandmetadata_kv_count, each a u64. A count whose least size (12 bytes a pair, 28 a tensor description) is past the file's length isheader_truncated, before anything is reserved for it.- A string is a u64 length and that many bytes, UTF-8 (
header_malformed). - Each key-value pair: the key, a string of
.-separated non-empty segments ofatoz,0to9and_, at most 65 535 bytes, no key twice (header_malformed); the value type, a u32 from 0 to 12 (header_malformed); the value. A boolean (type 7) is 0 or 1 (header_malformed). An array (type 9) is an element type, a u64 count and the elements; this reader reads one level of array, so an element type of 9 isheader_malformed(checked before the count), and a count whose least size is past the bytes left isheader_truncated.general.alignment, where present, is a u32 (type 4), non-zero and a multiple of 8 (header_malformed); without it the alignment is 32. - Each tensor description: the name, 1 to 64 bytes, no name twice
(
tensor_malformed);n_dimensions, a u32 from 1 to 4, and that many dimensions, u64 each, innermost first, none 0 (tensor_malformed); the type, a u32 from the table below (dtype_unknown); the offset, a u64, a multiple of the alignment (offset_mismatch). AQ8_0tensor's innermost dimension is a multiple of 32, and an element count or a size in bytes past 2^64 − 1 is malformed (tensor_malformed). - The tensor data begins at the first multiple of the alignment at or after the end of
the descriptions; the bytes between are zero (
header_malformed), and the file ending before the data begins isheader_truncated. A tensor's bytes are at its offset from there; no two tensors overlap and none runs past the end of the file (offset_mismatch).
A tensor's shape, as §10 and §11 use it, is its GGUF dimensions in reverse order (outermost first); its bytes are its elements in that row-major order.
| type | number | element | bytes |
|---|---|---|---|
F32 |
0 | binary32 | 4 |
F16 |
1 | binary16 | 2 |
Q8_0 |
8 | blocks of 32 elements: a binary16 scale and 32 signed bytes | 34 per block |
I8 |
24 | signed integer | 1 |
I16 |
25 | signed integer | 2 |
I32 |
26 | signed integer | 4 |
I64 |
27 | signed integer | 8 |
F64 |
28 | binary64 | 8 |
BF16 |
30 | the 16-bit truncation of binary32 | 2 |
GGUF defines further types, which this reader refuses (dtype_unknown): their sizes
are not stated here. The file's string values are its metadata; a value of another type
is read and checked and not kept, so a file that holds one is not canonical (§11.2).
9.3 What a reader obtains
The metadata, a map from strings to strings, and the tensors, each a name, an element type, a shape and bytes, in ascending byte order of their names. Two readers that follow this section obtain the same tensors from the same file: no rule leaves a choice, since a name twice and overlapping or uncovered bytes are refused. That is why the registry lists both forms as established exact (§5): reading hands the decoder the bytes, and what the decoder makes of a floating-point tensor is its own determinism contract.
10. The integer-quantised profile wai-int/1
A prior is in the profile when it is a tensor file, in a form of §9 or the NNC form of §12, whose every tensor is a signed integer the kernel reads as it is. No scale is applied on load; any scale the kernel uses (a multiplier, a shift) is itself an integer tensor; nothing is floating point. The profile is how a network trained in floating point reaches an integer kernel: whoever quantises it makes the integers once (the integer decoders' priors round half to even at a fixed scale), and every sink then reads the same integers, with no rounding on load.
10.1 Metadata and element types
| form | the metadata, exactly |
|---|---|
| safetensors, NNC | wai.binding: a binding of §10.2; wai.profile: wai-int/1 |
| GGUF | the same, and general.architecture: wai, a key the GGUF layout requires |
In NNC the metadata travels in the topology unit (§12.3). The element types are I8,
I16, I32 and I64 (GGUF types 24 to 27), little-endian two's complement. NNC
carries every integer tensor as I32 (§12.1), and the binding's type then bounds its
values.
10.2 Bindings
A binding maps a native artifact, byte for byte, to named integer tensors and back.
wsr1/1: the WSR1 prior of exact-upscale §3.1, with layers
0 to n − 1, n from 1 to 8, <i> written in plain decimal:
| tensor | type | shape | values |
|---|---|---|---|
act_hi |
I16 |
[1] | 1 to 32767 |
layer.<i>.weight |
I8 |
[cout, cin, k, k] | any |
layer.<i>.bias |
I32 |
[cout] | any |
layer.<i>.mult |
I32 |
[cout] | 1 to 65535 |
layer.<i>.shift |
I8 |
[1] | 0 to 31 |
cin is 3 in layer 0 and the previous layer's cout after it; k is 1, 3 or 5; cout
is 1 to 64, and 12 in the last layer. The native bytes are written as exact-upscale §3.1
lays them out from these values. A native artifact that does not read under that section
is native_malformed.
whm1/1: the WHM1 model of integer-payloads §9 (the magic,
afrac, gf, shift, the layer count, then each layer by its tag), with layers 000
to n − 1 written in three digits, n at most 1000:
| tensor | type | shape | values |
|---|---|---|---|
header |
I64 |
[3] | afrac, gf, shift: each 0 to 2^32 − 1 |
layer.NNN.conv (tag 1) |
I64 |
[5] | in, out, k: each 1 to 2^32 − 1; stride, pad: each 0 to 2^32 − 1 |
layer.NNN.convt (tag 0) |
I64 |
[6] | in, out, k as for conv; stride, pad, opad: each 0 to 2^32 − 1 |
layer.NNN.weight |
I16 |
conv: [out, in, k, k]; convt: [in, out, k, k] |
any |
layer.NNN.mult, layer.NNN.bias |
I64 |
[out] | any |
layer.NNN.igdn (tag 2) |
I64 |
[1] | c: 1 to 2^32 − 1 |
layer.NNN.gamma |
I32 |
[c, c] | any |
layer.NNN.beta |
I64 |
[c] | any |
layer.NNN.relu (tag 3) |
I8 |
[1] | 0 |
layer.NNN.leaky (tag 4) |
I64 |
[2] | slope_q: any; slope_frac: 0 to 2^32 − 1 |
A layer is the kind its one kind tensor (conv, convt, igdn, relu or leaky)
names, with the tensors listed under that kind. A WHM1 layer with a dimension of 0 has
no tensor form (binding_mismatch), and more than 1000 layers neither; a native artifact
that does not read as integer-payloads §9 lays it out is native_malformed.
10.3 Loading and encoding
A sink loads a tensor file of the profile in these steps, refusing at the first that fails:
- Read the file in its form (§9, §12), with the form's codes.
- Require the file canonical (§11):
header_not_canonical. - Require §10.1's metadata exactly,
wai.profilebeingwai-int/1(profile_absent); the binding registered (binding_unknown); every tensor of a profile integer type,I32in NNC (dtype_not_integer). - The binding's structure, then its names. For
wsr1/1:nis the number oflayer.<i>.weighttensors (<i>in plain decimal); the indices are0ton − 1withnfrom 1 to 8, and each weight's first and last dimensions give acoutand akthe table admits (binding_mismatch). Forwhm1/1: each layer index (three digits) has exactly one kind tensor and the indices are000ton − 1(binding_mismatch); then, layer by layer, its kind tensor (butrelu's) has the type and shape the table gives (binding_mismatch), and its in, out and k (conv,convt) or c (igdn) are from 1 to 2^32 − 1 (value_out_of_range). Then the tensors are exactly the names the binding gives for those layers (binding_mismatch), and then each, in the order of step 5, has the type and shape it gives (binding_mismatch). - Every value within its type and the binding's range (
value_out_of_range), tensor by tensor in this order: forwsr1/1,act_hi, then layer by layer itsweight,bias,multandshift; forwhm1/1,header, then layer by layer its kind tensor and, forconvandconvt,weight,mult,bias, forigdn,gamma,beta. - Write the native artifact from the values.
The kernel reads the native artifact's bytes through the capability's own reader. A prior loaded this way is the native artifact, byte for byte, so it decodes as the native artifact decodes. Encoding is the reverse: read the native artifact into tensors, add §10.1's metadata, and write the file canonically (§11).
11. The digest rule
A pin in a tensor-file form covers the whole file: the tensors' bytes and the canonical header that names them. The forms are canonical as stated below, and a sink refuses a file that is not (§10.3 step 2). So, in the profile, a file is a function of its tensors (their names, types, shapes and bytes) and its metadata, and one prior has exactly one file in each form: the digest identifies the tensors, and the tensors the digest. One prior in three forms has three digests; a pin names one, and §2 forbids resolving it with another.
11.1 safetensors
The header is {, then "__metadata__":{ with the metadata's members in ascending byte
order of their names and } (omitted when there is no metadata), then the tensors in
ascending byte order of their names, each "<name>":{"dtype":"<dtype>","shape":[<d>,…], "data_offsets":[<begin>,<end>]}, every two members separated by ,, then }: no
whitespace, strings escaped as RFC 8785 §3.2.2.2 escapes them, integers in plain
decimal. Then 0x20 bytes until 8 + N is a multiple of 8. The tensors' bytes follow in
the same order, contiguous from offset 0.
11.2 GGUF
Version 3. The metadata pairs are string values (type 8), in ascending byte order of
their keys, with no general.alignment (the alignment is 32). The tensor descriptions
are in ascending byte order of their names. The first tensor is at offset 0, and each
later one at the first multiple of 32 after the end of the one before; the bytes
between the header and the data, and between tensors, are zero; nothing follows the last
tensor.
11.3 NNC
The bitstream §12.3 writes.
12. The NNC reader and writer
12.1 What was read, the subset, and where it stops
Readable: the public sample of the 2022 edition (its scope, the definition of an NNR unit, its number conventions and its clause titles) and an open implementation that states compliance with the 2024 edition (§8): its high-level syntax and its arithmetic coder. The syntax of §12.2 is the syntax that implementation reads and writes, and Appendix A is its arithmetic coding. The standard's syntax, decoding and entropy-coding clauses were not read, so nothing here states a field's semantics beyond what that implementation does with it. The reader and writer were measured against that implementation (§12.4), not against the conformance bitstreams of ISO/IEC 15938-18.
The subset: one NNR_STR, first; one NNR_MPS, before any data unit, with no
performance map and topology elements referenced by name; at most one NNR_TPL, after
the MPS, of a storage format that carries its topology as a string (0, 3 or 4) with raw
compression; one NNR_NDU per tensor, each carrying one topology element by name, its
input parameters and dimensions in band, and a payload of one of two types:
NNR_PT_INT(0): integers, withdq_flag0,scan_order0 and the unary length in band, coded as Appendix A codes them; read back asI32.NNR_PT_RAW_FLOAT(2): 32-bit floats, little-endian, as they are; read back asF32.
Everything else is refused with nnc_not_read, naming what was met: the units
NNR_LPS, NNR_QNT and NNR_AGG; a partial data counter; a performance map; topology
elements referenced by index; another topology storage format or a compressed topology;
the payload types NNR_PT_FLOAT (quantised parameters) and NNR_PT_BLOCK; several
topology elements in one unit; a decompressed data format other than the payload's
own; input parameters, dimensions or the unary length out of band; compressed parameter
types other than 0; dq_flag 1 (dependent quantisation); a scan_order other than 0
(block scans with entry points); a tensor of more than 2^28 elements.
Where it stops, and why. A quantised payload reconstructs each parameter from a quantisation index and a step size, and that arithmetic, with whatever tolerance conformance allows it, is in the text that was not read; dependent quantisation and block scans add a state machine and entry points an integer prior does not need. Within the subset, decoding is integer arithmetic on fixed widths from the first bit to the last (Appendix A), so the integers a reader obtains are a function of the bits, and a prior in the profile round-trips exactly (§14).
12.2 Reading
Bits are read most significant first: u(n) is an n-bit unsigned integer, i(n) an n-bit
two's complement integer, ue(k) the order-k exponential-Golomb code (zero bits, each
adding 2^k to the value and raising k by one, until a one bit, then k more bits added to
the value), and a string a zero-terminated UTF-8 byte sequence that begins on a byte
(nnc_malformed otherwise). A byte alignment is a 1 bit, then 0 bits to the byte
boundary (nnc_malformed).
Units are read in order until the bitstream ends; an empty bitstream is
nnc_truncated. Each unit: nnr_unit_size_flag u(1), then nnr_unit_size, u(15) or,
with the flag, u(31), counting the whole unit, its size field included (a unit under 3
bytes or past the end of the bitstream: nnc_truncated); nnr_unit_type u(6);
independently_decodable_flag u(1); partial_data_counter_present_flag u(1). The first
unit is NNR_STR (nnc_malformed), and a partial data counter is not read
(nnc_not_read), in that order. A field that runs past its unit's end, and a string
with no zero byte before it, are nnc_truncated; an exponential-Golomb code longer than
40 bits is nnc_malformed. Then by type, fields in this order, each check where its
field is read:
NNR_STR(0): only the first unit (nnc_malformed);general_profile_idcu(8).NNR_MPS(1), once and before any data unit (nnc_malformed):topology_carriage_flag, four performance-map flags,mps_quantization_method_flagsu(3),mps_topology_indexed_reference_flag, seven reserved bits; the reserved bits are zero (nnc_malformed), then the performance-map flags are zero and then the indexed-reference flag is (eachnnc_not_read); with quantisation flags,mps_qp_densityu(3) andmps_quantization_parameteri(13); a byte alignment.NNR_TPL(3), once and after the MPS (nnc_malformed):topology_storage_formatu(8),topology_compression_formatu(8), the topology as a string. In storage format 0, a topology that is a JSON object of string values is the file's metadata (§10.1).NNR_NDU(5), after the MPS (nnc_malformed):nnr_compressed_data_unit_payload_typeu(5),nnr_multiple_topology_elements_present_flagu(1),nnr_decompressed_data_format_present_flagu(1),input_parameters_present_flagu(1), after which the payload type is 0 or 2 and then the multiple-elements flag is 0 (eachnnc_not_read); the topology element's name, a string, not empty (nnc_malformed); forNNR_PT_INT,dq_flagu(1), 0 (nnc_not_read);nnr_decompressed_data_formatu(7) where present, 0 forNNR_PT_INTand 1 forNNR_PT_RAW_FLOAT(nnc_not_read); the input parameters are in band (nnc_not_read);tensor_dimensions_flagu(1),cabac_unary_length_flagu(1),compressed_parameter_typesu(4), which is 0, and then the dimensions are in band (eachnnc_not_read); the dimension count ue(1), from 1 to 16, and each dimension ue(7), outermost first, whose product is from 1 to 2^64 − 1 (eachnnc_malformed) and at most 2^28 (nnc_not_read);cabac_unary_length_minus1u(8) where flagged; with more than one dimension,scan_orderu(4), 0 (nnc_not_read); a byte alignment; then the payload, to the unit's end. An integer payload with the unary length out of band isnnc_not_read.
A unit other than a data unit ends exactly where its fields end, and a data unit's
payload fills the rest of it: four bytes per element of raw floats, and an integer
payload whose decoding (Appendix A) ends exactly at the unit's end (nnc_malformed).
Two data units carrying one name are nnc_malformed, checked once every unit has been
read.
12.3 Writing
NNR_STR with profile 0. NNR_MPS with topology_carriage_flag 1 when there is
metadata and every other field 0. With metadata, NNR_TPL of storage format 0 and
compression 0, its topology the metadata as a JSON object, keys in ascending byte order,
no whitespace, strings escaped as RFC 8785 §3.2.2.2 escapes them. Then one data unit per
tensor, in ascending byte order of names: an integer tensor as NNR_PT_INT, a binary32
tensor as NNR_PT_RAW_FLOAT; independently_decodable_flag 1, no partial data counter,
one topology element, no decompressed data format, input parameters in band with both
flags 1, compressed_parameter_types 0, cabac_unary_length_minus1 10, scan_order 0
where present, dq_flag 0. Each unit takes the 15-bit size when the whole unit is under
32 768 bytes, else the 31-bit size. An integer past ±(2^31 − 1) has no NNR_PT_INT form
(value_out_of_range).
12.4 Measured against an open implementation
On 2026-10-22 the subset was measured against the open implementation of §12.1 (source
at commit 0e3dcd2961701572d253f175754c09952a4b44c7, its arithmetic coder compiled from
that source; arm64): for each of the seven priors of the corpus (§14), that
implementation's decoder read the NNC file this extension's writer made to the integers
the prior holds; its encoder, given those integers (one topology element per data unit,
dq_flag 0, scan_order 0, unary length 10, no parameter optimisation), wrote every
data unit byte-identical to this writer's; and this reader read that encoder's whole
bitstream to the same integers. Its bitstream differs from this writer's only in the
topology unit, which carries §10.1's metadata here. The check is
tools/nnc_interop_check.py; it runs the other
implementation's code, so it is a measurement this repository records, not a CI step.
13. Refusals
| code | where |
|---|---|
header_truncated |
§9.1 steps 1 and 2; §9.2: a field, a count or the data's start past the end |
header_too_large |
§9.1 step 2 |
header_malformed |
§9.1 steps 3 and 4; §9.2 steps 4, 5 and 7 |
bad_magic |
§9.2 step 1 |
version_unsupported |
§9.2 step 2 |
tensor_malformed |
§9.1 steps 5 and 6; §9.2 step 6 |
dtype_unknown |
§9.1 step 5; §9.2 step 6; a tensor of a type a writer's form has no name for |
offset_mismatch |
§9.1 steps 6 and 7; §9.2 steps 6 and 7 |
header_not_canonical |
§10.3 step 2: the file reads and is not its canonical form (§11) |
nnc_truncated |
§12.2: a unit past the end; an integer payload that ends inside its unit (Appendix A) |
nnc_malformed |
§12.2, Appendix A |
nnc_not_read |
§12.1: outside the subset, with what was met |
profile_absent |
§10.3 step 3 |
binding_unknown |
§10.3 step 3 |
dtype_not_integer |
§10.3 step 3 |
binding_mismatch |
§10.3 step 4; §10.2: a native artifact with no tensor form |
value_out_of_range |
§10.3 steps 4 and 5; §12.3: an integer NNC cannot carry |
native_malformed |
§10.2: a native artifact that does not read under its own format |
14. Conformance corpus
live-conformance/prior-forms/ holds, for seven
native artifacts other corpora commit, each pinned by SHA-256 (two trained WSR1
priors and a seeded one of the exact-upscale corpus, four generated WHM1 models of the
integer-operators corpus), the canonical file in each of the three forms; native
artifacts that do not read or that NNC cannot carry; and negative controls, each refused
with its code. A vector is an encode (a native artifact to a file, or a refusal) or a
load (§10.3 over a file: the binding and the native artifact, or a refusal), and a
verifier conforms when it reaches every vector's outcome, its file bytes and its native
bytes. wai_prior_forms_vectors gen|verify writes and checks it, tests/prior_wire.rs
regenerates it and loads every prior in every form into the integer kernels, and
live-conformance/prior_forms_verify.py, written from this text and sharing no code
with wai-rs, is its second implementation.
Appendix A — The integer payload's arithmetic coding (normative)
The integers of an NNR_PT_INT payload are binarised and coded with context-adaptive
binary arithmetic coding. Every quantity is an integer; >> is an arithmetic shift
(division by a power of two rounding toward negative infinity), and lz(x) is the number
of leading zero bits of x as a 32-bit word.
Contexts. With G = cabac_unary_length_minus1 + 1 greater-than flags there are
L = 24 + 3 + 2G + 32 + 4 contexts, numbered from 0: 0 to 23 for significance (three for
each of eight quantiser states; with dq_flag 0 only 0 to 2 are used), 24 to 26 for the
sign, 27 to 26 + 2G for the greater-than flags (two per flag, positive then negative),
27 + 2G to 58 + 2G for the remainder's prefix, and four more; the one numbered
L − 2 codes the initialisations. A context holds s0 (a signed 8-bit integer), s01
(a signed 16-bit integer) and two shifts h0, h1. Initialisation j (0 to 8), the row
[b0, b1, v01, v0] of table A.3, sets s0 = v0, s01 = v01, h0 = b0 + 4,
h1 = b1. A context's most probable symbol is 1 when s01 ≥ 0, else 0. After a bin
b, with σ = +1 when b = 1 and −1 otherwise:
s1 = s01 − 16·s0
s0 = wrap8 ( s0 + σ·( C[16 + ((σ·s0) >> 3)] >> h0 ) )
s1 = s1 + σ·( C[16 + ((σ·s1) >> 7)] >> h1 )
s01 = wrap16( s1 + 16·s0 )
where wrap8 and wrap16 keep the low 8 and 16 bits as two's complement and C is
table A.2 (an index outside it is nnc_malformed). A context's least-probable range for
a coder range R is T[|s01 >> 7| + (R & 0xE0)], T being table A.1 (an index
outside it is nnc_malformed).
Decoding. The coder starts with range = 510, value = 256·B0 + B1 from the
payload's first two bytes (fewer: nnc_truncated), need = −8, and two bytes read. Reading
a byte past the payload is nnc_truncated.
bin(ctx): r = T(ctx, range); m = range − r
if value < m·128: b = mps(ctx); range = m
else: value −= m·128; range = r; b = 1 − mps(ctx)
n = lz(range) − 23; range <<= n; value <<= n; need += n
if need ≥ 0: value += readbyte() << need; need −= 8
update(ctx, b); return b
bypass(): value = 2·value; need += 1
if need ≥ 0: value += readbyte(); need = −8
if value ≥ range·128: value −= range·128; return 1
return 0
bypass(k): k bypass bins, the first the most significant bit of a k-bit number
finish(): range −= 2
value ≥ range·128, else nnc_malformed (the terminating bin)
((last byte read) << (8 + need)) & 0xFF = 0x80, else nnc_malformed
the payload's length is the number of bytes read
The payload. First the initialisations: for each context i from 0 to L − 5,
skipping 3 to 23, decode a bin with context L − 2; when it is 1, j = 1 + bypass(3),
else j = 0; initialise context i with j (context L − 2 keeps adapting). Then each
of the tensor's elements in row-major order, p being the previous element's value (0
before the first) and c(p) 0, 1 or 2 as p is zero, negative or positive:
if bin(c(p)) = 0: value 0
else:
v = 1; s = bin(24 + c(p))
g = bin(27 + s); k = 1
while g = 1 and k < G: v += 1; g = bin(27 + s + 2k); k += 1
if g = 1:
v += 1; q = 27 + 2G; rem = 0
if bin(q) = 1:
rem = 1; q += 1; e = 1
loop: q ≥ 59 + 2G → nnc_malformed
if bin(q) = 0: stop
rem += 2^e; q += 1; e += 1
rem += bypass(e)
v += rem
v past 2^31 − 1 → nnc_malformed
value v, or −v when s = 1
then finish().
Encoding. The writer's coder starts with low = 0 (a 32-bit word: every addition and
left shift below keeps its low 32 bits), range = 510, left = 23, a buffered byte
B = 0 and n = 0 bytes buffered.
out(): lead = low >> (24 − left); left += 8; low &= 0xFFFFFFFF >> left
if lead = 0xFF: n += 1
else if n > 0: c = lead >> 8; emit (B + c) mod 256; B = lead mod 256
emit (0xFF + c) mod 256, n − 1 times; n = 1
else: n = 1; B = lead
bin(b, ctx): r = T(ctx, range); range −= r
if b = mps(ctx): if range < 256: range = 2·range; low = 2·low; left −= 1
else: k = lz(r) − 23; low += range; range = r << k; low <<= k; left −= k
if left < 12: out()
update(ctx, b)
bypass(b): low = 2·low; if b = 1: low += range
left −= 1; if left < 12: out()
finish(): range −= 2; low += range; low <<= 7; range = 256; left −= 7
if left < 12: out()
if low >> (32 − left) ≠ 0: emit (B + 1) mod 256; emit 0x00, n − 1 times
low −= 2^(32 − left)
else: if n > 0: emit B
emit 0xFF, n − 1 times
low = 2·(low >> 8) + 1; left −= 1; low <<= 32 − (24 − left)
for i = 0, 8, 16, … while i < 24 − left: emit (low >> (24 − i)) & 0xFF
The writer codes every initialisation as 0: for each context i from 0 to L − 5,
skipping 3 to 23, a bin 0 with context L − 2. Then each value v, p as above:
bin(v ≠ 0, c(p))
if v ≠ 0:
s = 1 if v < 0 else 0; bin(s, 24 + c(p))
r = |v| − 1; g = (r > 0); bin(g, 27 + s); k = 1
while g = 1 and k < G: r −= 1; g = (r > 0); bin(g, 27 + s + 2k); k += 1
if g = 1:
r −= 1; q = 27 + 2G
if r = 0: bin(0, q)
else: bin(1, q); base = 1; e = 1; q += 1
while r > base + 2^e − 1: bin(1, q); base += 2^e; q += 1; e += 1
bin(0, q); r − base as e bypass bins, most significant first
then finish().
Table A.1 — T, 256 entries, row x holding entries 32x to 32x + 31:
128 112 97 84 74 65 57 50 45 39 34 30 27 23 20 18 15 14 12 11 10 9 7 7 5 5 4 4 3 3 2 2
142 125 108 93 82 72 63 56 50 43 38 33 30 26 22 20 17 16 13 12 11 10 8 8 6 6 5 5 3 3 2 2
156 137 119 103 90 79 70 61 55 48 42 37 33 28 24 22 19 17 15 13 12 11 9 9 6 6 5 5 4 4 2 2
171 150 130 112 99 87 76 67 60 52 46 40 36 31 27 24 21 19 16 15 13 12 10 10 7 7 6 6 4 4 3 3
185 162 141 121 107 94 82 73 65 56 50 43 39 34 29 26 22 21 17 16 14 13 11 11 8 8 6 6 4 4 3 3
199 175 152 131 115 101 89 78 70 61 54 47 42 36 31 28 24 22 19 17 15 14 12 12 8 8 7 7 5 5 3 3
213 187 163 140 123 108 95 84 75 65 58 50 45 39 33 30 26 24 20 18 16 15 13 13 9 9 7 7 5 5 3 3
228 200 174 150 132 116 102 90 80 70 62 54 48 42 36 32 28 26 22 20 18 16 14 14 10 10 8 8 6 6 4 4
Table A.2 — C, 32 entries:
2512 2288 2064 1840 1616 1392 1168 944 720 560 464 368 272 208 144 80
64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 0
Table A.3 — the nine initialisations [b0, b1, v01, v0]:
0: [1, 4, 0, 0] 1: [1, 4, -1310, -41] 2: [1, 4, 3039, 95]
3: [0, 5, 0, 0] 4: [2, 6, 962, 30] 5: [2, 6, 3039, 95]
6: [2, 6, -673, -21] 7: [3, 5, 0, 0] 8: [3, 5, 962, 30]