diff --git a/.github/workflows/wai-neural-gates.yml b/.github/workflows/wai-neural-gates.yml index 638c73d26..deb814350 100644 --- a/.github/workflows/wai-neural-gates.yml +++ b/.github/workflows/wai-neural-gates.yml @@ -1,10 +1,12 @@ name: WAI neural sink gates # The neural sink refuses some envelopes before it touches a model: a pinned -# prior whose hash does not match the registered model -# (ModelRegistry::resolve_checked), and a `replicate` that SPEC §7 does not back -# — over a capability whose registry class cannot back it, or with a prior that -# is float or absent (decode_envelope_strict). Those refusals live behind the +# prior whose hash does not match the registered model, or that declares a form +# the sink cannot read (ModelRegistry::resolve_checked), and a `replicate` that +# SPEC §7 does not back (decode_envelope_strict): over a capability whose registry +# class cannot back it, or with a prior that is float or absent, or that declares a +# form not established to reconstruct the same tensors at every reader. Those +# refusals live behind the # `neural` feature, which wai-conformance does not build, and the tests that # exercise them need no model, so wai-neural-e2e's reference filter never selects # them. This job runs exactly those tests and fails if any of them is not run: @@ -17,6 +19,7 @@ on: - 'wai/wai-rs/src/neural/**' - 'wai/wai-rs/src/container.rs' - 'wai/wai-rs/src/codecs/mod.rs' + - 'wai/wai-rs/src/codecs/prior_form.rs' - 'wai/wai-rs/tests/neural_native_decode.rs' - 'wai/wai-rs/Cargo.toml' - 'wai/wai-rs/Cargo.lock' @@ -26,6 +29,7 @@ on: - 'wai/wai-rs/src/neural/**' - 'wai/wai-rs/src/container.rs' - 'wai/wai-rs/src/codecs/mod.rs' + - 'wai/wai-rs/src/codecs/prior_form.rs' - 'wai/wai-rs/tests/neural_native_decode.rs' - 'wai/wai-rs/Cargo.toml' - 'wai/wai-rs/Cargo.lock' diff --git a/wai/CHANGELOG.md b/wai/CHANGELOG.md index 7e9e76b20..304190916 100644 --- a/wai/CHANGELOG.md +++ b/wai/CHANGELOG.md @@ -6,6 +6,96 @@ wire format may change and such changes are recorded here. ## v0.4.0 — current +### Prior carriage and derived-track operations — registered with their limits + +- New [`extensions/prior-carriage.md`]: a `Prior` MAY declare the wire form of + the artifact its digest covers (`prior.form`, additive on the wire; manifests + without it round-trip byte for byte). When declared, the digest covers the coded + bitstream, and a sink that cannot read the form refuses the prior rather than + substitute another packaging. `wai.prior.nnc` (ISO/IEC 15938-17:2024) is the + registered form. It is optional, not recommended: royalty status is not + established, and whether conforming decoders reconstruct identical tensors is + not established, so a `replicate` cannot rest on a prior read out of a form not + listed as exact (none is). As a payload, NNC takes the replicate class of a + standard whose decoder conformance the registry has not checked, as MPEG-I + Haptics does; its own conformance clauses were not read. `wai.tensor.nnc` is + registered as a candidate for general tensors; its coding is in an unpublished + amendment, which + is tracked, not pinned. The extension also fixes the precedence of in-band + data-unit signatures: WAI's digest covers every byte, and an in-band verdict is + recorded beside it, never instead of it. +- New [`extensions/derived-tracks.md`]: the ten derivation operations of + ISO/IEC 23001-16:2021 are registered as `wai.derive.*` strings, each with a + per-operation determinism tier. Dissolve and Scaling are untiered. No operation + is exact unconditionally, Identity included. An executed instance is exact only + when no sample value is converted into another representation or computed, its + output does not change over the span the claim covers, and its operation's own + conditions hold; the sink lists each condition it checked. A derived sample + holds the tier of its weakest step. A claim over it names the operations + actually executed, the parameter values each applied, the span it covers and + its inputs, visual and parameter. These are not payload capabilities, and WAI + does not composite. Clause 8 of the standard, which defines each operation, was + not read. Every tier rests on conditions checked on the executed instance; the + geometric conditions listed per operation come from the operation's title, as + does whether it reads a visual input, and reading clause 8 may show a list to be + incomplete. Clause 4 crops or fills an input of another size implicitly for + every operation, and no list carries a condition on where such an input is + placed. +- `wai-rs`: `codecs::derivation` (registry, `Condition`, `derived_sample_tier`), + `codecs::DeterminismTier` (the two tiers as values, re-exported by + `codecs::derivation`), `codecs::prior_form`, `Prior::{new, with_form, form, + check_form, form_preserves_exactness}`, and `container::replicate_prior_refusal`, + which states why a `replicate` is refused, including over a declared form. +- **Behaviour change:** `Prior::is_cross_hardware_exact` now also requires the + declared form, if any, to be established exact (`form_preserves_exactness`), so + a prior that declares a form is not cross-hardware exact whatever its contract + says; `determinism_class` still reads the contract alone. A prior without a form, + which is every prior written before `form` existed, gets the same answer as + before. The rule sits in that predicate because every `replicate` check reads + it, `container::replicate_is_sound` and `Wai::replicate_is_sound` included. +- Each `ExecutedOp` names the visual inputs its instance read (`reads`, as an + entry of `inputs` or the output of an earlier executed operation), because + clause 4 of ISO/IEC 23001-16 lets an operation that is not first read the + output of any previous operation, only new inputs, or both, so its position + does not say what it read. An instance of an operation that reads a visual + input and names nothing gives no tier, wherever it sits: it read an input that + nothing names, whether a track, item, track group or preceding output the file + lists or, where nothing lists one, the default fill picture (clause 5.1). When + at least one operation executed, a call that contradicts itself (a reference to + an unlisted input or to an operation that does not precede the reader, or a + listed input no instance names) is a `DerivedSampleError`; an empty executed + list gives no tier before those checks. `neural::ModelRegistry::resolve_checked` + refuses a declared form the sink cannot read, and the reference sink reads + none. A new CI workflow, `wai-neural-gates`, is configured to run the two + model-free tests of those neural-sink refusals, which no workflow ran before, + in an Ubuntu 24.04 container image; it has not yet run on a runner. +- **Rust API, source-breaking.** `container::Prior` gained a public field, + `form`. Its fields are all public and it is not `#[non_exhaustive]`, so Rust + code outside the crate that builds a `Prior` with a struct literal, or + destructures one without `..`, stops compiling. The fix is to add + `form: None`, or to build it with the new `Prior::new(id, sha256, + determinism)` (and `.with_form(form)`), which names no optional field and so + is unaffected by the next optional field added. Reading fields, serde, and + functional update (`Prior { form, ..other }`) are unaffected. SPEC §8 versions + the wire format, not this API. On the wire, `form` is an optional key, added + as `prior` itself was, and absent it the manifest's bytes are unchanged. SPEC §8 + also says the manifest schema is frozen for all of 1.x; how an optional key is + added under that freeze is not settled here, for `form` or for `prior`. For the + crate, Cargo's semver rules treat a source-breaking change in a 0.x crate as + needing a new minor version: the release that carries this must be 0.4.0 or + later (`wai-rs/Cargo.toml` still says 0.3.0). `#[non_exhaustive]` was not + added. It would break the same literals now and, in addition, rule out the + `form: None` fix and functional update from outside the crate, which the + crate's own integration tests use. It would also make `Prior` the only + manifest struct that code outside the crate cannot build with a literal; the + others are `Manifest`, `ModelRequirement`, `Conditioning`, `ManifestV2` and + `RenditionMeta`. Whether they should all become `#[non_exhaustive]`, so that + adding an optional key never breaks Rust callers, is a decision for all of them + together. +- Open: the operations' four-character codes are not recorded (clause 8 of + ISO/IEC 23001-16 was not read); the `wai` CLI does not list derivation + operations; there is no NNC reader. + ### Broadcast and streaming codecs — registered, licence-classified, never defaults - SPEC §5 registers the elementary video and audio bitstreams distribution chains diff --git a/wai/SPEC.md b/wai/SPEC.md index f4d653c73..f1af42599 100644 --- a/wai/SPEC.md +++ b/wai/SPEC.md @@ -7,8 +7,10 @@ file: - the **neural condition** — the sink regenerates the media from a compact conditioning payload against a *shared ambient prior* (a model). The model is a **requirement of existence**, never - transported and never hash-pinned, exactly as `.mp3` does not ship - its decoder. + transported in the envelope that decodes against it, exactly as `.mp3` + does not ship its decoder. Its identity MAY be pinned by digest (a + `Prior`), and the pinned artifact's wire form MAY be declared + ([prior-carriage](extensions/prior-carriage.md)). - the **zeroth condition** — a *menu of registered SOTA standard codecs*. Always satisfiable on any device that ships ffmpeg or equivalent. WAI does **not** define new codecs; it dispatches to the @@ -283,6 +285,21 @@ companion energy assertion: the joules a reconstruction cost, bound to that reconstruction, with the acquisition class and coverage C2PA 2.4's `c2pa.environmental-sustainability` has no field for. +### Prior artifacts, general tensors and derivation operations + +| capability | payload | status · extension | +|---|---|---| +| `wai.prior.nnc` | a trained network's parameters as an ISO/IEC 15938-17:2024 (NNC, 2nd edition) bitstream | the registered wire form a `Prior` MAY declare (`prior.form`); its digest then covers the coded bitstream. Optional, not recommended: royalty status is not established, and cross-decoder exactness of the reconstruction is not established, so a `replicate` over a neural capability cannot rest on a prior read out of it. As a payload it takes the replicate class of a standard whose decoder conformance the registry has not checked · [prior-carriage](extensions/prior-carriage.md) | +| `wai.tensor.nnc` | general tensorial data (split-inference feature maps, 3D/4D scene parameters) coded with NNC | **candidate**: general-tensor coding is in an unpublished amendment, so the payload format is not fixed and MUST NOT be emitted until pinned · [prior-carriage](extensions/prior-carriage.md) | + +The ten derivation operations of ISO/IEC 23001-16:2021 (derived visual tracks) +are registered as `wai.derive.*` strings, each with a per-operation determinism +tier. They are **not payload capabilities**: an operation acts on samples another +capability decoded. WAI does not composite; the registration states what a +reproducibility claim over a derived sample needs. The geometric conditions a +tier lists for each operation are read from the operation's title: clause 8 of +ISO/IEC 23001-16, which defines each operation, was not read · [derived-tracks](extensions/derived-tracks.md). + ### Interactive, temporal & computed content (extensions) Media that consumes input, moves over time, or is *computed* from a @@ -793,7 +810,7 @@ cited are the ones read; the evidence, and what was not read, is in | | `wai.video.av2` | as AV1: identical output unless film grain is signalled, then one of the same two options (AV2 v1.0.0 §7.1) | | | `wai.audio.ivas` | output bit-exact to the test sequences, or, for an implementation based on the floating-point reference code whose output is not, a non-bit-exact procedure: TS 26.444's for mono operation, a per-frame loudness-difference corridor for stereo and immersive operation (3GPP TS 26.252 V19.2.0 §7.1, §7.2.1) | | | `wai.audio.iamf` | its substreams include Opus (above), and renderers are out of its scope | -| `StandardUnclassified` | `wai.video.evc`, `wai.video.evc.baseline`, `wai.video.avs3`, `wai.video.lcevc`, `wai.audio.mpegh`, `wai.audio.avs3`, `wai.audio.ac4`, `wai.mesh.draco`, `wai.pc.gpcc`, `wai.pc.vpcc`, `wai.splat.sog`, `wai.haptic.mihs`, `wai.meta.hdr10plus`, `wai.meta.dovi` | not checked in this revision (all sink-supplied; `wai.splat.sog` names no frozen standard yet) | +| `StandardUnclassified` | `wai.video.evc`, `wai.video.evc.baseline`, `wai.video.avs3`, `wai.video.lcevc`, `wai.audio.mpegh`, `wai.audio.avs3`, `wai.audio.ac4`, `wai.mesh.draco`, `wai.pc.gpcc`, `wai.pc.vpcc`, `wai.splat.sog`, `wai.haptic.mihs`, `wai.meta.hdr10plus`, `wai.meta.dovi`, `wai.prior.nnc`, `wai.tensor.nnc` | not checked in this revision (all sink-supplied; `wai.splat.sog` names no frozen standard yet) | | | `wai.audio.aac` | not checked in this revision; three decoders on one host gave three different PCM streams from one ADTS stream, so it is not placed where byte identity would be promised | | | `wai.mesh.glb`, `wai.splat.spz` | the payload bytes pass through intact; what a sink decodes from them is not classified here (a glTF binary may carry JPEG images in its buffer and compressed geometry through extensions, and glTF 2.0 specifies an asset format, not a decoder conformance over what is rendered; SPZ stores quantised attributes) | @@ -874,7 +891,11 @@ reference impl checks this per-envelope in `decode_envelope_strict` class: only `NeuralIntegerExact` and the three standard classes pass, whatever the envelope declares (`ReplicateClass::replicate_promise` states what each promises). Then, for a `NeuralIntegerExact` capability, the pinned prior: it must -declare an integer-exact contract (`DeterminismClass`). The prior check reads the +declare an integer-exact contract (`DeterminismClass`), and a prior that declares +the wire form of its artifact counts as integer-exact only if that form is +established to reconstruct identical tensors at every reader +([prior-carriage](extensions/prior-carriage.md) §5); no form is yet, so a +`replicate` whose prior declares one is refused. The prior check reads the sender's declaration; the class check is what ties it to the capability, so an envelope whose prior declares an integer contract over a float or entropy-only capability is refused. `decode_envelope_strict` decodes only through diff --git a/wai/extensions/derived-tracks.md b/wai/extensions/derived-tracks.md new file mode 100644 index 000000000..19735aa91 --- /dev/null +++ b/wai/extensions/derived-tracks.md @@ -0,0 +1,323 @@ +# WAI Extension: Derived Tracks + +> Status: Draft. Registers the derivation operations of ISO/IEC 23001-16:2021 +> as `wai.derive.*` capability strings, each with a **per-operation determinism +> tier**, and states how a tier composes over a derived sample. No wire format, +> and no compositing. The geometric conditions listed per operation are read +> from the operations' titles: clause 8 of ISO/IEC 23001-16, which defines each +> operation, was not read. Reference impl: `wai-rs` `codecs::derivation`. +> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174. + +## 1. Prior art, stated plainly + +A track that carries a process the receiver evaluates, rather than the processed +result, is not new. **ISO/IEC 23001-16:2021**, *Derived visual tracks in the ISO +base media file format* (1st edition, 2021-11), ratified it. A derived visual +track (sample entry `dtrk`) carries derived samples, each active for a duration +of the track's composition timeline. Each derived sample is an ordered list of +derivation operations, each applying a derivation transformation, with its +parameters, to an ordered list of inputs. An input is either a **visual input** +(an image item, an interval of an input track or track group possibly spanning +several samples, the visual output of a preceding operation, or a default fill +picture) or a **parameter input** (a metadata item, or an interval of a metadata +track possibly spanning several samples). A parameter takes the value the derived +sample gives it, else the value in the sample entry, else the default its +transformation's clause defines. A visual output is one frame or a sequence of +frames, and any transformation may have internal time structure (the standard's +example is a cross-fade), so the picture may change during the sample's duration. +Operations chain, and a derived track can be an input to another. The first +edition defines a base set of ten transformations, each identified by a +four-character code, with `'uuid'` reserved for vendor transformations identified +by UUID. + +The parts of that specification read for this extension (clauses 1–4, 5.1 and +5.2, and the table of contents, from the published sample, which ends at 5.2) do +not say whether an operation's output reproduces byte for byte across conforming +readers. The rest was not read: clause 5.3 (the semantics of the derivation +boxes), clauses 6 and 7 (the sample entry, the configuration record and the +sample format), clause 8, which gives each operation's definition, syntax and +semantics, and so any blend or resampling formula, and Annex A. This extension +states, per operation, when a +claim over its output may hold, in the vocabulary of +[determinism-tiers](determinism-tiers.md), so a receipt or binding over a derived +sample can say which tier it actually holds. Every tier rests on conditions that +the sink which executed an instance checks on that instance, in space and in +time. Clause 8 decides what an instance does, and so which conditions it can +meet. What the registry takes from outside clause 8 is each operation's purpose, +read from its title, which decides the geometric conditions listed for it (§2, +*Basis of the tiers*). The tiers are not uniform. Dissolve and Scaling compute +new sample values by their nature. The others are exact only for instances that +copy samples without converting them, and whose output does not change over the +span a claim covers. + +**Scope.** WAI does not composite. Presentation stays with the sink, as in +[linear-film](linear-film.md) and [interactive-worlds](interactive-worlds.md). +Registering grid and overlay composition states what a reproducibility claim over +their output would need; it does not make WAI a renderer. Registering the +operations aligns WAI's vocabulary with the standard; it says nothing about which +players evaluate derived visual tracks. + +## 2. Registered operations + +An executed instance of an operation is **exact** when every output sample is a +copy of an input sample, or of a constant the claim names, placed by integer +geometry, with no arithmetic on sample values, no change of sample +representation, and no change over the span the claim covers. Inputs at +`decode-equivalence` give an exact instance's output at `decode-equivalence`. A +per-operation tier is one of: + +- **exact when (conditions)**: an executed instance is exact when it meets the + three universal conditions below and every condition the table lists for its + operation, and untiered otherwise. The conditions are properties of the + executed instance, checked by the sink that executed it. **No operation is + exact unconditionally**, Identity included. +- **untiered**: the operation computes new sample values, by blending or + resampling, and WAI pins no arithmetic for it. The output carries no tier, + whatever its inputs held, until an arithmetic is pinned and an instance is + shown to follow it. Dependence on time is not what makes an operation + untiered: every operation needs time-invariant. + +The three **universal conditions**, required of every tiered operation: + +- **same-representation**: every visual input, and every constant the instance + writes (a fill colour, the default fill picture), is already in the output's + sample representation: the same bit depth, chroma format and chroma siting, + colour primaries, transfer characteristics, matrix coefficients, range and + component set. No sample value is converted. Clause 4 requires the visual + inputs of a derived sample to share pixel aspect ratio and bit depth with each + other; the clauses read say nothing about the other properties, or about the + output's representation relative to its inputs. So an 8-bit input composed into + a 10-bit output, a BT.709 input composed into a BT.2020 output, or an Identity + whose output representation differs from its input's holds no tier. +- **copies-only**: every output sample value is, bit for bit, the value of an + input sample or of a constant whose value the claim names. The instance + computed no sample value. +- **time-invariant**: over the span the claim covers, every visual input the + instance reads is one frame and every parameter it uses has one value. The + instance also has no time structure of its own, such as a transition or a + parameter interpolated over time. So its output over the span is one frame, + fixed by those frames and values. The span is either the derived sample's + whole duration or one output frame at a composition instant the claim names + (§3 rule 3). A parameter counts whatever its source: the derived sample, the + sample entry, the default of the operation's clause, or a parameter input. + Clause 4 lets one derivation duration span several samples of an input track, + and lets any transformation have internal time structure. A visual or + parameter input may be an interval of a track spanning several samples, and a + visual output may be a sequence of frames. So two instances hold no tier for a + claim over the whole duration: an ROI selection whose region is driven by an + interval of a metadata track whose value changes within the sample, and an + Identity over an input interval holding several samples. Either can hold one + for a claim over one output frame, at an instant at which nothing it reads + changes, if it has no time structure of its own. + +Because every tiered operation requires all three, an instance is untiered, not +over-claimed, if it uses any parameter that clause 8 allows to convert or compute +sample values, or to vary its output over time. This holds as long as the +executing sink lists only the conditions it actually checked. + +| capability | 23001-16 clause | operation | tier | conditions beyond the universal three (read from the operation's title; clause 8 was not read), or why untiered | +|---|---|---|---|---| +| `wai.derive.identity` | 8.2 | Identity | exact when | none. It copies its input; uncovered output samples come from the default fill picture, a constant the claim must name (§3 rule 5). Whether clause 8 lets Identity emit a representation other than its input's was not read, so same-representation is checked, not assumed | +| `wai.derive.srgb_fill` | 8.3 | sRGB Fill | exact when | none. The colour is stated in sRGB; writing it into any other representation is a conversion, which same-representation excludes. As its title is read, it reads no visual input, and it is the only operation registered as reading none | +| `wai.derive.dissolve` | 8.4 | Dissolve | untiered | a transition that mixes two inputs, so its output values are computed, not copied. A transition is also time structure of its own, which time-invariant excludes for every operation (clause 4's example of internal time structure is a cross-fade). Untiered until a blend arithmetic, and how its weights depend on the instant, are pinned; clause 8.4, which may fix them, was not read | +| `wai.derive.crop` | 8.5 | Crop | exact when | the kept region lies on the sample grid of every component (no component is resampled) | +| `wai.derive.rotation` | 8.6 | Rotation | exact when | the angle is a multiple of 90°, and every component is sampled at the same resolution, so the rotation permutes samples | +| `wai.derive.mirror` | 8.7 | Mirror | exact when | the reflection maps every component's sample grid, including its siting, onto itself (always so without chroma subsampling) | +| `wai.derive.scaling` | 8.8 | Scaling | untiered | resamples, so its output values are computed, not copied. Untiered until an interpolation filter and its rounding are pinned; clause 8.8, which may fix them, was not read | +| `wai.derive.roi_selection` | 8.9 | Region of interest (ROI) selection | exact when | the region lies on the sample grid of every component, and is emitted at its own size | +| `wai.derive.grid_composition` | 8.10 | Grid composition | exact when | every input is placed at its own size, at offsets on every component's sample grid | +| `wai.derive.overlay_composition` | 8.11 | Overlay composition | exact when | every input is placed at its own size, at offsets on every component's sample grid, and opaquely (no alpha, opacity or blending) | + +- **Identification.** Each capability string stands for the transformation its + clause defines, including that clause's four-character code. The registry does + not yet record the codes: clause 8 of the published text was not available when + this was written. A sink maps each capability to the code its clause defines, + and MUST NOT map one by a code taken from any other source. Codes from earlier + drafts circulate in secondary sources. +- **Basis of the tiers.** Tiers are assigned from what each operation does to + sample values, in space and in time, not from clause 8, which was not read. + Each condition is a property of an executed instance, not of parameter syntax. + The three universal conditions exclude any conversion or computation of sample + values, and any change over the span a claim covers. What is taken from outside + clause 8 is each operation's purpose, read from its title (a crop selects, a + rotation rotates), which decides the geometric conditions listed for it, and + whether it reads a visual input at all (only sRGB Fill is registered as reading + none). Reading clause 8 may show one of those lists to be incomplete. It may + also make some conditions unnecessary (for example, if Rotation admits only + quarter turns, or if the output representation is always the inputs'). Clause 8 + may also fix a blend or resampling arithmetic bit-exactly; that would be the + start of a pin for Dissolve or Scaling, not a tier in itself. Clause 4, which + was read, already shows where a list may fall short: wherever an input's size + differs from the output's, the input is cropped or filled implicitly, for every + operation unless the operation's clause overrides that. The fill is covered by + §3 rule 5. The clauses read do not say where such an input is placed, and no + list, Identity's included, carries a condition on that placement. +- **Correction.** An earlier revision said that no tier here rested on anything + clause 8 might fix. The time dimension did. Only Dissolve was untiered for + depending on time, as if clause 4's cross-fade were particular to it, but + clause 4 states generally that a derivation transformation may have internal + time structure. Time-invariant is now a universal condition. +- **Not payload capabilities.** An operation names no payload format; it acts on + samples another capability decoded. SPEC §4 assigns every payload exactly one + capability, so these strings are not in `codecs::known_capabilities` and a + manifest cannot name one as its payload's capability. A sink advertises the + operations it evaluates the way it advertises codecs. +- **Licence.** The Introduction of ISO/IEC 23001-16:2021 records a patent claimed + to be required for compliance, whose holder has given a + reasonable-and-non-discriminatory licensing assurance. The registry classifies + the operations as patent-pool, conservatively + (`codecs::derivation::derivation_license`). + +## 3. Composing a tier over a derived sample + +1. **Weakest step.** A derived sample holds `decode-equivalence` only if every + visual input holds `decode-equivalence` and every executed operation instance + is exact: it meets the three universal conditions and every condition listed + for its operation. Otherwise it holds no tier, and a receipt over it MUST NOT + carry a reproducible derived-sample hash. +2. **`entropy-consistency` does not pass through.** Derivation operations act on + reconstructed samples, which that tier does not fix. A derived sample over an + input at `entropy-consistency` holds no tier. +3. **The claim is about what was executed, with what, and when.** ISO/IEC + 23001-16 lets a reader treat an unsupported non-essential operation as a null + operation, so two conforming readers can emit different samples from one + derived sample. A parameter can come from a metadata track, and the output + can change within the sample's duration. A tier claim or receipt over a + derived sample MUST name: + - the operations executed, in order; + - for each instance, the parameter values it applied, whatever their source + (the derived sample, the sample entry, the default of the operation's + clause, or a parameter input), with any region, offset or size expressed in + samples; + - the span it covers. For a claim over the whole duration, that is the derived + sample's composition time and duration on the derived track's timeline, in + its timescale. Otherwise it is the composition instant of each output frame + the claim covers; + - its inputs, visual and parameter, by content hash. For an interval of a + track, the hash covers the samples the instance read. + + Two sinks' derived-sample hashes are comparable only when all four are equal. + A sink MUST NOT present a hash computed with an operation skipped as the hash + of the sample with that operation, and MUST NOT present the hash of one output + frame as the hash of a derived sample whose output changes over its duration. +4. **Transforms applied before derivation count.** ISO/IEC 23001-16 applies the + transformative properties of inputs (clean aperture, track matrix, and the + like) before the derivation operation. For a tier claim each counts as a step, + tiered as the registered operation it performs. A unity matrix is identity; a + quarter-turn or reflection matrix is rotation or mirror; a clean aperture is a + crop. Any other matrix is untiered, whether or not it resamples (a scale, a + shear or a sub-sample translation does): the registry pins no arithmetic for + it. +5. **The default fill picture is an input.** Where output samples come from the + configuration record's default fill (black, grey or white), whatever the + operation, the claim MUST name the fill's sample values in the output + representation. A fill whose values the claim does not name is an input with + no tier, and an instance that writes it does not meet copies-only. An input + that neither the derived sample nor the sample entry lists is the default fill + picture (clause 5.1). So an operation reads the fill for every visual input it + reads that neither lists, and a claim over it names the fill as an input. + Where an instance sits in the executed list does not say what it read: clause + 4 lets an operation that is not first read the output of any previous + operation, only new inputs, or both. A claim therefore names, for each + instance, the visual inputs it read: an input, or the output of an operation + executed before it. An instance of an operation that reads a visual input and + names none holds no tier, wherever it sits in the list. +6. **Unregistered operations cannot be tiered.** A `'uuid'` vendor operation, or + any transformation this registry does not name, makes the derived sample + untiered. The reference implementation returns an error rather than guess. +7. **Relation to [determinism-tiers](determinism-tiers.md) §2.** That section's + rule is that a determinism claim covers what the sink emits. A derived sample + is an emitted medium, and this section says when a claim over it holds. It + does not relax the rule for the inputs: a hash over a decoded input is not a + hash over the derived sample. + +## 4. Receipt + +A receipt over a derived sample SHOULD carry: + +- the executed operation list, as capability strings in order, with the + conditions each instance met, the universal ones included, the visual inputs + each instance read (§3 rule 5), and the parameter values each instance applied + (§3 rule 3); +- the span the claim covers: the derived sample's composition time and + duration, or the composition instant of each output frame covered; +- the content hashes of the inputs, visual and parameter; +- the resulting tier; +- only when that tier is `decode-equivalence`, the hash of each output frame the + claim covers. A claim over the whole duration covers one frame. + +The energy of evaluating the graph is measured and attributed as for any other +work ([energy-measurement](energy-measurement.md)). The tier and the energy are +separate claims, and neither implies the other. This extension defines the +content of such a receipt, not a field layout. + +## 5. Reference implementation + +`wai-rs`, module `codecs::derivation`, with no feature gate: + +- `derivation_operations()`, the ten registered operations in clause order + (`DerivationOp { capability, name, clause, fourcc, tier, basis, + reads_visual_input }`, with `fourcc` `None` throughout and + `reads_visual_input` false only for sRGB Fill); +- `derivation_op(cap)` and `is_derivation_operation(cap)`; +- `Condition` (`SameRepresentation`, `CopiesOnly`, `TimeInvariant` — together + `UNIVERSAL_CONDITIONS` — and the per-operation conditions), with + `Condition::statement()`; +- `OperationTier { ExactWhen(&[Condition]), Untiered }`, + `DerivationOp::exact_conditions()` (the universal conditions followed by the + operation's own) and `DeterminismTier { EntropyConsistency, DecodeEquivalence }` + (the registry's `codecs::DeterminismTier`, re-exported); +- `derived_sample_tier(inputs, executed) -> Result, + DerivedSampleError>`, which implements rules 1, 2, 5 and 6 of §3 over an + executed list of `ExecutedOp { capability, conditions_met, reads }`. The + conditions met are listed one by one, so a sink that checked only geometry + cannot meet same-representation or time-invariant by omission. `inputs` lists + the tier of every visual input read other than a preceding operation's output, + the default fill picture included. `reads` names what each instance read, as + `VisualInput::Listed(k)` (entry `k` of `inputs`) or `VisualInput::Preceding(j)` + (the output of executed operation `j`, which comes before it). An instance of + an operation that reads a visual input and names nothing gives no tier, + wherever it sits in the list. When at least one operation executed, a call that + contradicts itself is an error: a reference to an input that is not listed, to + an operation that does not precede the reader, or a listed input that no + instance names. An empty executed list gives no tier before any of these + checks, whatever `inputs` holds. What cannot be + detected is a `reads` that leaves out an input its instance read, unless that + input is listed and no instance names it. The result is as sound as the sink's + report of what each instance read, as it is as sound as its report of the + conditions it checked. An earlier revision + refused a tier on an empty `inputs` only when the first executed operation + read a visual input, so an operation after an sRGB Fill could still read an + input nobody listed and be reported exact; +- `derivation_license()`. + +The crate evaluates no derivation operation, and has no receipt emitter. Rules 3 +and 4 of §3, and the receipt content of §4, are therefore not implemented. +`ExecutedOp` carries only what the tier depends on (the conditions met and the +inputs read), not the parameter values, span or input hashes a claim binds. The +`wai` command-line tool does not yet list these operations. + +## 6. Sources + +- ISO/IEC 23001-16:2021, *Information technology — MPEG systems technologies — + Part 16: Derived visual tracks in the ISO base media file format*. The published + sample (clauses 1–4, 5.1 and 5.2, and the table of contents; it ends at 5.2) is + the source for the operation + names and clause numbers, 4CC identification and the `'uuid'` escape, the + essential and non-essential rule, the default fill picture, pre-derivation + transforms, the rule that visual inputs share pixel aspect ratio and bit depth, + the kinds of input and output (definitions 3.5–3.8: parameter inputs, and + visual and parameter inputs that are intervals of a track spanning several + samples, and visual outputs that are sequences of frames), the statement that + any derivation transformation may have internal time structure (clause 4, with + a cross-fade as its example), the order in which a parameter or an input takes + its value, an unlisted input being the default fill picture (clause 5.1), the + implicit crop and fill of an input whose size differs from the output's + (clause 4), and the patent statement. + The parts read say nothing about bit-exactness either way. What the parts not + read say about it is unknown: clause 5.3 (the semantics of the derivation + boxes), clauses 6 and 7 (the sample entry, the configuration record and the + sample format), clause 8 (each operation's definition, syntax and semantics, + §8.2–§8.11) and Annex A. +- [research/2030-transport-landscape.md](../research/2030-transport-landscape.md) + §3.1 and §8 item 12. diff --git a/wai/extensions/prior-carriage.md b/wai/extensions/prior-carriage.md new file mode 100644 index 000000000..dfa195d76 --- /dev/null +++ b/wai/extensions/prior-carriage.md @@ -0,0 +1,290 @@ +# WAI Extension: Prior Carriage + +> Status: Draft. Lets a `Prior` declare the **wire form** of the artifact its +> digest covers, registers ISO/IEC 15938-17 (neural-network coding, "NNC") as +> that form, registers a **candidate** general-tensor payload, and fixes how an +> in-band signature inside such an artifact relates to WAI's digest. Reference +> impl: `wai-rs` — `container::Prior::{new, with_form, form, check_form, +> form_preserves_exactness}`, `codecs::prior_form`, `codecs::{CAP_PRIOR_NNC, +> CAP_TENSOR_NNC}`. +> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174. + +## 1. A prior has a digest, and had no form + +A neural capability decodes against a model the envelope does not carry +([SPEC.md §1](../SPEC.md)). WAI pins that model's identity: a `Prior` in the +conditioning names an `id`, a `sha256` over the artifact, and a `determinism` +contract. The reference sink recomputes the digest over the model it registered +and refuses to decode on a mismatch (`neural::ModelRegistry::resolve_checked`), +and `replicate` soundness is gated on the determinism contract (SPEC §7). + +The digest fixes **which bytes**. Until now nothing fixed **what those bytes +are**: the sink hashes whatever single file it registered — a serialized graph, +an integer table bundle — so two implementations can hold the same network under +different digests, or agree on a digest while reading the bytes differently. +A determinism tier is a claim about reproducing a computation; the computation's +operand needs a form both ends can name. + +## 2. The `form` field + +```json +"prior": { + "id": "int_hyper/q4", + "sha256": "", + "determinism": "int-rans/p16+int-synth", + "form": "wai.prior.nnc" +} +``` + +- `form` is OPTIONAL. Absent, the form is undeclared and the digest covers the + artifact as the sink registered it — the existing behaviour, unchanged, and a + manifest without `form` round-trips byte for byte. +- Present, `form` MUST be a registered prior form (§3), and `sha256` MUST be + computed over the artifact serialized in exactly that form: the coded + bitstream, not an unpacking, a transcoding, or a directory of its parts. +- A sink that cannot read the declared form MUST refuse the prior, as it refuses + a digest mismatch. It MUST NOT resolve the prior against an artifact in another + form, even one whose unpacked parameters compare equal: equality after + unpacking is a different fact from the digest the manifest pins. +- A declared form names a **self-contained** artifact. An NNC incremental-update + bitstream decodes only against its base, so it MUST NOT be pinned as a prior on + its own. No form is registered for a base-plus-updates chain. +- A form is a capability string. When a prior artifact travels in a WAI envelope + it does so under its form's capability, so the envelope's per-object content + hash and the `Prior` digest cover the same bytes. + +## 3. Registered form and candidate payload + +| capability | payload | status | +|---|---|---| +| `wai.prior.nnc` | the parameters of a trained network as a bitstream conforming to **ISO/IEC 15938-17:2024** (NNC, 2nd edition: parameters and their incremental updates, DeepCABAC entropy coding) | registered; the only prior form | +| `wai.tensor.nnc` | general tensorial data — split-inference feature maps, 3D/4D scene parameters — coded with NNC | **candidate**: payload format not fixed | + +Both are sink-supplied: `wai-rs` contains no NNC reader or writer. The +standard's reference software is itself standardized, in **ISO/IEC 15938-18**, +*Conformance and reference software for compression of neural networks* (1st +edition 2023, 2nd edition 2025). That part also supplies the conformance +bitstreams and the procedures for testing decoders and bitstreams. Its scope +makes the software an integral part of it, gives the requirements of +ISO/IEC 15938-17 precedence over the software's behaviour, and does not require +an implementation to use it. + +Registry classification (`codecs::capability_*`): + +- **Fidelity tier: lossy, by primary role.** The registry's tier names a + capability's primary role, as it does for AVIF, which also has a lossless mode; + it is not a statement that every NNC artifact approximates its source. NNC's + primary role is compressing a network by parameter reduction and quantisation. + The published standard is not quantisation-only, though. The 2022 edition + defines an NNR unit as carrying compressed or uncompressed network data. Its + table of contents lists decoding methods for an integer payload type + (`NNR_PT_INT`) and a raw-float payload type (`NNR_PT_RAW_FLOAT`), and a decoding + process for an integer weight tensor. Those decoding clauses were not read, and + neither was the 2024 edition. An open implementation that states compliance + with the 2024 edition reads an `NNR_PT_RAW_FLOAT` payload as raw 32-bit floats + and passes int32 tensors through with no quantisation step. So an integer + network, or a float network carried raw, need not be approximated at all. + Lossy is kept because a capability-level `lossless` would present every NNC + artifact as reproducing its source, and whether a given artifact does depends + on its payload types, and on the decoding process of §5. +- **Licence: patent-pool, conservatively.** Royalty status is not established. + The Introduction of ISO/IEC 15938-17:2022 records a patent claimed to be + required for compliance, whose holder has given a + reasonable-and-non-discriminatory licensing assurance; the corresponding + statement in the 2024 edition was not read. No royalty-free commitment was + found, and an openly available implementation that states compliance with the + 2024 edition is released under a licence that expressly grants no patent rights + (see the research note in §8). The registry places NNC where it places MPEG-I + Haptics for the same reason, and never presents it as royalty-free. +- **Replicate class: the class the registry gives a standard whose decoder + conformance it has not checked**, as it gives MPEG-I Haptics. The tensors are + reconstructed by the standard's decoding process, not by a model WAI runs, and + NNC's decoder conformance (§5) was not read, so nothing here establishes that + every conforming decoder reconstructs bit-identical tensors. A registry that has + one class for every standard codec gives NNC that class, which then says only + that the named standard specifies the reconstruction. This concerns an NNC + payload. How a declared form bounds `replicate` over a neural capability is §5. + +**`wai.tensor.nnc` is a candidate.** Coding of general tensorial data is part of +an NNC amendment that has not been published (§4), so this payload's format is +not fixed. Until the capability is pinned to a published edition, an encoder +MUST NOT emit it and a sink MUST treat it as unsupported. SPEC §8's rule that a +registered payload format never changes applies from the edition it is pinned to. +It is registered now because split-inference feature maps and scene tensors are a +payload class WAI otherwise has no name for. For Gaussian-splat attributes +specifically, `wai.splat.int_codec` is the path with a portable hash +(splat-equivalence); `wai.tensor.nnc` would be an interchange form with no +determinism guarantee. + +## 4. What is pinned, what is tracked, and why `form` is optional + +The standard is at two different stages, and WAI treats them differently. + +**Pinned — the published editions.** ISO/IEC 15938-17 is a published +International Standard: 1st edition 2022, 2nd edition 2024. `wai.prior.nnc` is +defined against the 2nd edition. Incremental-update coding is in that edition; +it is not new, and an implementation MUST NOT describe it as forthcoming. + +**Tracked, not pinned — the amendment.** MPEG's press release for its 154th +meeting (Santa Eulària, 27 April – 1 May 2026) states that a Committee Draft of +an amendment to NNC was published. It adds lossless compression, including mixed +lossy and lossless compression in one bitstream; source-data-type indication for +models with limited bit depth; finer-grained configuration of the quantisation; +configurable extensions for initialising, ordering and packing data; coding of +other tensorial data, such as split-inference feature maps and the parameters of +3D and 4D scene representations; and signing and verification of data units, +which the release says lets a verifier check the authenticity of a set of +network parameters, part of a network, or an update. The amendment's text was +not read, so what a data-unit signature covers and how its key is identified are +not known here. A committee draft can still change. No rule in this extension +depends on any of it. When a published edition carries these features: + +- it gets a **new** capability string — SPEC §8 forbids redefining + `wai.prior.nnc`; +- `wai.tensor.nnc` is pinned to that edition, or withdrawn; +- nothing about exactness changes by that alone. Integer and raw-float payload + types are already in the published text (§3). What decides whether a form can + be listed as exact is the decoding process and its conformance requirement + (§5), for any edition, lossless compression included. + +**Optional, not recommended.** `form` is MAY, and NNC is not a SHOULD, for three +reasons: + +1. WAI treats royalty-free implementation as a precondition for adoption (SPEC + Appendix C). It does not recommend, for every prior, a form whose royalty + status it cannot establish. +2. WAI's reference implementation (`wai-rs`) cannot read or write NNC, so it + cannot demonstrate the form end to end. +3. Whether conforming NNC decoders reconstruct identical tensors is not + established (§5), so the form cannot yet carry WAI's strongest claims. + +## 5. Determinism through a form + +- Declaring a form does not change the prior's `determinism` contract, which + still states how the network's own decode reproduces. +- Reading an artifact out of a coded form is itself a decode. For a quantised + payload, NNC's entropy decode produces integer quantisation indices, which the + decoding process turns into parameter values. An integer or raw-float payload + carries its values without quantisation (§3). Whether every conforming decoder + reconstructs bit-identical parameter values, from any of these payload types, + was not established here. So an integer-exact network can still reach two sinks + as different weights. +- Where that would be settled: ISO/IEC 15938-18 states that decoder conformance + to ISO/IEC 15938-17 is specified in clause 7 of 15938-17 (headed *Decoding + process* in the 2022 edition), and supplies reference bitstreams and a decoder + test procedure (15938-18 §4.5). Neither clause 7 nor that procedure was read, + so whether conformance requires identical reconstructed parameters, or permits + a tolerance, is open. What was read, in the 2022 edition's public sample: + its scope requires only a decoder's externally observable output to conform; + its number conventions define `float` by ISO/IEC 60559 and make operator + results mathematically exact unless a float result is explicitly specified; + and its clause 7 includes decoding methods for integer, float and raw-float + payload types. Clause 7 is where a form would be established exact, for the + published editions and for any later one. +- Therefore an envelope with `intent = replicate` MUST NOT rest on a prior whose + declared form is not established exact. The registry lists the forms that are + established (`codecs::prior_form::form_reconstruction_is_exact`), and **none is + listed today**. In the reference implementation the rule is part of + `Prior::is_cross_hardware_exact`, which requires `Prior::form_preserves_exactness` + as well as an integer-exact contract. Every `replicate` check reads that + predicate, so `container::replicate_is_sound` and `Wai::replicate_is_sound` + reject such a `replicate`, and `decode_envelope_strict` refuses it with a reason + naming the form. +- The same bound applies to the tiers of + [determinism-tiers](determinism-tiers.md): `decode-equivalence` needs identical + tensors at every sink, so it cannot be claimed through a form not established + exact. +- The digest is unaffected. It is computed over bytes, so it is portable whatever + the reconstruction does. + +## 6. In-band data-unit signatures and WAI's digest + +MPEG's press release for its 154th meeting states that the NNC amendment adds +signing and verification of data units (§4). Its text was not read, and nothing +here depends on its details. This section fixes the precedence now, so the two +cannot diverge later. It takes the posture +[reconstruction-binding](reconstruction-binding.md) takes toward a C2PA manifest: +WAI's binding sits beside the standard's, never in place of it. + +- **Scope.** `Prior.sha256`, and a [JWP](jwp-receipts.md) per-object content + hash over a `wai.prior.nnc` or `wai.tensor.nnc` payload, cover **every byte** of + the coded bitstream, including any unit that carries an in-band signature. + Neither is computed over a subset chosen by an in-band signing scope. +- **An in-band signature is an attested input.** A verifier MAY check it and MAY + record the verdict in a receipt, naming the signature scheme and the key + location as found. The verdict never substitutes for the WAI digest check. A + matching WAI digest attests that these are the bytes the manifest pinned; an + in-band signature attests that a key holder signed these units. Neither implies + the other. +- **A failed in-band signature is recorded as failed.** It does not unpin a prior + whose WAI digest matches: the pinned artifact is the one that carries the bad + signature. +- **Stripping is not repackaging.** Removing signature units changes the bytes, + so the result no longer matches its pin. It is a different artifact. +- **Keys.** WAI has no key-discovery mechanism of its own. A key location found + in-band is recorded, not trusted. + +## 7. Reference implementation + +- `container::Prior::form` (field, serde-skipped when absent) and + `Prior::form()`; `Prior::new(id, sha256, determinism)`, which leaves the form + absent, and `Prior::with_form(form)`. The new public field breaks Rust code + outside the crate that builds a `Prior` with a struct literal; the CHANGELOG + (v0.4.0) states the break, the fix, and why `Prior` is not + `#[non_exhaustive]`; `Prior::check_form(readable) -> Result<(), PriorFormError>` + (`NotAPriorForm` / `Unreadable`); `Prior::form_preserves_exactness()`, which + `Prior::is_cross_hardware_exact()` requires alongside an integer-exact contract. + `container::replicate_prior_refusal(intent, prior) -> Option` + gives the reason: `PriorRefusal::FormNotExact(form)` when an integer-exact + contract declares a form not listed as exact, with a message that says whether + the form is registered at all. `container::replicate_is_sound` and + `Wai::replicate_is_sound` hold exactly when it returns `None`, and a unit test + checks that the two agree. +- `codecs::prior_form::{prior_forms, is_prior_form, + form_reconstruction_is_exact}`; `codecs::{CAP_PRIOR_NNC, CAP_TENSOR_NNC}` with + their tier and licence entries. They have no replicate-class entry of their own: + a unit test holds their class equal to MPEG-I Haptics' (§3). +- `neural::ModelRegistry::resolve_checked` runs `check_form` against + `neural::READABLE_PRIOR_FORMS`, which is empty: the reference sink refuses any + prior that declares a form. `neural::decode_envelope_strict` reports the + `PriorRefusal` as the reason for `UnsoundReplicate`. +- CI: the library unit tests (`wai-conformance`) cover `check_form`, + `form_preserves_exactness`, every `PriorRefusal`, and the form rule in each + `replicate` check (`Prior::is_cross_hardware_exact`, both `replicate_is_sound` + functions and `replicate_prior_refusal`). The two model-free tests of the + neural-sink wiring (`resolve_checked_verifies_prior_hash`, + `strict_decode_gates_unsound_replicate`) are configured to run in + `wai-neural-gates`, in an Ubuntu 24.04 container image. That job has not yet + run on a runner. +- Not implemented: an NNC reader or writer, and any cross-decoder measurement of + NNC reconstruction. + +## 8. Sources + +- ISO/IEC 15938-17:2022 and ISO/IEC 15938-17:2024, *Compression of neural + networks for multimedia content description and analysis*. Read: the public sample + of the 2022 edition (Introduction, clauses 1–4 and the table of contents), + which is the source for the scope, the definition of an NNR unit, the number + conventions, the clause-7 headings (including the integer and raw-float payload + types) and the patent statement. Not read: clause 7 itself, and the 2024 + edition. +- MPEG, press release of the 154th meeting, *Compressing Tensor Data in AI-based + Media Coding: MPEG publishes Committee Draft of Amendment of Neural Network + Coding (NNC)* (mpeg.org, 4 June 2026), and MPEG's meeting list (mpeg.org) for + the meeting's place and dates: the source for the amendment's stage and + features, including signing and verification of data units. +- An open implementation that states compliance with ISO/IEC 15938-17:2024: its + high-level-syntax source reads an `NNR_PT_RAW_FLOAT` payload as raw 32-bit + floats, and its integer path gives int32 tensors a `skip` approximation with no + quantisation. Named in the research note. +- ISO/IEC 15938-18:2023 and ISO/IEC 15938-18:2025, *Conformance and reference + software for compression of neural networks*: the reference software, the + conformance bitstreams and the decoder and bitstream test procedures. Read: the + public sample of its draft (DIS, 2022: Introduction, clauses 1–4.3, part of + 4.4, and the table of contents), and the published 2023 edition's catalogue + abstract. Not read: the + decoder test procedure (§4.5) and either published edition's full text. +- [research/2030-transport-landscape.md](../research/2030-transport-landscape.md) + §6.4 (the amendment's contents and stage, the reference software, and the open + implementation whose licence grants no patent rights), §8 item 9, §9 item 5 + (the signing flag, resolved by the press release). diff --git a/wai/research/2030-transport-landscape.md b/wai/research/2030-transport-landscape.md index 2a8e679e9..2162f2a63 100644 --- a/wai/research/2030-transport-landscape.md +++ b/wai/research/2030-transport-landscape.md @@ -201,10 +201,27 @@ op log does. **The honest form of the claim, for use everywhere in the tree:** *a track that carries a process the receiver evaluates is not novel — MPEG-B Part 16 ratified -it in 2021. What Part 16 leaves unspecified is a byte-exact determinism contract -per operation and a signed receipt binding the executed graph to measured -joules.* Note that the base set includes interpolating and blending operations -with no bit-exactness requirement, so the tier work is real, not decorative. +it in 2021. The parts of Part 16 read here (clauses 1–4, 5.1 and 5.2, and the +table of contents) state neither a byte-exact determinism contract per operation +nor a signed receipt binding the executed graph to measured joules; clause 5.3, +clauses 6 and 7, clause 8, which defines each operation, and Annex A were not +read.* Note that the base set includes operations that +compute new sample values (Dissolve, Scaling), so the tier work is real, not +decorative. + +> **Correction.** An earlier revision of this section said those operations carry +> "no bit-exactness requirement". That was not verified. Only the published +> sample of the standard was read (clauses 1–4, 5.1 and 5.2, and the table of +> contents; the sample ends at 5.2), and those parts say nothing about +> bit-exactness either way. Clause 5.3, clauses 6 and 7, clause 8, which defines +> each operation, and Annex A were not read, so whether they fix the operations' +> arithmetic — and so whether Part 16 leaves byte-exactness unspecified at all — +> is open. +> `extensions/derived-tracks.md` does not rely on either answer for its three +> universal conditions, which every tier needs and the executing sink checks. It +> does rely on the operations' titles, not on clause 8, for the geometric +> conditions it lists per operation and for which operations read a visual +> input, and reading clause 8 may show one of those lists to be incomplete. *Caveat for any outward copy:* ISOBMFF is universally shipped, but derived visual tracks have negligible deployed player support. The collision is @@ -796,9 +813,30 @@ incremental/federated update coding landed in **Ed. 2**, so it is not new). 27 April – 1 May 2026); MPEG separately lists an ongoing Edition 3 project with the same tensor-coding objective, into which the work is expected to be integrated — so cite it as *"the NNC amendment (CDAM, MPEG 154)"*, not "Ed. 3". -Reference implementation: NNCodec (Fraunhofer HHI); 2.0-beta is compliant with -15938-17:2024. Describe NNC as the **first** international standard on neural -network compression, not the only one. +Reference software: specified by a separate part, ISO/IEC 15938-18 (*Conformance +and reference software for compression of neural networks*; 1st ed. 2023, 2nd +ed. 2025), which also carries the conformance bitstreams and the decoder and +bitstream test procedures. NNCodec (Fraunhofer HHI) is a separate open +implementation: its README calls it "an efficient implementation of NNC" and +states that 2.0-beta is compliant with 15938-17:2024, and its licence file states +that no patent rights are granted. Describe NNC as the **first** international +standard on neural network compression, not the only one. *(Corrected: an +earlier revision of this note called NNCodec the reference implementation.)* + +**NNC is not quantisation-only.** The 2022 edition's public sample defines an NNR +unit as carrying "compressed or uncompressed" network data. Its table of contents +lists decoding methods for payload types `NNR_PT_INT`, `NNR_PT_FLOAT`, +`NNR_PT_RAW_FLOAT` and `NNR_PT_BLOCK` (§7.3.2–7.3.5), and a decoding process for +an integer weight tensor (§7.3.6). NNCodec's source (HEAD, read September 2026) +reads an `NNR_PT_RAW_FLOAT` payload as raw 32-bit floats +(`nnc_core/hls/__init__.py`). Its approximator gives int32 tensors a `skip` +method with `dq_flag` 0, so they are not quantised +(`nnc_core/approximator/integer.py`). Its payload-type enum marks `NNR_PT_INT` +"not yet implemented". So integer and raw-float carriage without quantisation is +already in the published text. On that evidence, the amendment's "lossless +compression" adds compression for values the published text can already carry +exactly, rather than introducing exact carriage. The decoding clauses themselves +(§7.3) and the 2024 edition were not read. The amendment adds: lossless and mixed lossy+lossless compression in one bitstream; source-data-type indication for limited-bit-depth models; finer-grained @@ -808,18 +846,28 @@ data** (split-inference feature maps, federated-learning updates, 3D/4D scene parameters including Gaussian splats); and **signing and verification of individual coded data units**. -⚠ *Still unverified after checking (Sept 2026), and the flag is load-bearing.* +*Verified (Sept 2026) against a primary source.* MPEG's press release for its +154th meeting (mpeg.org, 4 June 2026), in the section headed "MPEG publishes +Committee Draft of Amendment of Neural Network Coding (NNC)", states the stage and +the features listed above, including: "it adds support for signing and +verification of data units, which allows verifying the authenticity of a set of +neural network parameters, as well as parts of a neural network or its updates." +MPEG's meeting list gives MPEG 154 as Santa Eularia, 2026-04-27 to 2026-05-01. +*(Corrected: an earlier revision of this note said the signing feature could not +be corroborated from public sources. The press release corroborates it.)* The +amendment's text and its WG document number were not seen. So what a data-unit +signature covers, which scheme it uses and how its key is identified are unknown. The **base standard is confirmed**: ISO/IEC 15938-17 (1st ed. 2022, 2nd ed. 2024), *Compression of neural networks for multimedia content description and analysis* — DeepCABAC entropy coding, pruning/sparsification/low-rank -preprocessing, incremental-update support, with an open reference implementation -(Fraunhofer HHI, NNCodec). What could **not** be corroborated from public sources -is the **signing-and-verification-of-coded-data-units amendment**. Until an -amendment text or a WG document number is seen, do not present NNC as a -competing authenticity architecture; the defensible claim is the narrower one — -NNC is a ratified wire format for the artifact a `Prior` names, which WAI -currently leaves as an informal directory. That claim stands on the published -editions alone and needs no amendment. +preprocessing, incremental-update support, with reference software in +ISO/IEC 15938-18 and a separate open implementation (Fraunhofer HHI, NNCodec). +The claim that needs no amendment at all is the carriage one: NNC is a ratified +wire format for the artifact a `Prior` names, which WAI currently leaves as an +informal directory. The signing feature is a second surface. It concerns how +NNC's in-band signatures relate to WAI's digest, which +`extensions/prior-carriage.md` §6 fixes without depending on the amendment's +details. **What WAI already has, stated correctly.** WAI *does* pin model **identity** normatively: the reference implementation's `Prior { id, sha256, determinism }` @@ -834,9 +882,15 @@ bundle directory. **Three concrete surfaces.** (1) Name NNC as the standards-track normative form for the artifact a `Prior` identifies, so `Prior.sha256` covers a portable -bitstream rather than an ad-hoc directory — and note that the amendment's -lossless mode makes that bitstream byte-exact, hence eligible for -`decode-equivalence` rather than only `entropy-consistency`. (2) Register a +bitstream rather than an ad-hoc directory. Whether a bitstream reconstructs its +network identically at every decoder, and so could carry `decode-equivalence` +rather than only `entropy-consistency`, is decided by the decoding process +(clause 7) and its conformance requirement, not by the amendment. Integer and +raw-float payload types already carry values unquantised in the published text. +*(Corrected: an earlier revision said the amendment's lossless mode would make the +bitstream byte-exact. Exact values are already carriable. Byte-exact +reconstruction at every decoder is the open question, and clause 7 is where it is +settled.)* (2) Register a capability for the general-tensor payload: split-inference feature maps and 3D/4D scene tensors are a payload class WAI lacks and which fits the instruction/latent thesis — and it directly abuts the existing `wai.splat.int_codec`. (3) Reconcile @@ -1047,14 +1101,16 @@ above. They are listed so the document can be checked rather than believed. (confirmed); and the 23888 series structure (Part 3 confirmed, **Part 7 corrected — no such part appears in the catalogue**). - **Two remain open and both are consequential.** (a) The NNC - signed-data-unit amendment (§6.4) is uncorroborated; if it does not exist, - NNC is a carriage format, not a rival authenticity architecture, and that - entry weakens to its narrow form. (b) ABNT NBR 25608's clause-level detail - (§7.2) was read only through public companions because the standard is - paywalled; buy it before quoting a clause number outward. - Any one of these failing catalogue confirmation removes that row from the - queue but does not disturb the others; they are independent. + A sixth was **resolved** later in September 2026. The NNC + signed-data-unit feature (§6.4) is stated in MPEG's own press release for + MPEG 154, as part of the Committee Draft of Amendment. Its text was not + read, so the feature's details remain unknown. + + **One remains open and it is consequential.** ABNT NBR 25608's clause-level + detail (§7.2) was read only through public companions because the standard + is paywalled; buy it before quoting a clause number outward. If it fails + catalogue confirmation, that row leaves the queue without disturbing the + others. 6. **The regional mechanisms are already expressible.** If a predicate capability turns out to be a straightforward instance of existing WAI state-feed diff --git a/wai/wai-rs/src/codecs/derivation.rs b/wai/wai-rs/src/codecs/derivation.rs new file mode 100644 index 000000000..43fa50bbc --- /dev/null +++ b/wai/wai-rs/src/codecs/derivation.rs @@ -0,0 +1,949 @@ +//! Derivation operations of ISO/IEC 23001-16:2021 (*Derived visual tracks in the +//! ISO base media file format*), registered with a per-operation determinism tier. +//! +//! A derived visual track carries derived samples, each an ordered list of +//! derivation operations applied to decoded inputs — input tracks, image items, +//! the output of a preceding operation, or a default fill picture. Carrying a +//! process the receiver evaluates is therefore ratified practice, not something +//! WAI introduces. The parts of the published text read here (clauses 1–4, 5.1 +//! and 5.2, and the table of contents) do not say whether an operation's output +//! reproduces byte for byte across conforming readers. The rest was not read: +//! clause 5.3 (the semantics of the derivation boxes), clauses 6 and 7 (the sample +//! entry, the configuration record and the sample format), clause 8, which +//! defines each operation, and Annex A. This +//! registry states, per operation, when a claim over its output may hold, in the +//! vocabulary of `extensions/determinism-tiers.md`, so a receipt or binding over a +//! derived sample can say which tier it actually holds. Normative text: +//! `extensions/derived-tracks.md`. +//! +//! These strings are **not payload capabilities**. A derivation operation names +//! no payload format — it acts on samples some other capability decoded — so the +//! strings are deliberately absent from [`super::known_capabilities`], which lists +//! what a manifest may name as its payload's capability (SPEC §4). A sink +//! advertises the operations it evaluates the same way it advertises codecs. +//! +//! WAI does not composite. Registering the grid and overlay compositions and +//! giving them a tier states what a reproducibility claim over their output would +//! need; it does not make presentation WAI's job (presentation stays with the sink, +//! as in `linear-film` and `interactive-worlds`). +//! +//! What was read, and what was not: the operation names, their clause numbers, +//! 4CC identification, the essential / non-essential rule, the default-fill rule, +//! the order in which a parameter or an input takes its value (an unlisted input +//! is the default fill picture), the kinds of input (visual inputs and parameter +//! inputs, either of which may be an interval of a track spanning several +//! samples), the rule that a derived sample's visual inputs share +//! pixel aspect ratio and bit depth, and the rule that any derivation +//! transformation may change its picture during the sample's duration are taken +//! from the published first edition (clauses 1–4, 5.1 and 5.2, and the table of +//! contents; the published sample ends at 5.2). +//! Clause 8, which gives each operation's definition, four-character code, syntax +//! and semantics, was not available, so no 4CC is recorded here +//! ([`DerivationOp::fourcc`] is `None` throughout). +//! +//! Each tier is assigned from what an operation does to sample values, in space +//! and in time. Everything a tier needs is either a [`Condition`] that the sink +//! which executed an instance checks on that instance, or the visual inputs that +//! instance read ([`ExecutedOp::reads`]), which the sink names. Three conditions +//! ([`UNIVERSAL_CONDITIONS`]) apply to every operation: no sample value is +//! converted into another representation, none is computed rather than copied, +//! and the output does not change over the span the claim covers. Clause 8 +//! decides what an instance does, and so which conditions it can meet; an instance +//! that fails one is untiered whatever clause 8 says, as long as the executing +//! sink lists only the conditions it actually checked. What the registry does take +//! from outside clause 8 is each operation's purpose, read from its title (a crop +//! selects, a rotation rotates). **The geometric conditions listed for each +//! operation come from its title, not from clause 8 of ISO/IEC 23001-16, which was +//! not read**, and so does whether an operation reads a visual input at all +//! ([`DerivationOp::reads_visual_input`]). Reading clause 8 may show one of those +//! lists to be incomplete. Clause 4, which was read, already shows where a list +//! may fall short: wherever an input's size differs from the output's, the input +//! is cropped or filled implicitly, for every operation unless its own clause +//! overrides that. The clauses read do not say where such an input is placed, and +//! no list, Identity's included, carries a condition on that placement. +//! +//! An earlier revision said that no tier rested on anything clause 8 might fix. +//! The time dimension did: only Dissolve was untiered for depending on time, +//! although clause 4 states generally that a derivation transformation may have +//! internal time structure. Time is now a universal condition. +//! +//! A later revision refused a tier on an empty `inputs` only when the first +//! executed operation read a visual input. Clause 4 lets an operation that is not +//! first read the output of any previous operation, only new inputs, or both, so a +//! composition after an sRGB Fill could still read an input nobody listed and be +//! reported exact. Each instance now names what it read. + +use super::LicenseClass; + +// ---- Capability strings, one per clause of ISO/IEC 23001-16:2021 §8 ---- +pub const CAP_DERIVE_IDENTITY: &str = "wai.derive.identity"; // §8.2 +pub const CAP_DERIVE_SRGB_FILL: &str = "wai.derive.srgb_fill"; // §8.3 +pub const CAP_DERIVE_DISSOLVE: &str = "wai.derive.dissolve"; // §8.4 +pub const CAP_DERIVE_CROP: &str = "wai.derive.crop"; // §8.5 +pub const CAP_DERIVE_ROTATION: &str = "wai.derive.rotation"; // §8.6 +pub const CAP_DERIVE_MIRROR: &str = "wai.derive.mirror"; // §8.7 +pub const CAP_DERIVE_SCALING: &str = "wai.derive.scaling"; // §8.8 +pub const CAP_DERIVE_ROI_SELECTION: &str = "wai.derive.roi_selection"; // §8.9 +pub const CAP_DERIVE_GRID_COMPOSITION: &str = "wai.derive.grid_composition"; // §8.10 +pub const CAP_DERIVE_OVERLAY_COMPOSITION: &str = "wai.derive.overlay_composition"; // §8.11 + +/// The registry's tier type, re-exported: a derived sample is tiered in the same +/// vocabulary as the capabilities that decoded its inputs. +pub use super::DeterminismTier; + +/// A property of one **executed instance** of a derivation operation that its +/// tier depends on. The sink that executed the instance checks it, and reports +/// the ones that hold in [`ExecutedOp::conditions_met`]. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub enum Condition { + /// Every visual input, and every constant the instance writes (a fill colour, + /// the default fill picture), is already in the output's sample + /// representation. Required of every operation ([`UNIVERSAL_CONDITIONS`]). + SameRepresentation, + /// Every output sample value is, bit for bit, an input sample value or a + /// constant whose value the claim names. Required of every operation + /// ([`UNIVERSAL_CONDITIONS`]). + CopiesOnly, + /// Over the span the claim covers — the derived sample's whole duration, or + /// one output frame at a composition instant the claim names — every visual + /// input the instance reads is one frame, every parameter it uses has one + /// value, and the instance has no time structure of its own, so its output + /// over the span is one frame. Required of every operation + /// ([`UNIVERSAL_CONDITIONS`]). + TimeInvariant, + /// Every region read or placed lies on every component's sample grid. + OnSampleGrid, + /// Every region or input is emitted at its own size. + OwnSize, + /// The rotation is a multiple of 90 degrees. + QuarterTurn, + /// Every component is sampled at the same resolution. + NoSubsampling, + /// The reflection maps every component's sample grid, with its siting, onto + /// itself. + SitingPreserved, + /// Every input is placed opaquely. + Opaque, +} + +impl Condition { + /// The condition as a verifier checks it. + pub fn statement(self) -> &'static str { + match self { + Self::SameRepresentation => { + "every visual input, and every constant the instance writes (a fill colour, \ + the default fill picture), is already in the output's sample representation: \ + the same bit depth, chroma format and chroma siting, colour primaries, transfer \ + characteristics, matrix coefficients, range and component set, so no sample \ + value is converted" + } + Self::CopiesOnly => { + "every output sample value is, bit for bit, the value of an input sample or of a \ + constant whose value the claim names: the instance computed no sample value" + } + Self::TimeInvariant => { + "over the span the claim covers (the derived sample's whole duration, or one \ + output frame at a composition instant the claim names), every visual input the \ + instance reads is one frame; every parameter it uses has one value, whether \ + carried in the derived sample or the sample entry, defaulted by its clause, or \ + read from a metadata item or an interval of a metadata track; and the instance \ + has no time structure of its own, such as a transition or a parameter \ + interpolated over time. So its output over the span is one frame, fixed by those \ + input frames and parameter values" + } + Self::OnSampleGrid => { + "every region the instance reads or places lies on the sample grid of every \ + component, so no component is resampled" + } + Self::OwnSize => "every region or input is emitted at its own size (a resize is a scaling)", + Self::QuarterTurn => "the rotation angle is a multiple of 90 degrees", + Self::NoSubsampling => "every component is sampled at the same resolution", + Self::SitingPreserved => { + "the reflection maps every component's sample grid, including its siting, onto \ + itself (always so without chroma subsampling)" + } + Self::Opaque => "every input is placed opaquely: no alpha, opacity or blending", + } + } +} + +/// The conditions every operation needs before it can hold a tier, in addition to +/// its own ([`OperationTier::ExactWhen`]). No registered operation is exact +/// without them. +/// +/// Representation. Nothing in the clauses read requires a derived sample's output +/// to share its inputs' representation: clause 4 requires the visual inputs to +/// share pixel aspect ratio and bit depth with each other, and says nothing, in +/// those clauses, about colour primaries, transfer characteristics, matrix +/// coefficients, range or chroma format. So an instance that converts a +/// representation — an 8-bit input into a 10-bit output, one set of colour +/// primaries into another — holds no tier, Identity included. +/// +/// Time. Nothing read ties a derived sample's output to one instant either. Clause +/// 4 lets one derivation duration span several samples of an input track, and +/// lets any derivation transformation have internal time structure, so that the +/// picture may change during the sample's duration. A visual input or a parameter +/// input may be an interval of a track spanning several samples (definitions 3.6 +/// and 3.7), and a visual output may be a sequence of frames (3.8). So an instance +/// whose output changes over the span its claim covers holds no tier, whatever +/// operation it is. Examples are a region driven by an interval of a metadata +/// track whose value changes within the sample, or an input interval that holds +/// several samples. The same instance can hold a tier for a claim that covers one +/// output frame at a named composition instant, when nothing it reads changes at +/// that instant and it has no time structure of its own. +/// +/// A `static`, not a `const`: cbindgen reads public constants into `include/wai.h` +/// and skips statics that are not `no_mangle`. +pub static UNIVERSAL_CONDITIONS: [Condition; 3] = + [Condition::SameRepresentation, Condition::CopiesOnly, Condition::TimeInvariant]; + +/// What one derivation operation does to the tier of the samples it consumes. +/// +/// Every derivation operation acts on **reconstructed samples**, so the only tier +/// that can pass through one is `decode-equivalence`; `entropy-consistency` does +/// not fix samples, and nothing derived from them inherits it. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum OperationTier { + /// Exact for an executed instance that meets every [`UNIVERSAL_CONDITIONS`] + /// entry and every condition listed here, and untiered for any other instance. + /// An exact instance copies input samples (or constants the claim names) into + /// place by integer geometry, with no arithmetic on sample values, and its + /// output does not change over the span its claim covers: inputs at + /// `decode-equivalence` give an output at `decode-equivalence`. An empty list + /// means the universal conditions alone. The list is read from the + /// operation's title; clause 8 of ISO/IEC 23001-16, which defines the + /// operation, was not read, and may show the list to be incomplete. + ExactWhen(&'static [Condition]), + /// The operation computes new sample values, by blending or resampling, and WAI + /// pins no arithmetic for it. Its output carries no tier, whatever its inputs + /// held, until an arithmetic is pinned and an instance is shown to follow it. + /// (Dependence on time is not what makes an operation untiered: every + /// operation needs [`Condition::TimeInvariant`].) + Untiered, +} + +/// One registered derivation operation. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct DerivationOp { + /// The WAI capability string a sink advertises to say it evaluates this + /// operation. + pub capability: &'static str, + /// The operation's title in ISO/IEC 23001-16:2021. + pub name: &'static str, + /// The clause of ISO/IEC 23001-16:2021 that defines it. + pub clause: &'static str, + /// The operation's four-character code as defined in `clause`, once it has + /// been read from the published text. `None` means not recorded: a sink maps + /// this capability to the code the named clause defines. + pub fourcc: Option<[u8; 4]>, + /// The per-operation determinism tier. Its geometric conditions come from the + /// operation's title; clause 8, which defines the operation, was not read. + pub tier: OperationTier, + /// Why the operation has that tier. + pub basis: &'static str, + /// Whether the operation reads a visual input. Taken from the operation's + /// title, as its conditions are; clause 8, which defines its inputs, was not + /// read. Only sRGB Fill, which writes a constant colour, is registered as + /// reading none. Under ISO/IEC 23001-16 clause 5.1, an input that neither the + /// derived sample nor the sample entry lists is the default input fill picture, + /// so an operation that reads a visual input always reads one, listed or not. + /// An executed instance of such an operation that names nothing in + /// [`ExecutedOp::reads`] therefore holds no tier. + pub reads_visual_input: bool, +} + +impl DerivationOp { + /// Every condition an executed instance must meet to be exact — the + /// [`UNIVERSAL_CONDITIONS`] followed by the operation's own — or `None` for an + /// [`OperationTier::Untiered`] operation, which no instance makes exact. + pub fn exact_conditions(&self) -> Option> { + match self.tier { + OperationTier::ExactWhen(own) => { + Some(UNIVERSAL_CONDITIONS.iter().chain(own).copied().collect()) + } + OperationTier::Untiered => None, + } + } +} + +/// Implementation terms for the derivation operations. The published standard +/// records, in its Introduction, a patent claimed to be required for compliance +/// whose holder has given a reasonable-and-non-discriminatory licensing assurance; +/// no royalty-free commitment was found. Classified conservatively, as the +/// registry classifies MPEG-I Haptics. +pub fn derivation_license() -> LicenseClass { + LicenseClass::PatentPool +} + +use Condition::{NoSubsampling, OnSampleGrid, Opaque, OwnSize, QuarterTurn, SitingPreserved}; + +static REGISTRY: [DerivationOp; 10] = [ + DerivationOp { + capability: CAP_DERIVE_IDENTITY, + name: "Identity", + clause: "8.2", + fourcc: None, + reads_visual_input: true, + tier: OperationTier::ExactWhen(&[]), + basis: "copies its input; output samples the input does not cover come from the \ + default fill picture, a constant the claim must name. Whether Identity may \ + emit a representation other than its input's is for clause 8, which was not \ + read, so the representation condition is checked, not assumed", + }, + DerivationOp { + capability: CAP_DERIVE_SRGB_FILL, + name: "sRGB Fill", + clause: "8.3", + fourcc: None, + reads_visual_input: false, + tier: OperationTier::ExactWhen(&[]), + basis: "writes a constant colour, stated in sRGB, and reads no visual input (both \ + read from the title; clause 8.3 was not read); writing the colour into any \ + other sample representation is a conversion, which the representation \ + condition excludes", + }, + DerivationOp { + capability: CAP_DERIVE_DISSOLVE, + name: "Dissolve", + clause: "8.4", + fourcc: None, + reads_visual_input: true, + tier: OperationTier::Untiered, + basis: "a transition that mixes two inputs, so its output values are computed, not \ + copied. A transition is also time structure of its own, which the time \ + condition excludes for every operation (clause 4's example of internal time \ + structure is a cross-fade). Untiered until a blend arithmetic, and how its \ + weights depend on the instant, are pinned; clause 8.4, which may fix them, was \ + not read", + }, + DerivationOp { + capability: CAP_DERIVE_CROP, + name: "Crop", + clause: "8.5", + fourcc: None, + reads_visual_input: true, + tier: OperationTier::ExactWhen(&[OnSampleGrid]), + basis: "selects samples; a region off a subsampled component's grid needs a \ + resampling or siting choice", + }, + DerivationOp { + capability: CAP_DERIVE_ROTATION, + name: "Rotation", + clause: "8.6", + fourcc: None, + reads_visual_input: true, + tier: OperationTier::ExactWhen(&[QuarterTurn, NoSubsampling]), + basis: "a quarter turn of unsubsampled components permutes samples; any other angle \ + resamples, and with subsampled chroma a quarter turn moves the chroma grid and \ + siting, which needs a resampling or siting choice", + }, + DerivationOp { + capability: CAP_DERIVE_MIRROR, + name: "Mirror", + clause: "8.7", + fourcc: None, + reads_visual_input: true, + tier: OperationTier::ExactWhen(&[SitingPreserved]), + basis: "permutes samples; with subsampled chroma whether the reflection keeps the \ + chroma siting depends on the siting and on the picture dimension's parity", + }, + DerivationOp { + capability: CAP_DERIVE_SCALING, + name: "Scaling", + clause: "8.8", + fourcc: None, + reads_visual_input: true, + tier: OperationTier::Untiered, + basis: "resamples, so its output values are computed, not copied. Untiered until an \ + interpolation filter and its rounding are pinned; clause 8.8, which may fix \ + them, was not read", + }, + DerivationOp { + capability: CAP_DERIVE_ROI_SELECTION, + name: "Region of interest (ROI) selection", + clause: "8.9", + fourcc: None, + reads_visual_input: true, + tier: OperationTier::ExactWhen(&[OnSampleGrid, OwnSize]), + basis: "selects samples, like Crop; an ROI emitted at another size is a scaling", + }, + DerivationOp { + capability: CAP_DERIVE_GRID_COMPOSITION, + name: "Grid composition", + clause: "8.10", + fourcc: None, + reads_visual_input: true, + tier: OperationTier::ExactWhen(&[OnSampleGrid, OwnSize]), + basis: "places samples; a cell whose input must be resized is a scaling, and \ + uncovered output samples take the default fill, a constant the claim must name", + }, + DerivationOp { + capability: CAP_DERIVE_OVERLAY_COMPOSITION, + name: "Overlay composition", + clause: "8.11", + fourcc: None, + reads_visual_input: true, + tier: OperationTier::ExactWhen(&[OnSampleGrid, OwnSize, Opaque]), + basis: "places samples; any blending of overlapping inputs computes new sample values", + }, +]; + +/// Every registered derivation operation, in clause order. +pub fn derivation_operations() -> &'static [DerivationOp] { + ®ISTRY +} + +/// The registered derivation operation for a capability string, if any. +pub fn derivation_op(cap: &str) -> Option<&'static DerivationOp> { + REGISTRY.iter().find(|op| op.capability == cap) +} + +/// Whether `cap` is a registered derivation operation. +pub fn is_derivation_operation(cap: &str) -> bool { + derivation_op(cap).is_some() +} + +/// A visual input that one executed instance read, named the way ISO/IEC 23001-16 +/// clause 5.1 lets a derivation operation name an input: through the track +/// reference box, an input track, an image item or, where unified IDs are in use, +/// a track group; the visual output of a previous derivation operation in the same +/// derived sample; or the default input fill picture. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub enum VisualInput { + /// Entry `k` of the `inputs` passed to [`derived_sample_tier`]: a decoded + /// track, image item or track group, or the default fill picture. + Listed(usize), + /// The visual output of entry `j` of the executed list, which must come before + /// the instance that reads it. Clause 5.1 names a previous operation by a + /// relative index into the derived sample's list, which can name an operation + /// that was skipped. A skipped non-essential operation outputs what the + /// operation before it output, or, with none before it, the default fill + /// picture. So the executing sink resolves such a reference to the executed + /// operation whose output was read, or to a `Listed` fill picture. + Preceding(usize), +} + +/// One derivation operation as a sink **executed** it for a derived sample: the +/// [`Condition`]s the sink checked on this instance and found to hold, and the +/// visual inputs the instance read. +/// +/// The conditions are listed, not summarised in one flag, so a sink that checked +/// only the geometry cannot claim the representation condition by omission: an +/// instance is exact only when every condition its operation requires +/// ([`DerivationOp::exact_conditions`]) is listed here. +/// +/// What the instance read is listed for a similar reason. Where an instance sits +/// in the list does not say what it read. ISO/IEC 23001-16 clause 4 lets an +/// operation that is not first read the output of any previous operation, only new +/// inputs, or both. And an operation that reads a visual input always reads one: +/// by clause 5.1, an input takes the one the derived sample lists, else the default +/// the sample entry lists, else the default fill picture. So an instance of an +/// operation registered as reading a visual input +/// ([`DerivationOp::reads_visual_input`]) that names nothing in `reads` holds no +/// tier. [`derived_sample_tier`] trusts `reads` as it +/// trusts `conditions_met`: a sink that leaves out an input an instance read +/// over-claims, just as one that lists a condition it did not check does. +/// +/// The list of executed operations is what was executed, not what the sample +/// declared: ISO/IEC 23001-16 lets a reader skip an unsupported non-essential +/// operation as a null operation, so two conforming readers can emit different +/// samples from one derived sample. A tier claim is about an executed list, over a +/// span of composition time, with the parameter values each instance applied. +/// Two sinks' hashes are comparable only when all of these are equal: the executed +/// lists, the parameter values each instance applied, the span, and the content +/// of the inputs, visual and parameter. This type carries only what the tier +/// depends on. Binding the rest is the receipt's job +/// (`extensions/derived-tracks.md` §3 rule 3 and §4); this crate has no receipt +/// emitter. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct ExecutedOp<'a> { + pub capability: &'a str, + pub conditions_met: &'a [Condition], + /// Every visual input this instance read, whether the derived sample, the + /// sample entry or neither of them listed it. + pub reads: &'a [VisualInput], +} + +/// Why [`derived_sample_tier`] states nothing at all, as opposed to stating that +/// a derived sample holds no tier. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum DerivedSampleError { + /// A derived sample used an operation this registry does not name, including + /// a vendor operation identified by `'uuid'`. A sink cannot state a tier for an + /// operation whose arithmetic nothing registers. + UnregisteredOperation(String), + /// Executed operation `op` read [`VisualInput::Listed`]`(input)`, and `inputs` + /// has no such entry. + NoSuchInput { op: usize, input: usize }, + /// Executed operation `op` read [`VisualInput::Preceding`]`(output)`, and + /// `output` is not an operation executed before it. + NotPreceding { op: usize, output: usize }, + /// Entry `input` of `inputs` is named in no executed operation's `reads`. + /// Either it was listed by mistake, or an instance read it and its `reads` + /// leaves it out; the function cannot tell which. + UnreadInput { input: usize }, +} + +impl std::fmt::Display for DerivedSampleError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::UnregisteredOperation(cap) => { + write!(f, "'{cap}' is not a registered derivation operation") + } + Self::NoSuchInput { op, input } => { + write!(f, "executed operation {op} reads input {input}, which is not listed") + } + Self::NotPreceding { op, output } => write!( + f, + "executed operation {op} reads the output of operation {output}, which does \ + not precede it" + ), + Self::UnreadInput { input } => { + write!(f, "input {input} is listed, but no executed operation reads it") + } + } + } +} +impl std::error::Error for DerivedSampleError {} + +/// The strongest tier a derived sample's emitted samples can claim, or `None`. +/// +/// `inputs` are the tiers of the visual inputs the executed operations read, other +/// than the output of a preceding executed operation: decoded tracks, image items, +/// and the default fill picture when it is read, which counts as an input at +/// `decode-equivalence` only when the claim names its sample values. Every entry +/// is read by some instance, which names it in [`ExecutedOp::reads`] as +/// [`VisualInput::Listed`]; an instance names the output of an earlier executed +/// operation as [`VisualInput::Preceding`]. `None` in `inputs` is an input with no +/// tier. A parameter input (a metadata item, or an interval of a metadata track) +/// has no tier of its own: the claim binds its content and the values the instance +/// applied, and [`Condition::TimeInvariant`] requires each parameter to have one +/// value over the span. +/// +/// A derived sample is exactly as reproducible as its least reproducible step: +/// the result is `decode-equivalence` only when every input is at +/// `decode-equivalence`, every executed operation is [`OperationTier::ExactWhen`] +/// with all of its conditions, the universal ones included, listed as met, and +/// every instance of an operation that reads a visual input names what it read. +/// Anything else gives `None`, including: +/// +/// - an input at `entropy-consistency`, because an operation over reconstructed +/// samples cannot inherit a tier that does not fix them; +/// - an instance of an operation registered as reading a visual input +/// ([`DerivationOp::reads_visual_input`]) whose `reads` is empty, at any +/// position in the list. It read something that nothing names: a track, an +/// item, a track group or a preceding output that the file lists, or, where +/// nothing lists an input, the default fill picture (clause 5.1). An input the +/// claim does not name vouches for nothing (`extensions/derived-tracks.md` §3 +/// rule 5); +/// - an empty executed list. Nothing was executed, so there is nothing to tier. +/// ISO/IEC 23001-16 gives an empty derived sample no visual output; when every +/// listed operation was skipped as an unsupported non-essential one, the output +/// is the default fill picture (clause 5.1), which this function does not tier. +/// +/// When at least one operation executed, a call that contradicts itself is an +/// error, not `None` ([`DerivedSampleError`]): an instance that reads an entry +/// `inputs` does not have, or the output of an operation that does not precede it, +/// or an entry of `inputs` that no instance reads. So is an operation this registry +/// does not name. An empty executed list gives `None` before any of these checks, +/// whatever `inputs` holds. +/// +/// What cannot be detected is a `reads` that leaves out an input its instance +/// read, unless that input is listed and no instance names it. The result is as +/// sound as the executing sink's report of what each instance read, as it is as +/// sound as its report of the conditions it checked. +pub fn derived_sample_tier( + inputs: &[Option], + executed: &[ExecutedOp<'_>], +) -> Result, DerivedSampleError> { + if executed.is_empty() { + return Ok(None); + } + let mut ops_exact = true; + let mut reads_named = true; + let mut read = vec![false; inputs.len()]; + for (i, step) in executed.iter().enumerate() { + let op = derivation_op(step.capability) + .ok_or_else(|| DerivedSampleError::UnregisteredOperation(step.capability.to_string()))?; + for r in step.reads { + match *r { + VisualInput::Listed(k) => { + *read.get_mut(k).ok_or(DerivedSampleError::NoSuchInput { op: i, input: k })? = + true; + } + VisualInput::Preceding(j) if j < i => {} + VisualInput::Preceding(j) => { + return Err(DerivedSampleError::NotPreceding { op: i, output: j }); + } + } + } + // The position of an instance does not say what it read (clause 4), so an + // empty `reads` vouches for nothing, whichever operation came before. + reads_named &= !(op.reads_visual_input && step.reads.is_empty()); + ops_exact &= op + .exact_conditions() + .is_some_and(|required| required.iter().all(|c| step.conditions_met.contains(c))); + } + if let Some(k) = read.iter().position(|was_read| !was_read) { + return Err(DerivedSampleError::UnreadInput { input: k }); + } + let inputs_exact = inputs.iter().all(|t| *t == Some(DeterminismTier::DecodeEquivalence)); + Ok((ops_exact && reads_named && inputs_exact).then_some(DeterminismTier::DecodeEquivalence)) +} + +#[cfg(test)] +mod tests { + use super::*; + use Condition::*; + use DeterminismTier::*; + use VisualInput::{Listed, Preceding}; + + const DE: Option = Some(DecodeEquivalence); + const UNIVERSAL: &[Condition] = &[SameRepresentation, CopiesOnly, TimeInvariant]; + + // What an instance read: nothing, the first listed input, the first two, or + // the output of executed operation 0 or 1. + const NOTHING: &[VisualInput] = &[]; + const L0: &[VisualInput] = &[Listed(0)]; + const L01: &[VisualInput] = &[Listed(0), Listed(1)]; + const P0: &[VisualInput] = &[Preceding(0)]; + const P1: &[VisualInput] = &[Preceding(1)]; + + fn run<'a>( + cap: &'a str, + conditions_met: &'a [Condition], + reads: &'a [VisualInput], + ) -> ExecutedOp<'a> { + ExecutedOp { capability: cap, conditions_met, reads } + } + + /// One listed input at decode-equivalence, read by the operation, for an + /// operation that reads a visual input; nothing listed or read otherwise. + fn one_input(op: &DerivationOp) -> (&'static [Option], &'static [VisualInput]) { + if op.reads_visual_input { (&[DE], L0) } else { (&[], NOTHING) } + } + + #[test] + fn registry_is_the_published_base_set_in_clause_order() { + // The ten operations of ISO/IEC 23001-16:2021 §8.2–§8.11, titled as the + // published table of contents titles them. + let want = [ + ("8.2", "Identity"), + ("8.3", "sRGB Fill"), + ("8.4", "Dissolve"), + ("8.5", "Crop"), + ("8.6", "Rotation"), + ("8.7", "Mirror"), + ("8.8", "Scaling"), + ("8.9", "Region of interest (ROI) selection"), + ("8.10", "Grid composition"), + ("8.11", "Overlay composition"), + ]; + let got: Vec<_> = derivation_operations().iter().map(|op| (op.clause, op.name)).collect(); + assert_eq!(got, want); + + let mut seen = std::collections::HashSet::new(); + for op in derivation_operations() { + assert!(op.capability.starts_with("wai.derive."), "{}", op.capability); + assert!(seen.insert(op.capability), "duplicate {}", op.capability); + assert!(!op.basis.is_empty(), "{} states no basis for its tier", op.capability); + assert_eq!(derivation_op(op.capability), Some(op)); + assert!(is_derivation_operation(op.capability)); + } + assert!(!is_derivation_operation("wai.derive.nope")); + assert!(!is_derivation_operation(super::super::CAP_IMAGE_PNG)); + } + + #[test] + fn derivation_operations_are_not_payload_capabilities() { + // A derivation operation names no payload format, so a manifest cannot + // name one as its payload's capability. + for op in derivation_operations() { + assert!(!super::super::is_known_capability(op.capability), "{}", op.capability); + } + } + + #[test] + fn recorded_fourccs_are_well_formed() { + // None is recorded yet (clause 8 was not read). Any code recorded later + // must be four printable ASCII bytes, unique, and not the vendor escape. + let mut seen = std::collections::HashSet::new(); + for op in derivation_operations() { + if let Some(code) = op.fourcc { + assert!(code.iter().all(|b| b.is_ascii_graphic() || *b == b' '), "{}", op.capability); + assert_ne!(&code, b"uuid", "{} uses the vendor escape", op.capability); + assert!(seen.insert(code), "{} repeats a 4CC", op.capability); + } + } + } + + #[test] + fn operations_that_compute_sample_values_are_untiered() { + // Dissolve mixes two inputs and Scaling resamples: their output values are + // computed, and WAI pins no arithmetic for either, so the registry must not + // tier them. + let untiered: Vec<_> = derivation_operations().iter() + .filter(|op| op.tier == OperationTier::Untiered) + .map(|op| op.capability) + .collect(); + assert_eq!(untiered, [CAP_DERIVE_DISSOLVE, CAP_DERIVE_SCALING]); + for op in derivation_operations() { + assert_eq!(op.exact_conditions().is_none(), op.tier == OperationTier::Untiered); + } + } + + #[test] + fn no_operation_is_exact_without_the_universal_conditions() { + // Identity included: nothing read says an operation's output shares its + // inputs' representation, or stays one frame over a derived sample's + // duration, so every tiered operation requires that no sample value is + // converted, none is computed, and the output does not change over the + // span the claim covers. + for op in derivation_operations() { + let Some(required) = op.exact_conditions() else { continue }; + let (inputs, reads) = one_input(op); + assert_eq!(&required[..UNIVERSAL.len()], UNIVERSAL, "{}", op.capability); + assert_eq!(&UNIVERSAL_CONDITIONS[..], UNIVERSAL); + // An instance that meets every other condition, but not a universal + // one, holds no tier. + for dropped in UNIVERSAL { + let met: Vec<_> = required.iter().copied().filter(|c| c != dropped).collect(); + assert_eq!(derived_sample_tier(inputs, &[run(op.capability, &met, reads)]), Ok(None), + "{} without {dropped:?}", op.capability); + } + assert_eq!(derived_sample_tier(inputs, &[run(op.capability, &required, reads)]), Ok(DE), + "{}", op.capability); + } + for c in [SameRepresentation, CopiesOnly, TimeInvariant, OnSampleGrid, OwnSize, + QuarterTurn, NoSubsampling, SitingPreserved, Opaque] { + assert!(!c.statement().is_empty(), "{c:?}"); + } + } + + #[test] + fn a_representation_change_voids_the_tier() { + // An 8-bit input composed into a 10-bit output, or a BT.709 input into a + // BT.2020 output: the geometry is sample-exact, but values are converted. + let geometry_only = [CopiesOnly, TimeInvariant, OnSampleGrid, OwnSize]; + let grid = [run(CAP_DERIVE_GRID_COMPOSITION, &geometry_only, L01)]; + assert_eq!(derived_sample_tier(&[DE, DE], &grid), Ok(None)); + let overlay = [run(CAP_DERIVE_OVERLAY_COMPOSITION, + &[CopiesOnly, TimeInvariant, OnSampleGrid, OwnSize, Opaque], L01)]; + assert_eq!(derived_sample_tier(&[DE, DE], &overlay), Ok(None)); + // An Identity whose output representation differs from its input's. + let id = [run(CAP_DERIVE_IDENTITY, &[CopiesOnly, TimeInvariant], L0)]; + assert_eq!(derived_sample_tier(&[DE], &id), Ok(None)); + // The same instances with the representation unchanged. + let grid_ok = [run(CAP_DERIVE_GRID_COMPOSITION, + &[SameRepresentation, CopiesOnly, TimeInvariant, OnSampleGrid, OwnSize], + L01)]; + assert_eq!(derived_sample_tier(&[DE, DE], &grid_ok), Ok(DE)); + assert_eq!(derived_sample_tier(&[DE], &[run(CAP_DERIVE_IDENTITY, UNIVERSAL, L0)]), Ok(DE)); + } + + #[test] + fn a_change_over_the_span_voids_the_tier() { + // ISO/IEC 23001-16 clause 4: any derivation transformation may have + // internal time structure, and one derived sample may span several input + // samples; a parameter may come from an interval of a metadata track. + // Space-exact conditions do not make up for an output that changes over + // the span the claim covers. + let space_exact = [SameRepresentation, CopiesOnly, OnSampleGrid, OwnSize]; + // An ROI whose region is driven by a metadata-track interval whose value + // changes within the derived sample, claimed over the whole duration. + let roi = [run(CAP_DERIVE_ROI_SELECTION, &space_exact, L0)]; + assert_eq!(derived_sample_tier(&[DE], &roi), Ok(None)); + // An Identity over an input interval holding several samples, claimed + // over the whole duration. + let id = [run(CAP_DERIVE_IDENTITY, &[SameRepresentation, CopiesOnly], L0)]; + assert_eq!(derived_sample_tier(&[DE], &id), Ok(None)); + // Crop, grid composition and overlay composition likewise. + assert_eq!(derived_sample_tier(&[DE], &[run(CAP_DERIVE_CROP, &space_exact, L0)]), Ok(None)); + let grid = [run(CAP_DERIVE_GRID_COMPOSITION, &space_exact, L01)]; + assert_eq!(derived_sample_tier(&[DE, DE], &grid), Ok(None)); + let overlay = [SameRepresentation, CopiesOnly, OnSampleGrid, OwnSize, Opaque]; + let overlay = [run(CAP_DERIVE_OVERLAY_COMPOSITION, &overlay, L01)]; + assert_eq!(derived_sample_tier(&[DE, DE], &overlay), Ok(None)); + + // The same instances, claimed for one output frame at a named instant at + // which nothing they read changes, where the sink found the condition to hold. + let at_instant = [SameRepresentation, CopiesOnly, TimeInvariant, OnSampleGrid, OwnSize]; + assert_eq!(derived_sample_tier(&[DE], &[run(CAP_DERIVE_ROI_SELECTION, &at_instant, L0)]), + Ok(DE)); + assert_eq!(derived_sample_tier(&[DE], &[run(CAP_DERIVE_IDENTITY, UNIVERSAL, L0)]), Ok(DE)); + + // One step that changes over the span voids the whole chain. + let chain = [run(CAP_DERIVE_CROP, &at_instant, L0), + run(CAP_DERIVE_ROI_SELECTION, &space_exact, P0), + run(CAP_DERIVE_IDENTITY, UNIVERSAL, P1)]; + assert_eq!(derived_sample_tier(&[DE], &chain), Ok(None)); + + // A time condition does not rescue an operation that computes values. + let all = [SameRepresentation, CopiesOnly, TimeInvariant]; + assert_eq!(derived_sample_tier(&[DE, DE], &[run(CAP_DERIVE_DISSOLVE, &all, L01)]), Ok(None)); + } + + #[test] + fn a_derived_sample_holds_its_weakest_step() { + let id = run(CAP_DERIVE_IDENTITY, UNIVERSAL, L0); + assert_eq!(derived_sample_tier(&[DE], &[id]), Ok(DE)); + + // A conditional operation holds the tier only where its conditions are met, + // in any order. + let crop_ok = [OnSampleGrid, TimeInvariant, CopiesOnly, SameRepresentation]; + assert_eq!(derived_sample_tier(&[DE], &[run(CAP_DERIVE_CROP, &crop_ok, L0)]), Ok(DE)); + assert_eq!(derived_sample_tier(&[DE], &[run(CAP_DERIVE_CROP, UNIVERSAL, L0)]), Ok(None)); + // Rotation needs both of its own conditions. + let quarter = [SameRepresentation, CopiesOnly, TimeInvariant, QuarterTurn]; + assert_eq!(derived_sample_tier(&[DE], &[run(CAP_DERIVE_ROTATION, &quarter, L0)]), Ok(None)); + + // One untiered step voids the chain, whatever came before it; listing + // conditions does not rescue an untiered operation. + let all = [SameRepresentation, CopiesOnly, TimeInvariant, OnSampleGrid, OwnSize, + QuarterTurn, NoSubsampling, SitingPreserved, Opaque]; + let chain = [run(CAP_DERIVE_CROP, &crop_ok, L0), run(CAP_DERIVE_SCALING, &all, P0), + run(CAP_DERIVE_IDENTITY, UNIVERSAL, P1)]; + assert_eq!(derived_sample_tier(&[DE], &chain), Ok(None)); + assert_eq!(derived_sample_tier(&[DE, DE], &[run(CAP_DERIVE_DISSOLVE, &all, L01)]), Ok(None)); + // The same chain with every step exact holds the tier. + let chain = [run(CAP_DERIVE_CROP, &crop_ok, L0), run(CAP_DERIVE_CROP, &crop_ok, P0), + run(CAP_DERIVE_IDENTITY, UNIVERSAL, P1)]; + assert_eq!(derived_sample_tier(&[DE], &chain), Ok(DE)); + + // One input without the tier voids it too. + let place = [run(CAP_DERIVE_GRID_COMPOSITION, + &[SameRepresentation, CopiesOnly, TimeInvariant, OnSampleGrid, OwnSize], + L01)]; + assert_eq!(derived_sample_tier(&[DE, DE], &place), Ok(DE)); + assert_eq!(derived_sample_tier(&[DE, None], &place), Ok(None)); + } + + #[test] + fn an_instance_that_reads_an_input_must_name_it() { + // An operation that reads a visual input reads one even when nothing lists + // it: an input that neither the derived sample nor the sample entry lists + // is the default fill picture (ISO/IEC 23001-16 clause 5.1). So an instance + // that names nothing it read vouches for nothing. + let non_readers: Vec<_> = derivation_operations().iter() + .filter(|op| !op.reads_visual_input) + .map(|op| op.capability) + .collect(); + assert_eq!(non_readers, [CAP_DERIVE_SRGB_FILL]); + let fill = run(CAP_DERIVE_SRGB_FILL, UNIVERSAL, NOTHING); + assert_eq!(derived_sample_tier(&[], &[fill]), Ok(DE)); + + for op in derivation_operations() { + let Some(required) = op.exact_conditions() else { continue }; + if !op.reads_visual_input { + continue; + } + let cap = op.capability; + // First in the list, reading nothing named. + assert_eq!(derived_sample_tier(&[], &[run(cap, &required, NOTHING)]), Ok(None), "{cap}"); + // Named, at decode-equivalence (a track, or a fill whose values the + // claim names): the tier holds. Named with no tier (a fill whose values + // the claim does not name): it does not. + assert_eq!(derived_sample_tier(&[DE], &[run(cap, &required, L0)]), Ok(DE), "{cap}"); + assert_eq!(derived_sample_tier(&[None], &[run(cap, &required, L0)]), Ok(None), "{cap}"); + + // Clause 4: an operation that is not first may read the output of any + // previous operation, only new inputs, or both. So following an sRGB + // Fill, which reads nothing, does not tell what an instance read, and + // an instance that names nothing holds no tier there either. + let after_fill = [fill, run(cap, &required, NOTHING)]; + assert_eq!(derived_sample_tier(&[], &after_fill), Ok(None), "{cap} after a fill"); + // Reading only the fill's output, it holds the tier with no input listed. + let of_fill = [fill, run(cap, &required, P0)]; + assert_eq!(derived_sample_tier(&[], &of_fill), Ok(DE), "{cap} of a fill"); + // Reading the fill's output and a new input, the new input decides. + let both = [Preceding(0), Listed(0)]; + let mixed = [fill, run(cap, &required, &both)]; + assert_eq!(derived_sample_tier(&[DE], &mixed), Ok(DE), "{cap}"); + assert_eq!(derived_sample_tier(&[None], &mixed), Ok(None), "{cap}"); + assert_eq!(derived_sample_tier(&[], &mixed), + Err(DerivedSampleError::NoSuchInput { op: 1, input: 0 }), "{cap}"); + } + + // An operation registered as reading nothing may still name a read; what it + // names is checked like any other. + let fill_reading = [run(CAP_DERIVE_SRGB_FILL, UNIVERSAL, L0)]; + assert_eq!(derived_sample_tier(&[DE], &fill_reading), Ok(DE)); + assert_eq!(derived_sample_tier(&[None], &fill_reading), Ok(None)); + } + + #[test] + fn a_report_that_contradicts_itself_is_an_error() { + let crop = [SameRepresentation, CopiesOnly, TimeInvariant, OnSampleGrid]; + let fill = run(CAP_DERIVE_SRGB_FILL, UNIVERSAL, NOTHING); + // A listed input that is not there. + assert_eq!(derived_sample_tier(&[], &[run(CAP_DERIVE_CROP, &crop, L0)]), + Err(DerivedSampleError::NoSuchInput { op: 0, input: 0 })); + assert_eq!(derived_sample_tier(&[DE], &[run(CAP_DERIVE_GRID_COMPOSITION, &[], L01)]), + Err(DerivedSampleError::NoSuchInput { op: 0, input: 1 })); + // The first operation has no preceding output to read, and no operation + // reads its own output or a later one's. + assert_eq!(derived_sample_tier(&[], &[run(CAP_DERIVE_CROP, &crop, P0)]), + Err(DerivedSampleError::NotPreceding { op: 0, output: 0 })); + assert_eq!(derived_sample_tier(&[], &[fill, run(CAP_DERIVE_CROP, &crop, P1)]), + Err(DerivedSampleError::NotPreceding { op: 1, output: 1 })); + // A listed input that no instance names: a grid that read two inputs but + // names one. + let grid = [SameRepresentation, CopiesOnly, TimeInvariant, OnSampleGrid, OwnSize]; + assert_eq!(derived_sample_tier(&[DE, DE], &[run(CAP_DERIVE_GRID_COMPOSITION, &grid, L0)]), + Err(DerivedSampleError::UnreadInput { input: 1 })); + // A fill that reads nothing does not account for a listed input either. + assert_eq!(derived_sample_tier(&[DE], &[fill]), + Err(DerivedSampleError::UnreadInput { input: 0 })); + // The order of the list matters: a crop before the fill reads an input. + let crop_then_fill = [run(CAP_DERIVE_CROP, &crop, L0), fill]; + assert_eq!(derived_sample_tier(&[DE], &crop_then_fill), Ok(DE)); + assert_eq!(derived_sample_tier(&[], &crop_then_fill), + Err(DerivedSampleError::NoSuchInput { op: 0, input: 0 })); + // An operation the registry does not name is still an error. + let vendor = "wai.derive.vendor-uuid"; + assert_eq!(derived_sample_tier(&[], &[run(vendor, UNIVERSAL, NOTHING)]), + Err(DerivedSampleError::UnregisteredOperation(vendor.to_string()))); + for e in [DerivedSampleError::UnregisteredOperation(vendor.to_string()), + DerivedSampleError::NoSuchInput { op: 0, input: 0 }, + DerivedSampleError::NotPreceding { op: 0, output: 0 }, + DerivedSampleError::UnreadInput { input: 0 }] { + assert!(!e.to_string().is_empty(), "{e:?}"); + } + } + + #[test] + fn entropy_consistency_does_not_survive_derivation() { + // The operations act on reconstructed samples, which entropy-consistency + // does not fix. + let id = [run(CAP_DERIVE_IDENTITY, UNIVERSAL, L0)]; + assert_eq!(derived_sample_tier(&[Some(EntropyConsistency)], &id), Ok(None)); + let grid = [run(CAP_DERIVE_GRID_COMPOSITION, + &[SameRepresentation, CopiesOnly, TimeInvariant, OnSampleGrid, OwnSize], + L01)]; + assert_eq!(derived_sample_tier(&[DE, Some(EntropyConsistency)], &grid), Ok(None)); + } + + #[test] + fn edge_cases() { + // A fill with no visual input is exact where its colour is written as is. + assert_eq!(derived_sample_tier(&[], &[run(CAP_DERIVE_SRGB_FILL, UNIVERSAL, NOTHING)]), Ok(DE)); + let converted = [run(CAP_DERIVE_SRGB_FILL, &[CopiesOnly, TimeInvariant], NOTHING)]; + assert_eq!(derived_sample_tier(&[], &converted), Ok(None)); + // Nothing executed: nothing to tier. + assert_eq!(derived_sample_tier(&[DE], &[]), Ok(None)); + assert_eq!(derived_sample_tier(&[], &[]), Ok(None)); + // An operation nothing registers (a vendor 'uuid' operation, or a payload + // capability misused as an operation) cannot be tiered at all. + for bad in ["wai.derive.vendor-uuid", super::super::CAP_IMAGE_PNG] { + assert_eq!( + derived_sample_tier(&[DE], &[run(bad, UNIVERSAL, L0)]), + Err(DerivedSampleError::UnregisteredOperation(bad.to_string())) + ); + } + } + + #[test] + fn derivation_license_is_not_royalty_free() { + assert_eq!(derivation_license(), LicenseClass::PatentPool); + } +} diff --git a/wai/wai-rs/src/codecs/mod.rs b/wai/wai-rs/src/codecs/mod.rs index 6fb98d82a..d09940ff4 100644 --- a/wai/wai-rs/src/codecs/mod.rs +++ b/wai/wai-rs/src/codecs/mod.rs @@ -16,6 +16,12 @@ pub mod text; #[cfg(feature = "codecs")] pub mod video; +// Registry-only modules, no codec dependency: the derivation operations of +// ISO/IEC 23001-16 with a determinism tier each (extensions/derived-tracks), and +// the wire forms a `Prior` may declare (extensions/prior-carriage). +pub mod derivation; +pub mod prior_form; + // ---- Capability strings (the values that go in manifest.model_requirement.capability) // Image pub const CAP_IMAGE_PNG: &str = "wai.image.png"; // universal floor @@ -195,6 +201,22 @@ pub const CAP_QUANTUM_CIRCUIT: &str = "wai.quantum.circuit"; // determinism = WAI's deterministic floor (StandardDefined replicate class). pub const CAP_TERNARY_TT: &str = "wai.ternary.tt"; +// Neural-network coding, ISO/IEC 15938-17 ("NNC"; extensions/prior-carriage). +// wai.prior.nnc: a trained network's parameters as an NNC bitstream conforming to +// the published second edition (ISO/IEC 15938-17:2024) — the registered wire form +// a `Prior` MAY declare for the artifact its digest covers, and the capability the +// artifact travels under when it is carried at all. Decoder sink-supplied; this +// crate reads no NNC. The standard's reference software and decoder-conformance +// tests are specified by ISO/IEC 15938-18. +pub const CAP_PRIOR_NNC: &str = "wai.prior.nnc"; +// wai.tensor.nnc: general tensorial data (split-inference feature maps, 3D/4D +// scene parameters) coded with NNC. A CANDIDATE, like wai.splat.sog: general-tensor +// coding is in an NNC amendment at committee-draft stage, so the payload format is +// not fixed and an encoder MUST NOT emit it until it is pinned to a published +// edition. For splat attributes, wai.splat.int_codec is the byte-exact path; this +// is an interchange form with no determinism guarantee. +pub const CAP_TENSOR_NNC: &str = "wai.tensor.nnc"; + // Linear film (extensions/linear-film): a film is a sealed camera-on-rails // world. A WFLM embeds a sealed wai.world.replay + a keyframed camera track // (Fx floor) + a frame schedule; every sink samples the world at each frame @@ -449,6 +471,8 @@ pub fn known_capabilities() -> &'static [&'static str] { CAP_QUANTUM_CIRCUIT, // quantum-inspired ternary-MPS transform (byte-exact recon, lossy) CAP_TERNARY_TT, + // neural-network coding: the prior artifact form + the general-tensor candidate + CAP_PRIOR_NNC, CAP_TENSOR_NNC, // immersive audio (IAMF royalty-free; AC-4 / IVAS license-flagged) CAP_AUDIO_IAMF, CAP_AUDIO_AC4, CAP_AUDIO_IVAS, // cinema mastering (JPEG 2000 / DCP) @@ -622,6 +646,12 @@ pub enum FidelityTier { pub fn capability_tier(cap: &str) -> Option { use FidelityTier::*; let tier = match cap { + // NNC: lossy by primary role (compression by parameter reduction and + // quantisation), as AVIF is. The published standard also has integer and + // raw-float payload types that carry values unquantised, so an NNC + // artifact can reproduce its source; whether every conforming decoder + // reconstructs the same values is open (extensions/prior-carriage.md §5). + CAP_PRIOR_NNC | CAP_TENSOR_NNC => Lossy, // Guaranteed / ceiling-lossless. CAP_IMAGE_PNG | CAP_IMAGE_JXL | CAP_AUDIO_FLAC | CAP_VIDEO_AV1_LOSSLESS | CAP_TEXT_ZSTD | CAP_TEXT_XZ @@ -727,6 +757,11 @@ pub fn capability_license(cap: &str) -> Option { return None; } Some(match cap { + // NNC: royalty status not established — ISO/IEC 15938-17:2022 records a + // patent claimed essential under a RAND assurance, an open implementation + // is licensed with no patent rights granted, and no royalty-free + // commitment was found. Classified conservatively, as MPEG-I Haptics is. + CAP_PRIOR_NNC | CAP_TENSOR_NNC => PatentPool, // Single-vendor proprietary (Dolby). CAP_AUDIO_AC4 | CAP_META_DOVI => Proprietary, // Patent-pool / SEP royalties: IVAS (Via-LA Voice Codec pool, with @@ -947,7 +982,10 @@ pub enum ReplicateClass { /// The two reproducibility tiers a neural capability can be registered at /// (`extensions/determinism-tiers.md` §2). Each is a property of the capability as /// registered — it holds on every conforming sink — not of one implementation. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] +/// Ordered weakest first. One type serves the registry and +/// [`derivation::derived_sample_tier`], so a tier read from one can be passed to +/// the other. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] pub enum DeterminismTier { /// The entropy decode reproduces byte-for-byte on every conforming sink; the /// reconstruction may differ. Does not back `replicate`. @@ -1118,6 +1156,12 @@ pub fn capability_replicate_class(cap: &str) -> Option { | CAP_MESH_DRACO | CAP_PC_GPCC | CAP_PC_VPCC | CAP_SPLAT_SOG | CAP_HAPTIC_MIHS | CAP_META_HDR10PLUS | CAP_META_DOVI | CAP_MESH_GLB | CAP_SPLAT_SPZ => StandardUnclassified, + // Neural-network coding (ISO/IEC 15938-17). Whether every conforming NNC + // decoder reconstructs the same parameter values is not established + // (extensions/prior-carriage.md §5), so NNC is held to the conservative + // class like the other unchecked standards. A prior's declared NNC form + // is refused for replicate by Prior::is_cross_hardware_exact regardless. + CAP_PRIOR_NNC | CAP_TENSOR_NNC => StandardUnclassified, // Registered but unclassified: a registry defect, surfaced as `None`. _ => return None, @@ -1137,7 +1181,7 @@ pub fn is_neural_integer_exact(cap: &str) -> bool { } /// Why SPEC §7 refuses an `intent = replicate` envelope. See [`replicate_refusal`]. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[derive(Debug, Clone, PartialEq, Eq)] pub enum ReplicateRefusal { /// The capability is not registered, so nothing states what it reproduces. Unregistered, @@ -1145,9 +1189,12 @@ pub enum ReplicateRefusal { /// ([`ReplicateClass::replicate_promise`] is `None`), whatever the envelope's /// prior declares. Class(ReplicateClass), - /// The class is [`ReplicateClass::NeuralIntegerExact`], but the envelope pins no - /// prior (`None`), or its prior's declared contract is not integer-exact. - Prior(Option), + /// The class is [`ReplicateClass::NeuralIntegerExact`], but the envelope's prior + /// does not make the decode exact across hardware: none is pinned, its declared + /// contract is not integer-exact, or it declares a form whose reconstruction is + /// not established exact at every reader + /// ([`crate::container::replicate_prior_refusal`] gives the reason). + Prior(crate::container::PriorRefusal), } impl std::fmt::Display for ReplicateRefusal { @@ -1167,9 +1214,8 @@ impl std::fmt::Display for ReplicateRefusal { "registry class GenerativeDeterministic: a generator (intent = create) has no \ source to replicate"), Self::Class(c) => write!(f, "registry class {c:?} does not back replicate"), - Self::Prior(None) => write!(f, "no pinned prior"), - Self::Prior(Some(d)) => write!(f, - "the pinned prior declares a {d:?} contract, not an integer-exact one"), + Self::Prior(crate::container::PriorRefusal::Absent) => write!(f, "no pinned prior"), + Self::Prior(r) => write!(f, "{r}"), } } } @@ -1207,11 +1253,11 @@ pub fn replicate_refusal( } match capability_replicate_class(cap) { None => Some(ReplicateRefusal::Unregistered), - Some(ReplicateClass::NeuralIntegerExact) => match prior { - Some(p) if p.is_cross_hardware_exact() => None, - Some(p) => Some(ReplicateRefusal::Prior(Some(p.determinism_class()))), - None => Some(ReplicateRefusal::Prior(None)), - }, + // The prior's reason, including a declared form that is not established + // exact, comes from the one place that defines it. + Some(ReplicateClass::NeuralIntegerExact) => { + crate::container::replicate_prior_refusal(intent, prior).map(ReplicateRefusal::Prior) + } Some(c) if c.replicate_promise().is_some() => None, Some(c) => Some(ReplicateRefusal::Class(c)), } @@ -1482,6 +1528,8 @@ mod tests { CAP_SEMANTIC_PERCO2, CAP_VIDEO_VVC, CAP_VIDEO_HEVC, CAP_VIDEO_AVC, CAP_VIDEO_EVC, CAP_VIDEO_LCEVC, CAP_VIDEO_AVS3, CAP_AUDIO_MPEGH, CAP_AUDIO_AVS3, CAP_AUDIO_AAC, + // NNC (ISO/IEC 15938-17): royalty status not established, so pooled. + CAP_PRIOR_NNC, CAP_TENSOR_NNC, ]; want.sort_unstable(); assert_eq!(non_rf, want); @@ -1730,7 +1778,9 @@ mod tests { CAP_AUDIO_MPEGH, CAP_AUDIO_AVS3, CAP_AUDIO_AAC, CAP_AUDIO_AC4, CAP_MESH_DRACO, CAP_PC_GPCC, CAP_PC_VPCC, CAP_SPLAT_SOG, CAP_HAPTIC_MIHS, CAP_META_HDR10PLUS, CAP_META_DOVI, - CAP_MESH_GLB, CAP_SPLAT_SPZ])); + CAP_MESH_GLB, CAP_SPLAT_SPZ, + // NNC: reconstruction equality across decoders not established. + CAP_PRIOR_NNC, CAP_TENSOR_NNC])); // Every natively decoded standard codec is classified: none is unclassified. for &c in sink_capabilities() { assert_ne!(capability_replicate_class(c), Some(StandardUnclassified), "{c}"); @@ -1758,7 +1808,7 @@ mod tests { #[test] fn replicate_follows_the_registry_class_not_the_declared_prior() { use crate::container::{DeterminismClass, Prior}; - let p = |det: &str| Prior { id: "m".into(), sha256: "ab".repeat(32), determinism: det.into() }; + let p = |det: &str| Prior { id: "m".into(), sha256: "ab".repeat(32), determinism: det.into(), form: None }; let integer = p("int-rans/p16+int-synth/i64"); let floaty = p("int-rans/p16+onnx-cpu-f32/1thread"); @@ -1794,9 +1844,9 @@ mod tests { // An integer-exact capability still needs its integer-exact prior. assert_eq!(replicate_refusal(CAP_NEURAL_INT_SYNTH, "replicate", Some(&integer)), None); assert_eq!(replicate_refusal(CAP_NEURAL_INT_SYNTH, "replicate", Some(&floaty)), - Some(ReplicateRefusal::Prior(Some(DeterminismClass::FloatReproducible)))); + Some(ReplicateRefusal::Prior(crate::container::PriorRefusal::NotIntegerExact(DeterminismClass::FloatReproducible)))); assert_eq!(replicate_refusal(CAP_NEURAL_INT_SYNTH, "replicate", None), - Some(ReplicateRefusal::Prior(None))); + Some(ReplicateRefusal::Prior(crate::container::PriorRefusal::Absent))); // A standard-defined floor codec carries no prior and is not refused here. assert_eq!(replicate_refusal(CAP_IMAGE_PNG, "replicate", None), None); @@ -1980,14 +2030,14 @@ mod tests { assert_eq!(default.is_ok(), matches!(class, ReplicateClass::StandardDefined | ReplicateClass::StandardTolerance | ReplicateClass::StandardUnclassified), "{c} ({class:?})"); - match default { + match &default { // What the encoder writes, the sink's check accepts … Ok(intent) => { - assert_eq!(intent, "replicate", "{c}"); + assert_eq!(*intent, "replicate", "{c}"); assert_eq!(envelope(c, intent).replicate_refusal(), None, "{c}"); } // … and what it refuses is what the sink would refuse, for the same reason. - Err(why) => assert_eq!(envelope(c, "replicate").replicate_refusal(), Some(why), "{c}"), + Err(why) => assert_eq!(envelope(c, "replicate").replicate_refusal(), Some(why.clone()), "{c}"), } // An explicit `replicate` is held to the same rule; any other intent passes. assert_eq!(unpinned_envelope_intent(c, Some("replicate")), default, "{c}"); @@ -2007,7 +2057,7 @@ mod tests { // An integer-exact neural codec backs replicate only with a pinned prior, // which an unpinned envelope does not carry. assert_eq!(unpinned_envelope_intent(CAP_NEURAL_INT_HYPER, None), - Err(ReplicateRefusal::Prior(None))); + Err(ReplicateRefusal::Prior(crate::container::PriorRefusal::Absent))); // Every capability `wai encode` writes (all with `replicate`, no prior) is // one of the standard classes, so its envelopes stay acceptable. diff --git a/wai/wai-rs/src/codecs/prior_form.rs b/wai/wai-rs/src/codecs/prior_form.rs new file mode 100644 index 000000000..8a822b428 --- /dev/null +++ b/wai/wai-rs/src/codecs/prior_form.rs @@ -0,0 +1,160 @@ +//! Registered wire forms for the artifact a [`Prior`](crate::container::Prior) +//! names. Normative text: `extensions/prior-carriage.md`. +//! +//! A `Prior` pins a model's identity by digest. The digest fixes *which bytes*; +//! a declared form fixes *what those bytes are*, so two parties holding the same +//! digest also agree on how to read it. A form is named by a capability string +//! from the registry, so the artifact can travel in a WAI envelope under the same +//! name, and a sink decides whether it can read a form by the same dispatch it +//! uses for payloads. + +use super::CAP_PRIOR_NNC; + +/// Every capability string a `Prior` may declare as its artifact's form. +pub fn prior_forms() -> &'static [&'static str] { + &[CAP_PRIOR_NNC] +} + +/// Whether `cap` is a registered prior form. +pub fn is_prior_form(cap: &str) -> bool { + prior_forms().contains(&cap) +} + +/// Prior forms whose decode to the tensors a network consumes is **established** +/// to give identical tensors at every conforming reader. +/// +/// Empty. ISO/IEC 15938-17 already has payload types that carry values without +/// quantisation: its 2022 edition lists decoding methods for an integer payload +/// type (`NNR_PT_INT`) and a raw-float payload type (`NNR_PT_RAW_FLOAT`), and a +/// decoding process for an integer weight tensor. What is not established here is +/// whether every conforming decoder reconstructs bit-identical parameter values, +/// from those payload types or from quantisation indices. Decoder conformance is +/// specified in its clause 7 and tested with the bitstreams and procedure of +/// ISO/IEC 15938-18, and neither text was read. That is what would settle it, for +/// the published editions and for any later one; a lossless mode added by an +/// amendment would not settle it by itself. Until a form is listed, a `replicate` +/// claim cannot rest on a prior carried in it (see +/// [`Prior::form_preserves_exactness`](crate::container::Prior::form_preserves_exactness)). +const EXACT_RECONSTRUCTION_FORMS: &[&str] = &[]; + +/// Whether reading an artifact out of `form` is established to reproduce the +/// same tensors at every conforming reader. `false` for an unregistered form. +pub fn form_reconstruction_is_exact(form: &str) -> bool { + is_prior_form(form) && EXACT_RECONSTRUCTION_FORMS.contains(&form) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::codecs::{ + capability_license, capability_replicate_class, capability_tier, is_known_capability, + is_neural_integer_exact, is_royalty_free, FidelityTier, LicenseClass, CAP_HAPTIC_MIHS, + CAP_IMAGE_PNG, CAP_NEURAL_INT_HYPER, CAP_TENSOR_NNC, + }; + use crate::container::{ + replicate_is_sound, replicate_prior_refusal, Conditioning, DeterminismClass, Manifest, + ModelRequirement, Prior, PriorRefusal, Wai, + }; + + #[test] + fn prior_forms_are_registered_capabilities() { + assert_eq!(prior_forms(), [CAP_PRIOR_NNC]); + for &f in prior_forms() { + assert!(is_known_capability(f), "{f} must be a registered capability"); + } + // A payload codec is not a model form, and a general tensor is not a + // network: neither may stand as a prior's form. + for not_a_form in [CAP_IMAGE_PNG, CAP_NEURAL_INT_HYPER, CAP_TENSOR_NNC, "wai.prior.nope"] { + assert!(!is_prior_form(not_a_form), "{not_a_form}"); + } + } + + #[test] + fn no_form_is_yet_established_exact() { + assert!(!form_reconstruction_is_exact(CAP_PRIOR_NNC)); + assert!(!form_reconstruction_is_exact("wai.prior.nope")); + } + + #[test] + fn nnc_capabilities_are_classified_conservatively() { + for c in [CAP_PRIOR_NNC, CAP_TENSOR_NNC] { + // Lossy by primary role (compression by parameter reduction and + // quantisation), as AVIF is. The published standard also has integer + // and raw-float payload types that carry values unquantised, so this + // is not a claim that every NNC artifact approximates its source. + assert_eq!(capability_tier(c), Some(FidelityTier::Lossy), "{c}"); + // Royalty status not established (a RAND patent declaration, and no + // royalty-free commitment found): never presented as royalty-free. + assert_eq!(capability_license(c), Some(LicenseClass::PatentPool), "{c}"); + assert!(!is_royalty_free(c), "{c}"); + // Not a neural decode WAI runs, so never in the integer-exact class. + assert!(capability_replicate_class(c).is_some(), "{c}"); + assert!(!is_neural_integer_exact(c), "{c}"); + // Replicate class: the one the registry gives an MPEG standard whose + // decoder conformance it has not checked, as it gives MPEG-I Haptics. + // NNC's decoder conformance (ISO/IEC 15938-17 clause 7, tested per + // ISO/IEC 15938-18) was not read, so NNC takes no class that a checked + // conformance would justify. Where the registry has one class for every + // standard codec, that class; where it separates the standard codecs by + // what their decoder conformance fixes, the unchecked one. + assert_eq!(capability_replicate_class(c), capability_replicate_class(CAP_HAPTIC_MIHS), + "{c} must be classed as a standard whose conformance was not checked"); + } + } + + /// The declared-form rule is part of [`Prior::is_cross_hardware_exact`], the + /// predicate every `replicate` check reads, so it holds whichever check runs: + /// the free function, the envelope's own check, and the reason a strict decoder + /// reports. The capability is integer-exact, so only the prior can refuse it. + #[test] + fn every_replicate_check_applies_the_form_rule() { + let integer = Prior::new("int_hyper/q4", "ef".repeat(32), "int-rans/p16+int-synth"); + let nnc = integer.clone().with_form(CAP_PRIOR_NNC); + let envelope = |intent: &str, prior: &Prior| { + Wai::new( + Manifest { + wai: "1.0".into(), + media: "image".into(), + intent: intent.into(), + model_requirement: ModelRequirement { + capability: CAP_NEURAL_INT_HYPER.into(), + fallback: None, + min_sim: None, + sim_metric: None, + }, + conditioning: Conditioning { kind: "int_hyper".into(), prior: Some(prior.clone()) }, + target: serde_json::Value::Null, + }, + vec![], + ) + }; + assert!(is_neural_integer_exact(CAP_NEURAL_INT_HYPER)); + assert_eq!(nnc.determinism_class(), DeterminismClass::IntegerExact); + + // Sound when the artifact is the bytes the digest covers … + assert!(integer.is_cross_hardware_exact()); + assert!(replicate_is_sound("replicate", Some(&integer))); + assert!(envelope("replicate", &integer).replicate_is_sound()); + // … and refused by every check once it must be read out of NNC. + assert!(!nnc.is_cross_hardware_exact()); + assert!(!replicate_is_sound("replicate", Some(&nnc))); + assert!(!envelope("replicate", &nnc).replicate_is_sound()); + assert_eq!(envelope("replicate", &nnc).replicate_prior_refusal(), + Some(PriorRefusal::FormNotExact(CAP_PRIOR_NNC.into()))); + // Only `replicate` is gated. + assert!(envelope("improve", &nnc).replicate_is_sound()); + + // The yes/no answer and the reason never disagree. + let floaty = Prior::new("m", "ab".repeat(32), "int-rans/p16+onnx-cpu-f32/1thread"); + let floaty_nnc = floaty.clone().with_form(CAP_PRIOR_NNC); + let unregistered = integer.clone().with_form("wai.prior.nope"); + let priors = [None, Some(&integer), Some(&nnc), Some(&floaty), Some(&floaty_nnc), + Some(&unregistered)]; + for intent in ["replicate", "improve", "create"] { + for prior in priors { + assert_eq!(replicate_is_sound(intent, prior), + replicate_prior_refusal(intent, prior).is_none(), "{intent} {prior:?}"); + } + } + } +} diff --git a/wai/wai-rs/src/container.rs b/wai/wai-rs/src/container.rs index 6342ed8d1..7bb9a5d52 100644 --- a/wai/wai-rs/src/container.rs +++ b/wai/wai-rs/src/container.rs @@ -11,8 +11,11 @@ //! family name (`wai.neural.bmshj2018`) can front different checkpoints //! that reconstruct differently. So the conditioning MAY carry a [`Prior`]: //! the model's content hash plus a determinism contract. This pins -//! identity, not transport — the weights are still never shipped, exactly -//! as gzip never ships its decoder; the manifest only names *which* one. +//! identity, not transport — the envelope that decodes against the weights +//! never ships them, exactly as gzip never ships its decoder; the manifest +//! names *which* one, and MAY declare the artifact's wire form +//! ([`Prior::form`], `extensions/prior-carriage.md`) so the digest means the +//! same bytes to both ends. //! The `prior` key is additive and absent for every floor capability, so //! files written before it existed round-trip byte for byte. //! @@ -60,8 +63,34 @@ pub struct ModelRequirement { /// Pins *which prior, exactly* a neural capability decodes against, so two /// sinks that honor the same `determinism` contract reproduce the same -/// output bit for bit. Identity, not transport: the weights are never -/// carried — only named and verified. +/// output bit for bit. Identity, not transport: the envelope never carries +/// the weights — it names them, verifies them, and may declare their form. +/// +/// Build one with [`Prior::new`] and, to declare a form, [`Prior::with_form`]: +/// +/// ``` +/// use wai::container::{DeterminismClass, Prior}; +/// +/// let plain = Prior::new("int_hyper/q4", "ef".repeat(32), "int-rans/p16+int-synth"); +/// let nnc = plain.clone().with_form(wai::codecs::CAP_PRIOR_NNC); +/// assert_eq!(nnc.form(), Some("wai.prior.nnc")); +/// // The same integer-exact contract, but no form is yet established to +/// // reconstruct identical tensors at every reader. +/// assert_eq!(nnc.determinism_class(), DeterminismClass::IntegerExact); +/// assert!(!nnc.form_preserves_exactness()); +/// assert!(plain.is_cross_hardware_exact()); +/// assert!(!nnc.is_cross_hardware_exact()); +/// ``` +/// +/// **Source compatibility.** The fields are public and the struct is not +/// `#[non_exhaustive]`, so adding a field (as `form` was added, for +/// `extensions/prior-carriage.md`) breaks Rust code outside this crate that +/// builds a `Prior` with a struct literal naming every field, or destructures one +/// without `..`. [`Prior::new`] leaves every optional field absent, so code that +/// builds a `Prior` with it and the `with_*` builders does not name the optional +/// fields and is unaffected when another one is added. The wire format is a +/// separate matter: an optional field is skipped when absent, so a manifest +/// without it round-trips byte for byte. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct Prior { /// Stable identifier of the prior, e.g. `"bmshj2018-factorized/q3"`. @@ -75,8 +104,41 @@ pub struct Prior { /// made reproducible; absent it, a "replicate" claim is unfounded. Parse /// its strength with [`Prior::determinism_class`]. pub determinism: String, + /// The wire form of the artifact `sha256` covers, as a registered prior-form + /// capability ([`crate::codecs::prior_form::prior_forms`], e.g. + /// `"wai.prior.nnc"`). Present: `sha256` is over the artifact serialized in + /// exactly that form — the coded bitstream, never an unpacking of it. + /// Absent: the form is undeclared and the digest covers the artifact as the + /// sink registered it. Additive on the wire: files written before it existed + /// round-trip byte for byte. In Rust it is a new public field (see *Source + /// compatibility* above). See `extensions/prior-carriage.md`. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub form: Option, } +/// Why a [`Prior`]'s declared form cannot be used at this sink. Either way the +/// prior is refused: a sink never resolves a pinned prior against an artifact +/// in a form other than the one declared, even one whose unpacked parameters +/// compare equal. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum PriorFormError { + /// The declared form is not a registered prior form (a payload capability, + /// or a form from a registry this sink does not know). + NotAPriorForm(String), + /// A registered prior form this sink has no reader for. + Unreadable(String), +} + +impl std::fmt::Display for PriorFormError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::NotAPriorForm(form) => write!(f, "declared prior form '{form}' is not a registered prior form"), + Self::Unreadable(form) => write!(f, "declared prior form '{form}' cannot be read by this sink"), + } + } +} +impl std::error::Error for PriorFormError {} + /// How reproducible a [`Prior`]'s decode actually is, parsed from its /// `determinism` contract. This distinction is load-bearing for any normative /// `replicate` claim. Float inference through a runtime is not reproducible @@ -105,6 +167,25 @@ pub enum DeterminismClass { } impl Prior { + /// A prior pinned by `id`, the `sha256` of its artifact and its + /// `determinism` contract, with every optional field absent (no declared + /// form). It serializes exactly as a prior written before `form` existed. + pub fn new( + id: impl Into, + sha256: impl Into, + determinism: impl Into, + ) -> Self { + Self { id: id.into(), sha256: sha256.into(), determinism: determinism.into(), form: None } + } + + /// The same prior, declaring the wire form of the artifact its digest covers + /// ([`Prior::form`]). The form is not checked here; see [`Prior::check_form`] + /// and [`Prior::form_preserves_exactness`]. + pub fn with_form(mut self, form: impl Into) -> Self { + self.form = Some(form.into()); + self + } + /// Classify the `determinism` contract. Heuristic: any float token /// (`f32`/`f64`/`float`) anywhere in the pipeline downgrades to /// [`DeterminismClass::FloatReproducible`] even if the entropy stage is @@ -125,8 +206,119 @@ impl Prior { /// True iff this prior's decode is byte-identical across *arbitrary* /// hardware — the bar a normatively-`replicate` neural capability must /// meet. Single-threaded float is NOT enough. + /// + /// Two things must hold: the `determinism` contract is integer-exact + /// ([`DeterminismClass::IntegerExact`]), and the network reaches every sink as + /// the same tensors. The second fails when the artifact must be read out of a + /// declared form not established to reconstruct identical tensors at every + /// reader ([`Prior::form_preserves_exactness`]): an integer-exact network read + /// as different weights decodes differently. The form rule lives here, not + /// only in the functions that report why a `replicate` is refused, so every + /// `replicate` check that reads this predicate applies it. pub fn is_cross_hardware_exact(&self) -> bool { self.determinism_class() == DeterminismClass::IntegerExact + && self.form_preserves_exactness() + } + + /// The declared wire form of the pinned artifact, if any. + pub fn form(&self) -> Option<&str> { + self.form.as_deref() + } + + /// Whether reading the pinned artifact out of its declared form is + /// established to hand every conforming sink the same tensors. Reading an + /// artifact out of a coded form is itself a decode, so a network that is + /// integer-exact can still reach two sinks as different weights. An + /// undeclared form is the bytes the digest covers, so it passes; a declared + /// form passes only once the registry lists it as exact + /// ([`form_reconstruction_is_exact`](crate::codecs::prior_form::form_reconstruction_is_exact)), + /// which no form is yet. + pub fn form_preserves_exactness(&self) -> bool { + self.form().is_none_or(crate::codecs::prior_form::form_reconstruction_is_exact) + } + + /// Check the declared form before resolving the artifact: it must be a + /// registered prior form, and one of the forms this sink can read. An + /// undeclared form passes (the digest alone identifies the artifact). + pub fn check_form(&self, readable: &[&str]) -> Result<(), PriorFormError> { + match self.form() { + None => Ok(()), + Some(f) if !crate::codecs::prior_form::is_prior_form(f) => { + Err(PriorFormError::NotAPriorForm(f.to_string())) + } + Some(f) if !readable.contains(&f) => Err(PriorFormError::Unreadable(f.to_string())), + Some(_) => Ok(()), + } + } +} + +/// Why a `(intent, prior)` pair does not make a sound normative-`replicate` claim. +/// See [`replicate_prior_refusal`]; its `Display` is the reason a strict decoder +/// reports. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum PriorRefusal { + /// `replicate` with no pinned prior. + Absent, + /// The pinned prior's declared contract is not integer-exact. + NotIntegerExact(DeterminismClass), + /// The pinned prior's contract is integer-exact, but it declares a form + /// (named here) whose reconstruction is not established to give identical + /// tensors at every reader ([`Prior::form_preserves_exactness`]), or that is + /// not a registered prior form at all; `Display` says which. + FormNotExact(String), +} + +impl std::fmt::Display for PriorRefusal { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + const FLOAT: &str = "float decode is not byte-identical across hardware — tag this \ + perceptual/improve, or pin an integer-only prior"; + match self { + Self::Absent => write!(f, "no prior is pinned; {FLOAT}"), + Self::NotIntegerExact(class) => { + write!(f, "the prior is not integer-exact (determinism {class:?}); {FLOAT}") + } + Self::FormNotExact(form) if !crate::codecs::prior_form::is_prior_form(form) => write!( + f, + "the prior's contract is integer-exact, but its declared form '{form}' is not a \ + registered prior form, so nothing states what reading the artifact out of it \ + reconstructs — tag this perceptual/improve, or pin the artifact without a \ + declared form" + ), + Self::FormNotExact(form) => write!( + f, + "the prior's contract is integer-exact, but its declared form '{form}' is not \ + established to reconstruct identical tensors at every reader, so two sinks can \ + read different weights out of it — tag this perceptual/improve, or pin the \ + artifact without a declared form" + ), + } + } +} + +/// The prior half of the `replicate`-soundness rule, with its reason: `None` +/// exactly when [`replicate_is_sound`] holds for `(intent, prior)` (or `intent` is +/// not `replicate`), otherwise why not. `replicate` requires a pinned prior whose +/// decode is byte-identical across hardware ([`Prior::is_cross_hardware_exact`]): +/// its contract is integer-exact, and its declared form, if any, is established +/// to deliver the same tensors to every sink ([`Prior::form_preserves_exactness`]). +/// The contract is checked first, so a float contract is refused as float whatever +/// form it declares. A `replicate` intent backed by a float-reproducible (or +/// absent) prior is only "exact on this build" and should be tagged a perceptual / +/// `improve` tier instead — this is the WAI honesty rule the determinism work +/// exists to enforce. +pub fn replicate_prior_refusal(intent: &str, prior: Option<&Prior>) -> Option { + if intent != "replicate" { + return None; + } + match prior { + None => Some(PriorRefusal::Absent), + Some(p) if p.determinism_class() != DeterminismClass::IntegerExact => { + Some(PriorRefusal::NotIntegerExact(p.determinism_class())) + } + Some(p) if !p.form_preserves_exactness() => { + Some(PriorRefusal::FormNotExact(p.form().unwrap_or_default().to_string())) + } + Some(_) => None, } } @@ -330,6 +522,12 @@ impl Wai { } Ok(()) } + + /// Why this envelope's `replicate` claim is unsound, or `None` if + /// [`Self::replicate_is_sound`]. See [`replicate_prior_refusal`]. + pub fn replicate_prior_refusal(&self) -> Option { + replicate_prior_refusal(&self.manifest.intent, self.prior()) + } } // ---- v1.1: multi-rendition envelope (WAI2) ---------------------- @@ -851,6 +1049,7 @@ mod tests { id: "bmshj2018-factorized/q3".into(), sha256: "ab".repeat(32), determinism: "int-rans/p16+onnx-cpu-f32/1thread".into(), + form: None, }), }, target: serde_json::json!({"w": 256, "h": 256}), @@ -868,7 +1067,7 @@ mod tests { #[test] fn determinism_class_and_replicate_soundness() { let mk = |det: &str| Prior { - id: "m".into(), sha256: "ab".repeat(32), determinism: det.into(), + id: "m".into(), sha256: "ab".repeat(32), determinism: det.into(), form: None, }; // Integer rANS + FLOAT synthesis → only "exact here", not everywhere. let float_synth = mk("int-rans/p16+onnx-cpu-f32/1thread"); @@ -909,7 +1108,7 @@ mod tests { }, conditioning: Conditioning { kind: "k".into(), - prior: det.map(|d| Prior { id: "m".into(), sha256: "ab".repeat(32), determinism: d.into() }), + prior: det.map(|d| Prior { id: "m".into(), sha256: "ab".repeat(32), determinism: d.into(), form: None }), }, target: serde_json::json!({}), }, vec![]) @@ -935,6 +1134,140 @@ mod tests { assert!(env("wai.image.jpegai", "improve", None).replicate_is_sound()); } + const NNC: &str = crate::codecs::CAP_PRIOR_NNC; + + fn with_form(form: Option<&str>) -> Prior { + Prior { + id: "int_hyper/q4".into(), + sha256: "ef".repeat(32), + determinism: "int-rans/p16+int-synth".into(), + form: form.map(String::from), + } + } + + #[test] + fn prior_without_a_form_round_trips_byte_for_byte() { + // A manifest written before `form` existed parses, and re-serializes to + // the identical bytes: the field is absent, not `null`. + let legacy = r#"{"wai":"1.0","media":"image","intent":"replicate","model_requirement":{"capability":"wai.neural.int_hyper"},"conditioning":{"kind":"int_hyper","prior":{"id":"int_hyper/q4","sha256":"efef","determinism":"int-rans/p16+int-synth"}},"target":null}"#; + let m: Manifest = serde_json::from_str(legacy).unwrap(); + assert_eq!(m.conditioning.prior.as_ref().unwrap().form(), None); + assert_eq!(serde_json::to_string(&m).unwrap(), legacy); + } + + #[test] + fn constructor_builds_the_prior_a_literal_would() { + // `Prior::new` leaves the optional form absent, so it serializes exactly + // as a prior written before `form` existed; `with_form` declares one. + let p = Prior::new("int_hyper/q4", "ef".repeat(32), "int-rans/p16+int-synth"); + assert_eq!(p, with_form(None)); + assert_eq!( + serde_json::to_string(&p).unwrap(), + format!(r#"{{"id":"int_hyper/q4","sha256":"{}","determinism":"int-rans/p16+int-synth"}}"#, + "ef".repeat(32)) + ); + assert_eq!(p.clone().with_form(NNC), with_form(Some(NNC))); + assert_eq!(p.with_form(NNC).form(), Some(NNC)); + } + + #[test] + fn declared_form_round_trips_through_the_envelope() { + let mut w = Wai::new(Manifest { + wai: "1.0".into(), media: "image".into(), intent: "replicate".into(), + model_requirement: ModelRequirement { capability: "wai.neural.int_hyper".into(), + fallback: None, min_sim: None, sim_metric: None }, + conditioning: Conditioning { kind: "int_hyper".into(), prior: None }, + target: serde_json::Value::Null, + }, vec![1, 2, 3]); + w.manifest.conditioning.prior = Some(with_form(Some(NNC))); + let back = Wai::from_bytes(&w.to_bytes().unwrap()).unwrap(); + assert_eq!(back.prior(), Some(&with_form(Some(NNC)))); + assert_eq!(back.prior().unwrap().form(), Some(NNC)); + } + + #[test] + fn a_declared_form_must_be_registered_and_readable() { + // Undeclared: the digest alone identifies the artifact. + assert_eq!(with_form(None).check_form(&[]), Ok(())); + // Registered, but this sink has no reader for it: refused, never + // resolved against an artifact in some other form. + assert_eq!(with_form(Some(NNC)).check_form(&[]), Err(PriorFormError::Unreadable(NNC.into()))); + assert_eq!(with_form(Some(NNC)).check_form(&[NNC]), Ok(())); + // Not a prior form at all: a payload codec, or an unknown string. + for bad in ["wai.image.png", "wai.tensor.nnc", "wai.prior.nope"] { + assert_eq!( + with_form(Some(bad)).check_form(&[bad]), + Err(PriorFormError::NotAPriorForm(bad.into())), + "{bad}" + ); + } + } + + #[test] + fn replicate_cannot_rest_on_a_form_not_established_exact() { + // The same integer-exact network: sound when the artifact is the bytes + // the digest covers; not sound once it must be decoded out of a form + // whose reconstruction is not established exact at every reader. + let plain = with_form(None); + let nnc = with_form(Some(NNC)); + // One contract, integer-exact for both … + assert_eq!(plain.determinism_class(), DeterminismClass::IntegerExact); + assert_eq!(nnc.determinism_class(), DeterminismClass::IntegerExact); + assert!(plain.form_preserves_exactness()); + assert!(!nnc.form_preserves_exactness()); + // … but only the prior read as the bytes its digest covers is exact across + // hardware. The predicate every `replicate` check reads carries the form rule. + assert!(plain.is_cross_hardware_exact()); + assert!(!nnc.is_cross_hardware_exact()); + assert!(replicate_is_sound("replicate", Some(&plain))); + assert!(!replicate_is_sound("replicate", Some(&nnc))); + // Only `replicate` is gated. + assert!(replicate_is_sound("improve", Some(&nnc))); + } + + #[test] + fn replicate_refusal_names_its_reason() { + // The reason a strict decoder reports. The form refusal must not fall back + // to the float wording: that network's contract IS integer-exact. + let mk = |det: &str, form: Option<&str>| Prior { + id: "m".into(), sha256: "ab".repeat(32), determinism: det.into(), + form: form.map(String::from), + }; + let integer = mk("int-rans/p16+int-synth", None); + let floaty = mk("int-rans/p16+onnx-cpu-f32/1thread", None); + let integer_nnc = mk("int-rans/p16+int-synth", Some(NNC)); + + assert_eq!(replicate_prior_refusal("replicate", Some(&integer)), None); + assert_eq!(replicate_prior_refusal("improve", None), None); + assert_eq!(replicate_prior_refusal("improve", Some(&integer_nnc)), None); + assert_eq!(replicate_prior_refusal("replicate", None), Some(PriorRefusal::Absent)); + assert_eq!( + replicate_prior_refusal("replicate", Some(&floaty)), + Some(PriorRefusal::NotIntegerExact(DeterminismClass::FloatReproducible)) + ); + // A float contract is refused as float even when it also declares a form. + assert_eq!( + replicate_prior_refusal("replicate", Some(&mk("onnx-cpu-f32", Some(NNC)))), + Some(PriorRefusal::NotIntegerExact(DeterminismClass::FloatReproducible)) + ); + let why = replicate_prior_refusal("replicate", Some(&integer_nnc)); + assert_eq!(why, Some(PriorRefusal::FormNotExact(NNC.into()))); + let text = why.unwrap().to_string(); + assert!(text.contains(NNC) && text.contains("declared form"), "{text}"); + assert!(text.contains("not established"), "{text}"); + assert!(!text.contains("not integer-exact"), "{text}"); + // A form outside the registry is refused as such, not as a form whose + // reconstruction is merely unestablished. + let nope = replicate_prior_refusal("replicate", Some(&mk("int-rans/p16+int-synth", + Some("wai.prior.nope")))); + assert_eq!(nope, Some(PriorRefusal::FormNotExact("wai.prior.nope".into()))); + let text = nope.unwrap().to_string(); + assert!(text.contains("not a registered prior form"), "{text}"); + assert!(PriorRefusal::NotIntegerExact(DeterminismClass::Unknown) + .to_string().contains("not integer-exact")); + assert!(PriorRefusal::Absent.to_string().contains("no prior is pinned")); + } + // ---- v1.1: multi-rendition WAI2 ---- fn sample_multi() -> WaiMulti { @@ -946,7 +1279,8 @@ mod tests { payload: b"NEURAL-TOKENS-2317-BYTES".to_vec(), prior: Some(Prior { id: "encodec32/v1".into(), sha256: "cd".repeat(32), - determinism: "int-tokens".into() }), + determinism: "int-tokens".into(), + form: None }), role: None, companion_of: None }, Rendition { capability: "wai.audio.opus".into(), kind: "opus".into(), diff --git a/wai/wai-rs/src/neural/mod.rs b/wai/wai-rs/src/neural/mod.rs index b47dae1e5..d1ec2df1b 100644 --- a/wai/wai-rs/src/neural/mod.rs +++ b/wai/wai-rs/src/neural/mod.rs @@ -86,6 +86,11 @@ impl ModelRegistry { let path = self.paths.get(capability) .ok_or_else(|| DecodeError::ModelNotRegistered(capability.to_string()))?; if let Some(p) = prior { + // A declared form this sink cannot read is refused before anything is + // hashed: resolving it against the registered artifact would mean + // reading bytes in some other form than the one the pin names. + p.check_form(READABLE_PRIOR_FORMS) + .map_err(|e| DecodeError::PriorMismatch(format!("{capability}: {e}")))?; let got = file_sha256_hex(path)?; if !got.eq_ignore_ascii_case(&p.sha256) { return Err(DecodeError::PriorMismatch(format!( @@ -98,6 +103,11 @@ impl ModelRegistry { } } +/// The declared prior forms (`Prior::form`) this sink can read. None: the +/// registry loads each model artifact as registered and contains no reader for a +/// coded form such as `wai.prior.nnc`, so a prior that declares one is refused. +pub const READABLE_PRIOR_FORMS: &[&str] = &[]; + /// Lowercase-hex SHA-256 of a model artifact — compute the value to pin in a /// neural capability's [`Prior::sha256`](crate::container::Prior). Minting a /// neural envelope and verifying one at decode time use the same hash, so the @@ -292,7 +302,9 @@ fn try_jpegai_exact(payload: &[u8], dir: &Path) -> Option { /// /// 1. its capability's registry class is `NeuralIntegerExact` (integer end to end, /// entropy and synthesis), and -/// 2. it pins a prior that declares an integer-exact contract. +/// 2. it pins a prior that declares an integer-exact contract, and whose declared +/// form, if any, is established to reconstruct identical tensors at every reader +/// ([`crate::container::Prior::form_preserves_exactness`]; none is yet). /// /// The class is checked first and does not depend on anything the sender declares: /// a `replicate` envelope over `wai.image.jpegai` (`NeuralEntropyExact`: integer @@ -304,16 +316,23 @@ fn try_jpegai_exact(payload: &[u8], dir: &Path) -> Option { /// build and on the process's floating-point environment, which neither the /// envelope nor the registry can state. /// -/// Returns [`DecodeError::UnsoundReplicate`] *before* touching the model otherwise, -/// and [`DecodeError::UnknownCapability`] for an unregistered capability. -/// `decode_envelope` stays lenient (it decodes regardless, honest about being -/// "exact on this build"). +/// Returns [`DecodeError::UnsoundReplicate`], with the reason, *before* touching the +/// model otherwise, and [`DecodeError::UnknownCapability`] for an unregistered +/// capability. `decode_envelope` stays lenient (it decodes regardless, honest about +/// being "exact on this build"). pub fn decode_envelope_strict(env: &Wai, registry: &ModelRegistry) -> Result { use crate::codecs::ReplicateRefusal; let cap = &env.manifest.model_requirement.capability; match env.replicate_refusal() { None => decode_envelope(env, registry), Some(ReplicateRefusal::Unregistered) => Err(DecodeError::UnknownCapability(cap.clone())), + // The capability is integer-exact; the pinned prior is what fails. The reason + // (none pinned, a contract that is not integer-exact, or a declared form not + // established exact) comes from the library, whose unit tests run without + // this feature. + Some(ReplicateRefusal::Prior(why)) => Err(DecodeError::UnsoundReplicate(format!( + "intent=replicate over {cap} but {why}" + ))), Some(why) => Err(DecodeError::UnsoundReplicate(format!( "intent=replicate over {cap}: {why}. Over this capability `replicate` would \ promise the same reconstruction bytes on every conforming sink, which its \ diff --git a/wai/wai-rs/tests/neural_native_decode.rs b/wai/wai-rs/tests/neural_native_decode.rs index 50066605e..fede9ea4a 100644 --- a/wai/wai-rs/tests/neural_native_decode.rs +++ b/wai/wai-rs/tests/neural_native_decode.rs @@ -168,6 +168,7 @@ fn pinned_prior_is_enforced() { id: "bmshj2018-factorized/q3".into(), sha256: sha.into(), determinism: "int-rans/p16+onnx-cpu-f32/1thread".into(), + form: None, }); Wai::new(m, base.payload.clone()) }; @@ -205,16 +206,35 @@ fn resolve_checked_verifies_prior_hash() { "absent prior should resolve unverified"); // Correct pin → pass. - let good = Prior { id: "m/q3".into(), sha256: real.clone(), determinism: "x".into() }; + let good = Prior { id: "m/q3".into(), sha256: real.clone(), determinism: "x".into(), form: None }; assert!(reg.resolve_checked("wai.neural.bmshj2018", Some(&good)).is_ok(), "true hash should resolve"); // Wrong pin → refused, loudly. - let bad = Prior { id: "m/q3".into(), sha256: "00".repeat(32), determinism: "x".into() }; + let bad = Prior { id: "m/q3".into(), sha256: "00".repeat(32), determinism: "x".into(), form: None }; assert!(matches!(reg.resolve_checked("wai.neural.bmshj2018", Some(&bad)), Err(DecodeError::PriorMismatch(_))), "wrong hash must be a PriorMismatch"); + // True hash, but the pin declares a form this sink cannot read → refused: + // the registered bytes are not an artifact in the declared form. + let nnc = Prior { form: Some(wai::codecs::CAP_PRIOR_NNC.into()), ..good.clone() }; + match reg.resolve_checked("wai.neural.bmshj2018", Some(&nnc)) { + Err(DecodeError::PriorMismatch(why)) => assert!( + why.contains("wai.prior.nnc") && why.contains("cannot be read by this sink"), + "the refusal must name the unreadable form, not a hash mismatch: {why}" + ), + other => panic!("an unreadable declared form must be refused, got {other:?}"), + } + // A declared form that is not a registered prior form is refused too. + let png = Prior { form: Some("wai.image.png".into()), ..good.clone() }; + match reg.resolve_checked("wai.neural.bmshj2018", Some(&png)) { + Err(DecodeError::PriorMismatch(why)) => assert!( + why.contains("is not a registered prior form"), "{why}" + ), + other => panic!("a non-form declared as a form must be refused, got {other:?}"), + } + // Unknown capability → unregistered, regardless of prior. assert!(matches!(reg.resolve_checked("wai.neural.nope", None), Err(DecodeError::ModelNotRegistered(_)))); @@ -275,7 +295,46 @@ fn strict_decode_gates_unsound_replicate() { }; Wai::new(manifest, vec![]) } - let p = |det: &str| Prior { id: "m".into(), sha256: "ab".repeat(32), determinism: det.into() }; + let p = |det: &str| Prior { id: "m".into(), sha256: "ab".repeat(32), determinism: det.into(), form: None }; + // An integer-exact network read out of a form not established exact: refused + // before any model is touched, and the reason names the form. The envelope is + // built here, over an integer-exact capability, so neither the capability nor + // the contract can be what refuses it; the same prior without a form passes. + { + let over_int_synth = |prior: Prior| { + Wai::new( + Manifest { + wai: "1.0".into(), + media: "image".into(), + intent: "replicate".into(), + model_requirement: ModelRequirement { + capability: "wai.neural.int_synth".into(), + fallback: None, + min_sim: None, + sim_metric: None, + }, + conditioning: Conditioning { kind: "int_synth".into(), prior: Some(prior) }, + target: serde_json::json!({"w": 256, "h": 256}), + }, + vec![], + ) + }; + let exact = p("int-rans/p16+onnx-int8-integerized"); + let nnc = exact.clone().with_form(wai::codecs::CAP_PRIOR_NNC); + let reg = ModelRegistry::new(); + assert!(matches!( + decode_envelope_strict(&over_int_synth(exact), ®), + Err(DecodeError::ModelNotRegistered(_)) + )); + match decode_envelope_strict(&over_int_synth(nnc), ®) { + Err(DecodeError::UnsoundReplicate(why)) => { + assert!(why.contains("wai.prior.nnc"), "the refusal must name the form: {why}"); + assert!(!why.contains("not integer-exact"), "the contract is integer-exact: {why}"); + } + Err(e) => panic!("expected UnsoundReplicate, got {e:?}"), + Ok(_) => panic!("expected UnsoundReplicate, got a decode"), + } + } let integer = p("int-rans/p16+onnx-int8-integerized"); let floaty = p("int-rans/p16+onnx-cpu-f32/1thread"); const FLOAT_CODEC: &str = "wai.neural.bmshj2018";