wai/CHANGELOG.md

@@ -6,6 +6,32 @@ wire format may change and such changes are recorded here.
 
 ## v0.4.0 — current
 
+### Integer operators (Draft), and the payload text of two integer codecs
+
+- **A new extension, [integer-operators](extensions/integer-operators.md) (Draft).**
+  `wai-ops/1` is the integer operator set the integer-exact capabilities decode
+  through: each operator's exact arithmetic over the integers, with rounding, bias
+  placement and shift scope as parameters of requantisation; a range policy on every
+  tensor (`derived`, `clamp` or `check`); the `acc64` and `acc32` accumulator profiles;
+  static bounds rules, saturating at `±2^255` so that a 512-bit integer holds every
+  value they form, and a canonical bounds certificate that decides which profile a
+  parameter set meets; the accelerator subset `wai-ops/1-npu`; the `WHG1` graph form;
+  the stage operators of the drivers, among them the mANS decode of
+  `wai.image.jpegai` (E8) with its scale path (§13.6.1); and canonical lowerings of
+  the existing network forms (`int_synth.json`, `WHM1`, `WAM1`, `WMB1`) to `WHG1`.
+  The reference crate, the tool and the conformance corpus it names follow this
+  Draft; until they land, each capability's reference is the decoder its payload
+  extension names.
+- **[integer-payloads](extensions/integer-payloads.md) writes down two more formats.**
+  §13 is the `wai.neural.int_synth` payload and its set, `int_synth.json`; §14 is the
+  `WIM1` payload and the `WMB1` bundle of `wai.neural.int_mlicpp`, including that the
+  entropy network reads every element of `y` not yet decoded as 0, and that the three
+  networks and the tables share one `afrac` in `1..=30`. §9 gains the members and
+  checks of `entropy.json`, and states that the layers' arithmetic is
+  integer-operators §13.2 and §13.3. No decoder changes: §13 lists the
+  `wai.neural.int_synth` checks the reference decoder makes once it decodes through
+  its lowering.
+
 ### Staged measurement and the session claim in the browser
 
 - **The measurement core, bound for JS** ([staged-measurement](extensions/staged-measurement.md)).

wai/README.md

@@ -48,7 +48,7 @@ cargo check -p wai --no-default-features
 | [`wai-rs/`](wai-rs/) | Rust reference implementation (lib + cdylib + staticlib + C FFI) |
 | [`REGISTRY.md`](REGISTRY.md), [`registry/capabilities.json`](registry/capabilities.json) | The registry: every registered string with its kind, status, classes and edition, generated from the reference implementation (`wai registry export`), with its JSON Schema and the policy for registering a string |
 | [`vocab/`](vocab/) | RDF vocabulary: v2 generated from the registry, v1 frozen |
-| [`extensions/`](extensions/) | Sub-specifications (confidentiality, receipts, worlds, film, quantum operations, …) |
+| [`extensions/`](extensions/) | Sub-specifications (confidentiality, receipts, worlds, film, quantum operations, integer operators, …) |
 | [`research/`](research/) | Research notes and dated records: graded, not normative |
 | [`glossary/`](glossary/) | Reference catalog of media codecs surveyed for capability registration. [`codecs.json`](glossary/codecs.json) is the dataset, linked to the registry; [`SCHEMA.md`](glossary/SCHEMA.md) defines the shape and contribution rules. Rendered at [wai.transaction.science/glossary](https://wai.transaction.science/glossary). |
 | [`CONTRIBUTING.md`](CONTRIBUTING.md) | How to change the specification, register a capability and add tests CI runs |
@@ -114,7 +114,11 @@ hash: a deployer can require `intent=replicate` to dispatch only to these.
 Their payloads are binary formats SPEC §5 pins — `WIV1` for the integer video
 codecs, `WIH1`, `WIA1` and `WIS1`
 ([`extensions/integer-payloads.md`](extensions/integer-payloads.md)). The JSON the
-encode and export tools write is converted to them, never carried.
+encode and export tools write is converted to them, never carried. The same extension
+writes down the `wai.neural.int_synth` payload and the `WIM1` payload of
+`wai.neural.int_mlicpp`. Their networks' integer arithmetic is written down as the
+operator set `wai-ops/1`
+([`extensions/integer-operators.md`](extensions/integer-operators.md), Draft).
 
 ## Interactive & temporal content (extensions)
 

wai/extensions/integer-payloads.md

@@ -7,7 +7,9 @@
 > `wai.splat.int_codec`. Defines each layout, its decode, the checks a sink makes and
 > the closed list of refusal codes; names each capability's parameter-set members;
 > and makes the JSON the encode and export tools write a converter input, never a
-> payload. Reference impl: `wai-rs` — `int_payload::{wire, decode_int, resolve}`,
+> payload. Writes down the payload formats and parameter sets that SPEC §5 keeps for
+> `wai.neural.int_synth` (§13) and `wai.neural.int_mlicpp` (`WIM1` and `WMB1`, §14).
+> Reference impl: `wai-rs` — `int_payload::{wire, decode_int, resolve}`,
 > `int_payload::{wiv1_from_json, wih1_from_json, wia1_from_json, wis1_from_json}`,
 > `codecs::PINNED_INT_PAYLOADS`, `container::check_int_payload_framing`.
 > Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
@@ -37,10 +39,10 @@ these strings from this revision on. The encode and export tools still write JSO
 that carries the same fields; it is a converter input (§10).
 
 `wai.neural.int_synth` and `wai.neural.int_mlicpp` keep the payload formats SPEC §5
-gives them. A sink decodes them and the five strings above (`wai.video.int_motion`,
-`wai.video.int_hyper`, `wai.neural.int_hyper`, `wai.audio.int_codec`,
-`wai.splat.int_codec`) through one path (`int_payload::decode_int`), and §8 names the
-codes it reports for those two.
+gives them, and §13 and §14 write those formats down. A sink decodes them and the five
+strings above (`wai.video.int_motion`, `wai.video.int_hyper`, `wai.neural.int_hyper`,
+`wai.audio.int_codec`, `wai.splat.int_codec`) through one path
+(`int_payload::decode_int`), and §8 names the codes it reports for those two.
 
 ## 2. Common rules
 
@@ -478,8 +480,32 @@ biases as `i64`), 2 an inverse GDN (channels as `u32`; `gamma` as `i32`, `beta`
 one-dimensional transposed convolution, and tag 2, an inverse GDN. In both forms
 nothing follows the last layer: a file with bytes after it is refused
 (`model_mismatch`), so one network has one file and one digest. The layers' integer
-arithmetic is the reference decoders' (`int_hyper_synth.rs`, `int_audio.rs`,
-`int_transform.rs`), which the conformance vectors pin.
+arithmetic is [integer-operators](integer-operators.md) §13.2 (`WHM1`) and §13.3
+(`WAM1`).
+
+**`entropy.json`**, the tables of `wai.neural.int_hyper` and `wai.video.int_hyper`: a
+JSON object with these members. Other members are ignored. An integer is a JSON number
+with no fraction and no exponent.
+
+| member | value |
+|---|---|
+| `scale_edges` | `N ≥ 1` integers, each fitting `i64`: the edges `e_0 … e_{N−1}` of the `ceil` bucket rule ([integer-operators](integer-operators.md) E3) |
+| `cdfs` | `N` arrays of integers, each fitting `i32`: the latent tables, one per bucket |
+| `cdf_lengths` | `N` non-negative integers: table `b`'s length `n_b` |
+| `offsets` | `N` integers, each fitting `i32`: table `b`'s offset |
+| `z_cdfs` | `K ≥ 1` arrays of integers, each fitting `i32`: the hyper-latent tables, one per channel |
+| `z_cdf_lengths` | `K` non-negative integers |
+| `z_offsets` | `K` integers, each fitting `i32` |
+| `z_median_q` | `K` integers, each fitting `i64`: channel `k`'s median at `2^afrac` |
+
+Each table, over its first `n_b` entries, is a table of §2: `3 ≤ n_b ≤ 4097`, `n_b` no
+more than its array's length, `cdf[0] = 0`, `cdf[n_b − 1] = 65536`, strictly
+increasing. Entries after `n_b` are not read. A set whose tables break any of this is
+refused (`model_mismatch`). A payload has no more hyper-latent channels than `K` (§7).
+Hyper-latent channel `k` decodes against `z_cdfs[k]`, `z_cdf_lengths[k]` and
+`z_offsets[k]`, and its elements are `s·2^afrac + z_median_q[k]` (§4 step 1). A latent
+element decodes against table `b`, the `ceil` bucket of its `σ` over `scale_edges`
+(§4 step 3).
 
 ## 10. The tools' JSON forms
 
@@ -557,3 +583,103 @@ was made.
   decodes the `WIV1` keyframe-0 payloads in full; and the cross-architecture goldens of
   `byte-exact-conformance`, which parse and decode the `WIV1` and `WIH1` payloads on
   every platform of its matrix.
+
+## 13. The `wai.neural.int_synth` payload and set
+
+SPEC §5 keeps this capability's payload format; this section writes it down.
+
+| off | size | field | rule |
+|---|---|---|---|
+| 0 | 4 | `H` | ≥ 1: the image's height |
+| 4 | 4 | `W` | ≥ 1: the image's width |
+| 8 | 2 | `C` | ≥ 1: latent channels |
+| 10 | 1 | `S_log2` | the latent's side is `S = 2^S_log2` |
+| 11 | 1 | reserved | `0` |
+| 12 | `C·S·S` | the latents | one signed 8-bit value each, `[C, S, S]` in CHW order |
+
+The format has no magic and no version. The payload is exactly `12 + C·S·S` bytes, and
+`C·S·S ≤ 2^26`.
+
+The set is one file, `int_synth.json`: a JSON object whose member `layers` is an array
+of layer objects. A layer has `in_ch`, `out_ch`, `k`, `stride` and `pad`, each a
+non-negative integer; `out_pad`, a non-negative integer, 0 when absent; `weight`,
+`in_ch·out_ch·k·k` integers each fitting `i32`, in the layout `[in_ch][out_ch][k][k]`;
+`bias`, `out_ch` integers each fitting `i32`; `shift`, in `0..=127`, 0 when absent; and
+`relu`, a boolean, false when absent. Other members are ignored.
+
+The decode is [integer-operators](integer-operators.md) §13.1's graph on the latents.
+Its output must be `[3, H, W]`, and it is presented as row-major HWC bytes. Every
+refusal of the payload, the set or the decode is `payload_malformed` (§8).
+
+**Checks that wait for the lowering.** The reference decoder
+(`int_synth::decode_bytes`) makes every check of this section except these, which it
+makes once the capability decodes through its lowering
+([integer-operators](integer-operators.md) §15.3): a reserved byte of 0; `C ≥ 1`; the
+caps; `k ≥ 1`, `stride ≥ 1` and a shift of at most 127; and the certificate. Until
+then it decodes a payload or set that fails one of them where it can, and with a shift
+from 64 to 127 its result depends on how it was built.
+
+## 14. `WIM1` and `WMB1` — `wai.neural.int_mlicpp`
+
+SPEC §5 keeps this capability's payload format and its one-file parameter set, the
+`WMB1` bundle (read from a set as `model.wmb`); this section writes both down.
+
+| off | size | field | rule |
+|---|---|---|---|
+| 0 | 4 | magic `WIM1` | |
+| 4 | 4 | `H` | ≥ 1 |
+| 8 | 4 | `W` | ≥ 1 |
+| 12 | 2 | `Cz` | ≥ 1: hyper-latent channels |
+| 14 | 2 | `Zh` | ≥ 1 |
+| 16 | 2 | `Zw` | ≥ 1 |
+| 18 | 2 | `Cy` | ≥ 1: latent channels, a multiple of `K` |
+| 20 | 2 | `Yh` | ≥ 1 |
+| 22 | 2 | `Yw` | ≥ 1 |
+| 24 | 2 | `K` | ≥ 1: channel slices |
+| 26 | 2 | reserved | `0` |
+| 28 | 4 | `Lz` | the `z` stream's length in bytes |
+| 32 | 4 | `Ly` | the `y` stream's length in bytes |
+| 36 | `Lz` | the `z` stream | §2, with no length prefix |
+| 36 + `Lz` | `Ly` | the `y` stream | §2, with no length prefix |
+
+A parser checks, in order: at least 36 bytes; the magic; the reserved field; no
+dimension 0; `Cy` a multiple of `K`; `Cz·Zh·Zw`, `3·Cy·Yh·Yw` and `3·H·W` each at most
+`2^26`; and the payload exactly `36 + Lz + Ly` bytes. Each stream's length is checked (a
+multiple of 4, at least 8) when the decode reaches it.
+
+The set is one file, a `WMB1` bundle: the magic `WMB1`, then four sections, each a
+`u32` length followed by that many bytes, in the order `h_s`, `entropy`, `g_s` (each a
+`WHM1` network, §9) and the tables. The tables are a JSON object with the members of
+`entropy.json` (§9), and `afrac`. Nothing follows the fourth section. The bundle's
+rules: the three networks' `afrac` are equal, lie in `1..=30`, and equal the tables'
+`afrac`; `1 ≤ N ≤ 65536`; and every table, over its length, follows
+[integer-operators](integer-operators.md) E1's `t2` rule (`2 ≤ n_b`, `n_b` no more than
+its array's length, `cdf[0] = 0`, `cdf[n_b − 1] = 65536`, strictly increasing), in
+place of §2's rule.
+
+The decode:
+
+1. **Plan**, before any stream: `Cz` is at most the number of `z` tables; `h_s` maps
+   `[Cz, Zh, Zw]` to `[2·Cy, Yh, Yw]`; `entropy` maps `[3·Cy, Yh, Yw]` to
+   `[2·Cy, Yh, Yw]`; `g_s` maps `[Cy, Yh, Yw]` to `[3, H, W]`; and no tensor any of them
+   forms exceeds `2^26` elements.
+2. **`z`.** For each channel `k`, and each of its `Zh·Zw` elements in raster order,
+   decode a symbol `s` against `z` table `k`, and the element is
+   `dequantize(s, afrac, z_median_q[k])` (integer-operators E2). After the last `z`
+   symbol, the stream must not have run out.
+3. `hyper = h_s(z)`.
+4. **`y`.** Let `y` be `[Cy, Yh, Yw]`, all 0. For each slice `j = 0 … K − 1`, and for
+   each pass (the anchors, `(row + col)` even, then the rest):
+   `σ‖μ = entropy(concat(y, hyper))`, where every element of `y` not yet decoded is
+   still 0. Then, for each channel of the slice (`j·Cy/K` to `(j + 1)·Cy/K − 1`), and
+   each of its positions in raster order that belongs to the pass: `b` is the `ceil`
+   bucket of `σ` over `scale_edges` (integer-operators E3); decode `r` against table
+   `b`; the element is `dequantize(r, afrac, μ)`. One `y` stream carries every slice's
+   two passes, in that order. After the last `y` symbol, the stream must not have run
+   out.
+5. `g_s(y)`, then the RGB8 output step (integer-operators §13.0), as row-major HWC
+   bytes.
+
+The streams decode with `t2`, `lenient` and `final_only`
+([integer-operators](integer-operators.md) E1). Every refusal of the payload, the
+bundle or the decode is `payload_malformed` (§8).