# WAI Extension: Integer Neural Fields > Status: Draft. Registers `wai.field.int_grid`, a computed capability: a volumetric scene held > as a **neural field**, a multi-resolution integer feature grid and a small integer network, > carried together in one payload, `WIG1` (§3). The sink computes the content itself. It decodes > the field to its values at any point of its domain, a density and a colour (§5), and renders it > at any view by integer ray marching (§6); the payload names the view it is presented at (§4). > Every value is an integer and every rounding is stated: a level's table is read through a > direct index or a spatial hash with a stated wrap, the interpolation weights are fixed point, > the network's layers round half up and clamp, and the ray march's samples, its stop and its > compositing are pinned. So the values at a point and the render at a view are the same bytes on > every machine (**field-equivalence**, §9), and a digest of a render can be signed and checked > where it is presented. A payload carries the coarsest levels of its grid and declares the > rest; each further level is a refinement, `wai.field.int_refine` (`WIL1`, §7), delivered as > its own object under [staged delivery](staged-delivery.md) in the level form, and a level the > sink does not hold reads as zero, so the coarse field is presented first and every stage is > presentable. The reference implementation is the `int_field` module of `wai-rs` (feature `field_int`); the > encoder is `tools/wai_int_field_export.py`; the corpus is > [`volumetric-conformance/fields/`](../volumetric-conformance/fields/), and its second > implementation is `volumetric-conformance/field_verify.py`. > Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174. ## 1. Scope A **field** maps a point of the unit cube `[0, 1]³`, its **domain**, to a density and a colour. A `wai.field.int_grid` payload holds: - `L` grid **levels**, from coarse to fine. Level `l` divides each axis of the domain into `R_l` cells, so it has `(R_l + 1)³` vertices, and holds a **table** of entries, each `F` integer features. A level whose vertices fit its table gives each vertex its own entry; a finer one shares entries among vertices through a spatial hash (§5.2); - a **network** of `N` integer layers that reads the `L · F` features interpolated at a point and gives four values: the density and the three colour channels (§5.4); - a **view** (§4): a camera, the frame's size, the samples a ray takes and a background colour. A payload carries the coarsest `K` levels. Each of the others is a payload of its own, a `WIL1` of `wai.field.int_refine` (§7.2), which refines the field by one level. A sink decodes the payload once and then evaluates the field: at points (§5), and as a frame at the payload's view or at any other view (§6). The field is the content: the payload carries no pixels, and every frame is computed by the sink. A field decodes against no parameter set (SPEC §3.1): everything its decode reads is in its payload. The arithmetic is that of [integer-operators](integer-operators.md) §2, and each layer of the network is that extension's `conv` (§5.2 there) with a 1 × 1 kernel, followed by its requantisation `rq` (§5.1 there) and `relu` or a `clamp` (§5.5 there), with the settings §5.4 below states. The camera is [splat-exact](splat-exact.md) §5.3's, from a pose of that extension's §2 members. The payload's framing, check order and refusal codes are [integer-payloads](integer-payloads.md) §2's and §8's. **Out of scope.** How the field was made (§10 describes one encoder, informatively); a colour that depends on the direction a point is seen from; a field whose domain is not the unit cube (a sink places the cube in its scene by its own transform, outside the decode); and how a sink presents a frame after it is computed. ## 2. Integer functions `⌊a / b⌋`, `R_half_up(p, s)`, `isqrt`, `clamp` and `exp_neg_q16` are integer-operators §2's; `rdiv(a, b) = ⌊(2a + b) / 2b⌋` for `b > 0` is splat-exact §5.1's. `a >> s` is `⌊a / 2^s⌋`. `·` and `×` are exact products, of integers or, between vectors, the dot and cross products. No formula wraps or saturates unless it says so. A value "at 2^k" is the integer `v` that stands for `v / 2^k`; a position in the domain is at 2^16, so the domain is `[0, 2^16]³`. ## 3. The field payload `WIG1` ### 3.1 Layout Every multi-byte field is little-endian. | off | size | field | rule | |---|---|---|---| | 0 | 4 | magic | `WIG1` | | 4 | 1 | version | 1 | | 5 | 1 | `L`, the levels the field declares | 1–16 | | 6 | 1 | `K`, the levels the payload carries | 1–`L` | | 7 | 1 | `F`, the features of an entry | 1–8 | | 8 | 1 | `T`, the log2 of a level's table size | 4–19 | | 9 | 1 | `ffrac`, the fraction bits of a feature and of an activation | 8–16 | | 10 | 1 | `wfrac`, the fraction bits of a weight | 0–15 | | 11 | 1 | `N`, the network's layers | 1–8 | | 12 | 4 | reserved | 0 | | 16 | 2·`L` | `R_0` … `R_(L−1)`, each level's cells an axis, `u16` | each at least 1; strictly increasing | | 16 + 2·`L` | 2·(`N` + 1) | the widths `w_0` … `w_N`, `u16` | each 1–256; `w_0 = L · F`; `w_N = 4` | | | 56 | the view (§4) | §4 | | | | the network: for each layer `j` from 0 to `N − 1`, its weights, `w_(j+1)` rows of `w_j` `i16` at 2^wfrac, row by row; then its biases, `w_(j+1)` `i32` at 2^(ffrac + wfrac) | | | | | the levels carried: for each level `l` from 0 to `K − 1`, its table, `E_l` entries of `F` `i16` features at 2^ffrac, an entry's features together, entry by entry | | A level's table has `E_l = min((R_l + 1)³, 2^T)` entries. The levels `K` to `L − 1` are declared and not carried: each is a `WIL1` of its own (§7). ### 3.2 Rules The rules of the header's fields are the table's, read in this order: `L`, `K`, `F`, `T`, `ffrac`, `wfrac`, `N`; then each side `R_l` in order, at least 1 and greater than the one before; then the widths: each in 1–256, then `w_0 = L · F` (the network reads every level's features, so a field whose network reads another number of levels is refused), then `w_N = 4`; then the view (§4). A field that breaks one is refused (`field_range`). The weights, biases and features take any value of their type. ### 3.3 Caps and declared cost The network's multiply-accumulates a point, `Σ_j w_j · w_(j+1)`, are at most 2^16; a payload past it is refused (`too_large`). A level's table holds at most `2^19 · 8` features by the rules of §3.2. The header declares the work of a render at a view of `W × H` pixels and `S` samples a ray: at most `W · H · S` evaluations of the field, each `8 · L · F` features read and `Σ_j w_j · w_(j+1)` multiply-accumulates, and `3 · W · H` bytes out. A sink MAY apply a limit to it and refuse a payload past the limit before it renders (`cost_over_limit`, which routes as integer-payloads §8 says). ## 4. The view A view is 56 bytes: | off | size | field | rule | |---|---|---|---| | 0 | 12 | `eye`, three `i32` at 2^16 | each magnitude at most 2^24 | | 12 | 12 | `target`, three `i32` at 2^16 | each magnitude at most 2^24 | | 24 | 12 | `up`, three `i32` at 2^16 | each magnitude at most 2^24 | | 36 | 2 | `focal`, the focal length in pixels, `u16` | 1–4096 | | 38 | 2 | `W`, the frame's width in pixels, `u16` | 1–1024 | | 40 | 2 | `H`, the frame's height in pixels, `u16` | 1–1024 | | 42 | 2 | `S`, the samples a ray takes, `u16` | 1–1024 | | 44 | 4 | `t_near`, `i32` at 2^16 | 0–2^22 | | 48 | 4 | `t_far`, `i32` at 2^16 | greater than `t_near`; at most 2^22 | | 52 | 3 | `bg`, the background, RGB8 | any | | 55 | 1 | reserved | 0 | `eye`, `target`, `up`, `focal`, `W` and `H` are a pose of splat-exact §2, in the field's domain. The rules are read in the table's order (`eye`, `target` and `up` component by component), and then the pose MUST NOT be degenerate: `n_f` and `n_r` of §6.1 are not 0. A view that breaks one is refused (`field_range`). The payload's view is the one the field is presented at. A sink MAY render the field at any other view that meets these rules (an interactive one, say); it is computed by §6 as the payload's is, so every conforming sink computes the same frame there too. ## 5. Values at a point ### 5.1 The domain A point `p = (p_x, p_y, p_z)` is at 2^16. A point with a coordinate outside `[0, 2^16]` is **empty**: its values are `(0, 0, 0, 0)`, and nothing below is computed for it. The faces of the domain are inside it. ### 5.2 The table index At level `l`, with `R = R_l`, vertex `(x, y, z)` (each in `0..=R`) reads one entry of the level's table: - if `(R + 1)³ ≤ 2^T`, entry `x + (R + 1) · (y + (R + 1) · z)`: every vertex has its own; - otherwise entry `h mod 2^T`, where ``` h = (x · 1 mod 2^32) XOR (y · 2654435761 mod 2^32) XOR (z · 805459861 mod 2^32) ``` each product taken modulo 2^32 before the exclusive-or (the **wrap**): vertices share entries, and an encoder fits the shared entries to every vertex that reads them. ### 5.3 Interpolation For each level `l` the sink holds (§7), with `R = R_l`, and for each axis `a` of `x`, `y`, `z`: ``` s_a = p_a · R the position in cells, at 2^16 c_a = min(⌊s_a / 2^16⌋, R − 1) the cell w_a = s_a − 2^16 · c_a the fraction, in [0, 2^16] ``` The cell's eight corners are `(c_x + d_x, c_y + d_y, c_z + d_z)` for `d` in `{0, 1}³`, and corner `d`'s **weight** is ``` ω_d = Π_a (d_a = 1 ? w_a : 2^16 − w_a) at 2^48 ``` The weights are non-negative and sum to 2^48 exactly. Feature `c` of the level at `p` is ``` x_(l·F + c) = R_half_up(Σ_d ω_d · G_l[e_d][c], 48) at 2^ffrac ``` where `e_d` is the entry corner `d` reads (§5.2) and `G_l[e][c]` feature `c` of entry `e`. A level the sink does not hold gives `F` zeros. The network's input is the `L · F` values `x_0 … x_(L·F − 1)`, level by level. ### 5.4 The network Layer `j` reads `a_j` (`a_0` is the input of §5.3) and gives, for each output `o`: ``` acc = B_j[o] + Σ_i W_j[o][i] · a_j[i] y = clamp(R_half_up(acc, wfrac), −2^31, 2^31 − 1) a_(j+1)[o] = max(y, 0) for every layer but the last a_N[o] = y for the last ``` So every activation is at 2^ffrac, and the only rounding a layer makes is `R_half_up` by `wfrac` bits: a tie rounds toward +∞, whatever the sign. In the terms of integer-operators: each layer is `conv` with a 1 × 1 kernel over `[w_j, 1, 1]`; its `rq` has no multiplier, one shift `wfrac`, a `pre` bias `B_j` and the rounding `half_up`; its output tensor has the policy `clamp` to `[−2^31, 2^31 − 1]`; and each layer but the last is followed by `relu`. ### 5.5 The values With `y = a_N` and `sh = 16 − ffrac`, the values at `p` are, each at 2^16: ``` density σ = min(max(y_0, 0) · 2^sh, 2^24) colour c_k = clamp(y_k, 0, 2^ffrac) · 2^sh k = 1, 2, 3: red, green, blue ``` A density is an absorption per unit length of the domain: one at 2^16 absorbs a fraction `1 − e^(−1)` of light over the domain's side. A colour is in `[0, 2^16]`. ### 5.6 Widths Every weight `ω_d` is at most 2^48 and every feature fits `i16`, and the weights sum to 2^48, so every partial sum of §5.3 lies in `[−2^63, (2^15 − 1) · 2^48]` and fits `i64` in any order. In §5.4 an input fits `i32`, a weight `i16` and a bias `i32`, with at most 256 terms, so every partial sum fits in 55 bits plus a sign. An implementation forms every value of §5 in 64 bits and none of them wraps. ## 6. The render The render at a view is `W × H` pixels of RGB8, rows top to bottom and pixels left to right, three bytes a pixel and no padding: the canonical buffer of [reconstruction-binding](reconstruction-binding.md) §2. ### 6.1 The camera From `eye` **E**, `target` and `up` **U**, as splat-exact §5.3 forms it: ``` F = target − E n_f = isqrt(F·F) f_k = rdiv(F_k · 2^14, n_f) the forward row at 2^14 Rv = F × U n_r = isqrt(Rv·Rv) r_k = rdiv(Rv_k · 2^14, n_r) the right row at 2^14 u_k = R_half_up((r × f)_k, 14) the up row at 2^14 ``` ### 6.2 A pixel's ray The pixel in column `i` and row `j` has its centre at `(256·i + 128, 256·j + 128)` at 2^8, as in splat-exact §5.4. Its ray leaves **E** in the direction ``` dx = 256·i + 128 − 128·W dy = 128·H − 256·j − 128 D_k = 256 · focal · f_k + dx · r_k + dy · u_k n_d = isqrt(D·D) dir_k = rdiv(D_k · 2^16, n_d) at 2^16, of length 2^16 to rounding ``` A pixel with `n_d = 0` takes no sample and shows the background (§6.4). ### 6.3 The samples The march from `t_near` to `t_far` is cut into `S` pieces with the edges ``` e_k = t_near + ⌊k · (t_far − t_near) / S⌋ k = 0 … S ``` Sample `k` (for `k` from 0 to `S − 1`, in order) has the length `δ_k = e_(k+1) − e_k`, so the lengths sum to `t_far − t_near` exactly, and its point is at `m_k = e_k + ⌊δ_k / 2⌋` along the ray: ``` p_c = E_c + R_half_up(dir_c · m_k, 16) c = x, y, z ``` A sample of length 0 changes nothing. ### 6.4 Compositing Each pixel starts with transmittance `T = 2^16` and accumulators `A_r = A_g = A_b = 0`, and takes its samples in order. Before each, if `T < 7` (0.0001 at 2^16), the ray is **done** and takes no further sample. Otherwise, with `(σ, c_r, c_g, c_b)` the values at the sample's point (§5): ``` τ = R_half_up(σ · δ_k, 16) the optical depth, at 2^16 α = 2^16 − min(2^16, exp_neg_q16(−τ)) the fraction absorbed, at 2^16 c = R_half_up(α · T, 16) T = T − c A_k = A_k + R_half_up(c_k · c, 16) k = r, g, b ``` `exp_neg_q16(0)` is 65553, past 2^16, so a sample of no density absorbs nothing. `c` never exceeds `T`, so `T` stays in `[0, 2^16]`, and `A_k ≤ 2^16 − T`. The pixel's bytes are ``` min(255, R_half_up(255 · A_k + T · bg_k, 16)) k = r, g, b ``` ### 6.5 Widths `|E_c|` is at most 2^24, so `F` and **U** fit 26 bits, `Rv` fits 52 and `Rv·Rv` fits 106: the camera is formed in 128-bit arithmetic, once a view. `D` fits 37 bits and `D·D` 76, so a ray's direction is formed in 128 bits too. Everything a sample forms fits in 64 bits: `|dir_c · m_k|` is below 2^39, `σ · δ_k` at most 2^46, and `α · T`, `c_k · c` and `255 · A_k + T · bg_k` below 2^33. ## 7. Levels as refinements ### 7.1 A level not held A payload carries levels 0 to `K − 1` and declares levels `K` to `L − 1`. A sink holds the levels the payload carries and the levels it has applied (§7.3), always a prefix `0` to `h − 1` of the field's levels. A level it does not hold gives zero features at every point (§5.3), so the field is defined, and presentable, whatever prefix it holds. The network reads every level, so an encoder that sends a coarse prefix fits the coarse levels to be presentable alone and each finer level to what the levels below it leave. ### 7.2 The level payload `WIL1` The payload of `wai.field.int_refine`: one level of a field. | off | size | field | rule | |---|---|---|---| | 0 | 4 | magic | `WIL1` | | 4 | 1 | version | 1 | | 5 | 1 | `l`, the level it carries | 1–15 | | 6 | 1 | `F`, the features of an entry | 1–8 | | 7 | 1 | `T`, the log2 of the level's table size | 4–19 | | 8 | 2 | `R`, the level's cells an axis, `u16` | at least 1 | | 10 | 2 | reserved | 0 | | 12 | | the level's table: `min((R + 1)³, 2^T)` entries of `F` `i16` features at 2^ffrac, an entry's features together, entry by entry | | Level 0 is in every `WIG1` (`K ≥ 1`), so a `WIL1` carries level 1 or later. Its rules are read in the table's order (`field_range`). ### 7.3 Applying a level A level applies to a field only as its **next** level: with the field holding levels 0 to `h − 1` (`h < L`), the level's `l` is `h`, its `F` and `T` are the field's, and its `R` is `R_h`, the side the field declares for level `h`. Otherwise it is refused (`geometry_mismatch`) and the field is as it was: a level never applies out of order, twice, past `L`, or in another shape than the one the field declared. Applied, it is the field's level `h`, and the field holds levels 0 to `h`. ### 7.4 Levels as layers `wai.field.int_refine` is refinement-only (SPEC §5, *Refinement capabilities*), of the **level form** of [staged-delivery](staged-delivery.md) §6.2, with the one admitted base `wai.field.int_grid` (SPEC §5, *Base admission*). A sink that holds a field base processes each level layer by staged-delivery §4.1: - the base's output has one **unit**, the whole field, so a layer names no span, and its `prev` is one producer; - a field whose payload carries `K` levels is at stage 0; layer `k` brings it to stage `k`, and carries level `K + k − 1`; - step 6's parse is §7.2's (`payload-malformed`, with the payload's code), and a level is one unit; - step 9 is §7.3: a level that is not the field's next level fails it (`geometry-mismatch`); - the next stage is the field with that level applied. A unit's output, for the stage hash of staged-delivery §5, is the render at the field's own view (§6), and its stage hash is that frame's SHA-256. The base step is `decode-equivalence` (`wai.field.int_grid` is `StandardDefined`), and so is each layer step: a stage's values and frames are exact given the levels it holds. Every layer boundary is a truncation point. A level reads no parameter set: everything it reads is in its payload and in the field below. ## 8. Order of the checks, and refusal codes A parser of `WIG1` checks, in this order, and refuses with the first that fails (integer-payloads §2): 1. the magic: `WIG1`, or else `format_capability_mismatch` if it is `WIL1`, or `bad_magic`; 2. the version byte: present (`stream_length`), and 1 (`version`); 3. the fixed header, 16 bytes, present; then the header through the view, `74 + 2·L + 2·N` bytes, present (`stream_length`); 4. the reserved fields: the header's, at 12, and the view's last byte (`reserved_nonzero`); 5. the header's fields in the order of §3.2, the view's in the order of §4 (`field_range`); 6. the cap of §3.3 (`too_large`); 7. the network, layer by layer, its weights then its biases, then the levels carried, level by level, each present (`stream_length`); 8. nothing after the last level (`trailing`). A parser of `WIL1` checks, in this order: 1. the magic: `WIL1`, or else `format_capability_mismatch` if it is `WIG1`, or `bad_magic`; 2. the version byte: present (`stream_length`), and 1 (`version`); 3. the header, 12 bytes, present (`stream_length`); 4. the reserved field at 10 (`reserved_nonzero`); 5. the header's fields in the order of §7.2 (`field_range`); 6. the table present (`stream_length`); 7. nothing after it (`trailing`). Applying a level then checks §7.3 (`geometry_mismatch`). The codes are integer-payloads §8's: | code | when | |---|---| | `bad_magic` | the payload does not start with `WIG1` or `WIL1` | | `format_capability_mismatch` | a `WIL1` where a `WIG1` is read, or a `WIG1` where a `WIL1` is | | `version` | a version this revision does not define | | `stream_length` | the payload ends before a field or section it declares | | `reserved_nonzero` | a reserved field that is not zero | | `field_range` | a field of the header or the view outside its rule, or a degenerate pose | | `too_large` | a network past the cap of §3.3 | | `trailing` | bytes after the last level, or after a `WIL1`'s table | | `geometry_mismatch` | a level that is not the next level of the field it is applied to (§7.3) | A refusal is final, as integer-payloads §8 says of a payload's refusal. A sink that does not implement this extension does not advertise `wai.field.int_grid` and routes it as any capability it does not decode (SPEC §4); a level layer it receives is not applied (staged-delivery §4.1). An emitter checks the framing before it writes the envelope, with the parser a sink runs (`container::check_int_payload_framing`). ## 9. Digests and field-equivalence - The **values digest** of a list of points is the SHA-256 (FIPS 180-4) of their values, each a little-endian `i64`, the points in order and a point's four values in order (density, red, green, blue). - The **frame digest** of a render is the SHA-256 of its bytes (§6). The frame digest at the field's own view is its stage hash (§7.4). **Field-equivalence.** Two conforming sinks that hold the same levels of a field compute the same values at every point, so the same values digest of any list of points, and the same frame at any view, so the same frame digest. The criterion has no tolerance. `wai.field.int_grid` is registered with the class `StandardDefined` (SPEC §7): this text fixes every integer of its decode, and the payload carries its network, so no parameter set or pin is involved. Its fidelity tier is lossy: the features and weights are quantized from whatever the encoder made. ## 10. The encoder (informative) ### 10.1 The JSON form `tools/wai_int_field_export.py` reads a field in a JSON form, `"form": "wai-int-field/1"`, with `ffrac`, `wfrac`, `log2_table` (`T`), `carried` (`K`, so the tool writes a `WIL1` for each later level); `levels`, each a `side` and a `table` of `E_l` entries of `F` floats; `network`, each layer's `weights` (rows) and `biases` as floats; and `view`, its positions and `t_near`, `t_far` as floats in units of the domain's side, its other members as integers. The JSON form is a converter's input, never a payload. ### 10.2 Quantization A feature is `⌊v · 2^ffrac + 0.5⌋`, clamped to `i16`; a weight `⌊v · 2^wfrac + 0.5⌋`, clamped to `i16`; a bias `⌊v · 2^(ffrac + wfrac) + 0.5⌋`, clamped to `i32`; a position `⌊v · 2^16 + 0.5⌋`. The tool builds the corpus's scenes itself, with no download: a field it fits to a procedural scene (a ball and a box of absorbing medium, coloured by position) by stochastic gradient descent on each level's table in turn, coarse to fine, through the level's own index and hash and against what the quantized levels below it leave; a seeded random field, the network's general case; and a field of saturating values. Its float steps are additions, subtractions, multiplications, divisions and comparisons, so it writes the same bytes on every machine. ## 11. Conformance A conforming sink of this extension decodes every scene of the corpus [`volumetric-conformance/fields/`](../volumetric-conformance/fields/), at every prefix of its levels, to the values digest it states at its points and the frame digest it states at each view; refuses every fault payload with the code it states; gives every level applied to a field the outcome it states (§7.3); and, offered each staged sequence's envelopes in order, reports for each the stage or the reason it states and reaches the final stage, freeze and stage hash stated (§7.4, with a hold bound of zero). The corpus's `README.md` lists its files. A conforming decode reproduces every statement; a decode with any one of these faults fails at least one, and both implementations plant each in their own decode and require it: | fault | what it changes | |---|---| | `weight_off_by_one` | every interpolation fraction `w_a` one more than §5.3 gives | | `hash_constant` | the third multiplier of §5.2's hash, 805459861, changed by 2 | | `activation_rounding` | a layer's `R_half_up` by `wfrac` replaced by `⌊acc / 2^wfrac⌋` | | `ray_step` | one sample more a ray than `S` | | `level_order` | a level applied at its own index whatever levels the field holds, not only as the next one (§7.3) | The reference implementation is `wai-rs`: `int_field::{parse_wig1, parse_wil1, Field, View, FieldStages}`, `Field::{query, render, apply_level}`, `int_field::corpus`, the tool `wai_int_field_vectors gen|verify`, and `tests/int_field.rs`. The second implementation, `volumetric-conformance/field_verify.py`, is written from this text and shares no code with it. ## 12. Security considerations A payload is bounded by its header before any section is read: at most 16 levels of at most `2^19 · 8` features, and a network of at most 2^16 multiply-accumulates a point; a level is bounded by its own header, and applies only in the shape its field declared. A render's work is declared before it starts (§3.3), and a sink that renders at views of its own choosing bounds `W · H · S` by its own limit. Every value fits the width §5.6 and §6.5 give it, so a hostile payload cannot make an implementation overflow, and none of its values steers a loop past those bounds. The spatial hash is fixed and public: shared entries are a property of the field, which its encoder fitted, and a frame digest identifies what every conforming sink shows at a view, not how faithfully the field reproduces any scene it was fitted to.