open-standards/wai/research/dcvc-drift.md
Transaction Science e718ccdb9b
Some checks failed
repository checks / lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text (push) Has been cancelled
WAI breaking changes and the conformance runner / WAI breaking · does this change touch WAI (push) Successful in 6s
WAI breaking changes and the conformance runner / WAI breaking · wire, record and API against main · linux (push) Successful in 53s
WAI breaking changes and the conformance runner / WAI breaking and conform · their tests, and the independent verifiers through wai conform · linux (push) Successful in 3m22s
The drift page says every fabric was run again with the tool as committed
The four x86_64 fabrics were run again on both fixtures with the tool as committed, as the arm64
ones were: every frame, feature and node was the same, and `record` over the four runs wrote each
fabric's reading again field for field, the adapter and the driver it now adds aside.

Changelog: none
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 03:34:51 -05:00

347 lines
33 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Drift and desync of `wai.neural.dcvc_rt`, per frame and fabric
> **Status.** A measured record and its procedure, informative. Every figure below is read
> from the readings in [`dcvc-drift/`](dcvc-drift/), and
> [`dcvc-drift/verify.py`](dcvc-drift/verify.py), which shares no code with `tools/dcvc-drift`,
> the tool that took them, decodes every y stream again with an rANS decoder of its own,
> recounts every desync event under the buckets each reading states and checks every figure
> this page quotes (each is followed by a `dcvc` marker naming its place); §7 says what it
> recomputes and what it can only hold together. A statement of how a run went, with nothing
> committed behind it, is marked *(as run)*. Measured on 6 November 2026.
> `wai.neural.dcvc_rt` is `NeuralFloat` (SPEC §7): it claims no tier, and this record measures
> what that leaves a sink with. It changes no registration.
## 1. What is measured, and what is not
`wai.neural.dcvc_rt` decodes frame after frame: an entropy decode whose per-element Gaussian
bucket comes from a float network, `hyper_synthesis`, then a float temporal decoder that also
reads the reference feature the frame before it left (`neural/video_dcvc.rs`). Two things can
go wrong across machines, and this record counts them apart, per frame and per fabric:
1. **Drift.** The float networks round differently, so a frame's samples and its reference
feature move, and the feature carries the movement into the next frame.
2. **Desync.** A scale lands on the other side of a bucket edge from where the encoder had it.
The entropy stage then decodes with a distribution the encoder did not use.
Not measured here, and why:
- **A trained DCVC-RT model.** The repository holds none. The model is the seeded one that
`tools/wai_validate_synthetic.py` builds for the end-to-end job, with random weights, so its
frames are noise and no figure here speaks to picture quality. Its entropy model is a plain
hyperprior that reads nothing from the reference feature, so here a frame's entropy decode
stands alone. The rates of drift and desync are this model's; what carries over is the
mechanism.
- **A neural processor.** No reading of one was kept. The compute plan the platform's
machine-learning provider in the runtime reported took none of the transposed convolutions,
which hold nearly all of both graphs' arithmetic, and it gave the same figures whether asked
for the GPU or the neural processor *(as run)*.
- **`wai.neural.dcvc_fm`**, which decodes through the same file, was not run.
- **Time and energy.** Neither was read.
## 2. Terms
- **Fabric.** Where a decode's graphs run: an instruction set and a device, through a provider
of the runtime, with a thread count (§3.3).
- **The sink's route.** The decode the reference sink runs: `wai::neural::decode_envelope`, whose
sessions come from `ort_session::load_deterministic`, on the CPU provider at one thread.
- **Baseline.** The sink's route on the fabric that encoded the fixture.
- **Drift** is read against the baseline: the 8-bit samples that differ, the histogram of their
differences, and the largest difference in the reference feature.
- **A desync event** is a frame whose entropy decode gives any symbol other than the one the
encoder coded. Only the entropy stage is read for it, so drift alone never counts as one.
## 3. Procedure
### 3.1 Fixtures
`tools/dcvc-drift/make_fixture.py` runs the end-to-end job's generator: its model, seed, export,
encoder and session settings. Both fixtures, with the sets they decode against, are committed
under `dcvc-drift/fixture/`, because a stream's bytes depend on the machine that encoded it and
every fabric has to read one stream. Each was encoded once, on the x86_64 fabric.
- **`ci`** is the job's own fixture: 3<!--dcvc:sets.ci.frames--> frames at 256×256, its
weights untouched. `--check-harness` also runs the job's own generator and requires the two to
write one envelope and one reference. On the arm64 host they did, and that envelope is the one
committed here, byte for byte *(as run)*. Most of its samples are clamped, and a clamped sample
shows no difference.
- **`gop`** is the same model and the same motion for 64<!--dcvc:sets.gop.frames--> frames,
with two changes that a long run needs. The job's weights grow the reference feature by about
the square root of 3 a frame, which saturates a run of this length, so the reference feature's
weights are scaled by 0.6<!--dcvc:sets.gop.ref_gain-->. The feature's RMS then rises slowly,
with no saturation: 5.31<!--dcvc:fixture.gop.frames.0.feature_rms--> at the first frame,
13.02<!--dcvc:fixture.gop.frames.47.feature_rms--> at frame 47 and
14.13<!--dcvc:fixture.gop.frames.63.feature_rms--> at the last. The image head is scaled by
0.008<!--dcvc:sets.gop.image_gain--> and centred on 0.5, so that samples stay in range and
a difference can reach them. The two gains came from a sweep of the encoder's own figures,
which is not committed *(as run)*; `fixture.json` states each frame's RMS.
```sh
python make_fixture.py --out ci --frames 3 --check-harness
python make_fixture.py --out gop --frames 64 --ref-gain 0.6 --image-gain 0.008
```
The stack is `tools/requirements-sota-int.txt`'s torch, compressai and numpy, with onnxruntime
1.28.0, the version the decoder links. `fixture.json` states it, and each file's SHA-256. Run
twice on the arm64 host, the script wrote the stream, the set, the reference and the buckets byte
for byte the same, and the `gop` stream it wrote there is not the committed one *(as run)*. A
fixture is regenerated only with this stack on the fabric that encoded it; every step below reads
what is committed.
### 3.2 A run
`dcvc-drift run` decodes a fixture on one fabric. It decodes the stream twice through a loop of
its own that keeps what the sink's decode throws away, each element's scale, bucket and symbol,
the latent and the feature, and the reading says whether the two passes agreed. On the sink's
route it also decodes twice through `decode_envelope` itself, and refuses to write a reading
unless the two are one output and the loop's frames are that output byte for byte.
`load_deterministic` is private to the decoder, so the loop builds its sessions with the same
settings and that comparison is what holds it to them.
```sh
cargo build --release --locked # CPU fabrics
cargo build --release --locked --features webgpu # WebGPU fabrics
xz -dk fixture/<set>/encoder-buckets.bin.xz
dcvc-drift run --fixture <set> --fabric <name> [--threads N] [--force F:E:B] --out <run>
dcvc-drift run --fixture <set> --fabric <name> --provider webgpu --adapter <class> --out <run>
dcvc-drift record --fixture <set> --run <baseline> --run <run> ... --out readings/<set>
dcvc-drift event --fixture <set> --against <run> --run <forced run> ... --out readings/<set>/events.json
dcvc-drift finish --root research/dcvc-drift --set ci --set gop --measured <date>
```
The CPU fabrics ran on the first build and the WebGPU fabrics on the second. The `webgpu` feature
links the provider's shared library beside the binary and gives the binary a search path to that
directory, and on macOS the second build runs from where cargo put it *(as run)*. `record` reads
`encoder-buckets.bin` and the per-frame digests in `fixture.json`, and no other file of the
encoder's. It takes no copy of the encoder's reference: the baseline's frames, rounded half to
even, must have those digests or it writes nothing, so the baseline has to be a run on the fabric
that encoded the fixture, and a run on any other fabric is read against it. `--force` gives one
element of one frame a bucket whatever its scale says, and changes nothing else; `event` reads
such a run against the unforced run of its own fabric, so no drift lies under the reading (§4.5).
Each run ran under a memory limit on its host.
### 3.3 Fabrics
| fabric | instruction set | operating system | graphs run on |
|---|---|---|---|
| `x86_64-cpu` | x86_64 with AVX-512 | Linux 7.0 | CPU provider, one thread: the sink's route, and the baseline |
| `x86_64-cpu-t8` | as above | Linux 7.0 | CPU provider, eight threads |
| `arm64-cpu` | arm64 | macOS 27.2 | CPU provider, one thread: the sink's route |
| `arm64-cpu-t8` | arm64 | macOS 27.2 | CPU provider, eight threads |
| `x86_64-webgpu` | x86_64, integrated GPU | Linux 7.0 | WebGPU provider, over Vulkan |
| `arm64-webgpu` | arm64, integrated GPU | macOS 27.2 | WebGPU provider, over the platform's GPU interface |
| `x86_64-webgpu-software` | x86_64 CPU | Linux 7.0 | WebGPU provider, over a software Vulkan driver |
The WebGPU fabrics, and any thread count but one, are sessions no sink's decode opens. They run
in the loop alone, held to nothing, and they show what happens if a sink puts these graphs
elsewhere. The runtime's own profile, kept in the readings as `nodes`, shows every node of both
graphs on the named provider. On Linux each WebGPU run was pinned to one Vulkan driver with
`VK_DRIVER_FILES`, and the process mapped that driver; on macOS the run loaded the platform's GPU
driver *(as run)*. `run` writes the adapter's class, which `--adapter` gives as the host's own tools
report it, and the driver its process mapped, and `record` carries both into a reading; the
readings committed here were taken without them. The readings come from the tool as it first
stood. Every fabric was run again with the tool as committed, and every frame, feature and node was
the same; on `x86_64`, `record` over the four new runs wrote each fabric's reading again field for
field, the adapter and the driver aside *(as run)*. No reading names a device.
## 4. Results
### 4.1 The encoder's fabric
On the baseline the decoder reproduces the encoder's float image exactly: rounded the encoder's
way, every frame of both fixtures is the encoder's reconstruction byte for byte. The encoder
rounds to 8 bits half to even and the decoder half away from zero, so the two differ only on
exact ties: 56<!--dcvc:sets.gop.tie_samples--> samples over
38<!--dcvc:sets.gop.tie_frames--> of the `gop` frames, each by 1, and
0<!--dcvc:sets.ci.tie_samples--> in `ci`. The tables below read from the baseline, so
no tie counts as drift.
### 4.2 `ci`
| fabric | graphs run on | frames equal to the baseline, of 3<!--dcvc:ci.frames--> | desync events | first | before the first event: largest sample difference | most samples changed in a frame | largest feature difference |
|---|---|---|---|---|---|---|---|
| `x86_64-cpu` | CPU, one thread: the sink's route, and the baseline | 3<!--dcvc:ci.fabric:x86_64-cpu.summary.frames_exact--> | 0<!--dcvc:ci.fabric:x86_64-cpu.summary.desync_events--> | none | 0<!--dcvc:ci.fabric:x86_64-cpu.summary.before_first_desync.rgb_max--> | 0<!--dcvc:ci.fabric:x86_64-cpu.summary.before_first_desync.rgb_changed_max--> | 0<!--dcvc:ci.fabric:x86_64-cpu.summary.before_first_desync.feature_max--> |
| `x86_64-cpu-t8` | CPU, eight threads | 3<!--dcvc:ci.fabric:x86_64-cpu-t8.summary.frames_exact--> | 0<!--dcvc:ci.fabric:x86_64-cpu-t8.summary.desync_events--> | none | 0<!--dcvc:ci.fabric:x86_64-cpu-t8.summary.before_first_desync.rgb_max--> | 0<!--dcvc:ci.fabric:x86_64-cpu-t8.summary.before_first_desync.rgb_changed_max--> | 0<!--dcvc:ci.fabric:x86_64-cpu-t8.summary.before_first_desync.feature_max--> |
| `arm64-cpu` | CPU, one thread: the sink's route | 1<!--dcvc:ci.fabric:arm64-cpu.summary.frames_exact--> | 0<!--dcvc:ci.fabric:arm64-cpu.summary.desync_events--> | none | 1<!--dcvc:ci.fabric:arm64-cpu.summary.before_first_desync.rgb_max--> | 7<!--dcvc:ci.fabric:arm64-cpu.summary.before_first_desync.rgb_changed_max--> | 3.8e-05<!--dcvc:ci.fabric:arm64-cpu.summary.before_first_desync.feature_max--> |
| `arm64-cpu-t8` | CPU, eight threads | 1<!--dcvc:ci.fabric:arm64-cpu-t8.summary.frames_exact--> | 0<!--dcvc:ci.fabric:arm64-cpu-t8.summary.desync_events--> | none | 1<!--dcvc:ci.fabric:arm64-cpu-t8.summary.before_first_desync.rgb_max--> | 9<!--dcvc:ci.fabric:arm64-cpu-t8.summary.before_first_desync.rgb_changed_max--> | 5.3e-05<!--dcvc:ci.fabric:arm64-cpu-t8.summary.before_first_desync.feature_max--> |
| `x86_64-webgpu` | integrated GPU, WebGPU over Vulkan | 0<!--dcvc:ci.fabric:x86_64-webgpu.summary.frames_exact--> | 0<!--dcvc:ci.fabric:x86_64-webgpu.summary.desync_events--> | none | 1<!--dcvc:ci.fabric:x86_64-webgpu.summary.before_first_desync.rgb_max--> | 30<!--dcvc:ci.fabric:x86_64-webgpu.summary.before_first_desync.rgb_changed_max--> | 2.1e-04<!--dcvc:ci.fabric:x86_64-webgpu.summary.before_first_desync.feature_max--> |
| `arm64-webgpu` | integrated GPU, WebGPU over the platform's GPU interface | 0<!--dcvc:ci.fabric:arm64-webgpu.summary.frames_exact--> | 0<!--dcvc:ci.fabric:arm64-webgpu.summary.desync_events--> | none | 1<!--dcvc:ci.fabric:arm64-webgpu.summary.before_first_desync.rgb_max--> | 30<!--dcvc:ci.fabric:arm64-webgpu.summary.before_first_desync.rgb_changed_max--> | 2.1e-04<!--dcvc:ci.fabric:arm64-webgpu.summary.before_first_desync.feature_max--> |
| `x86_64-webgpu-software` | CPU, WebGPU over a software Vulkan driver | 0<!--dcvc:ci.fabric:x86_64-webgpu-software.summary.frames_exact--> | 0<!--dcvc:ci.fabric:x86_64-webgpu-software.summary.desync_events--> | none | 1<!--dcvc:ci.fabric:x86_64-webgpu-software.summary.before_first_desync.rgb_max--> | 31<!--dcvc:ci.fabric:x86_64-webgpu-software.summary.before_first_desync.rgb_changed_max--> | 2.3e-04<!--dcvc:ci.fabric:x86_64-webgpu-software.summary.before_first_desync.feature_max--> |
### 4.3 `gop`
| fabric | graphs run on | frames equal to the baseline, of 64<!--dcvc:gop.frames--> | desync events | first | before the first event: largest sample difference | most samples changed in a frame | largest feature difference |
|---|---|---|---|---|---|---|---|
| `x86_64-cpu` | CPU, one thread: the sink's route, and the baseline | 64<!--dcvc:gop.fabric:x86_64-cpu.summary.frames_exact--> | 0<!--dcvc:gop.fabric:x86_64-cpu.summary.desync_events--> | none | 0<!--dcvc:gop.fabric:x86_64-cpu.summary.before_first_desync.rgb_max--> | 0<!--dcvc:gop.fabric:x86_64-cpu.summary.before_first_desync.rgb_changed_max--> | 0<!--dcvc:gop.fabric:x86_64-cpu.summary.before_first_desync.feature_max--> |
| `x86_64-cpu-t8` | CPU, eight threads | 64<!--dcvc:gop.fabric:x86_64-cpu-t8.summary.frames_exact--> | 0<!--dcvc:gop.fabric:x86_64-cpu-t8.summary.desync_events--> | none | 0<!--dcvc:gop.fabric:x86_64-cpu-t8.summary.before_first_desync.rgb_max--> | 0<!--dcvc:gop.fabric:x86_64-cpu-t8.summary.before_first_desync.rgb_changed_max--> | 0<!--dcvc:gop.fabric:x86_64-cpu-t8.summary.before_first_desync.feature_max--> |
| `arm64-cpu` | CPU, one thread: the sink's route | 2<!--dcvc:gop.fabric:arm64-cpu.summary.frames_exact--> | 0<!--dcvc:gop.fabric:arm64-cpu.summary.desync_events--> | none | 1<!--dcvc:gop.fabric:arm64-cpu.summary.before_first_desync.rgb_max--> | 42<!--dcvc:gop.fabric:arm64-cpu.summary.before_first_desync.rgb_changed_max--> | 8.7e-04<!--dcvc:gop.fabric:arm64-cpu.summary.before_first_desync.feature_max--> |
| `arm64-cpu-t8` | CPU, eight threads | 1<!--dcvc:gop.fabric:arm64-cpu-t8.summary.frames_exact--> | 0<!--dcvc:gop.fabric:arm64-cpu-t8.summary.desync_events--> | none | 1<!--dcvc:gop.fabric:arm64-cpu-t8.summary.before_first_desync.rgb_max--> | 45<!--dcvc:gop.fabric:arm64-cpu-t8.summary.before_first_desync.rgb_changed_max--> | 1.0e-03<!--dcvc:gop.fabric:arm64-cpu-t8.summary.before_first_desync.feature_max--> |
| `x86_64-webgpu` | integrated GPU, WebGPU over Vulkan | 0<!--dcvc:gop.fabric:x86_64-webgpu.summary.frames_exact--> | 3<!--dcvc:gop.fabric:x86_64-webgpu.summary.desync_events--> | 17<!--dcvc:gop.fabric:x86_64-webgpu.summary.first_desync_frame--> | 1<!--dcvc:gop.fabric:x86_64-webgpu.summary.before_first_desync.rgb_max--> | 26<!--dcvc:gop.fabric:x86_64-webgpu.summary.before_first_desync.rgb_changed_max--> | 2.9e-04<!--dcvc:gop.fabric:x86_64-webgpu.summary.before_first_desync.feature_max--> |
| `arm64-webgpu` | integrated GPU, WebGPU over the platform's GPU interface | 0<!--dcvc:gop.fabric:arm64-webgpu.summary.frames_exact--> | 3<!--dcvc:gop.fabric:arm64-webgpu.summary.desync_events--> | 17<!--dcvc:gop.fabric:arm64-webgpu.summary.first_desync_frame--> | 1<!--dcvc:gop.fabric:arm64-webgpu.summary.before_first_desync.rgb_max--> | 26<!--dcvc:gop.fabric:arm64-webgpu.summary.before_first_desync.rgb_changed_max--> | 2.9e-04<!--dcvc:gop.fabric:arm64-webgpu.summary.before_first_desync.feature_max--> |
| `x86_64-webgpu-software` | CPU, WebGPU over a software Vulkan driver | 0<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.frames_exact--> | 2<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.desync_events--> | 17<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.first_desync_frame--> | 1<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.before_first_desync.rgb_max--> | 28<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.before_first_desync.rgb_changed_max--> | 3.5e-04<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.before_first_desync.feature_max--> |
Two passes of every fabric agreed with each other, so each fabric repeats itself. Eight threads
gave the one-thread frames on x86_64 and did not on arm64. The two GPUs gave one another's frames
and features byte for byte, on both fixtures, across two hosts and two GPU interfaces, and the
software driver, running the same provider, did not (`record.json`,
`fabrics_with_equal_frames`).
### 4.4 Drift by frame
On the sink's route across the two instruction sets, drift is never a desync: 0<!--dcvc:gop.fabric:arm64-cpu.summary.frames_scale_changed--> of the
64<!--dcvc:gop.frames--> frames hold a scale that differs, so every scale is one float, bit for bit, on
both, and the symbols and latents with it. Only the temporal decoder moves, carried by the
reference feature. The difference grows over the run, with variation from frame to frame:
`arm64-cpu` is back to 0<!--dcvc:gop.fabric:arm64-cpu.readings.5.rgb_changed--> samples changed at
frame 5, and its largest feature difference peaks at
8.7e-04<!--dcvc:gop.fabric:arm64-cpu.readings.60.feature_max--> at frame 60, not at the last:
| frame | `arm64-cpu`: samples changed | `arm64-cpu-t8`: samples changed | `arm64-cpu`: largest feature difference | `arm64-cpu-t8`: largest feature difference |
|---|---|---|---|---|
| 0 | 0<!--dcvc:gop.fabric:arm64-cpu.readings.0.rgb_changed--> | 0<!--dcvc:gop.fabric:arm64-cpu-t8.readings.0.rgb_changed--> | 9.5e-06<!--dcvc:gop.fabric:arm64-cpu.readings.0.feature_max--> | 1.6e-05<!--dcvc:gop.fabric:arm64-cpu-t8.readings.0.feature_max--> |
| 7 | 5<!--dcvc:gop.fabric:arm64-cpu.readings.7.rgb_changed--> | 4<!--dcvc:gop.fabric:arm64-cpu-t8.readings.7.rgb_changed--> | 3.4e-05<!--dcvc:gop.fabric:arm64-cpu.readings.7.feature_max--> | 6.6e-05<!--dcvc:gop.fabric:arm64-cpu-t8.readings.7.feature_max--> |
| 15 | 7<!--dcvc:gop.fabric:arm64-cpu.readings.15.rgb_changed--> | 5<!--dcvc:gop.fabric:arm64-cpu-t8.readings.15.rgb_changed--> | 7.7e-05<!--dcvc:gop.fabric:arm64-cpu.readings.15.feature_max--> | 8.8e-05<!--dcvc:gop.fabric:arm64-cpu-t8.readings.15.feature_max--> |
| 31 | 14<!--dcvc:gop.fabric:arm64-cpu.readings.31.rgb_changed--> | 23<!--dcvc:gop.fabric:arm64-cpu-t8.readings.31.rgb_changed--> | 1.8e-04<!--dcvc:gop.fabric:arm64-cpu.readings.31.feature_max--> | 2.7e-04<!--dcvc:gop.fabric:arm64-cpu-t8.readings.31.feature_max--> |
| 47 | 19<!--dcvc:gop.fabric:arm64-cpu.readings.47.rgb_changed--> | 27<!--dcvc:gop.fabric:arm64-cpu-t8.readings.47.rgb_changed--> | 3.5e-04<!--dcvc:gop.fabric:arm64-cpu.readings.47.feature_max--> | 4.0e-04<!--dcvc:gop.fabric:arm64-cpu-t8.readings.47.feature_max--> |
| 63 | 42<!--dcvc:gop.fabric:arm64-cpu.readings.63.rgb_changed--> | 41<!--dcvc:gop.fabric:arm64-cpu-t8.readings.63.rgb_changed--> | 7.5e-04<!--dcvc:gop.fabric:arm64-cpu.readings.63.feature_max--> | 7.3e-04<!--dcvc:gop.fabric:arm64-cpu-t8.readings.63.feature_max--> |
The samples that change do so by 1. By the last frame the largest feature difference is
7.5e-04<!--dcvc:gop.fabric:arm64-cpu.readings.63.feature_max-->, on a
feature whose RMS is 14.1<!--dcvc:sets.gop.feature_rms_last-->, and
nothing in the stream brings it back: the payload has no frame that resets the reference feature.
### 4.5 Desync events
Each of the 786,432 scales of `gop` chooses a bucket. On the CPU fabrics every choice was
the encoder's. On every WebGPU fabric some were not, and each of them changes every scale a little
(64<!--dcvc:gop.fabric:x86_64-webgpu.summary.frames_scale_changed--> frames of 64<!--dcvc:gop.frames-->):
| fabric | frame | element | scale, baseline | scale, this fabric | table edge | wrong symbols | stream ends canonically |
|---|---|---|---|---|---|---|---|
| `x86_64-webgpu` | 17 | 9663<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.first_bucket_change.element--> | 1.8644881<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.first_bucket_change.baseline_scale--> | 1.8644860<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.first_bucket_change.scale--> | 1.8644862<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.first_bucket_change.edge--> | 2558<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.symbol_changed--> | no |
| `x86_64-webgpu` | 20 | 3805<!--dcvc:gop.fabric:x86_64-webgpu.readings.20.first_bucket_change.element--> | 10.4411297<!--dcvc:gop.fabric:x86_64-webgpu.readings.20.first_bucket_change.baseline_scale--> | 10.4411278<!--dcvc:gop.fabric:x86_64-webgpu.readings.20.first_bucket_change.scale--> | 10.4411287<!--dcvc:gop.fabric:x86_64-webgpu.readings.20.first_bucket_change.edge--> | 8225<!--dcvc:gop.fabric:x86_64-webgpu.readings.20.symbol_changed--> | no |
| `x86_64-webgpu` | 36 | 9454<!--dcvc:gop.fabric:x86_64-webgpu.readings.36.first_bucket_change.element--> | 0.6159989<!--dcvc:gop.fabric:x86_64-webgpu.readings.36.first_bucket_change.baseline_scale--> | 0.6160025<!--dcvc:gop.fabric:x86_64-webgpu.readings.36.first_bucket_change.scale--> | 0.6160004<!--dcvc:gop.fabric:x86_64-webgpu.readings.36.first_bucket_change.edge--> | 1<!--dcvc:gop.fabric:x86_64-webgpu.readings.36.symbol_changed--> | yes |
| `arm64-webgpu` | 17 | 9663<!--dcvc:gop.fabric:arm64-webgpu.readings.17.first_bucket_change.element--> | 1.8644881<!--dcvc:gop.fabric:arm64-webgpu.readings.17.first_bucket_change.baseline_scale--> | 1.8644860<!--dcvc:gop.fabric:arm64-webgpu.readings.17.first_bucket_change.scale--> | 1.8644862<!--dcvc:gop.fabric:arm64-webgpu.readings.17.first_bucket_change.edge--> | 2558<!--dcvc:gop.fabric:arm64-webgpu.readings.17.symbol_changed--> | no |
| `arm64-webgpu` | 20 | 3805<!--dcvc:gop.fabric:arm64-webgpu.readings.20.first_bucket_change.element--> | 10.4411297<!--dcvc:gop.fabric:arm64-webgpu.readings.20.first_bucket_change.baseline_scale--> | 10.4411278<!--dcvc:gop.fabric:arm64-webgpu.readings.20.first_bucket_change.scale--> | 10.4411287<!--dcvc:gop.fabric:arm64-webgpu.readings.20.first_bucket_change.edge--> | 8225<!--dcvc:gop.fabric:arm64-webgpu.readings.20.symbol_changed--> | no |
| `arm64-webgpu` | 36 | 9454<!--dcvc:gop.fabric:arm64-webgpu.readings.36.first_bucket_change.element--> | 0.6159989<!--dcvc:gop.fabric:arm64-webgpu.readings.36.first_bucket_change.baseline_scale--> | 0.6160025<!--dcvc:gop.fabric:arm64-webgpu.readings.36.first_bucket_change.scale--> | 0.6160004<!--dcvc:gop.fabric:arm64-webgpu.readings.36.first_bucket_change.edge--> | 1<!--dcvc:gop.fabric:arm64-webgpu.readings.36.symbol_changed--> | yes |
| `x86_64-webgpu-software` | 17 | 9663<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.17.first_bucket_change.element--> | 1.8644881<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.17.first_bucket_change.baseline_scale--> | 1.8644859<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.17.first_bucket_change.scale--> | 1.8644862<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.17.first_bucket_change.edge--> | 2558<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.17.symbol_changed--> | no |
| `x86_64-webgpu-software` | 36 | 9454<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.36.first_bucket_change.element--> | 0.6159989<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.36.first_bucket_change.baseline_scale--> | 0.6160018<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.36.first_bucket_change.scale--> | 0.6160004<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.36.first_bucket_change.edge--> | 1<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.36.symbol_changed--> | yes |
Each event starts from one scale on the wrong side of an edge, by a few parts in a million at
most, and one bucket. What follows differs, and on a WebGPU fabric a later event falls inside
what an earlier one left. So each is also read alone: `--force` gives `arm64-cpu`, the sink's
route, that one bucket and no other, and `event` reads the run against the same fabric's run
without it, so neither drift nor another event lies under the figures
(`dcvc-drift/readings/gop/events.json`). Each forced run repeated itself, and no frame before its
event moved.
| bucket forced at frame | wrong symbols | stream ends canonically | frames from the event to the last | of those, frames that differ | largest sample difference | most samples changed in a frame | lowest PSNR against the run without it | largest feature difference |
|---|---|---|---|---|---|---|---|---|
| 17 | 2,558<!--dcvc:gop.events.frame:17.summary.symbol_changes--> | no | 47<!--dcvc:gop.events.frame:17.summary.from_first_desync.frames--> | 47<!--dcvc:gop.events.frame:17.summary.from_first_desync.frames_changed--> | 255<!--dcvc:gop.events.frame:17.summary.from_first_desync.rgb_max--> | 192,920<!--dcvc:gop.events.frame:17.summary.from_first_desync.rgb_changed_max--> | 7.4 dB<!--dcvc:gop.events.frame:17.summary.from_first_desync.psnr_min_db--> | 9.0e+08<!--dcvc:gop.events.frame:17.summary.from_first_desync.feature_max--> |
| 20 | 8,225<!--dcvc:gop.events.frame:20.summary.symbol_changes--> | no | 44<!--dcvc:gop.events.frame:20.summary.from_first_desync.frames--> | 44<!--dcvc:gop.events.frame:20.summary.from_first_desync.frames_changed--> | 255<!--dcvc:gop.events.frame:20.summary.from_first_desync.rgb_max--> | 192,930<!--dcvc:gop.events.frame:20.summary.from_first_desync.rgb_changed_max--> | 7.3 dB<!--dcvc:gop.events.frame:20.summary.from_first_desync.psnr_min_db--> | 9.5e+08<!--dcvc:gop.events.frame:20.summary.from_first_desync.feature_max--> |
| 36 | 1<!--dcvc:gop.events.frame:36.summary.symbol_changes--> | yes | 28<!--dcvc:gop.events.frame:36.summary.from_first_desync.frames--> | 28<!--dcvc:gop.events.frame:36.summary.from_first_desync.frames_changed--> | 1<!--dcvc:gop.events.frame:36.summary.from_first_desync.rgb_max--> | 4,610<!--dcvc:gop.events.frame:36.summary.from_first_desync.rgb_changed_max--> | 64.4 dB<!--dcvc:gop.events.frame:36.summary.from_first_desync.psnr_min_db--> | 0.30<!--dcvc:gop.events.frame:36.summary.from_first_desync.feature_max--> |
The three are two kinds:
- **The coder's state is lost** (frames 17 and 20). Everything decoded after that element is noise,
with latents as large as 2.15e+09<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.latent_max-->,
and the stream does not end canonically. The frame is lost, and so is every frame after it,
through the reference feature. The frames' y streams are independent, so a later frame's entropy
decode is right again, but its picture is not.
- **One symbol is wrong and the state survives** (frame 36). The stream ends canonically and nothing
in it shows that anything happened. The picture is not lost: read alone, this event moves no
sample by more than 1. It does not go away either. Every frame from it on differs, in
1,163<!--dcvc:gop.events.frame:36.readings.36.rgb_changed--> samples at its own frame and
4,587<!--dcvc:gop.events.frame:36.readings.63.rgb_changed--> at the last, and at the last frame
its feature difference is 0.094<!--dcvc:gop.events.frame:36.readings.63.feature_max-->, where
drift's is 7.5e-04<!--dcvc:gop.fabric:arm64-cpu.readings.63.feature_max-->.
On the fabrics that read these events, the first is frame 17's, so the first kind sets what they
show. On `x86_64-webgpu` that event and the frames after it make
47<!--dcvc:gop.fabric:x86_64-webgpu.summary.from_first_desync.frames--> frames, and every one
differs, by up to 255<!--dcvc:gop.fabric:x86_64-webgpu.summary.from_first_desync.rgb_max--> in a
sample and in as many as
192,930<!--dcvc:gop.fabric:x86_64-webgpu.summary.from_first_desync.rgb_changed_max--> of a frame's
196,608<!--dcvc:gop.samples_per_frame--> samples, and the PSNR against the baseline falls to
7.4<!--dcvc:gop.fabric:x86_64-webgpu.summary.from_first_desync.psnr_min_db--> dB at the worst.
Frame 36's event falls inside that, where only the forced run can read it. The sink's decode reads
the end of no y stream (`hyperprior::decode_yhat`), so it would report neither kind.
## 5. What the decoder can be offered as
The verifier derives each row from the readings (`record.json`, `verdicts`):
| property | sink's route, both instruction sets | every fabric read |
|---|---|---|
| Every frame byte for byte the baseline's (`decode-equivalence`) | does not hold | does not hold |
| Every symbol the encoder's (`entropy-consistency`) | held on this fixture | does not hold |
| One fabric repeats itself | held | held |
So the measurement supports what SPEC §7 says of `NeuralFloat`, and no more:
- **"Exact on this build" is what it can be offered as.** Each fabric repeated its own frames, and
a stream decoded on the encoder's own fabric gives the encoder's float image.
- **Not `replicate`, and no tier.** `decode-equivalence` fails between the two instruction sets
at the second frame (4<!--dcvc:gop.fabric:arm64-cpu.readings.1.rgb_changed--> samples of `arm64-cpu`'s frame 1 in `gop`).
`entropy-consistency` held on the sink's route here only because two CPUs gave one float for
every scale of this small network, which nothing in the capability promises, and it fails as
soon as the same graph runs on a GPU.
- **Across fabrics it is `perceptual` at best, and only until a desync loses the coder's state.**
Drift alone stays within 1 in a sample over this run. So does the event that cost one symbol,
read alone, though its error stays to the last frame. An event that loses the coder's state does
not degrade a picture a little: it ends the stream's usefulness until something resets the
reference feature, which nothing in this payload does. Both kinds start the same way, from one
scale past one edge (§4.5).
- **Nothing tells the sink.** The payload carries no check of a frame, and an event can end
canonically, so even a check of the stream's end, which the decode does not make, would miss it.
Holding the entropy stage takes what `entropy-consistency` asks: integer entropy arithmetic, or
parameters the codestream carries (determinism-tiers §2).
## 6. What this does not show
- **A trained model, or real pictures.** The rates here are those of a small seeded network on
noise. A trained model's scales and recurrence differ, and its entropy model may read the
reference feature, in which case a desync would not stay in its frame.
- **Rates.** 786,432 scales a fabric is one stream. The count of events is what happened
here; it is not an estimate of a rate on another stream, device or driver.
- **Every kind of event.** Three events were read alone, on one stream and one fabric. The two
kinds of §4.5 are what those three did, not a list of what an event can do, and the split
between them says nothing about how often either comes.
- **Every CPU.** Two instruction sets were read, one host each. The equal scales are a reading of
these two, and of the prebuilt runtime each linked.
- **Longer runs.** 64<!--dcvc:gop.frames--> frames, with no reset of the reference feature.
## 7. Verify
```sh
python3 -I research/dcvc-drift/verify.py --selftest research/dcvc-drift research/dcvc-drift.md
```
The verifier needs the standard library alone. It reads each fixture's envelope, decodes each
frame's y stream under the encoder's buckets, requires a canonical end and the encoder's symbols,
then decodes it again under the buckets each reading states wherever they differ, and from that
counts buckets, symbols, ends and events. `--selftest` then changes copies of the record and
requires each change to be refused for the reason it is planted for, with no help from this page:
a changed reading, one desync too many, one too few, a real event hidden, an event erased together
with its bucket and a forced run that does not repeat an event among them. The wai-conformance
step `dcvc-drift` runs exactly this, over committed files, with no runtime and no build.
What it establishes differs by field, and the float networks draw the line:
- **Recomputed from committed bytes:** every symbol, stream end and desync count under a reading's
buckets, each first bucket change against the scale table, and every summary and verdict.
- **Taken from the reading:** a fabric's scales, and so its buckets. Recomputing them takes that
fabric's float network, which the verifier does not run. It holds them to `latent_max`, because a
latent is a symbol plus a float mean: a wrong symbol moves it by a whole step and drift by far
less, so an event erased with its bucket is refused while its latent stands.
- **Held only to one another and to the files' digests:** the sample histograms and `rgb_changed`,
`rgb_max`, `rgb_sse` and `rgb_sha256`, because a frame's pixels are not committed; `feature_max`,
`feature_changed` and `feature_sha256`, likewise; `latent_max`, `scale_max` and their counts;
and `nodes`. A forced run's figures are of this class too: the verifier recomputes its symbols
and its end, and holds its digests to the run it is read against. A change to any one of them
alone is refused. A change made to all of a reading's
related fields together, with the digests here and in `record.json` written again, is not: such
a reading is shown false only by taking the run again.
`record.json`, which names every reading file by digest, has SHA-256
`730dc3d000d96247b05004ee9a04a0c64ba8c3b240b46f6eb3d6daf9488129ad`<!--dcvc-file:record.json-->.