Some checks failed
repository checks / lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text (push) Has been cancelled
WAI breaking changes and the conformance runner / WAI breaking · does this change touch WAI (push) Successful in 6s
WAI breaking changes and the conformance runner / WAI breaking · wire, record and API against main · linux (push) Successful in 53s
WAI breaking changes and the conformance runner / WAI breaking and conform · their tests, and the independent verifiers through wai conform · linux (push) Successful in 3m22s
The four x86_64 fabrics were run again on both fixtures with the tool as committed, as the arm64 ones were: every frame, feature and node was the same, and `record` over the four runs wrote each fabric's reading again field for field, the adapter and the driver it now adds aside. Changelog: none Co-Authored-By: Claude Opus 5.5 <[email protected]>
347 lines
33 KiB
Markdown
347 lines
33 KiB
Markdown
# Drift and desync of `wai.neural.dcvc_rt`, per frame and fabric
|
||
|
||
> **Status.** A measured record and its procedure, informative. Every figure below is read
|
||
> from the readings in [`dcvc-drift/`](dcvc-drift/), and
|
||
> [`dcvc-drift/verify.py`](dcvc-drift/verify.py), which shares no code with `tools/dcvc-drift`,
|
||
> the tool that took them, decodes every y stream again with an rANS decoder of its own,
|
||
> recounts every desync event under the buckets each reading states and checks every figure
|
||
> this page quotes (each is followed by a `dcvc` marker naming its place); §7 says what it
|
||
> recomputes and what it can only hold together. A statement of how a run went, with nothing
|
||
> committed behind it, is marked *(as run)*. Measured on 6 November 2026.
|
||
> `wai.neural.dcvc_rt` is `NeuralFloat` (SPEC §7): it claims no tier, and this record measures
|
||
> what that leaves a sink with. It changes no registration.
|
||
|
||
## 1. What is measured, and what is not
|
||
|
||
`wai.neural.dcvc_rt` decodes frame after frame: an entropy decode whose per-element Gaussian
|
||
bucket comes from a float network, `hyper_synthesis`, then a float temporal decoder that also
|
||
reads the reference feature the frame before it left (`neural/video_dcvc.rs`). Two things can
|
||
go wrong across machines, and this record counts them apart, per frame and per fabric:
|
||
|
||
1. **Drift.** The float networks round differently, so a frame's samples and its reference
|
||
feature move, and the feature carries the movement into the next frame.
|
||
2. **Desync.** A scale lands on the other side of a bucket edge from where the encoder had it.
|
||
The entropy stage then decodes with a distribution the encoder did not use.
|
||
|
||
Not measured here, and why:
|
||
|
||
- **A trained DCVC-RT model.** The repository holds none. The model is the seeded one that
|
||
`tools/wai_validate_synthetic.py` builds for the end-to-end job, with random weights, so its
|
||
frames are noise and no figure here speaks to picture quality. Its entropy model is a plain
|
||
hyperprior that reads nothing from the reference feature, so here a frame's entropy decode
|
||
stands alone. The rates of drift and desync are this model's; what carries over is the
|
||
mechanism.
|
||
- **A neural processor.** No reading of one was kept. The compute plan the platform's
|
||
machine-learning provider in the runtime reported took none of the transposed convolutions,
|
||
which hold nearly all of both graphs' arithmetic, and it gave the same figures whether asked
|
||
for the GPU or the neural processor *(as run)*.
|
||
- **`wai.neural.dcvc_fm`**, which decodes through the same file, was not run.
|
||
- **Time and energy.** Neither was read.
|
||
|
||
## 2. Terms
|
||
|
||
- **Fabric.** Where a decode's graphs run: an instruction set and a device, through a provider
|
||
of the runtime, with a thread count (§3.3).
|
||
- **The sink's route.** The decode the reference sink runs: `wai::neural::decode_envelope`, whose
|
||
sessions come from `ort_session::load_deterministic`, on the CPU provider at one thread.
|
||
- **Baseline.** The sink's route on the fabric that encoded the fixture.
|
||
- **Drift** is read against the baseline: the 8-bit samples that differ, the histogram of their
|
||
differences, and the largest difference in the reference feature.
|
||
- **A desync event** is a frame whose entropy decode gives any symbol other than the one the
|
||
encoder coded. Only the entropy stage is read for it, so drift alone never counts as one.
|
||
|
||
## 3. Procedure
|
||
|
||
### 3.1 Fixtures
|
||
|
||
`tools/dcvc-drift/make_fixture.py` runs the end-to-end job's generator: its model, seed, export,
|
||
encoder and session settings. Both fixtures, with the sets they decode against, are committed
|
||
under `dcvc-drift/fixture/`, because a stream's bytes depend on the machine that encoded it and
|
||
every fabric has to read one stream. Each was encoded once, on the x86_64 fabric.
|
||
|
||
- **`ci`** is the job's own fixture: 3<!--dcvc:sets.ci.frames--> frames at 256×256, its
|
||
weights untouched. `--check-harness` also runs the job's own generator and requires the two to
|
||
write one envelope and one reference. On the arm64 host they did, and that envelope is the one
|
||
committed here, byte for byte *(as run)*. Most of its samples are clamped, and a clamped sample
|
||
shows no difference.
|
||
- **`gop`** is the same model and the same motion for 64<!--dcvc:sets.gop.frames--> frames,
|
||
with two changes that a long run needs. The job's weights grow the reference feature by about
|
||
the square root of 3 a frame, which saturates a run of this length, so the reference feature's
|
||
weights are scaled by 0.6<!--dcvc:sets.gop.ref_gain-->. The feature's RMS then rises slowly,
|
||
with no saturation: 5.31<!--dcvc:fixture.gop.frames.0.feature_rms--> at the first frame,
|
||
13.02<!--dcvc:fixture.gop.frames.47.feature_rms--> at frame 47 and
|
||
14.13<!--dcvc:fixture.gop.frames.63.feature_rms--> at the last. The image head is scaled by
|
||
0.008<!--dcvc:sets.gop.image_gain--> and centred on 0.5, so that samples stay in range and
|
||
a difference can reach them. The two gains came from a sweep of the encoder's own figures,
|
||
which is not committed *(as run)*; `fixture.json` states each frame's RMS.
|
||
|
||
```sh
|
||
python make_fixture.py --out ci --frames 3 --check-harness
|
||
python make_fixture.py --out gop --frames 64 --ref-gain 0.6 --image-gain 0.008
|
||
```
|
||
|
||
The stack is `tools/requirements-sota-int.txt`'s torch, compressai and numpy, with onnxruntime
|
||
1.28.0, the version the decoder links. `fixture.json` states it, and each file's SHA-256. Run
|
||
twice on the arm64 host, the script wrote the stream, the set, the reference and the buckets byte
|
||
for byte the same, and the `gop` stream it wrote there is not the committed one *(as run)*. A
|
||
fixture is regenerated only with this stack on the fabric that encoded it; every step below reads
|
||
what is committed.
|
||
|
||
### 3.2 A run
|
||
|
||
`dcvc-drift run` decodes a fixture on one fabric. It decodes the stream twice through a loop of
|
||
its own that keeps what the sink's decode throws away, each element's scale, bucket and symbol,
|
||
the latent and the feature, and the reading says whether the two passes agreed. On the sink's
|
||
route it also decodes twice through `decode_envelope` itself, and refuses to write a reading
|
||
unless the two are one output and the loop's frames are that output byte for byte.
|
||
`load_deterministic` is private to the decoder, so the loop builds its sessions with the same
|
||
settings and that comparison is what holds it to them.
|
||
|
||
```sh
|
||
cargo build --release --locked # CPU fabrics
|
||
cargo build --release --locked --features webgpu # WebGPU fabrics
|
||
xz -dk fixture/<set>/encoder-buckets.bin.xz
|
||
dcvc-drift run --fixture <set> --fabric <name> [--threads N] [--force F:E:B] --out <run>
|
||
dcvc-drift run --fixture <set> --fabric <name> --provider webgpu --adapter <class> --out <run>
|
||
dcvc-drift record --fixture <set> --run <baseline> --run <run> ... --out readings/<set>
|
||
dcvc-drift event --fixture <set> --against <run> --run <forced run> ... --out readings/<set>/events.json
|
||
dcvc-drift finish --root research/dcvc-drift --set ci --set gop --measured <date>
|
||
```
|
||
|
||
The CPU fabrics ran on the first build and the WebGPU fabrics on the second. The `webgpu` feature
|
||
links the provider's shared library beside the binary and gives the binary a search path to that
|
||
directory, and on macOS the second build runs from where cargo put it *(as run)*. `record` reads
|
||
`encoder-buckets.bin` and the per-frame digests in `fixture.json`, and no other file of the
|
||
encoder's. It takes no copy of the encoder's reference: the baseline's frames, rounded half to
|
||
even, must have those digests or it writes nothing, so the baseline has to be a run on the fabric
|
||
that encoded the fixture, and a run on any other fabric is read against it. `--force` gives one
|
||
element of one frame a bucket whatever its scale says, and changes nothing else; `event` reads
|
||
such a run against the unforced run of its own fabric, so no drift lies under the reading (§4.5).
|
||
Each run ran under a memory limit on its host.
|
||
|
||
### 3.3 Fabrics
|
||
|
||
| fabric | instruction set | operating system | graphs run on |
|
||
|---|---|---|---|
|
||
| `x86_64-cpu` | x86_64 with AVX-512 | Linux 7.0 | CPU provider, one thread: the sink's route, and the baseline |
|
||
| `x86_64-cpu-t8` | as above | Linux 7.0 | CPU provider, eight threads |
|
||
| `arm64-cpu` | arm64 | macOS 27.2 | CPU provider, one thread: the sink's route |
|
||
| `arm64-cpu-t8` | arm64 | macOS 27.2 | CPU provider, eight threads |
|
||
| `x86_64-webgpu` | x86_64, integrated GPU | Linux 7.0 | WebGPU provider, over Vulkan |
|
||
| `arm64-webgpu` | arm64, integrated GPU | macOS 27.2 | WebGPU provider, over the platform's GPU interface |
|
||
| `x86_64-webgpu-software` | x86_64 CPU | Linux 7.0 | WebGPU provider, over a software Vulkan driver |
|
||
|
||
The WebGPU fabrics, and any thread count but one, are sessions no sink's decode opens. They run
|
||
in the loop alone, held to nothing, and they show what happens if a sink puts these graphs
|
||
elsewhere. The runtime's own profile, kept in the readings as `nodes`, shows every node of both
|
||
graphs on the named provider. On Linux each WebGPU run was pinned to one Vulkan driver with
|
||
`VK_DRIVER_FILES`, and the process mapped that driver; on macOS the run loaded the platform's GPU
|
||
driver *(as run)*. `run` writes the adapter's class, which `--adapter` gives as the host's own tools
|
||
report it, and the driver its process mapped, and `record` carries both into a reading; the
|
||
readings committed here were taken without them. The readings come from the tool as it first
|
||
stood. Every fabric was run again with the tool as committed, and every frame, feature and node was
|
||
the same; on `x86_64`, `record` over the four new runs wrote each fabric's reading again field for
|
||
field, the adapter and the driver aside *(as run)*. No reading names a device.
|
||
|
||
## 4. Results
|
||
|
||
### 4.1 The encoder's fabric
|
||
|
||
On the baseline the decoder reproduces the encoder's float image exactly: rounded the encoder's
|
||
way, every frame of both fixtures is the encoder's reconstruction byte for byte. The encoder
|
||
rounds to 8 bits half to even and the decoder half away from zero, so the two differ only on
|
||
exact ties: 56<!--dcvc:sets.gop.tie_samples--> samples over
|
||
38<!--dcvc:sets.gop.tie_frames--> of the `gop` frames, each by 1, and
|
||
0<!--dcvc:sets.ci.tie_samples--> in `ci`. The tables below read from the baseline, so
|
||
no tie counts as drift.
|
||
|
||
### 4.2 `ci`
|
||
|
||
| fabric | graphs run on | frames equal to the baseline, of 3<!--dcvc:ci.frames--> | desync events | first | before the first event: largest sample difference | most samples changed in a frame | largest feature difference |
|
||
|---|---|---|---|---|---|---|---|
|
||
| `x86_64-cpu` | CPU, one thread: the sink's route, and the baseline | 3<!--dcvc:ci.fabric:x86_64-cpu.summary.frames_exact--> | 0<!--dcvc:ci.fabric:x86_64-cpu.summary.desync_events--> | none | 0<!--dcvc:ci.fabric:x86_64-cpu.summary.before_first_desync.rgb_max--> | 0<!--dcvc:ci.fabric:x86_64-cpu.summary.before_first_desync.rgb_changed_max--> | 0<!--dcvc:ci.fabric:x86_64-cpu.summary.before_first_desync.feature_max--> |
|
||
| `x86_64-cpu-t8` | CPU, eight threads | 3<!--dcvc:ci.fabric:x86_64-cpu-t8.summary.frames_exact--> | 0<!--dcvc:ci.fabric:x86_64-cpu-t8.summary.desync_events--> | none | 0<!--dcvc:ci.fabric:x86_64-cpu-t8.summary.before_first_desync.rgb_max--> | 0<!--dcvc:ci.fabric:x86_64-cpu-t8.summary.before_first_desync.rgb_changed_max--> | 0<!--dcvc:ci.fabric:x86_64-cpu-t8.summary.before_first_desync.feature_max--> |
|
||
| `arm64-cpu` | CPU, one thread: the sink's route | 1<!--dcvc:ci.fabric:arm64-cpu.summary.frames_exact--> | 0<!--dcvc:ci.fabric:arm64-cpu.summary.desync_events--> | none | 1<!--dcvc:ci.fabric:arm64-cpu.summary.before_first_desync.rgb_max--> | 7<!--dcvc:ci.fabric:arm64-cpu.summary.before_first_desync.rgb_changed_max--> | 3.8e-05<!--dcvc:ci.fabric:arm64-cpu.summary.before_first_desync.feature_max--> |
|
||
| `arm64-cpu-t8` | CPU, eight threads | 1<!--dcvc:ci.fabric:arm64-cpu-t8.summary.frames_exact--> | 0<!--dcvc:ci.fabric:arm64-cpu-t8.summary.desync_events--> | none | 1<!--dcvc:ci.fabric:arm64-cpu-t8.summary.before_first_desync.rgb_max--> | 9<!--dcvc:ci.fabric:arm64-cpu-t8.summary.before_first_desync.rgb_changed_max--> | 5.3e-05<!--dcvc:ci.fabric:arm64-cpu-t8.summary.before_first_desync.feature_max--> |
|
||
| `x86_64-webgpu` | integrated GPU, WebGPU over Vulkan | 0<!--dcvc:ci.fabric:x86_64-webgpu.summary.frames_exact--> | 0<!--dcvc:ci.fabric:x86_64-webgpu.summary.desync_events--> | none | 1<!--dcvc:ci.fabric:x86_64-webgpu.summary.before_first_desync.rgb_max--> | 30<!--dcvc:ci.fabric:x86_64-webgpu.summary.before_first_desync.rgb_changed_max--> | 2.1e-04<!--dcvc:ci.fabric:x86_64-webgpu.summary.before_first_desync.feature_max--> |
|
||
| `arm64-webgpu` | integrated GPU, WebGPU over the platform's GPU interface | 0<!--dcvc:ci.fabric:arm64-webgpu.summary.frames_exact--> | 0<!--dcvc:ci.fabric:arm64-webgpu.summary.desync_events--> | none | 1<!--dcvc:ci.fabric:arm64-webgpu.summary.before_first_desync.rgb_max--> | 30<!--dcvc:ci.fabric:arm64-webgpu.summary.before_first_desync.rgb_changed_max--> | 2.1e-04<!--dcvc:ci.fabric:arm64-webgpu.summary.before_first_desync.feature_max--> |
|
||
| `x86_64-webgpu-software` | CPU, WebGPU over a software Vulkan driver | 0<!--dcvc:ci.fabric:x86_64-webgpu-software.summary.frames_exact--> | 0<!--dcvc:ci.fabric:x86_64-webgpu-software.summary.desync_events--> | none | 1<!--dcvc:ci.fabric:x86_64-webgpu-software.summary.before_first_desync.rgb_max--> | 31<!--dcvc:ci.fabric:x86_64-webgpu-software.summary.before_first_desync.rgb_changed_max--> | 2.3e-04<!--dcvc:ci.fabric:x86_64-webgpu-software.summary.before_first_desync.feature_max--> |
|
||
|
||
### 4.3 `gop`
|
||
|
||
| fabric | graphs run on | frames equal to the baseline, of 64<!--dcvc:gop.frames--> | desync events | first | before the first event: largest sample difference | most samples changed in a frame | largest feature difference |
|
||
|---|---|---|---|---|---|---|---|
|
||
| `x86_64-cpu` | CPU, one thread: the sink's route, and the baseline | 64<!--dcvc:gop.fabric:x86_64-cpu.summary.frames_exact--> | 0<!--dcvc:gop.fabric:x86_64-cpu.summary.desync_events--> | none | 0<!--dcvc:gop.fabric:x86_64-cpu.summary.before_first_desync.rgb_max--> | 0<!--dcvc:gop.fabric:x86_64-cpu.summary.before_first_desync.rgb_changed_max--> | 0<!--dcvc:gop.fabric:x86_64-cpu.summary.before_first_desync.feature_max--> |
|
||
| `x86_64-cpu-t8` | CPU, eight threads | 64<!--dcvc:gop.fabric:x86_64-cpu-t8.summary.frames_exact--> | 0<!--dcvc:gop.fabric:x86_64-cpu-t8.summary.desync_events--> | none | 0<!--dcvc:gop.fabric:x86_64-cpu-t8.summary.before_first_desync.rgb_max--> | 0<!--dcvc:gop.fabric:x86_64-cpu-t8.summary.before_first_desync.rgb_changed_max--> | 0<!--dcvc:gop.fabric:x86_64-cpu-t8.summary.before_first_desync.feature_max--> |
|
||
| `arm64-cpu` | CPU, one thread: the sink's route | 2<!--dcvc:gop.fabric:arm64-cpu.summary.frames_exact--> | 0<!--dcvc:gop.fabric:arm64-cpu.summary.desync_events--> | none | 1<!--dcvc:gop.fabric:arm64-cpu.summary.before_first_desync.rgb_max--> | 42<!--dcvc:gop.fabric:arm64-cpu.summary.before_first_desync.rgb_changed_max--> | 8.7e-04<!--dcvc:gop.fabric:arm64-cpu.summary.before_first_desync.feature_max--> |
|
||
| `arm64-cpu-t8` | CPU, eight threads | 1<!--dcvc:gop.fabric:arm64-cpu-t8.summary.frames_exact--> | 0<!--dcvc:gop.fabric:arm64-cpu-t8.summary.desync_events--> | none | 1<!--dcvc:gop.fabric:arm64-cpu-t8.summary.before_first_desync.rgb_max--> | 45<!--dcvc:gop.fabric:arm64-cpu-t8.summary.before_first_desync.rgb_changed_max--> | 1.0e-03<!--dcvc:gop.fabric:arm64-cpu-t8.summary.before_first_desync.feature_max--> |
|
||
| `x86_64-webgpu` | integrated GPU, WebGPU over Vulkan | 0<!--dcvc:gop.fabric:x86_64-webgpu.summary.frames_exact--> | 3<!--dcvc:gop.fabric:x86_64-webgpu.summary.desync_events--> | 17<!--dcvc:gop.fabric:x86_64-webgpu.summary.first_desync_frame--> | 1<!--dcvc:gop.fabric:x86_64-webgpu.summary.before_first_desync.rgb_max--> | 26<!--dcvc:gop.fabric:x86_64-webgpu.summary.before_first_desync.rgb_changed_max--> | 2.9e-04<!--dcvc:gop.fabric:x86_64-webgpu.summary.before_first_desync.feature_max--> |
|
||
| `arm64-webgpu` | integrated GPU, WebGPU over the platform's GPU interface | 0<!--dcvc:gop.fabric:arm64-webgpu.summary.frames_exact--> | 3<!--dcvc:gop.fabric:arm64-webgpu.summary.desync_events--> | 17<!--dcvc:gop.fabric:arm64-webgpu.summary.first_desync_frame--> | 1<!--dcvc:gop.fabric:arm64-webgpu.summary.before_first_desync.rgb_max--> | 26<!--dcvc:gop.fabric:arm64-webgpu.summary.before_first_desync.rgb_changed_max--> | 2.9e-04<!--dcvc:gop.fabric:arm64-webgpu.summary.before_first_desync.feature_max--> |
|
||
| `x86_64-webgpu-software` | CPU, WebGPU over a software Vulkan driver | 0<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.frames_exact--> | 2<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.desync_events--> | 17<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.first_desync_frame--> | 1<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.before_first_desync.rgb_max--> | 28<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.before_first_desync.rgb_changed_max--> | 3.5e-04<!--dcvc:gop.fabric:x86_64-webgpu-software.summary.before_first_desync.feature_max--> |
|
||
|
||
Two passes of every fabric agreed with each other, so each fabric repeats itself. Eight threads
|
||
gave the one-thread frames on x86_64 and did not on arm64. The two GPUs gave one another's frames
|
||
and features byte for byte, on both fixtures, across two hosts and two GPU interfaces, and the
|
||
software driver, running the same provider, did not (`record.json`,
|
||
`fabrics_with_equal_frames`).
|
||
|
||
### 4.4 Drift by frame
|
||
|
||
On the sink's route across the two instruction sets, drift is never a desync: 0<!--dcvc:gop.fabric:arm64-cpu.summary.frames_scale_changed--> of the
|
||
64<!--dcvc:gop.frames--> frames hold a scale that differs, so every scale is one float, bit for bit, on
|
||
both, and the symbols and latents with it. Only the temporal decoder moves, carried by the
|
||
reference feature. The difference grows over the run, with variation from frame to frame:
|
||
`arm64-cpu` is back to 0<!--dcvc:gop.fabric:arm64-cpu.readings.5.rgb_changed--> samples changed at
|
||
frame 5, and its largest feature difference peaks at
|
||
8.7e-04<!--dcvc:gop.fabric:arm64-cpu.readings.60.feature_max--> at frame 60, not at the last:
|
||
|
||
| frame | `arm64-cpu`: samples changed | `arm64-cpu-t8`: samples changed | `arm64-cpu`: largest feature difference | `arm64-cpu-t8`: largest feature difference |
|
||
|---|---|---|---|---|
|
||
| 0 | 0<!--dcvc:gop.fabric:arm64-cpu.readings.0.rgb_changed--> | 0<!--dcvc:gop.fabric:arm64-cpu-t8.readings.0.rgb_changed--> | 9.5e-06<!--dcvc:gop.fabric:arm64-cpu.readings.0.feature_max--> | 1.6e-05<!--dcvc:gop.fabric:arm64-cpu-t8.readings.0.feature_max--> |
|
||
| 7 | 5<!--dcvc:gop.fabric:arm64-cpu.readings.7.rgb_changed--> | 4<!--dcvc:gop.fabric:arm64-cpu-t8.readings.7.rgb_changed--> | 3.4e-05<!--dcvc:gop.fabric:arm64-cpu.readings.7.feature_max--> | 6.6e-05<!--dcvc:gop.fabric:arm64-cpu-t8.readings.7.feature_max--> |
|
||
| 15 | 7<!--dcvc:gop.fabric:arm64-cpu.readings.15.rgb_changed--> | 5<!--dcvc:gop.fabric:arm64-cpu-t8.readings.15.rgb_changed--> | 7.7e-05<!--dcvc:gop.fabric:arm64-cpu.readings.15.feature_max--> | 8.8e-05<!--dcvc:gop.fabric:arm64-cpu-t8.readings.15.feature_max--> |
|
||
| 31 | 14<!--dcvc:gop.fabric:arm64-cpu.readings.31.rgb_changed--> | 23<!--dcvc:gop.fabric:arm64-cpu-t8.readings.31.rgb_changed--> | 1.8e-04<!--dcvc:gop.fabric:arm64-cpu.readings.31.feature_max--> | 2.7e-04<!--dcvc:gop.fabric:arm64-cpu-t8.readings.31.feature_max--> |
|
||
| 47 | 19<!--dcvc:gop.fabric:arm64-cpu.readings.47.rgb_changed--> | 27<!--dcvc:gop.fabric:arm64-cpu-t8.readings.47.rgb_changed--> | 3.5e-04<!--dcvc:gop.fabric:arm64-cpu.readings.47.feature_max--> | 4.0e-04<!--dcvc:gop.fabric:arm64-cpu-t8.readings.47.feature_max--> |
|
||
| 63 | 42<!--dcvc:gop.fabric:arm64-cpu.readings.63.rgb_changed--> | 41<!--dcvc:gop.fabric:arm64-cpu-t8.readings.63.rgb_changed--> | 7.5e-04<!--dcvc:gop.fabric:arm64-cpu.readings.63.feature_max--> | 7.3e-04<!--dcvc:gop.fabric:arm64-cpu-t8.readings.63.feature_max--> |
|
||
|
||
The samples that change do so by 1. By the last frame the largest feature difference is
|
||
7.5e-04<!--dcvc:gop.fabric:arm64-cpu.readings.63.feature_max-->, on a
|
||
feature whose RMS is 14.1<!--dcvc:sets.gop.feature_rms_last-->, and
|
||
nothing in the stream brings it back: the payload has no frame that resets the reference feature.
|
||
|
||
### 4.5 Desync events
|
||
|
||
Each of the 786,432 scales of `gop` chooses a bucket. On the CPU fabrics every choice was
|
||
the encoder's. On every WebGPU fabric some were not, and each of them changes every scale a little
|
||
(64<!--dcvc:gop.fabric:x86_64-webgpu.summary.frames_scale_changed--> frames of 64<!--dcvc:gop.frames-->):
|
||
|
||
| fabric | frame | element | scale, baseline | scale, this fabric | table edge | wrong symbols | stream ends canonically |
|
||
|---|---|---|---|---|---|---|---|
|
||
| `x86_64-webgpu` | 17 | 9663<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.first_bucket_change.element--> | 1.8644881<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.first_bucket_change.baseline_scale--> | 1.8644860<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.first_bucket_change.scale--> | 1.8644862<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.first_bucket_change.edge--> | 2558<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.symbol_changed--> | no |
|
||
| `x86_64-webgpu` | 20 | 3805<!--dcvc:gop.fabric:x86_64-webgpu.readings.20.first_bucket_change.element--> | 10.4411297<!--dcvc:gop.fabric:x86_64-webgpu.readings.20.first_bucket_change.baseline_scale--> | 10.4411278<!--dcvc:gop.fabric:x86_64-webgpu.readings.20.first_bucket_change.scale--> | 10.4411287<!--dcvc:gop.fabric:x86_64-webgpu.readings.20.first_bucket_change.edge--> | 8225<!--dcvc:gop.fabric:x86_64-webgpu.readings.20.symbol_changed--> | no |
|
||
| `x86_64-webgpu` | 36 | 9454<!--dcvc:gop.fabric:x86_64-webgpu.readings.36.first_bucket_change.element--> | 0.6159989<!--dcvc:gop.fabric:x86_64-webgpu.readings.36.first_bucket_change.baseline_scale--> | 0.6160025<!--dcvc:gop.fabric:x86_64-webgpu.readings.36.first_bucket_change.scale--> | 0.6160004<!--dcvc:gop.fabric:x86_64-webgpu.readings.36.first_bucket_change.edge--> | 1<!--dcvc:gop.fabric:x86_64-webgpu.readings.36.symbol_changed--> | yes |
|
||
| `arm64-webgpu` | 17 | 9663<!--dcvc:gop.fabric:arm64-webgpu.readings.17.first_bucket_change.element--> | 1.8644881<!--dcvc:gop.fabric:arm64-webgpu.readings.17.first_bucket_change.baseline_scale--> | 1.8644860<!--dcvc:gop.fabric:arm64-webgpu.readings.17.first_bucket_change.scale--> | 1.8644862<!--dcvc:gop.fabric:arm64-webgpu.readings.17.first_bucket_change.edge--> | 2558<!--dcvc:gop.fabric:arm64-webgpu.readings.17.symbol_changed--> | no |
|
||
| `arm64-webgpu` | 20 | 3805<!--dcvc:gop.fabric:arm64-webgpu.readings.20.first_bucket_change.element--> | 10.4411297<!--dcvc:gop.fabric:arm64-webgpu.readings.20.first_bucket_change.baseline_scale--> | 10.4411278<!--dcvc:gop.fabric:arm64-webgpu.readings.20.first_bucket_change.scale--> | 10.4411287<!--dcvc:gop.fabric:arm64-webgpu.readings.20.first_bucket_change.edge--> | 8225<!--dcvc:gop.fabric:arm64-webgpu.readings.20.symbol_changed--> | no |
|
||
| `arm64-webgpu` | 36 | 9454<!--dcvc:gop.fabric:arm64-webgpu.readings.36.first_bucket_change.element--> | 0.6159989<!--dcvc:gop.fabric:arm64-webgpu.readings.36.first_bucket_change.baseline_scale--> | 0.6160025<!--dcvc:gop.fabric:arm64-webgpu.readings.36.first_bucket_change.scale--> | 0.6160004<!--dcvc:gop.fabric:arm64-webgpu.readings.36.first_bucket_change.edge--> | 1<!--dcvc:gop.fabric:arm64-webgpu.readings.36.symbol_changed--> | yes |
|
||
| `x86_64-webgpu-software` | 17 | 9663<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.17.first_bucket_change.element--> | 1.8644881<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.17.first_bucket_change.baseline_scale--> | 1.8644859<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.17.first_bucket_change.scale--> | 1.8644862<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.17.first_bucket_change.edge--> | 2558<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.17.symbol_changed--> | no |
|
||
| `x86_64-webgpu-software` | 36 | 9454<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.36.first_bucket_change.element--> | 0.6159989<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.36.first_bucket_change.baseline_scale--> | 0.6160018<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.36.first_bucket_change.scale--> | 0.6160004<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.36.first_bucket_change.edge--> | 1<!--dcvc:gop.fabric:x86_64-webgpu-software.readings.36.symbol_changed--> | yes |
|
||
|
||
Each event starts from one scale on the wrong side of an edge, by a few parts in a million at
|
||
most, and one bucket. What follows differs, and on a WebGPU fabric a later event falls inside
|
||
what an earlier one left. So each is also read alone: `--force` gives `arm64-cpu`, the sink's
|
||
route, that one bucket and no other, and `event` reads the run against the same fabric's run
|
||
without it, so neither drift nor another event lies under the figures
|
||
(`dcvc-drift/readings/gop/events.json`). Each forced run repeated itself, and no frame before its
|
||
event moved.
|
||
|
||
| bucket forced at frame | wrong symbols | stream ends canonically | frames from the event to the last | of those, frames that differ | largest sample difference | most samples changed in a frame | lowest PSNR against the run without it | largest feature difference |
|
||
|---|---|---|---|---|---|---|---|---|
|
||
| 17 | 2,558<!--dcvc:gop.events.frame:17.summary.symbol_changes--> | no | 47<!--dcvc:gop.events.frame:17.summary.from_first_desync.frames--> | 47<!--dcvc:gop.events.frame:17.summary.from_first_desync.frames_changed--> | 255<!--dcvc:gop.events.frame:17.summary.from_first_desync.rgb_max--> | 192,920<!--dcvc:gop.events.frame:17.summary.from_first_desync.rgb_changed_max--> | 7.4 dB<!--dcvc:gop.events.frame:17.summary.from_first_desync.psnr_min_db--> | 9.0e+08<!--dcvc:gop.events.frame:17.summary.from_first_desync.feature_max--> |
|
||
| 20 | 8,225<!--dcvc:gop.events.frame:20.summary.symbol_changes--> | no | 44<!--dcvc:gop.events.frame:20.summary.from_first_desync.frames--> | 44<!--dcvc:gop.events.frame:20.summary.from_first_desync.frames_changed--> | 255<!--dcvc:gop.events.frame:20.summary.from_first_desync.rgb_max--> | 192,930<!--dcvc:gop.events.frame:20.summary.from_first_desync.rgb_changed_max--> | 7.3 dB<!--dcvc:gop.events.frame:20.summary.from_first_desync.psnr_min_db--> | 9.5e+08<!--dcvc:gop.events.frame:20.summary.from_first_desync.feature_max--> |
|
||
| 36 | 1<!--dcvc:gop.events.frame:36.summary.symbol_changes--> | yes | 28<!--dcvc:gop.events.frame:36.summary.from_first_desync.frames--> | 28<!--dcvc:gop.events.frame:36.summary.from_first_desync.frames_changed--> | 1<!--dcvc:gop.events.frame:36.summary.from_first_desync.rgb_max--> | 4,610<!--dcvc:gop.events.frame:36.summary.from_first_desync.rgb_changed_max--> | 64.4 dB<!--dcvc:gop.events.frame:36.summary.from_first_desync.psnr_min_db--> | 0.30<!--dcvc:gop.events.frame:36.summary.from_first_desync.feature_max--> |
|
||
|
||
The three are two kinds:
|
||
|
||
- **The coder's state is lost** (frames 17 and 20). Everything decoded after that element is noise,
|
||
with latents as large as 2.15e+09<!--dcvc:gop.fabric:x86_64-webgpu.readings.17.latent_max-->,
|
||
and the stream does not end canonically. The frame is lost, and so is every frame after it,
|
||
through the reference feature. The frames' y streams are independent, so a later frame's entropy
|
||
decode is right again, but its picture is not.
|
||
- **One symbol is wrong and the state survives** (frame 36). The stream ends canonically and nothing
|
||
in it shows that anything happened. The picture is not lost: read alone, this event moves no
|
||
sample by more than 1. It does not go away either. Every frame from it on differs, in
|
||
1,163<!--dcvc:gop.events.frame:36.readings.36.rgb_changed--> samples at its own frame and
|
||
4,587<!--dcvc:gop.events.frame:36.readings.63.rgb_changed--> at the last, and at the last frame
|
||
its feature difference is 0.094<!--dcvc:gop.events.frame:36.readings.63.feature_max-->, where
|
||
drift's is 7.5e-04<!--dcvc:gop.fabric:arm64-cpu.readings.63.feature_max-->.
|
||
|
||
On the fabrics that read these events, the first is frame 17's, so the first kind sets what they
|
||
show. On `x86_64-webgpu` that event and the frames after it make
|
||
47<!--dcvc:gop.fabric:x86_64-webgpu.summary.from_first_desync.frames--> frames, and every one
|
||
differs, by up to 255<!--dcvc:gop.fabric:x86_64-webgpu.summary.from_first_desync.rgb_max--> in a
|
||
sample and in as many as
|
||
192,930<!--dcvc:gop.fabric:x86_64-webgpu.summary.from_first_desync.rgb_changed_max--> of a frame's
|
||
196,608<!--dcvc:gop.samples_per_frame--> samples, and the PSNR against the baseline falls to
|
||
7.4<!--dcvc:gop.fabric:x86_64-webgpu.summary.from_first_desync.psnr_min_db--> dB at the worst.
|
||
Frame 36's event falls inside that, where only the forced run can read it. The sink's decode reads
|
||
the end of no y stream (`hyperprior::decode_yhat`), so it would report neither kind.
|
||
|
||
## 5. What the decoder can be offered as
|
||
|
||
The verifier derives each row from the readings (`record.json`, `verdicts`):
|
||
|
||
| property | sink's route, both instruction sets | every fabric read |
|
||
|---|---|---|
|
||
| Every frame byte for byte the baseline's (`decode-equivalence`) | does not hold | does not hold |
|
||
| Every symbol the encoder's (`entropy-consistency`) | held on this fixture | does not hold |
|
||
| One fabric repeats itself | held | held |
|
||
|
||
So the measurement supports what SPEC §7 says of `NeuralFloat`, and no more:
|
||
|
||
- **"Exact on this build" is what it can be offered as.** Each fabric repeated its own frames, and
|
||
a stream decoded on the encoder's own fabric gives the encoder's float image.
|
||
- **Not `replicate`, and no tier.** `decode-equivalence` fails between the two instruction sets
|
||
at the second frame (4<!--dcvc:gop.fabric:arm64-cpu.readings.1.rgb_changed--> samples of `arm64-cpu`'s frame 1 in `gop`).
|
||
`entropy-consistency` held on the sink's route here only because two CPUs gave one float for
|
||
every scale of this small network, which nothing in the capability promises, and it fails as
|
||
soon as the same graph runs on a GPU.
|
||
- **Across fabrics it is `perceptual` at best, and only until a desync loses the coder's state.**
|
||
Drift alone stays within 1 in a sample over this run. So does the event that cost one symbol,
|
||
read alone, though its error stays to the last frame. An event that loses the coder's state does
|
||
not degrade a picture a little: it ends the stream's usefulness until something resets the
|
||
reference feature, which nothing in this payload does. Both kinds start the same way, from one
|
||
scale past one edge (§4.5).
|
||
- **Nothing tells the sink.** The payload carries no check of a frame, and an event can end
|
||
canonically, so even a check of the stream's end, which the decode does not make, would miss it.
|
||
|
||
Holding the entropy stage takes what `entropy-consistency` asks: integer entropy arithmetic, or
|
||
parameters the codestream carries (determinism-tiers §2).
|
||
|
||
## 6. What this does not show
|
||
|
||
- **A trained model, or real pictures.** The rates here are those of a small seeded network on
|
||
noise. A trained model's scales and recurrence differ, and its entropy model may read the
|
||
reference feature, in which case a desync would not stay in its frame.
|
||
- **Rates.** 786,432 scales a fabric is one stream. The count of events is what happened
|
||
here; it is not an estimate of a rate on another stream, device or driver.
|
||
- **Every kind of event.** Three events were read alone, on one stream and one fabric. The two
|
||
kinds of §4.5 are what those three did, not a list of what an event can do, and the split
|
||
between them says nothing about how often either comes.
|
||
- **Every CPU.** Two instruction sets were read, one host each. The equal scales are a reading of
|
||
these two, and of the prebuilt runtime each linked.
|
||
- **Longer runs.** 64<!--dcvc:gop.frames--> frames, with no reset of the reference feature.
|
||
|
||
## 7. Verify
|
||
|
||
```sh
|
||
python3 -I research/dcvc-drift/verify.py --selftest research/dcvc-drift research/dcvc-drift.md
|
||
```
|
||
|
||
The verifier needs the standard library alone. It reads each fixture's envelope, decodes each
|
||
frame's y stream under the encoder's buckets, requires a canonical end and the encoder's symbols,
|
||
then decodes it again under the buckets each reading states wherever they differ, and from that
|
||
counts buckets, symbols, ends and events. `--selftest` then changes copies of the record and
|
||
requires each change to be refused for the reason it is planted for, with no help from this page:
|
||
a changed reading, one desync too many, one too few, a real event hidden, an event erased together
|
||
with its bucket and a forced run that does not repeat an event among them. The wai-conformance
|
||
step `dcvc-drift` runs exactly this, over committed files, with no runtime and no build.
|
||
|
||
What it establishes differs by field, and the float networks draw the line:
|
||
|
||
- **Recomputed from committed bytes:** every symbol, stream end and desync count under a reading's
|
||
buckets, each first bucket change against the scale table, and every summary and verdict.
|
||
- **Taken from the reading:** a fabric's scales, and so its buckets. Recomputing them takes that
|
||
fabric's float network, which the verifier does not run. It holds them to `latent_max`, because a
|
||
latent is a symbol plus a float mean: a wrong symbol moves it by a whole step and drift by far
|
||
less, so an event erased with its bucket is refused while its latent stands.
|
||
- **Held only to one another and to the files' digests:** the sample histograms and `rgb_changed`,
|
||
`rgb_max`, `rgb_sse` and `rgb_sha256`, because a frame's pixels are not committed; `feature_max`,
|
||
`feature_changed` and `feature_sha256`, likewise; `latent_max`, `scale_max` and their counts;
|
||
and `nodes`. A forced run's figures are of this class too: the verifier recomputes its symbols
|
||
and its end, and holds its digests to the run it is read against. A change to any one of them
|
||
alone is refused. A change made to all of a reading's
|
||
related fields together, with the digests here and in `record.json` written again, is not: such
|
||
a reading is shown false only by taking the run again.
|
||
|
||
`record.json`, which names every reading file by digest, has SHA-256
|
||
`730dc3d000d96247b05004ee9a04a0c64ba8c3b240b46f6eb3d6daf9488129ad`<!--dcvc-file:record.json-->.
|