open-standards/wai/research/dcvc-drift.md

Drift and desync of wai.neural.dcvc_rt, per frame and fabric

Status. A measured record and its procedure, informative. Every figure below is read from the readings in dcvc-drift/, and dcvc-drift/verify.py, which shares no code with tools/dcvc-drift, the tool that took them, decodes every y stream again with an rANS decoder of its own, recounts every desync event under the buckets each reading states and checks every figure this page quotes (each is followed by a dcvc marker naming its place); §7 says what it recomputes and what it can only hold together. A statement of how a run went, with nothing committed behind it, is marked (as run). Measured on 6 November 2026. wai.neural.dcvc_rt is NeuralFloat (SPEC §7): it claims no tier, and this record measures what that leaves a sink with. It changes no registration.

1. What is measured, and what is not

wai.neural.dcvc_rt decodes frame after frame: an entropy decode whose per-element Gaussian bucket comes from a float network, hyper_synthesis, then a float temporal decoder that also reads the reference feature the frame before it left (neural/video_dcvc.rs). Two things can go wrong across machines, and this record counts them apart, per frame and per fabric:

1. Drift. The float networks round differently, so a frame's samples and its reference feature move, and the feature carries the movement into the next frame.
2. Desync. A scale lands on the other side of a bucket edge from where the encoder had it. The entropy stage then decodes with a distribution the encoder did not use.

Not measured here, and why:

• A trained DCVC-RT model. The repository holds none. The model is the seeded one that tools/wai_validate_synthetic.py builds for the end-to-end job, with random weights, so its frames are noise and no figure here speaks to picture quality. Its entropy model is a plain hyperprior that reads nothing from the reference feature, so here a frame's entropy decode stands alone. The rates of drift and desync are this model's; what carries over is the mechanism.
• A neural processor. No reading of one was kept. The compute plan the platform's machine-learning provider in the runtime reported took none of the transposed convolutions, which hold nearly all of both graphs' arithmetic, and it gave the same figures whether asked for the GPU or the neural processor (as run).
• wai.neural.dcvc_fm, which decodes through the same file, was not run.
• Time and energy. Neither was read.

2. Terms

• Fabric. Where a decode's graphs run: an instruction set and a device, through a provider of the runtime, with a thread count (§3.3).
• The sink's route. The decode the reference sink runs: wai::neural::decode_envelope, whose sessions come from ort_session::load_deterministic, on the CPU provider at one thread.
• Baseline. The sink's route on the fabric that encoded the fixture.
• Drift is read against the baseline: the 8-bit samples that differ, the histogram of their differences, and the largest difference in the reference feature.
• A desync event is a frame whose entropy decode gives any symbol other than the one the encoder coded. Only the entropy stage is read for it, so drift alone never counts as one.

3. Procedure

3.1 Fixtures

tools/dcvc-drift/make_fixture.py runs the end-to-end job's generator: its model, seed, export, encoder and session settings. Both fixtures, with the sets they decode against, are committed under dcvc-drift/fixture/, because a stream's bytes depend on the machine that encoded it and every fabric has to read one stream. Each was encoded once, on the x86_64 fabric.

• ci is the job's own fixture: 3 frames at 256×256, its weights untouched. --check-harness also runs the job's own generator and requires the two to write one envelope and one reference. On the arm64 host they did, and that envelope is the one committed here, byte for byte (as run). Most of its samples are clamped, and a clamped sample shows no difference.
• gop is the same model and the same motion for 64 frames, with two changes that a long run needs. The job's weights grow the reference feature by about the square root of 3 a frame, which saturates a run of this length, so the reference feature's weights are scaled by 0.6. The feature's RMS then rises slowly, with no saturation: 5.31 at the first frame, 13.02 at frame 47 and 14.13 at the last. The image head is scaled by 0.008 and centred on 0.5, so that samples stay in range and a difference can reach them. The two gains came from a sweep of the encoder's own figures, which is not committed (as run); fixture.json states each frame's RMS.

python make_fixture.py --out ci --frames 3 --check-harness
python make_fixture.py --out gop --frames 64 --ref-gain 0.6 --image-gain 0.008

The stack is tools/requirements-sota-int.txt's torch, compressai and numpy, with onnxruntime 1.28.0, the version the decoder links. fixture.json states it, and each file's SHA-256. Run twice on the arm64 host, the script wrote the stream, the set, the reference and the buckets byte for byte the same, and the gop stream it wrote there is not the committed one (as run). A fixture is regenerated only with this stack on the fabric that encoded it; every step below reads what is committed.

3.2 A run

dcvc-drift run decodes a fixture on one fabric. It decodes the stream twice through a loop of its own that keeps what the sink's decode throws away, each element's scale, bucket and symbol, the latent and the feature, and the reading says whether the two passes agreed. On the sink's route it also decodes twice through decode_envelope itself, and refuses to write a reading unless the two are one output and the loop's frames are that output byte for byte. load_deterministic is private to the decoder, so the loop builds its sessions with the same settings and that comparison is what holds it to them.

cargo build --release --locked                     # CPU fabrics
cargo build --release --locked --features webgpu   # WebGPU fabrics
xz -dk fixture/<set>/encoder-buckets.bin.xz
dcvc-drift run --fixture <set> --fabric <name> [--threads N] [--force F:E:B] --out <run>
dcvc-drift run --fixture <set> --fabric <name> --provider webgpu --adapter <class> --out <run>
dcvc-drift record --fixture <set> --run <baseline> --run <run> ... --out readings/<set>
dcvc-drift event --fixture <set> --against <run> --run <forced run> ... --out readings/<set>/events.json
dcvc-drift finish --root research/dcvc-drift --set ci --set gop --measured <date>

The CPU fabrics ran on the first build and the WebGPU fabrics on the second. The webgpu feature links the provider's shared library beside the binary and gives the binary a search path to that directory, and on macOS the second build runs from where cargo put it (as run). record reads encoder-buckets.bin and the per-frame digests in fixture.json, and no other file of the encoder's. It takes no copy of the encoder's reference: the baseline's frames, rounded half to even, must have those digests or it writes nothing, so the baseline has to be a run on the fabric that encoded the fixture, and a run on any other fabric is read against it. --force gives one element of one frame a bucket whatever its scale says, and changes nothing else; event reads such a run against the unforced run of its own fabric, so no drift lies under the reading (§4.5). Each run ran under a memory limit on its host.

3.3 Fabrics

fabric	instruction set	operating system	graphs run on
x86_64-cpu	x86_64 with AVX-512	Linux 7.0	CPU provider, one thread: the sink's route, and the baseline
x86_64-cpu-t8	as above	Linux 7.0	CPU provider, eight threads
arm64-cpu	arm64	macOS 27.2	CPU provider, one thread: the sink's route
arm64-cpu-t8	arm64	macOS 27.2	CPU provider, eight threads
x86_64-webgpu	x86_64, integrated GPU	Linux 7.0	WebGPU provider, over Vulkan
arm64-webgpu	arm64, integrated GPU	macOS 27.2	WebGPU provider, over the platform's GPU interface
x86_64-webgpu-software	x86_64 CPU	Linux 7.0	WebGPU provider, over a software Vulkan driver

The WebGPU fabrics, and any thread count but one, are sessions no sink's decode opens. They run in the loop alone, held to nothing, and they show what happens if a sink puts these graphs elsewhere. The runtime's own profile, kept in the readings as nodes, shows every node of both graphs on the named provider. On Linux each WebGPU run was pinned to one Vulkan driver with VK_DRIVER_FILES, and the process mapped that driver; on macOS the run loaded the platform's GPU driver (as run). run writes the adapter's class, which --adapter gives as the host's own tools report it, and the driver its process mapped, and record carries both into a reading; the readings committed here were taken without them. The readings come from the tool as it first stood. Every fabric was run again with the tool as committed, and every frame, feature and node was the same; on x86_64, record over the four new runs wrote each fabric's reading again field for field, the adapter and the driver aside (as run). No reading names a device.

4. Results

4.1 The encoder's fabric

On the baseline the decoder reproduces the encoder's float image exactly: rounded the encoder's way, every frame of both fixtures is the encoder's reconstruction byte for byte. The encoder rounds to 8 bits half to even and the decoder half away from zero, so the two differ only on exact ties: 56 samples over 38 of the gop frames, each by 1, and 0 in ci. The tables below read from the baseline, so no tie counts as drift.

4.2 ci

fabric	graphs run on	frames equal to the baseline, of 3	desync events	first	before the first event: largest sample difference	most samples changed in a frame	largest feature difference
x86_64-cpu	CPU, one thread: the sink's route, and the baseline	3	0	none	0	0	0
x86_64-cpu-t8	CPU, eight threads	3	0	none	0	0	0
arm64-cpu	CPU, one thread: the sink's route	1	0	none	1	7	3.8e-05
arm64-cpu-t8	CPU, eight threads	1	0	none	1	9	5.3e-05
x86_64-webgpu	integrated GPU, WebGPU over Vulkan	0	0	none	1	30	2.1e-04
arm64-webgpu	integrated GPU, WebGPU over the platform's GPU interface	0	0	none	1	30	2.1e-04
x86_64-webgpu-software	CPU, WebGPU over a software Vulkan driver	0	0	none	1	31	2.3e-04

4.3 gop

fabric	graphs run on	frames equal to the baseline, of 64	desync events	first	before the first event: largest sample difference	most samples changed in a frame	largest feature difference
x86_64-cpu	CPU, one thread: the sink's route, and the baseline	64	0	none	0	0	0
x86_64-cpu-t8	CPU, eight threads	64	0	none	0	0	0
arm64-cpu	CPU, one thread: the sink's route	2	0	none	1	42	8.7e-04
arm64-cpu-t8	CPU, eight threads	1	0	none	1	45	1.0e-03
x86_64-webgpu	integrated GPU, WebGPU over Vulkan	0	3	17	1	26	2.9e-04
arm64-webgpu	integrated GPU, WebGPU over the platform's GPU interface	0	3	17	1	26	2.9e-04
x86_64-webgpu-software	CPU, WebGPU over a software Vulkan driver	0	2	17	1	28	3.5e-04

Two passes of every fabric agreed with each other, so each fabric repeats itself. Eight threads gave the one-thread frames on x86_64 and did not on arm64. The two GPUs gave one another's frames and features byte for byte, on both fixtures, across two hosts and two GPU interfaces, and the software driver, running the same provider, did not (record.json, fabrics_with_equal_frames).

4.4 Drift by frame

On the sink's route across the two instruction sets, drift is never a desync: 0 of the 64 frames hold a scale that differs, so every scale is one float, bit for bit, on both, and the symbols and latents with it. Only the temporal decoder moves, carried by the reference feature. The difference grows over the run, with variation from frame to frame: arm64-cpu is back to 0 samples changed at frame 5, and its largest feature difference peaks at 8.7e-04 at frame 60, not at the last:

frame	arm64-cpu: samples changed	arm64-cpu-t8: samples changed	arm64-cpu: largest feature difference	arm64-cpu-t8: largest feature difference
0	0	0	9.5e-06	1.6e-05
7	5	4	3.4e-05	6.6e-05
15	7	5	7.7e-05	8.8e-05
31	14	23	1.8e-04	2.7e-04
47	19	27	3.5e-04	4.0e-04
63	42	41	7.5e-04	7.3e-04

The samples that change do so by 1. By the last frame the largest feature difference is 7.5e-04, on a feature whose RMS is 14.1, and nothing in the stream brings it back: the payload has no frame that resets the reference feature.

4.5 Desync events

Each of the 786,432 scales of gop chooses a bucket. On the CPU fabrics every choice was the encoder's. On every WebGPU fabric some were not, and each of them changes every scale a little (64 frames of 64):

fabric	frame	element	scale, baseline	scale, this fabric	table edge	wrong symbols	stream ends canonically
x86_64-webgpu	17	9663	1.8644881	1.8644860	1.8644862	2558	no
x86_64-webgpu	20	3805	10.4411297	10.4411278	10.4411287	8225	no
x86_64-webgpu	36	9454	0.6159989	0.6160025	0.6160004	1	yes
arm64-webgpu	17	9663	1.8644881	1.8644860	1.8644862	2558	no
arm64-webgpu	20	3805	10.4411297	10.4411278	10.4411287	8225	no
arm64-webgpu	36	9454	0.6159989	0.6160025	0.6160004	1	yes
x86_64-webgpu-software	17	9663	1.8644881	1.8644859	1.8644862	2558	no
x86_64-webgpu-software	36	9454	0.6159989	0.6160018	0.6160004	1	yes

Each event starts from one scale on the wrong side of an edge, by a few parts in a million at most, and one bucket. What follows differs, and on a WebGPU fabric a later event falls inside what an earlier one left. So each is also read alone: --force gives arm64-cpu, the sink's route, that one bucket and no other, and event reads the run against the same fabric's run without it, so neither drift nor another event lies under the figures (dcvc-drift/readings/gop/events.json). Each forced run repeated itself, and no frame before its event moved.

bucket forced at frame	wrong symbols	stream ends canonically	frames from the event to the last	of those, frames that differ	largest sample difference	most samples changed in a frame	lowest PSNR against the run without it	largest feature difference
17	2,558	no	47	47	255	192,920	7.4 dB	9.0e+08
20	8,225	no	44	44	255	192,930	7.3 dB	9.5e+08
36	1	yes	28	28	1	4,610	64.4 dB	0.30

The three are two kinds:

• The coder's state is lost (frames 17 and 20). Everything decoded after that element is noise, with latents as large as 2.15e+09, and the stream does not end canonically. The frame is lost, and so is every frame after it, through the reference feature. The frames' y streams are independent, so a later frame's entropy decode is right again, but its picture is not.
• One symbol is wrong and the state survives (frame 36). The stream ends canonically and nothing in it shows that anything happened. The picture is not lost: read alone, this event moves no sample by more than 1. It does not go away either. Every frame from it on differs, in 1,163 samples at its own frame and 4,587 at the last, and at the last frame its feature difference is 0.094, where drift's is 7.5e-04.

On the fabrics that read these events, the first is frame 17's, so the first kind sets what they show. On x86_64-webgpu that event and the frames after it make 47 frames, and every one differs, by up to 255 in a sample and in as many as 192,930 of a frame's 196,608 samples, and the PSNR against the baseline falls to 7.4 dB at the worst. Frame 36's event falls inside that, where only the forced run can read it. The sink's decode reads the end of no y stream (hyperprior::decode_yhat), so it would report neither kind.

5. What the decoder can be offered as

The verifier derives each row from the readings (record.json, verdicts):

property	sink's route, both instruction sets	every fabric read
Every frame byte for byte the baseline's (decode-equivalence)	does not hold	does not hold
Every symbol the encoder's (entropy-consistency)	held on this fixture	does not hold
One fabric repeats itself	held	held

So the measurement supports what SPEC §7 says of NeuralFloat, and no more:

• "Exact on this build" is what it can be offered as. Each fabric repeated its own frames, and a stream decoded on the encoder's own fabric gives the encoder's float image.
• Not replicate, and no tier. decode-equivalence fails between the two instruction sets at the second frame (4 samples of arm64-cpu's frame 1 in gop). entropy-consistency held on the sink's route here only because two CPUs gave one float for every scale of this small network, which nothing in the capability promises, and it fails as soon as the same graph runs on a GPU.
• Across fabrics it is perceptual at best, and only until a desync loses the coder's state. Drift alone stays within 1 in a sample over this run. So does the event that cost one symbol, read alone, though its error stays to the last frame. An event that loses the coder's state does not degrade a picture a little: it ends the stream's usefulness until something resets the reference feature, which nothing in this payload does. Both kinds start the same way, from one scale past one edge (§4.5).
• Nothing tells the sink. The payload carries no check of a frame, and an event can end canonically, so even a check of the stream's end, which the decode does not make, would miss it.

Holding the entropy stage takes what entropy-consistency asks: integer entropy arithmetic, or parameters the codestream carries (determinism-tiers §2).

6. What this does not show

• A trained model, or real pictures. The rates here are those of a small seeded network on noise. A trained model's scales and recurrence differ, and its entropy model may read the reference feature, in which case a desync would not stay in its frame.
• Rates. 786,432 scales a fabric is one stream. The count of events is what happened here; it is not an estimate of a rate on another stream, device or driver.
• Every kind of event. Three events were read alone, on one stream and one fabric. The two kinds of §4.5 are what those three did, not a list of what an event can do, and the split between them says nothing about how often either comes.
• Every CPU. Two instruction sets were read, one host each. The equal scales are a reading of these two, and of the prebuilt runtime each linked.
• Longer runs. 64 frames, with no reset of the reference feature.

7. Verify

python3 -I research/dcvc-drift/verify.py --selftest research/dcvc-drift research/dcvc-drift.md

The verifier needs the standard library alone. It reads each fixture's envelope, decodes each frame's y stream under the encoder's buckets, requires a canonical end and the encoder's symbols, then decodes it again under the buckets each reading states wherever they differ, and from that counts buckets, symbols, ends and events. --selftest then changes copies of the record and requires each change to be refused for the reason it is planted for, with no help from this page: a changed reading, one desync too many, one too few, a real event hidden, an event erased together with its bucket and a forced run that does not repeat an event among them. The wai-conformance step dcvc-drift runs exactly this, over committed files, with no runtime and no build.

What it establishes differs by field, and the float networks draw the line:

• Recomputed from committed bytes: every symbol, stream end and desync count under a reading's buckets, each first bucket change against the scale table, and every summary and verdict.
• Taken from the reading: a fabric's scales, and so its buckets. Recomputing them takes that fabric's float network, which the verifier does not run. It holds them to latent_max, because a latent is a symbol plus a float mean: a wrong symbol moves it by a whole step and drift by far less, so an event erased with its bucket is refused while its latent stands.
• Held only to one another and to the files' digests: the sample histograms and rgb_changed, rgb_max, rgb_sse and rgb_sha256, because a frame's pixels are not committed; feature_max, feature_changed and feature_sha256, likewise; latent_max, scale_max and their counts; and nodes. A forced run's figures are of this class too: the verifier recomputes its symbols and its end, and holds its digests to the run it is read against. A change to any one of them alone is refused. A change made to all of a reading's related fields together, with the digests here and in record.json written again, is not: such a reading is shown false only by taking the run again.

record.json, which names every reading file by digest, has SHA-256 730dc3d000d96247b05004ee9a04a0c64ba8c3b240b46f6eb3d6daf9488129ad.