wai-rs: e2e — recover the hyper-latent z from the real codestream (byte-exact) #20
2026-07-17
wai-rs: e2e — recover the hyper-latent z from the real codestream (byte-exact)
First stage chained straight off the bitstream rather than a fixture: parse the container, take the SOZ substream's bytes, and run the byte-exact mANS `fac` (factorized) decoder over it with the normative per-channel CDFs. Symbols are offset by 31 (the mid-point of the 63-symbol alphabet) to give signed z. SOZ carries both planes back to back — luma (160 ch) then chroma (96 ch), 16 samples per channel — decoded by ONE MansDecoder whose ANS state threads across the two calls. Validated on test256.bin: z[y] 2560/2560 exact (75 non-zero, range [-3,4]) and z[uv] 1536/1536 exact, both equal to the reference z that decode_hyper_scale consumes. This joins the two halves of the decoder: the container/entropy path now reproduces, from raw bytes, the exact tensor the neural scale path was validated against. Co-Authored-By: Claude Opus 4.8 <[email protected]>