wai-rs: jpegai_pipeline::sigma_index — close the scale path (byte-exact, both planes) #5

Merged
dcharlot merged 1 commit from jpegai-sigma-index into main 2026-07-12 12:22:40 -04:00

2026-07-10

Transaction Science
wai-rs: jpegai_pipeline::sigma_index — close the scale path (byte-exact, both planes)
Derives the per-position mANS distribution index from the hyper-scale decoder's
output, connecting decode_hyper_scale to the sgm entropy decode:

  sigma[c,p] = clamp( (i_s[c,p] + gain_per_ch[c] + rvs[c,p] + 64) >> 7, 0, 31 )

The `+64 >> 7` is a round-to-nearest-128 done as an arithmetic right shift (floor) —
the same reference-decoder semantics as the hyper_scale power-of-two division. The
form was recovered from the reference decoder's own fixtures: sigma is not a function
of i_s alone (the ranges overlap), but becomes an exact pure function once the
per-channel gain and per-position residual-variance scale are added.

Validated BYTE-EXACT against the reference decoder's entropy fixtures on real
test256.bin data, both planes: luma (SORP) 40960/40960 and chroma (SORS) 24576/24576
positions match the `indexes` the reference entropy coder actually consumed. Plus a
unit test pinning the 128-boundaries, the floor-on-negatives, and the [0,31] clamp.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-07-10 07:19:49 -04:00