Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions

wai-rs: jpegai_pipeline::sigma_index — close the scale path (byte-exact, both planes) #5

Merged
dcharlot merged 1 commit from jpegai-sigma-index into main 2026-07-12 12:22:40 -04:00
Conversation 0 Commits 1 Files changed 2 +100
dcharlot commented 2026-07-12 12:22:39 -04:00
Owner
Copy link

Derives the per-position mANS distribution index from the hyper-scale decoder output, connecting decode_hyper_scale to the sgm entropy decode:

sigma[c,p] = clamp( (i_s[c,p] + gain_per_ch[c] + rvs[c,p] + 64) >> 7, 0, 31 )

The +64 >>7 is a round-to-nearest-128 done as an arithmetic right shift (floor) — the same reference semantics as the hyper_scale division.

Recovered from the reference decoder fixtures: sigma is NOT a pure function of i_s (the ranges overlap), but becomes an exact one once the per-channel gain and per-position residual-variance scale are added — at which point the boundaries snap to clean multiples of 128.

Byte-exact vs the reference entropy fixtures on real test256.bin, both planes: luma (SORP) 40960/40960, chroma (SORS) 24576/24576 — matching the indexes the reference entropy coder actually consumed. Plus a unit test pinning the 128-boundaries, floor-on-negatives, and the [0,31] clamp.

This completes the entropy+scale chain byte-exact end to end: z → hyper_scale → i_s → sigma_index → mANS sgm → y_q → dequant → resi.

Derives the per-position mANS distribution index from the hyper-scale decoder output, connecting decode_hyper_scale to the sgm entropy decode: ``` sigma[c,p] = clamp( (i_s[c,p] + gain_per_ch[c] + rvs[c,p] + 64) >> 7, 0, 31 ) ``` The `+64 >>7` is a round-to-nearest-128 done as an arithmetic right shift (floor) — the same reference semantics as the hyper_scale division. Recovered from the reference decoder fixtures: sigma is NOT a pure function of i_s (the ranges overlap), but becomes an exact one once the per-channel gain and per-position residual-variance scale are added — at which point the boundaries snap to clean multiples of 128. **Byte-exact vs the reference entropy fixtures on real test256.bin, both planes:** luma (SORP) 40960/40960, chroma (SORS) 24576/24576 — matching the `indexes` the reference entropy coder actually consumed. Plus a unit test pinning the 128-boundaries, floor-on-negatives, and the [0,31] clamp. This completes the entropy+scale chain byte-exact end to end: z → hyper_scale → i_s → sigma_index → mANS sgm → y_q → dequant → resi.
Derives the per-position mANS distribution index from the hyper-scale decoder's
output, connecting decode_hyper_scale to the sgm entropy decode:

  sigma[c,p] = clamp( (i_s[c,p] + gain_per_ch[c] + rvs[c,p] + 64) >> 7, 0, 31 )

The `+64 >> 7` is a round-to-nearest-128 done as an arithmetic right shift (floor) —
the same reference-decoder semantics as the hyper_scale power-of-two division. The
form was recovered from the reference decoder's own fixtures: sigma is not a function
of i_s alone (the ranges overlap), but becomes an exact pure function once the
per-channel gain and per-position residual-variance scale are added.

Validated BYTE-EXACT against the reference decoder's entropy fixtures on real
test256.bin data, both planes: luma (SORP) 40960/40960 and chroma (SORS) 24576/24576
positions match the `indexes` the reference entropy coder actually consumed. Plus a
unit test pinning the 128-boundaries, the floor-on-negatives, and the [0,31] clamp.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Transaction-Science/open-standards!5
No description provided.