wai: the integer decode paths hold no floating point in their source, and read no JSON with a float parser #635
Loading…
Reference in a new issue
No description provided.
Delete branch "wai/det-source-fixes"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What this is
The integer capabilities decode to the same bytes on every machine because no step of their decode rounds. This change takes floating point out of the source of every integer decode path. It also reads the parameter sets' JSON with the strict reader, which converts no number to a float. A test of the sources keeps both true. No payload format and no corpus file changes.
Merge order. This branch sits on #632's commit,
946f3beda, which moves changelog entries intowai/changes/and the conformance workflow's lists into step files. Merge #632 first; until it merges, this pull request's diff also shows #632's change.What it carries. One commit,
e8f8e0033: the change ofwai/det-source-fixes(152aab5c7..9c4159958) in #632's layout. Its message lists the source commits. The branch's changelog entry is a file ofwai/changes/. It has no edit to the conformance workflow. Every other file is as the source branch has it.What it fixes
(n as f64).sqrt()), and the wide vector kernel took eight at a time with a vector square root; an integer correction then decided the result. The seed is now a fixed-point reciprocal square root, formed exactly inu64andu128and within one of the root: a table on the norm's top bits and two Newton steps. The vector kernel takes its roots digit by digit in integer lanes. The roots are the same integers.WIS1ranges are read, checked and written as binary32 bits (int_payload::wire::Binary32), andlo ≤ hiis compared as IEEE 754 compares two finite values, in integer arithmetic. The rule (integer-payloads §6) is unchanged.splat.json) is read from its text with the strict JSON reader: its integers as integer spellings, and each range byint_splat::binary32_ofin exact integer arithmetic. integer-payloads §10 now says how a number denotes a binary32 value. This is how the independent converter,int_payload_verify.py, reads a number. The general JSON reader used before disagreed with the converter in two places: it kept the sign of the integer-0, and it rounded an integer pastu64.splat_intno longer turns on that reader'sfloat_roundtripfeature for the whole crate;splat_exact, which reads an asset'sPOSITIONbounds as JSON numbers, does.int_entropy::fixed_scale_edgesandquantize_scaleform⌊x·2^frac⌋from the binary32 bits in integers, giving the values the binary64 product and its floor gave.entropy.jsonofwai.neural.int_hyperandwai.video.int_hyper, integer-payloads §9, and aWMB1bundle's tables, §14) is read by the strict reader, each integer from its spelling. §9 now says how the text is read. Two readings change.-0is zero, as §9's integer rule already gave. A member named twice, or a value more than 64 levels deep, refuses the set (model_mismatch). The committed sets read as before.IntDecoded::Attr's ranges andWis1::rangesare(Binary32, Binary32)bits.CondEntropy::try_from_valueis replaced byCondEntropy::read, which takes the file's bytes.SplatJson::rangesgives binary32 values.SplatJson::from_value,int_splat::decodeandint_payload::wis1_from_jsontake the file's bytes in place of a parsed value. The entry file names each with its migration.What the checks hold
float_free, a test of the sources (a library test underneural_int, run with the crate's unit tests). It fails on a floating-point type, literal or vector intrinsic in any module on the integer decode paths, outside tests and model-checker harnesses. Those modules are the kernels, the operator-set bridge, the payloads, the integer codecs, the rANS coder and its bucketing, the strict JSON reader, and the other integer entropy decoders and the decodes built on them. It reads every file of the operator set (wai-ops) too. It also fails on a call of the general JSON reader's parser in those modules, but for two named lines that are not decodes. Everyint_*file, and every file underint_kernels/andint_payload/, must be among those it reads, so a new one fails it until it is listed. It reads source, not a compiled decode.u64::isqrt. The test covers the edges ofu64: 0, 1,u64::MAX, and everyk² − 1,k²andk² + 1around the powers of two and across the range. An ignored test checks every boundary forkbelow2^32.frac.1and the next binary64 value, which reads as binary64 rounds it on each side of the tie. It is also tested at two spellings one last digit apart (4194304.0and4194304.5,8.388608e6and8.388609e6), which decode to the two adjacent values the payload presents.decode_intand through aWMB1bundle.How it was verified
This commit,
e8f8e0033.scripts/wai-conformance-lists.py selftestandcheck,scripts/wai-conformance-lists-selftest.sh,scripts/check-changelog-fragment.pyagainst main,scripts/changelog-assemble.py --checkandscripts/changelog-fragments-selftest.shall pass. Rebasing from the conversion's first base, #632's earlier tip 14f1abb7c, onto946f3bedaleft every file's change the same.wai/wai-rs,cargo check --locked --no-default-features --testswith each ofneural_int,splat_intandsplat_exact; and inwai/byte-exact-conformance,cargo check --locked --tests. All pass.git merge-tree). Merging all of them in forward, reverse and a mixed order gives one tree. On the build host, that tree passes these runs:cargo check --testsunder the derivedWAI_FEATURESand underWAI_PQ_FEATURES;cargo check --testsofquickstart,app; the catalogue's library tests; and thewaibinary's unit tests.The tests above,
float_freeamong them, and the corpora's regeneration did not run on this commit here; their results come from this pull request's CI.On the CI hosts. Results on the CI runners come from this pull request's own CI run. None of the runs above is a CI result.
🤖 Generated with Claude Code