22928147f3
SPEC §3.1 defines the pin a manifest carries (conditioning.prior, and a WAI2 rendition's prior): id, sha256, determinism, and the optional form, tier and supersedes. The digest covers the whole parameter set. A set of several files is pinned through its listing, the new registered prior form wai.prior.set. A sink verifies the pin before it decodes, reports a refusal with one of an ordered set of snake_case reasons, and decodes only from the set it verified. Tier is a closed vocabulary; supersedes is informative. A pin that does not resolve makes its capability unsupported for that envelope: SPEC §4 dispatch continues at the fallback, §7.1 selection at the next rendition, and an inert envelope's error names the reasons. A rendition entry may carry id, sha256 and, on a companion, companion_of_id. SPEC §7 gains a Digests subsection, and §8 says how a MINOR revision adds an optional member. In wai-rs: the manifest types are #[non_exhaustive] with builders; a new pin module verifies pins in memory and on disk; Wai::dispatch_verified, WaiMulti::pick and primary check pins and component digests; every neural decoder reads through ParamSource; a codec-free `ffi` feature carries the C envelope surface with wai_envelope_select and wai_neural_decode_ex. wai-web wraps wai::container. New vectors: the int_mlicpp set-form twin and the pin-conformance corpus with an independent Python verifier, plus sha256.json sidecars for the vectors of capabilities that decode against a parameter set. Squashed from wai/stream-a-parameter-set-identity onto link 12. Conflicts are resolved to keep both sides; the integration commits that follow make the two streams agree. Co-Authored-By: Claude Opus 5.5 <[email protected]>
72 lines
2.9 KiB
Python
72 lines
2.9 KiB
Python
#!/usr/bin/env python3
|
|
"""Write the set-form twin of the `wai.neural.int_mlicpp` conformance vector.
|
|
|
|
The committed `stream.wai` pins `model.wmb` form-less: its pin's `sha256` is the
|
|
SHA-256 of the one file. SPEC.md §3.1 also lets a pin cover a parameter set through
|
|
its listing (the prior form `wai.prior.set`). This tool writes, next to the vector:
|
|
|
|
model.sha256 the listing of the one-file set: the line
|
|
`<sha256 of model.wmb> model.wmb` and a line feed -- the text
|
|
`sha256sum model.wmb` prints, which `sha256sum -c` checks.
|
|
stream_set.wai `stream.wai` with the same payload, its pin replaced by one over
|
|
the listing: `sha256` = SHA-256(model.sha256), `form`
|
|
`wai.prior.set`, `tier` `decode-equivalence`.
|
|
|
|
Standard library only, and independent of the Rust reference implementation, which
|
|
checks these files in tests/int_mlicpp.rs and src/pin.rs.
|
|
|
|
python tools/wai_int_mlicpp_set_form.py [vector-dir]
|
|
|
|
`vector-dir` defaults to wai-rs/tests/vectors/int_mlicpp. `wai_int_mlicpp_export.py`
|
|
calls `write_set_form` after it writes the vector.
|
|
"""
|
|
import hashlib
|
|
import json
|
|
import os
|
|
import struct
|
|
import sys
|
|
|
|
|
|
def listing(files):
|
|
"""The listing of `files` (path -> bytes): sorted by path as bytes, one
|
|
`<64 hex> <path>\\n` line per file."""
|
|
out = b''
|
|
for path in sorted(files, key=lambda p: p.encode()):
|
|
out += hashlib.sha256(files[path]).hexdigest().encode() + b' ' + path.encode() + b'\n'
|
|
return out
|
|
|
|
|
|
def write_set_form(vector_dir):
|
|
with open(os.path.join(vector_dir, 'model.wmb'), 'rb') as f:
|
|
wmb = f.read()
|
|
with open(os.path.join(vector_dir, 'stream.wai'), 'rb') as f:
|
|
env = f.read()
|
|
assert env[:4] == b'WAI1'
|
|
ml = struct.unpack('<I', env[4:8])[0]
|
|
manifest = json.loads(env[8:8 + ml])
|
|
rest = env[8 + ml:]
|
|
lst = listing({'model.wmb': wmb})
|
|
prior = manifest['conditioning']['prior']
|
|
assert prior['sha256'] == hashlib.sha256(wmb).hexdigest(), 'stream.wai pins model.wmb'
|
|
prior['sha256'] = hashlib.sha256(lst).hexdigest()
|
|
prior['form'] = 'wai.prior.set'
|
|
prior['tier'] = 'decode-equivalence'
|
|
mb = json.dumps(manifest, separators=(',', ':')).encode()
|
|
set_env = b'WAI1' + struct.pack('<I', len(mb)) + mb + rest
|
|
with open(os.path.join(vector_dir, 'model.sha256'), 'wb') as f:
|
|
f.write(lst)
|
|
with open(os.path.join(vector_dir, 'stream_set.wai'), 'wb') as f:
|
|
f.write(set_env)
|
|
return lst, set_env
|
|
|
|
|
|
def main():
|
|
here = os.path.dirname(os.path.abspath(__file__))
|
|
d = sys.argv[1] if len(sys.argv) > 1 else os.path.join(here, '..', 'wai-rs', 'tests', 'vectors', 'int_mlicpp')
|
|
lst, set_env = write_set_form(d)
|
|
print(f'model.sha256 {len(lst)} B sha256 {hashlib.sha256(lst).hexdigest()}; '
|
|
f'stream_set.wai {len(set_env)} B sha256 {hashlib.sha256(set_env).hexdigest()}')
|
|
|
|
|
|
if __name__ == '__main__':
|
|
main()
|