Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 17 Packages Wiki Activity Actions
open-standards/wai/wai-rs/tests/vectors/int_hyper/natural_set.json
Transaction Science 3f9c3ef640 wai.neural.int_hyper and wai.neural.int_mlicpp: natural, sets trained on openly licensed natural pictures, held to one decode natively, through their lowerings, in WebAssembly and on GPUs 
int_hyper `natural` (wai-rs/tests/vectors/int_hyper/natural_*, record natural_set.json): a
mean-scale hyperprior, 64 synthesis and 64 latent channels, trained by
tools/wai_int_hyper_train.py at 661048492 from seed 20261112 on the 599 training pictures of
research/training-pictures/natural-1.json in 256 x 256 crops: 32,000 steps of 8, Adam 5e-4 then
5e-5 for the last quarter, lambda 0.0130, clip 1, deterministic algorithms, on the Linux x86_64
build host's integrated GPU (PCI 0x1002, gfx1151) through ROCm 7.2 and torch 2.14.0+rocm7.2, in
90 min (03:21:58-04:52:14Z); last logged step 0.5152 bpp at 26.23 dB (loss 2.53). Listing (its
identity) 3b3b56053917bf0b4a49b7011f4c725e3d413da865caec2b6bd57b953c115fb2. Coded picture: the
dead-leaves picture of that seed, stream 0, 128 x 128: 1,540 rANS bytes, 0.752 bpp, 18.74 dB,
decode 08b6bf5b...; WIH1 7922be10....

int_mlicpp `natural` (wai-rs/tests/vectors/int_mlicpp_natural/, record natural_set.json): 64
channels in 4 slices, a 48-channel hyper-latent, a 3 x 3 entropy network, trained by
tools/wai_int_mlicpp_train.py at 661048492 by the same recipe side by side on the same GPU, in 99
min (to 05:01:24Z); last logged loss 2.5833 (0.5009 bpp, 26.08 dB). model.wmb
27643187da6ba44cd5c8e3fc67ff4cf7e26026942414965cfcbd3a4b18bd4285 (1,988,549 B), listing
f37e2302bb2b26bb9a5483bf1f17e0210ecc2cf88b2b55e8b0aaaa6f4e79964b, prior id int_mlicpp/natural.
Coded picture: 1,512 rANS bytes, 0.7383 bpp, 18.65 dB, decode 1bbd9de6.... The synthetic vector
is unchanged under its own identity.

Both records read Open (check --open: tools at their blobs of 661048492, the manifest by its
digest under the licence rule); self-test 16 of 16 (int_hyper) and 14 of 14 (int_mlicpp).

Held: int-payload-conformance `natural` (WIH1); byte-exact goldens wih1_natural_set_is_golden and
int_mlicpp_golden::natural (by the sink's decode and through the lowering, reference and fast
kernels); wai-ops goldens (stage operators); ops-conformance certificates, both acc64, first
acc32 failure h_s input range at 43 bits, listed after the synthetic sets (committed_after) so
the index only gains entries; module corpus 123 cases; the browser crate's Node check; the GPU
capability test (ops_gpu.rs) with both mlicpp vectors. verifiers/trained-sets.sh runs the
manifest's check and self-test and both records' check --open and self-test.

Run through rb, macOS x86_64 (iMac): gens; wai_int_payload_vectors verify, wai-verify vectors
verify, wai_ops_check.py, int_payload_verify.py, corpus.py --check, all 0; wai-rs tests
int_hyper_ops, int_kernels_vectors, corpus_provenance, int_payload, int_mlicpp, int_mlicpp_ops
(release, no-fail-fast) and int_hyper_ops, int_payload (debug) 0; byte-exact-conformance
(release) 0; wai-ops tests 0; clippy -D warnings clean for neural_int and for the ops_gpu set
(no new lint in any other set: the union set's and byte-exact's and wai-ops's are pre-existing,
none in a touched line). Linux x86_64 (Framework): ops_gpu on gpu-vulkan-1002 and
swgpu-vulkan-10005, int_hyper 15 cases and int_mlicpp 2, graphs_differ 0, errors 0; the 48
test-picture payloads decode by wai_det_cost to the trainers' recorded digests.
Plants caught: the fast kernels' IGDN one off on 64-channel layers fails both trained sets'
goldens (3 tests) while 116 pass; one byte changed in leaves_g_s.bin and in int_mlicpp/model.wmb
(changed bytes under an existing set's name) each fail `wai breaking` as an unacknowledged wire
break; wai breaking against 88b0bdb4c passes (25 compatible changes).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 01:35:09 -05:00

488 lines
8.9 KiB
JSON
Raw Permalink Blame History

{
"format": "wai-int-hyper-set/1",
"capability": "wai.neural.int_hyper",
"about": "A wai.neural.int_hyper parameter set, its identity and where it came from (tools/wai_int_hyper_train.py writes and checks this record).",
"set": {
"entropy.json": {
"file": "natural_entropy.json",
"sha256": "e1e2d054fc4c40659eca20df364c42973d72372022503e0079a96ccfe53a93a7"
},
"g_s.whm": {
"file": "natural_g_s.bin",
"sha256": "8d571e86d6555b3e1b0d8a9e4afcd29d1e0e2f0f0b518e73e1eb43dde02a91dc"
},
"h_s.whm": {
"file": "natural_h_s.bin",
"sha256": "4d2be482912349570b10f3960f42626d79a77e227d3b820bcc5819adfaf4e321"
}
},
"listing_sha256": "3b3b56053917bf0b4a49b7011f4c725e3d413da865caec2b6bd57b953c115fb2",
"licence": "Apache-2.0",
"credits": "the training pictures' authors and licences, one a picture, in research/training-pictures/natural-1.json",
"provenance": {
"init": {
"from": "seed",
"seed": 20261112
},
"training": {
"content": "third-party",
"pictures": "research/training-pictures/natural-1.json",
"pictures_sha256": "c8af74063853fdc34c06478ed09d87f6fa78194a6f9bda692f2fb98db3123e53",
"split": "train",
"count": 599,
"licences": {
"by": 418,
"cc0": 168,
"pd": 13
},
"frames_sha256": "7d20861aac9fd112bbba54c8f091bca96b48d43cfaa6d21eb41d85d74fccbac0",
"prepare": "shorter side resized to 512 (Lanczos), centred crop to 512 x 512, RGB8; Pillow 12.3.0",
"input": "the 599 train pictures of research/training-pictures/natural-1.json, each under a licence its page states and the rule admits, credited there"
},
"coded": {
"content": "generated",
"generator": "tools/wai_int_synth_real.py",
"input": "the dead-leaves picture the tool generates from seed 20261112, stream 0",
"seed": 20261112,
"stream": 0
},
"recipe": {
"architecture": "mean-scale hyperprior (the exporter's topology)",
"n": 64,
"m": 64,
"side": 256,
"steps": 32000,
"batch": 8,
"lr": 0.0005,
"aux_lr": 0.001,
"lr_drop": "to a tenth for the last quarter of the steps",
"lambda": 0.013,
"loss": "lambda * 255^2 * MSE + bits per pixel; gradient norm clipped to 1",
"order_seed": 20261112,
"threads": 8,
"device": {
"kind": "gpu",
"name": "AMD Radeon 8060S",
"arch": "gfx1151",
"hip": "7.2.53211",
"cuda": null
},
"deterministic_algorithms": true,
"python": "3.11.17",
"torch": "2.14.0+rocm7.2",
"numpy": "1.26.4",
"compressai": "1.2.8"
},
"commit": "661048492afbee341062391036ea6121b6f62ab7",
"tools": {
"tools/wai_int_hyper_train.py": "32d8068d4673c77a3742a325a9ca612127925957",
"tools/wai_int_synth_real.py": "b556ac49a9fea4657c155ea126d526e56d9b1d9a",
"tools/wai_sota_int_synth_export.py": "f99c3cfe717acd4365e6990e174058ecb5d9e306",
"tools/wai_int_training_pictures.py": "54da4c43eaf3a4a91bff2daf55d37e24b662ffb2"
},
"float_model_sha256": "880ef5238adaceb193cbd92ead6b016d2b71ade04b19c0325cad4ec7120ca75e",
"weights_sha256": "58cf1e3f8df97755cd20c8c5f1ca8540d5340e6efe9d826863dcbc3ab1a3068d",
"log": [
{
"step": 0,
"bpp": 0.1297,
"psnr_db": 7.08,
"aux": 2633.75
},
{
"step": 500,
"bpp": 0.3777,
"psnr_db": 19.76,
"aux": 2206.6
},
{
"step": 1000,
"bpp": 0.4644,
"psnr_db": 19.54,
"aux": 1436.65
},
{
"step": 1500,
"bpp": 0.4182,
"psnr_db": 23.18,
"aux": 680.48
},
{
"step": 2000,
"bpp": 0.4176,
"psnr_db": 22.81,
"aux": 534.57
},
{
"step": 2500,
"bpp": 0.4604,
"psnr_db": 23.53,
"aux": 862.94
},
{
"step": 3000,
"bpp": 0.4573,
"psnr_db": 23.4,
"aux": 1184.58
},
{
"step": 3500,
"bpp": 0.5631,
"psnr_db": 22.86,
"aux": 1415.01
},
{
"step": 4000,
"bpp": 0.5342,
"psnr_db": 23.71,
"aux": 1631.43
},
{
"step": 4500,
"bpp": 0.5771,
"psnr_db": 23.44,
"aux": 1736.08
},
{
"step": 5000,
"bpp": 0.4728,
"psnr_db": 24.98,
"aux": 1768.7
},
{
"step": 5500,
"bpp": 0.5398,
"psnr_db": 23.24,
"aux": 1933.37
},
{
"step": 6000,
"bpp": 0.5673,
"psnr_db": 22.37,
"aux": 1784.32
},
{
"step": 6500,
"bpp": 0.5113,
"psnr_db": 24.05,
"aux": 1779.34
},
{
"step": 7000,
"bpp": 0.5881,
"psnr_db": 22.9,
"aux": 1517.31
},
{
"step": 7500,
"bpp": 0.5454,
"psnr_db": 24.72,
"aux": 1562.74
},
{
"step": 8000,
"bpp": 0.5551,
"psnr_db": 23.44,
"aux": 1470.59
},
{
"step": 8500,
"bpp": 0.4798,
"psnr_db": 26.74,
"aux": 1209.66
},
{
"step": 9000,
"bpp": 0.4414,
"psnr_db": 24.81,
"aux": 1182.33
},
{
"step": 9500,
"bpp": 0.589,
"psnr_db": 23.18,
"aux": 983.75
},
{
"step": 10000,
"bpp": 0.5728,
"psnr_db": 23.94,
"aux": 779.78
},
{
"step": 10500,
"bpp": 0.5342,
"psnr_db": 24.57,
"aux": 644.36
},
{
"step": 11000,
"bpp": 0.5193,
"psnr_db": 24.84,
"aux": 375.23
},
{
"step": 11500,
"bpp": 0.5805,
"psnr_db": 24.14,
"aux": 244.86
},
{
"step": 12000,
"bpp": 0.3962,
"psnr_db": 27.43,
"aux": 235.96
},
{
"step": 12500,
"bpp": 0.5271,
"psnr_db": 25.63,
"aux": 230.47
},
{
"step": 13000,
"bpp": 0.6874,
"psnr_db": 22.21,
"aux": 201.84
},
{
"step": 13500,
"bpp": 0.5372,
"psnr_db": 26.09,
"aux": 194.1
},
{
"step": 14000,
"bpp": 0.5445,
"psnr_db": 25.04,
"aux": 213.83
},
{
"step": 14500,
"bpp": 0.4165,
"psnr_db": 26.78,
"aux": 193.95
},
{
"step": 15000,
"bpp": 0.4949,
"psnr_db": 24.94,
"aux": 231.44
},
{
"step": 15500,
"bpp": 0.4865,
"psnr_db": 26.14,
"aux": 219.15
},
{
"step": 16000,
"bpp": 0.557,
"psnr_db": 23.62,
"aux": 219.26
},
{
"step": 16500,
"bpp": 0.4725,
"psnr_db": 25.81,
"aux": 211.41
},
{
"step": 17000,
"bpp": 0.5298,
"psnr_db": 24.52,
"aux": 223.45
},
{
"step": 17500,
"bpp": 0.5502,
"psnr_db": 25.33,
"aux": 204.74
},
{
"step": 18000,
"bpp": 0.4801,
"psnr_db": 26.44,
"aux": 209.7
},
{
"step": 18500,
"bpp": 0.5425,
"psnr_db": 24.67,
"aux": 198.64
},
{
"step": 19000,
"bpp": 0.5018,
"psnr_db": 26.47,
"aux": 171.1
},
{
"step": 19500,
"bpp": 0.4985,
"psnr_db": 25.05,
"aux": 163.62
},
{
"step": 20000,
"bpp": 0.5287,
"psnr_db": 24.15,
"aux": 164.98
},
{
"step": 20500,
"bpp": 0.4561,
"psnr_db": 25.94,
"aux": 163.97
},
{
"step": 21000,
"bpp": 0.511,
"psnr_db": 24.96,
"aux": 130.31
},
{
"step": 21500,
"bpp": 0.4726,
"psnr_db": 25.56,
"aux": 139.64
},
{
"step": 22000,
"bpp": 0.4201,
"psnr_db": 27.17,
"aux": 102.16
},
{
"step": 22500,
"bpp": 0.5139,
"psnr_db": 24.62,
"aux": 119.0
},
{
"step": 23000,
"bpp": 0.5901,
"psnr_db": 22.52,
"aux": 107.86
},
{
"step": 23500,
"bpp": 0.5282,
"psnr_db": 26.2,
"aux": 86.49
},
{
"step": 24000,
"bpp": 0.4606,
"psnr_db": 26.75,
"aux": 65.37
},
{
"step": 24500,
"bpp": 0.6026,
"psnr_db": 22.57,
"aux": 49.55
},
{
"step": 25000,
"bpp": 0.486,
"psnr_db": 25.15,
"aux": 39.87
},
{
"step": 25500,
"bpp": 0.5406,
"psnr_db": 25.75,
"aux": 26.83
},
{
"step": 26000,
"bpp": 0.5833,
"psnr_db": 23.71,
"aux": 19.87
},
{
"step": 26500,
"bpp": 0.466,
"psnr_db": 23.85,
"aux": 10.69
},
{
"step": 27000,
"bpp": 0.4958,
"psnr_db": 25.01,
"aux": 10.61
},
{
"step": 27500,
"bpp": 0.4433,
"psnr_db": 24.27,
"aux": 10.74
},
{
"step": 28000,
"bpp": 0.5864,
"psnr_db": 24.27,
"aux": 10.86
},
{
"step": 28500,
"bpp": 0.4423,
"psnr_db": 26.88,
"aux": 10.93
},
{
"step": 29000,
"bpp": 0.4262,
"psnr_db": 28.84,
"aux": 10.57
},
{
"step": 29500,
"bpp": 0.5314,
"psnr_db": 25.56,
"aux": 10.19
},
{
"step": 30000,
"bpp": 0.4868,
"psnr_db": 25.4,
"aux": 10.09
},
{
"step": 30500,
"bpp": 0.4996,
"psnr_db": 25.56,
"aux": 10.09
},
{
"step": 31000,
"bpp": 0.5138,
"psnr_db": 25.9,
"aux": 9.45
},
{
"step": 31500,
"bpp": 0.522,
"psnr_db": 26.09,
"aux": 8.69
},
{
"step": 31999,
"bpp": 0.5152,
"psnr_db": 26.23,
"aux": 8.42
}
]
},
"decode": {
"width": 128,
"height": 128,
"rgb_file": "natural_rgb.bin",
"rgb_sha256": "08b6bf5b93406cc9e17e487f564dacf0dde048114e0a3faf7375bfac42172be7",
"rgb_fnv1a": "7dce7099becc4a15",
"payload_rans_bytes": 1540,
"bpp": 0.752,
"psnr_db": 18.74
}
}
Reference in a new issue View git blame Copy permalink