fdce7d9172
The neural feature pulled `codecs`, dragging the AV1/JXL/opus/FLAC C chain (meson/ninja/nasm + dev libs) into any neural build — including the neural-e2e CI job, which failed building libdav1d-sys. But the Rust neural decoders need only `ort`; nothing in src/neural uses crate::codecs. The two couplings were incidental: the neural C FFI (ffi_neural) wraps the codec FFI's shared buffer/error types, and zstd-safe needed its `std` feature (which codecs' zstd was enabling transitively). Fix both: gate ffi_neural on all(neural, codecs) — the Rust decoders stay available with just `neural`, only the C FFI wrapper needs the codec FFI — and give zstd-safe `features=["std"]` directly. neural now drops `codecs`. So the neural-e2e job builds `--no-default-features --features neural`: no system codec deps, just ort. All 6 reference decoders (elic/mlicpp/glc/dcvc/ snac) reproduce their references end-to-end without the C chain. Both codecs+neural and neural-alone build clean. Co-Authored-By: Claude Opus 4.8 <[email protected]>
56 lines
2.7 KiB
YAML
56 lines
2.7 KiB
YAML
name: WAI neural decoders end-to-end
|
|
|
|
# The float neural decoders (elic / mlicpp / glc / dcvc / snac) cannot make
|
|
# WAI's cross-architecture byte-equality promise — their ONNX synthesis is
|
|
# float, so the low bits differ by hardware (this is exactly why the
|
|
# integer-exact int_synth path exists, and why only it is in the byte-equality
|
|
# matrix). What they CAN be is *correct end-to-end* on a real machine: entropy
|
|
# decode → ONNX synthesis → image assembly, reproducing a reference within ORT
|
|
# float rounding (±4). This job proves that, with no committed model blobs:
|
|
# the committed tool builds minimal synthetic CompressAI-compatible models +
|
|
# bitstreams, and the Rust decoders reproduce each reference. It runs only when
|
|
# the neural decode code or the generator changes — not on every cargo class.
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
paths:
|
|
- 'wai/wai-rs/src/neural/**'
|
|
- 'wai/wai-rs/tests/neural_native_decode.rs'
|
|
- 'wai/tools/wai_validate_synthetic.py'
|
|
- '.github/workflows/wai-neural-e2e.yml'
|
|
pull_request:
|
|
paths:
|
|
- 'wai/wai-rs/src/neural/**'
|
|
- 'wai/wai-rs/tests/neural_native_decode.rs'
|
|
- 'wai/tools/wai_validate_synthetic.py'
|
|
- '.github/workflows/wai-neural-e2e.yml'
|
|
|
|
jobs:
|
|
neural-e2e:
|
|
name: synthetic fixtures → Rust decode → reference (±4)
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
- uses: actions/setup-python@v5
|
|
with:
|
|
python-version: '3.11'
|
|
- name: ML stack (CPU torch + torchvision + compressai + onnxruntime)
|
|
run: |
|
|
python -m pip install --upgrade pip
|
|
# torch AND torchvision from the CPU index so their compiled ops match
|
|
# (compressai imports torchvision; a mismatched build fails at
|
|
# `torchvision::nms does not exist`).
|
|
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
|
|
# `onnx` (the exporter) is separate from `onnxruntime` (the decoder).
|
|
pip install compressai onnx onnxruntime numpy pillow
|
|
- name: Build the synthetic fixtures (elic / mlicpp / snac / dcvc / glc)
|
|
working-directory: wai
|
|
run: python tools/wai_validate_synthetic.py --size 256
|
|
- name: Rust 1.96.0
|
|
run: rustup toolchain install 1.96.0 --profile minimal && rustup default 1.96.0
|
|
- name: Each Rust decoder reproduces its reference end-to-end
|
|
working-directory: wai/wai-rs
|
|
# --no-default-features: the neural decoders need only `ort`, NOT the C
|
|
# codec chain (AV1/JXL/opus), so the job needs no system codec deps.
|
|
run: cargo test --no-default-features --features neural --test neural_native_decode -- _matches_reference --nocapture
|