- engine::job: a job planned, run, sealed with the host's key and returned
in the service's form (run), its check (verify), a key's did:key (did_of)
and a program's lint. The command line now calls these; the browser
binding calls the same.
- engine::draw: the circuit drawer, moved from the command line.
- The engine runs on wasm32-unknown-unknown: a result there carries no
elapsed_ms (a tab has no clock std reads), and the cut-rank order search
runs its seeds in turn where there are no threads, finding the same orders.
- A program that measures in the middle on the state vector is admitted
with the job's memory margin counted once, as trajectories count it, not
once per state: two 256 MiB margins had left no room in 512 MiB, and the
service refused a qubit it could hold.
- wai-web, feature quantum_engine (off by default, for the IDE's bundle):
QuantumEngine (run, plan, methods, signer_did) signing with the page's
key, and engine_lint, engine_draw and engine_verify. Tested natively and
in wasm32 under Node, where a job runs and verifies, a changed result is
refused, and every method a tab holds runs; wai-web-wasm.yml runs both
with their counts.
- wai-rs: feature quantum_engine re-exports the engine as wai::engine.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit aa516abdb1)
1023 lines
53 KiB
Rust
1023 lines
53 KiB
Rust
//! WAI envelope parser + capability dispatch for the browser.
|
||
//!
|
||
//! What this crate IS:
|
||
//! - A wasm-bindgen surface over the standard's container format
|
||
//! (`WAI1` magic + length-prefixed manifest + length-prefixed payload)
|
||
//! - A capability menu (the registered `wai.*` strings) the JS shim
|
||
//! uses to route to its handler map
|
||
//!
|
||
//! What this crate is NOT:
|
||
//! - A backend for the standard codecs. For those, a sink decodes each
|
||
//! capability with the decoder it registered for it (WASM, WebGPU, WebNN,
|
||
//! native ONNX, a JS library, custom); the deployer picks, and the JS shim
|
||
//! routes to that handler. The integer-exact and deterministic classes below
|
||
//! are decoded here, by the same Rust native sinks run.
|
||
|
||
use wasm_bindgen::prelude::*;
|
||
|
||
// Interactive worlds (extensions/interactive-worlds): the WorldSink —
|
||
// the SAME `wai` engine native sinks run, bound for the browser.
|
||
pub mod world;
|
||
pub mod audio;
|
||
pub mod haptic;
|
||
pub mod feed;
|
||
pub mod avatar;
|
||
pub mod splat4d;
|
||
pub mod score;
|
||
pub mod provenance;
|
||
pub mod film;
|
||
pub mod reel;
|
||
// Integer-exact neural decode (int_synth + int_entropy) — pure i64, ORT-free,
|
||
// so it runs in the browser with no ML runtime; one of the capability classes this
|
||
// crate decodes itself rather than dispatching out.
|
||
pub mod neural_int;
|
||
|
||
// Deterministic byte-exact generation (wai.generate.spectral) — pure i64, no ML
|
||
// runtime; the create-side of the integer contract, generated in-browser with a
|
||
// portable hash.
|
||
pub mod generate;
|
||
|
||
// The payloads SPEC §5 pins to the integer codecs (WIV1, WIH1, WIA1, WIS1), decoded
|
||
// through wai::int_payload::decode_int: the shared half of decodeWiv1, decodeWih1,
|
||
// decodeWia1 and decodeWis1.
|
||
mod int_payload;
|
||
|
||
// Byte-exact integer inter-frame VIDEO (wai.video.int_motion / int_hyper) — pure
|
||
// i64, no ML runtime; frame-state-equivalence, decoded + per-frame-verified in-page.
|
||
pub mod video;
|
||
|
||
// Browser sink for the OTT workflow (wai.video.manifest + wai.video.ssai):
|
||
// render a real manifest, verify its receipt in the tab, swap a creative to
|
||
// break it, and watch per-viewer energy amortise as the class dedups.
|
||
pub mod video_ott;
|
||
|
||
// Byte-exact LEARNED AUDIO codec (wai.audio.int_codec) — pure i64, no ML runtime;
|
||
// sample-equivalence, decoded to int16 PCM + hash-verified + played in-page.
|
||
pub mod audio_int;
|
||
|
||
// Byte-exact INTERACTIVE WORLD MODEL (wai.world.model integer renderer) — pure i64,
|
||
// no ML runtime; arrow-key-driven, every frame hash-verified, the signed action
|
||
// trace replaying to one portable session hash (replay-equivalence).
|
||
pub mod int_world;
|
||
|
||
// Byte-exact LEARNED GAUSSIAN-SPLAT codec (wai.splat.int_codec) — pure i64, no ML
|
||
// runtime; the decoded [14,S,S] attribute tensor is hash-verified in-page before a
|
||
// sink-supplied rasterizer presents the cloud (splat-equivalence).
|
||
pub mod int_splat;
|
||
|
||
// Deterministic quantum-circuit SIMULATION (wai.quantum.circuit2, and the frozen
|
||
// wai.quantum.circuit) — the SAME `wai::quantum` engine native sinks run, compiled
|
||
// to wasm32. A WQC gate op-log, in the phase table its magic names,
|
||
// is classically simulated in the fixed-point complex floor; the reconstructed
|
||
// statevector hashes byte-identically to the native reference
|
||
// (statevector-equivalence). Pure i64, no float, no ML runtime, no quantum
|
||
// hardware — a quantum computation made a transportable, receiptable object.
|
||
pub mod quantum;
|
||
|
||
// Quantum-operations ATTESTATION (wai.quantum.calibration / wai.quantum.job) — the
|
||
// verify-half applied to the quantum-control industry, in-browser: seal a signed,
|
||
// joule-metered, grant-authorized receipt; tamper it and watch verify() fail; blow
|
||
// the joule budget and watch verify() fail; bind a job to its calibration state.
|
||
// Gate ⊗ meter ⊗ receipt, demonstrated live. Reuses world Merkle + Ed25519.
|
||
pub mod quantum_ops;
|
||
|
||
// Autonomous CALIBRATION (wai.quantum.cal.*) — the same deterministic Rust a native
|
||
// sink runs against its device simulator, compiled to wasm32: run an autonomous
|
||
// tune-up (Rabi / Ramsey-T2 / RB / readout), watch it recover a hidden device
|
||
// parameter byte-exactly, and seal a wai.quantum.calibration receipt.
|
||
pub mod quantum_cal;
|
||
|
||
// Error MITIGATION (wai.quantum.mitigate) — the classical post-processing layer
|
||
// between raw QPU counts and the reported number, in-browser: zero-noise
|
||
// extrapolation, tensored readout inversion, and classical shadows recover a
|
||
// hidden ideal expectation from a deterministically-simulated noisy front-end,
|
||
// then seal a signed, joule-metered MitigationReceipt over that step.
|
||
pub mod quantum_mitigate;
|
||
|
||
// qLDPC DECODING (wai.quantum.decode) — a fixed-point Relay-BP (min-sum) decoder
|
||
// for quantum LDPC codes (toric + bivariate-bicycle), in-browser: decode a
|
||
// sampled syndrome byte-exactly, watch relay legs recover decodes plain BP leaves
|
||
// failed on a real qLDPC code, and seal a signed, joule-metered DecodeReceipt carrying joules-per-decode.
|
||
pub mod quantum_qec;
|
||
|
||
// FILTER-FUNCTION control + NOISE SPECTROSCOPY (wai.quantum.cal.*, frequency
|
||
// domain) — in-browser: reconstruct a hidden noise PSD from a DD filter bank, and
|
||
// reshape a DD sequence to push its filter's stopband over the noise, byte-exactly.
|
||
// Both fully offline, each sealing a wai.quantum.calibration receipt.
|
||
pub mod quantum_control;
|
||
|
||
// NOISE LEARNING (wai.quantum.cal.noise_learn) — Cycle Benchmarking + CER of a
|
||
// sparse Pauli–Lindblad model, in-browser: recover a hidden device's Pauli error
|
||
// rates from a simulated twirled-cycle experiment, seal the signed model, and
|
||
// watch a PEC of a Pauli observable bind that exact model (the loop close).
|
||
pub mod quantum_noise;
|
||
|
||
// Circuit COMPILATION + equivalence proof (wai.quantum.compile) — in-browser:
|
||
// route a random Clifford circuit onto a coupling graph, prove the routed circuit
|
||
// is equivalent to the source via a stabilizer-tableau check, seal a signed
|
||
// CompileReceipt, and watch the same check reject a tampered circuit.
|
||
pub mod quantum_compile;
|
||
|
||
// Neutral-atom REARRANGEMENT (wai.quantum.rearrange) — in-browser: load a
|
||
// stochastic ~half-full trap array and rearrange it into a defect-free target
|
||
// register by a Hungarian / LSAP solver at minimum total move distance, seal the
|
||
// signed plan, and watch the same check reject a tampered plan.
|
||
pub mod quantum_atom;
|
||
|
||
// The WAI Quantum IDE (wai.quantum.toolchain) — a full quantum development
|
||
// environment in the browser: algorithm library + state analysis + backend
|
||
// selection + OpenQASM interop over the byte-exact simulator, every run receipted.
|
||
pub mod quantum_toolchain;
|
||
|
||
// Variational quantum ML (wai.quantum.vml) — a Quantum Circuit Born Machine trained
|
||
// on the classical state vector (exact parameter-shift gradient) + a barren-plateau
|
||
// probe: the interference-based ML the thermodynamic pivot left behind, in-browser.
|
||
pub mod quantum_vml;
|
||
|
||
// Phasor-feature classifier (wai.quantum.kernel) — the quantum kernel as random
|
||
// Fourier / FHRR phasor features + a ridge classifier, in-browser.
|
||
pub mod quantum_kernel;
|
||
|
||
// Learning from quantum data (wai.quantum.qdata) — the exponential sample-complexity
|
||
// separation for Pauli-channel learning (Bell/quantum-memory vs entanglement-free),
|
||
// the advantage behind noise_learn's data-efficiency, in-browser.
|
||
pub mod quantum_qdata;
|
||
|
||
// The phasor bridge (wai.quantum.phasor) — a quantum feature-map model is a Fourier
|
||
// series of phasors; the identical function computes on the state vector, on a
|
||
// classical FHRR phasor sum, and on integer-quantized qFHRR phasors.
|
||
pub mod quantum_phasor;
|
||
|
||
// Quantum Bill of Materials (wai.quantum.qbom) — in-browser: assemble the stage
|
||
// receipts of one computation into one signed provenance manifest, with a root and
|
||
// a total-energy roll-up, and show a swapped stage break the bill.
|
||
pub mod quantum_qbom;
|
||
|
||
// The fabric's WebGPU member in a page (wai.quantum.fabric): the browser's adapter
|
||
// opened and probed, the gate kernel certified on it against the CPU reference's
|
||
// pinned corpus digests, and WQC containers run to their statevector hash. Only in
|
||
// the separate `quantum_webgpu` bundle; the main bundle does not carry it.
|
||
#[cfg(all(feature = "quantum_webgpu", target_arch = "wasm32"))]
|
||
mod quantum_webgpu;
|
||
|
||
// The job service's engine in the page: plan, run and sign a job in the tab, and
|
||
// check any job's receipt there, the service's included. The IDE's bundle only.
|
||
#[cfg(feature = "quantum_engine")]
|
||
pub mod quantum_engine;
|
||
|
||
// Staged delivery (extensions/staged-delivery.md): wai::staged::StageSink, the sink a
|
||
// native player runs, bound for JS — bases added, refinement layers offered, holds
|
||
// expired and bases removed in any order, every unit's stage, producer, tier, output
|
||
// and stage hash read back, and a layer's declared cost read before it is offered.
|
||
pub mod stage;
|
||
|
||
// Staged-delivery measurement (extensions/staged-measurement.md): wai::stage_measure, the
|
||
// code the native measurement tool runs, bound for JS — a title's counted bytes, quality
|
||
// integers, first presentable bytes, layering overhead, deduplication and output tree, the
|
||
// delivery simulator, and the reconstruct-session claim a browser sink seals over what it
|
||
// presented (jwp-receipts §2.4.12), with its verifier.
|
||
pub mod measure;
|
||
|
||
|
||
// ---- Capability menu ----
|
||
// The capabilities this browser sink advertises: a subset of the registry
|
||
// (wai-rs/src/codecs, published as wai/registry/capabilities.json), not the whole
|
||
// registry and not the native SDK's menu. Exported so JS can match capability
|
||
// strings without typing them from memory. A CI job checks that every string here
|
||
// is registered and that none is a candidate (SPEC §5).
|
||
#[wasm_bindgen]
|
||
pub fn known_capabilities() -> Box<[JsValue]> {
|
||
[
|
||
// image
|
||
"wai.image.png", "wai.image.jpeg", "wai.image.avif", "wai.image.jxl",
|
||
// audio
|
||
"wai.audio.opus", "wai.audio.flac",
|
||
// video
|
||
"wai.video.av1", "wai.video.av1.lossless",
|
||
// text
|
||
"wai.text.zstd", "wai.text.xz",
|
||
// neural (sink-installed model) — see SPEC.md §5 "Neural capabilities"
|
||
"wai.neural.encodec32", // EnCodec, 32 kHz audio
|
||
"wai.neural.dac", // DAC, 44.1 kHz music
|
||
"wai.neural.mimi", // Mimi, real-time speech
|
||
"wai.neural.wavtokenizer", // WavTokenizer, ultra-low bitrate audio
|
||
"wai.neural.bmshj2018", // bmshj2018-factorized image codec
|
||
"wai.neural.video_bmshj2018", // per-frame bmshj2018 video, browser-decodable
|
||
"wai.neural.glc", // Generative Latent Coding, ultra-low bpp images
|
||
// Inter-frame neural video (DCVC-RT/FM) is a native-sink capability —
|
||
// it carries a temporal net + reference-feature state too heavy for
|
||
// this browser sink. Per SPEC §5, a browser sink declares only
|
||
// `video_bmshj2018` for the video class, so it is not advertised here.
|
||
].iter().map(|s| JsValue::from_str(s)).collect()
|
||
}
|
||
|
||
/// Whether `cap` is a registered **candidate** capability: registered, but with no
|
||
/// payload format pinned, so an encoder MUST NOT emit it and a sink treats it as
|
||
/// unsupported (SPEC §5, *Candidate capabilities*). The registry's own predicate,
|
||
/// `wai::codecs::is_candidate`.
|
||
#[wasm_bindgen(js_name = isCandidate)]
|
||
pub fn is_candidate(cap: &str) -> bool {
|
||
wai::codecs::is_candidate(cap)
|
||
}
|
||
|
||
/// Whether `cap` is a string the registry registers as a payload or companion
|
||
/// capability (`wai::codecs::is_known_capability`).
|
||
#[wasm_bindgen(js_name = isRegistered)]
|
||
pub fn is_registered(cap: &str) -> bool {
|
||
wai::codecs::is_known_capability(cap)
|
||
}
|
||
|
||
// ---- Envelopes: the reference implementation's types, wrapped ----
|
||
//
|
||
// The parser is `wai::container` itself, so every manifest member it models —
|
||
// `role`, `companion_of`, component identity, the parameter-set pin — reaches JS,
|
||
// and selection is SPEC §7.1's, not a second implementation of it. The pure-Rust
|
||
// half (`core`) is what the native tests run; the `#[wasm_bindgen]` wrappers below
|
||
// only convert to and from JS values.
|
||
|
||
/// The envelope logic with no JS types, so it runs (and is tested) natively.
|
||
pub mod core {
|
||
use std::collections::BTreeMap;
|
||
|
||
use wai::container::{no_parameter_sets, PinRefusal, Prior, Wai, WaiMulti};
|
||
use wai::pin::{verify_bytes, HeldBytes, READABLE_FORMS};
|
||
|
||
/// One parameter set a sink holds: a file, or a listing with the files below
|
||
/// its root.
|
||
#[derive(Debug, Clone)]
|
||
pub enum Held {
|
||
File(Vec<u8>),
|
||
Set { listing: Vec<u8>, files: BTreeMap<String, Vec<u8>> },
|
||
}
|
||
|
||
/// The parameter sets a sink holds, by capability.
|
||
pub type HeldSets = BTreeMap<String, Vec<Held>>;
|
||
|
||
pub fn parse_v1(bytes: &[u8]) -> Result<Wai, String> {
|
||
Wai::from_bytes(bytes).map_err(|e| e.to_string())
|
||
}
|
||
|
||
pub fn parse_v2(bytes: &[u8]) -> Result<WaiMulti, String> {
|
||
WaiMulti::from_bytes(bytes).map_err(|e| e.to_string())
|
||
}
|
||
|
||
/// SPEC §3.1 verification of `prior` over `capability` against the sets held
|
||
/// for it: `Ok(index)` of the set that satisfied it, or the refusal.
|
||
pub fn verify(capability: &str, prior: &Prior, held: &HeldSets) -> Result<usize, PinRefusal> {
|
||
let sets = held.get(capability).map(Vec::as_slice).unwrap_or(&[]);
|
||
let views: Vec<BTreeMap<String, &[u8]>> = sets.iter()
|
||
.map(|h| match h {
|
||
Held::Set { files, .. } => files.iter().map(|(k, v)| (k.clone(), v.as_slice())).collect(),
|
||
Held::File(_) => BTreeMap::new(),
|
||
})
|
||
.collect();
|
||
let hb: Vec<HeldBytes<'_>> = sets.iter().zip(views.iter())
|
||
.map(|(h, v)| match h {
|
||
Held::File(b) => HeldBytes::File(b),
|
||
Held::Set { listing, .. } => HeldBytes::Set { listing, files: v },
|
||
})
|
||
.collect();
|
||
verify_bytes(capability, prior, &hb, READABLE_FORMS).map(|r| match r {
|
||
wai::pin::VerifiedRef::File { index } | wai::pin::VerifiedRef::Set { index, .. } => index,
|
||
})
|
||
}
|
||
|
||
/// The reason a pin check gives: `ok` or SPEC §3.1's reason.
|
||
pub fn check_pin(prior_json: &str, capability: &str, held: &HeldSets) -> Result<String, String> {
|
||
let prior: Prior = serde_json::from_str(prior_json).map_err(|e| format!("pin: {e}"))?;
|
||
Ok(match verify(capability, &prior, held) {
|
||
Ok(_) => "ok".into(),
|
||
Err(r) => r.code().into(),
|
||
})
|
||
}
|
||
|
||
/// What JS sees of each `WAI2` entry: every member the manifest carries, plus
|
||
/// `selectable` (whether SPEC §7.1 may present it as media).
|
||
pub fn renditions_meta(m: &WaiMulti) -> Vec<serde_json::Value> {
|
||
m.manifest.renditions.iter().map(|r| {
|
||
let mut v = serde_json::to_value(r).unwrap_or(serde_json::Value::Null);
|
||
if let Some(o) = v.as_object_mut() {
|
||
o.insert("selectable".into(), r.is_selectable().into());
|
||
}
|
||
v
|
||
}).collect()
|
||
}
|
||
|
||
/// SPEC §7.1: the picked index, or every entry's index and reason.
|
||
pub fn pick(m: &WaiMulti, advertised: &[&str], held: &HeldSets) -> Result<(usize, Vec<(usize, String)>), Vec<(usize, String)>> {
|
||
let reasons = |s: Vec<(usize, wai::container::Skip)>| s.into_iter().map(|(i, k)| (i, k.to_string())).collect::<Vec<_>>();
|
||
match m.pick(|c| advertised.contains(&c), |c, p| verify(c, p, held)) {
|
||
Ok(p) => Ok((p.index, reasons(p.skipped))),
|
||
Err(s) => Err(reasons(s)),
|
||
}
|
||
}
|
||
|
||
/// The companions applied with the entry at `index` (by id, or by capability).
|
||
pub fn companions_of_index(m: &WaiMulti, index: usize) -> Vec<usize> {
|
||
m.companions_of_index(index).into_iter().map(|(i, _, _)| i).collect()
|
||
}
|
||
|
||
/// SPEC §4 for a `WAI1` envelope: the capability that decodes it here, or why
|
||
/// it is inert. Only a capability `wai::codecs::is_dispatchable` accepts is
|
||
/// taken (SPEC §4): never a companion-only capability, a candidate, a record or
|
||
/// a derivation.
|
||
pub fn dispatch_capability(w: &Wai, advertised: &[&str], held: &HeldSets) -> Result<String, String> {
|
||
w.dispatch_verified(|c| advertised.contains(&c), |c, p| verify(c, p, held))
|
||
.map(|d| d.capability.to_string())
|
||
.map_err(|e| e.to_string())
|
||
}
|
||
|
||
/// The payload of entry `i` if SPEC §7.1 may present it as media: a selectable
|
||
/// entry whose payload matches its `sha256`, when it states one.
|
||
pub fn rendition_payload(m: &WaiMulti, i: usize) -> Result<&[u8], String> {
|
||
let meta = m.manifest.renditions.get(i)
|
||
.ok_or_else(|| format!("rendition index {i} out of range (have {})", m.renditions.len()))?;
|
||
if !meta.is_selectable() {
|
||
return Err(format!("entry {i} ({}) is not a selectable rendition: a companion, or a \
|
||
capability that is not dispatchable (SPEC \u{a7}4: companion-only, \
|
||
a candidate, a record or a derivation), is never presented as \
|
||
media", meta.capability));
|
||
}
|
||
checked_payload(m, i)
|
||
}
|
||
|
||
/// The payload of entry `i` if it is a companion whose payload matches its
|
||
/// `sha256`, when it states one. A companion is applied with a rendition, never
|
||
/// presented on its own.
|
||
pub fn companion_payload(m: &WaiMulti, i: usize) -> Result<&[u8], String> {
|
||
let meta = m.manifest.renditions.get(i)
|
||
.ok_or_else(|| format!("component index {i} out of range (have {})", m.renditions.len()))?;
|
||
if !meta.is_companion() {
|
||
return Err(format!("entry {i} is not a companion"));
|
||
}
|
||
checked_payload(m, i)
|
||
}
|
||
|
||
/// SPEC §7.1: a sink checks an entry's `sha256` before it decodes or applies
|
||
/// the entry.
|
||
fn checked_payload(m: &WaiMulti, i: usize) -> Result<&[u8], String> {
|
||
let payload = &m.renditions[i];
|
||
if !m.manifest.renditions[i].payload_digest_matches(payload) {
|
||
return Err(format!("component_digest_mismatch: entry {i}'s payload does not match its sha256"));
|
||
}
|
||
Ok(payload)
|
||
}
|
||
|
||
/// Selection with no parameter set held (for a page that only routes bytes).
|
||
pub fn pick_unverified(m: &WaiMulti, advertised: &[&str]) -> Option<usize> {
|
||
m.pick(|c| advertised.contains(&c), no_parameter_sets()).ok().map(|p| p.index)
|
||
}
|
||
}
|
||
|
||
fn js_err(e: impl std::fmt::Display) -> JsError {
|
||
JsError::new(&e.to_string())
|
||
}
|
||
|
||
/// Read the held parameter sets from JS: `{ "<capability>": [ {file: Uint8Array}
|
||
/// | {listing: Uint8Array, files: { "<path>": Uint8Array }} ] }`. `undefined` or
|
||
/// `null` holds nothing.
|
||
fn held_from_js(v: &JsValue) -> Result<core::HeldSets, JsError> {
|
||
use js_sys::{Array, Object, Reflect, Uint8Array};
|
||
let mut out = core::HeldSets::new();
|
||
if v.is_undefined() || v.is_null() {
|
||
return Ok(out);
|
||
}
|
||
let obj: &Object = v.dyn_ref().ok_or_else(|| JsError::new("held: expected an object"))?;
|
||
for entry in Object::entries(obj).iter() {
|
||
let pair: Array = entry.into();
|
||
let cap = pair.get(0).as_string().unwrap_or_default();
|
||
let list: Array = pair.get(1).dyn_into().map_err(|_| JsError::new("held: expected an array per capability"))?;
|
||
let mut sets = Vec::new();
|
||
for item in list.iter() {
|
||
let get = |k: &str| Reflect::get(&item, &JsValue::from_str(k)).unwrap_or(JsValue::UNDEFINED);
|
||
let file = get("file");
|
||
if !file.is_undefined() {
|
||
sets.push(core::Held::File(Uint8Array::new(&file).to_vec()));
|
||
continue;
|
||
}
|
||
let listing = Uint8Array::new(&get("listing")).to_vec();
|
||
let mut files = std::collections::BTreeMap::new();
|
||
if let Some(fo) = get("files").dyn_ref::<Object>() {
|
||
for f in Object::entries(fo).iter() {
|
||
let fp: Array = f.into();
|
||
files.insert(fp.get(0).as_string().unwrap_or_default(), Uint8Array::new(&fp.get(1)).to_vec());
|
||
}
|
||
}
|
||
sets.push(core::Held::Set { listing, files });
|
||
}
|
||
out.insert(cap, sets);
|
||
}
|
||
Ok(out)
|
||
}
|
||
|
||
fn strs(v: &[String]) -> Vec<&str> {
|
||
v.iter().map(String::as_str).collect()
|
||
}
|
||
|
||
/// Parsed WAI envelope returned to JS. The payload is a separate
|
||
/// Uint8Array so handlers can feed it straight into whatever decoder they
|
||
/// choose.
|
||
#[wasm_bindgen]
|
||
pub struct WaiEnvelope {
|
||
inner: wai::container::Wai,
|
||
manifest_json: String,
|
||
}
|
||
|
||
#[wasm_bindgen]
|
||
impl WaiEnvelope {
|
||
#[wasm_bindgen(getter)]
|
||
pub fn capability(&self) -> String { self.inner.capability().to_string() }
|
||
|
||
#[wasm_bindgen(getter)]
|
||
pub fn fallback(&self) -> Option<String> { self.inner.fallback().map(String::from) }
|
||
|
||
#[wasm_bindgen(getter)]
|
||
pub fn media(&self) -> String { self.inner.media().to_string() }
|
||
|
||
#[wasm_bindgen(getter)]
|
||
pub fn intent(&self) -> String { self.inner.intent().to_string() }
|
||
|
||
#[wasm_bindgen(getter)]
|
||
pub fn kind(&self) -> String { self.inner.kind().to_string() }
|
||
|
||
/// The manifest JSON exactly as the envelope carries it.
|
||
#[wasm_bindgen(getter)]
|
||
pub fn manifest(&self) -> String { self.manifest_json.clone() }
|
||
|
||
/// The pin (`conditioning.prior`) as JSON, or `undefined`.
|
||
#[wasm_bindgen(getter)]
|
||
pub fn prior(&self) -> Option<String> {
|
||
self.inner.prior().and_then(|p| serde_json::to_string(p).ok())
|
||
}
|
||
|
||
/// Returns the payload bytes — the codec-specific bytes the
|
||
/// handler for `capability` knows how to consume.
|
||
#[wasm_bindgen(getter)]
|
||
pub fn payload(&self) -> Box<[u8]> { self.inner.payload.clone().into_boxed_slice() }
|
||
|
||
#[wasm_bindgen(getter)]
|
||
pub fn payload_len(&self) -> usize { self.inner.payload.len() }
|
||
|
||
/// SPEC §4: the capability that decodes this envelope at a page that
|
||
/// advertises `advertised` and holds the parameter sets `held` (see
|
||
/// `verifyPin`); `undefined` when it is inert here — including when its
|
||
/// capability is companion-only, when its pin is refused and no fallback
|
||
/// remains, or when its manifest declares an encryption, which this page does not
|
||
/// open (SPEC §6.4 step 2).
|
||
#[wasm_bindgen(js_name = dispatchCapability)]
|
||
pub fn dispatch_capability(&self, advertised: Vec<String>, held: JsValue) -> Result<Option<String>, JsError> {
|
||
let held = held_from_js(&held)?;
|
||
Ok(core::dispatch_capability(&self.inner, &strs(&advertised), &held).ok())
|
||
}
|
||
|
||
/// The first static pin finding (SPEC §3.1, §7), or `undefined`.
|
||
#[wasm_bindgen(js_name = checkPin)]
|
||
pub fn check_pin(&self) -> Option<String> {
|
||
self.inner.check_pin().err().map(|f| f.to_string())
|
||
}
|
||
}
|
||
|
||
/// Parse a `Uint8Array` containing a complete WAI1 envelope. Throws a
|
||
/// JS Error with a descriptive message on bad magic / truncation /
|
||
/// invalid manifest JSON — the JS shim catches and surfaces it.
|
||
#[wasm_bindgen(js_name = parse)]
|
||
pub fn parse(bytes: &[u8]) -> Result<WaiEnvelope, JsError> {
|
||
let inner = core::parse_v1(bytes).map_err(js_err)?;
|
||
let m = wai::container::manifest_bytes(bytes).map_err(js_err)?;
|
||
Ok(WaiEnvelope { inner, manifest_json: String::from_utf8_lossy(m).into_owned() })
|
||
}
|
||
|
||
/// What [`pack`] refuses before it writes: a manifest that names a candidate as its
|
||
/// capability or fallback (SPEC §5, *Candidate capabilities*), and a payload that is
|
||
/// not the format SPEC §5 pins its integer capability or fallback to (*Payloads of the
|
||
/// WAI-native integer codecs*), checked by the wire parser a sink runs. The pure-Rust
|
||
/// half of the binding, which the native tests run.
|
||
fn pack_refusal(m: &wai::container::Manifest, payload: &[u8]) -> Result<(), String> {
|
||
let req = &m.model_requirement;
|
||
wai::container::check_candidates_v1(&req.capability, req.fallback.as_deref()).map_err(|e| e.to_string())?;
|
||
// An encrypted payload is ciphertext; a sink checks it after decrypting (SPEC §6.4).
|
||
if !m.security.as_ref().is_none_or(wai::container::Security::is_plaintext) {
|
||
return Ok(());
|
||
}
|
||
for cap in std::iter::once(req.capability.as_str()).chain(req.fallback.as_deref()) {
|
||
wai::container::check_int_payload_framing(cap, payload).map_err(|e| e.to_string())?;
|
||
}
|
||
Ok(())
|
||
}
|
||
|
||
/// A manifest a writer emits as given, read as every reader reads it (SPEC §2): a
|
||
/// member named twice in one object, at any depth, and a manifest past the value limit
|
||
/// (`manifest_too_many_values`) are refused, so a writer never emits an envelope the
|
||
/// reader refuses.
|
||
fn read_manifest<T: serde::de::DeserializeOwned>(json: &str) -> Result<T, String> {
|
||
wai::container::check_manifest_text(json.as_bytes()).map_err(|e| format!("manifest parse: {e}"))?;
|
||
serde_json::from_str(json).map_err(|e| format!("manifest parse: {e}"))
|
||
}
|
||
|
||
/// What [`pack_multi`] refuses before it writes, once it holds the payloads: a
|
||
/// `security` member that is not a security object (SPEC §6.1), and a component whose
|
||
/// payload is not the format SPEC §5 pins its integer capability to.
|
||
fn pack_multi_refusal(manifest: &wai::container::ManifestV2, payloads: &[Vec<u8>]) -> Result<(), String> {
|
||
wai::container::manifest_v2_security(manifest)?;
|
||
if !wai::container::manifest_v2_plaintext(manifest) {
|
||
return Ok(());
|
||
}
|
||
for (i, (r, p)) in manifest.renditions.iter().zip(payloads).enumerate() {
|
||
wai::container::check_int_payload_framing(&r.capability, p).map_err(|e| format!("component {i}: {e}"))?;
|
||
}
|
||
Ok(())
|
||
}
|
||
|
||
/// A length the writers put in a `u32` field (SPEC §2, §7.1), or an error naming the
|
||
/// field when it does not fit, as `wai-rs`'s writers refuse it. (On wasm32 a `usize`
|
||
/// is 32 bits, so every length fits; the check holds on any target.)
|
||
fn u32_field(len: usize, what: &str) -> Result<u32, JsError> {
|
||
u32::try_from(len).map_err(|_| JsError::new(&format!("{what} is {len} bytes, past the u32 field that holds it")))
|
||
}
|
||
|
||
/// Build a WAI envelope from a manifest JSON string + a payload byte
|
||
/// array. Mirrors `wai_envelope_pack` from the native SDK so JS code
|
||
/// can produce .wai files in the browser too: like it, refuses a manifest
|
||
/// that names a candidate capability as its capability or fallback, which an
|
||
/// encoder MUST NOT emit (SPEC §5, *Candidate capabilities*), and a payload that is
|
||
/// not the format SPEC §5 pins its integer capability or fallback to. The manifest is
|
||
/// written as given, so a manifest that names a member twice in one object is refused
|
||
/// (SPEC §2), as every reader refuses it, and so is one longer than SPEC §2's reader
|
||
/// limit (`manifest_too_large`).
|
||
#[wasm_bindgen(js_name = pack)]
|
||
pub fn pack(manifest_json: &str, payload: &[u8]) -> Result<Box<[u8]>, JsError> {
|
||
wai::container::check_manifest_len(manifest_json.len() as u64).map_err(|e| JsError::new(&e))?;
|
||
let m: wai::container::Manifest = read_manifest(manifest_json).map_err(|e| JsError::new(&e))?;
|
||
pack_refusal(&m, payload).map_err(|e| JsError::new(&e))?;
|
||
let mb = manifest_json.as_bytes();
|
||
let payload_len = u32_field(payload.len(), "the payload")?;
|
||
let mut out = Vec::with_capacity(12 + mb.len() + payload.len());
|
||
out.extend_from_slice(wai::container::MAGIC);
|
||
out.extend_from_slice(&(mb.len() as u32).to_le_bytes());
|
||
out.extend_from_slice(mb);
|
||
out.extend_from_slice(&payload_len.to_le_bytes());
|
||
out.extend_from_slice(payload);
|
||
Ok(out.into_boxed_slice())
|
||
}
|
||
|
||
/// Sniff the magic. Returns 1 for WAI1, 2 for WAI2, throws otherwise.
|
||
/// Use this to route between `parse` and `parseMulti`.
|
||
#[wasm_bindgen(js_name = detectVersion)]
|
||
pub fn detect_version(bytes: &[u8]) -> Result<u8, JsError> {
|
||
wai::container::detect_version(bytes).map_err(js_err)
|
||
}
|
||
|
||
/// SHA-256 of `bytes` as 64 lowercase hexadecimal digits (SPEC §7, *Digests*).
|
||
#[wasm_bindgen(js_name = sha256Hex)]
|
||
pub fn sha256_hex(bytes: &[u8]) -> String {
|
||
wai::pin::sha256_hex(bytes)
|
||
}
|
||
|
||
/// Check a pin (JSON) over `capability` against the parameter sets `held`:
|
||
/// `{ "<capability>": [ {file: Uint8Array} | {listing: Uint8Array, files: {"<path>":
|
||
/// Uint8Array}} ] }`. Returns `"ok"` or SPEC §3.1's reason.
|
||
#[wasm_bindgen(js_name = verifyPin)]
|
||
pub fn verify_pin(prior_json: &str, capability: &str, held: JsValue) -> Result<String, JsError> {
|
||
core::check_pin(prior_json, capability, &held_from_js(&held)?).map_err(js_err)
|
||
}
|
||
|
||
// ---- v1.1 (WAI2): multi-rendition envelope ----------------------
|
||
|
||
/// One component of a WAI2 envelope. `payload` is the codec bytes the handler
|
||
/// for `capability` consumes.
|
||
#[wasm_bindgen]
|
||
pub struct Rendition {
|
||
capability: String,
|
||
kind: String,
|
||
payload: Vec<u8>,
|
||
}
|
||
|
||
#[wasm_bindgen]
|
||
impl Rendition {
|
||
#[wasm_bindgen(getter)]
|
||
pub fn capability(&self) -> String { self.capability.clone() }
|
||
#[wasm_bindgen(getter)]
|
||
pub fn kind(&self) -> String { self.kind.clone() }
|
||
#[wasm_bindgen(getter)]
|
||
pub fn payload(&self) -> Box<[u8]> { self.payload.clone().into_boxed_slice() }
|
||
#[wasm_bindgen(getter)]
|
||
pub fn payload_len(&self) -> usize { self.payload.len() }
|
||
}
|
||
|
||
/// Parsed WAI2 envelope. Pick with `pick` (SPEC §7.1: component digests and pins
|
||
/// checked), present `rendition(index)`, and apply `companionsOf(index)` with it.
|
||
#[wasm_bindgen]
|
||
pub struct WaiMulti {
|
||
inner: wai::container::WaiMulti,
|
||
manifest_json: String,
|
||
}
|
||
|
||
#[wasm_bindgen]
|
||
impl WaiMulti {
|
||
#[wasm_bindgen(getter)]
|
||
pub fn manifest(&self) -> String { self.manifest_json.clone() }
|
||
#[wasm_bindgen(getter)]
|
||
pub fn media(&self) -> String { self.inner.media().to_string() }
|
||
#[wasm_bindgen(getter)]
|
||
pub fn intent(&self) -> String { self.inner.intent().to_string() }
|
||
#[wasm_bindgen(getter)]
|
||
pub fn n_renditions(&self) -> usize { self.inner.renditions.len() }
|
||
|
||
/// The rendition at `i`, to present as media. Throws for an index out of range,
|
||
/// for a component that is not a selectable rendition (a companion, or a
|
||
/// companion-only capability), which is never presented as media, and for one
|
||
/// whose payload does not match its `sha256` (`component_digest_mismatch`).
|
||
#[wasm_bindgen(js_name = rendition)]
|
||
pub fn rendition(&self, i: usize) -> Result<Rendition, JsError> {
|
||
let payload = core::rendition_payload(&self.inner, i).map_err(js_err)?;
|
||
let m = &self.inner.manifest.renditions[i];
|
||
Ok(Rendition { capability: m.capability.clone(), kind: m.kind.clone(), payload: payload.to_vec() })
|
||
}
|
||
|
||
/// The companion at `i` (applied with a rendition, never on its own). Throws
|
||
/// unless the entry is a companion, and when its payload does not match its
|
||
/// `sha256` (`component_digest_mismatch`): such a companion is not applied.
|
||
#[wasm_bindgen(js_name = companion)]
|
||
pub fn companion(&self, i: usize) -> Result<Rendition, JsError> {
|
||
let payload = core::companion_payload(&self.inner, i).map_err(js_err)?;
|
||
let m = &self.inner.manifest.renditions[i];
|
||
Ok(Rendition { capability: m.capability.clone(), kind: m.kind.clone(), payload: payload.to_vec() })
|
||
}
|
||
|
||
/// Every entry's members — `capability`, `kind`, and `id`, `sha256`, `prior`,
|
||
/// `role`, `companion_of`, `companion_of_id` when present — plus `selectable`.
|
||
#[wasm_bindgen(js_name = renditionsMeta)]
|
||
pub fn renditions_meta(&self) -> Result<JsValue, JsError> {
|
||
let metas = core::renditions_meta(&self.inner);
|
||
let json = serde_json::to_string(&metas).map_err(js_err)?;
|
||
js_sys::JSON::parse(&json).map_err(|_| JsError::new("renditionsMeta: JSON"))
|
||
}
|
||
|
||
/// SPEC §7.1 selection at a page that advertises `advertised` and holds the
|
||
/// parameter sets `held`. Returns `{index, capability, kind, skipped: [[i,
|
||
/// reason], …]}`; throws, naming every entry's reason, when nothing is picked.
|
||
#[wasm_bindgen(js_name = pick)]
|
||
pub fn pick(&self, advertised: Vec<String>, held: JsValue) -> Result<JsValue, JsError> {
|
||
let held = held_from_js(&held)?;
|
||
match core::pick(&self.inner, &strs(&advertised), &held) {
|
||
Ok((index, skipped)) => {
|
||
let m = &self.inner.manifest.renditions[index];
|
||
let v = serde_json::json!({ "index": index, "capability": m.capability, "kind": m.kind, "skipped": skipped });
|
||
js_sys::JSON::parse(&v.to_string()).map_err(|_| JsError::new("pick: JSON"))
|
||
}
|
||
Err(reasons) => Err(JsError::new(&format!("no rendition decodes here: {}",
|
||
reasons.iter().map(|(i, r)| format!("[{i}] {r}")).collect::<Vec<_>>().join("; ")))),
|
||
}
|
||
}
|
||
|
||
/// The indexes of the companions to apply with the rendition at `index`.
|
||
#[wasm_bindgen(js_name = companionsOf)]
|
||
pub fn companions_of(&self, index: usize) -> Vec<u32> {
|
||
core::companions_of_index(&self.inner, index).into_iter().map(|i| i as u32).collect()
|
||
}
|
||
|
||
/// The first static pin finding, `[index] reason: detail`, or `undefined`.
|
||
#[wasm_bindgen(js_name = checkPins)]
|
||
pub fn check_pins(&self) -> Option<String> {
|
||
self.inner.check_pins().err().map(|(i, f)| format!("[{i}] {f}"))
|
||
}
|
||
}
|
||
|
||
#[wasm_bindgen(js_name = parseMulti)]
|
||
pub fn parse_multi(bytes: &[u8]) -> Result<WaiMulti, JsError> {
|
||
let inner = core::parse_v2(bytes).map_err(js_err)?;
|
||
let m = wai::container::manifest_bytes(bytes).map_err(js_err)?;
|
||
Ok(WaiMulti { inner, manifest_json: String::from_utf8_lossy(m).into_owned() })
|
||
}
|
||
|
||
/// Pack a WAI2 envelope from a manifest JSON + array of payload byte arrays.
|
||
/// The manifest's `renditions` array MUST be the same length and order as the
|
||
/// payloads. The manifest is written as given. Like `wai_envelope_pack_multi`,
|
||
/// refuses a table with a component, in either role, whose capability is a
|
||
/// candidate, or whose payload is not the format SPEC §5 pins its integer capability
|
||
/// to (SPEC §5), and a manifest longer than SPEC §2's reader limit
|
||
/// (`manifest_too_large`).
|
||
#[wasm_bindgen(js_name = packMulti)]
|
||
pub fn pack_multi(manifest_json: &str, payloads: js_sys::Array) -> Result<Box<[u8]>, JsError> {
|
||
wai::container::check_manifest_len(manifest_json.len() as u64).map_err(|e| JsError::new(&e))?;
|
||
let manifest: wai::container::ManifestV2 = read_manifest(manifest_json).map_err(|e| JsError::new(&e))?;
|
||
wai::container::check_component_candidates(manifest.renditions.iter().map(|r| r.capability.as_str()))
|
||
.map_err(|e| JsError::new(&e.to_string()))?;
|
||
let n = payloads.length() as usize;
|
||
if n != manifest.renditions.len() {
|
||
return Err(JsError::new(&format!(
|
||
"{} payloads but manifest declares {} renditions",
|
||
n, manifest.renditions.len())));
|
||
}
|
||
if n > u16::MAX as usize {
|
||
return Err(JsError::new("too many renditions for u16 count field"));
|
||
}
|
||
let mut raw_payloads: Vec<Vec<u8>> = Vec::with_capacity(n);
|
||
for i in 0..n {
|
||
let val = payloads.get(i as u32);
|
||
let arr: js_sys::Uint8Array = val.dyn_into()
|
||
.map_err(|_| JsError::new(&format!(
|
||
"payload {i} is not a Uint8Array")))?;
|
||
raw_payloads.push(arr.to_vec());
|
||
}
|
||
pack_multi_refusal(&manifest, &raw_payloads).map_err(|e| JsError::new(&e))?;
|
||
let mb = manifest_json.as_bytes();
|
||
let table_bytes = 2 + n * 8;
|
||
let payload_bytes: usize = raw_payloads.iter().map(|p| p.len()).sum();
|
||
let mut out = Vec::with_capacity(8 + mb.len() + table_bytes + payload_bytes);
|
||
out.extend_from_slice(wai::container::MAGIC_V2);
|
||
out.extend_from_slice(&(mb.len() as u32).to_le_bytes());
|
||
out.extend_from_slice(mb);
|
||
out.extend_from_slice(&(n as u16).to_le_bytes());
|
||
let mut off: u32 = 0;
|
||
for (i, p) in raw_payloads.iter().enumerate() {
|
||
let len = u32_field(p.len(), &format!("rendition {i}'s payload"))?;
|
||
out.extend_from_slice(&off.to_le_bytes());
|
||
out.extend_from_slice(&len.to_le_bytes());
|
||
off = off.checked_add(len).ok_or_else(||
|
||
JsError::new("payload sum overflows u32"))?;
|
||
}
|
||
for p in &raw_payloads {
|
||
out.extend_from_slice(p);
|
||
}
|
||
Ok(out.into_boxed_slice())
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::core;
|
||
use wai::container::{Prior, Rendition, WaiMulti};
|
||
|
||
fn multi() -> Vec<u8> {
|
||
WaiMulti::new("video", "replicate", serde_json::Value::Null, vec![
|
||
Rendition::media("wai.video.lcevc", "lcevc", b"ENH-MISLABELLED".to_vec()),
|
||
Rendition::media("wai.video.hevc", "hevc", b"HEVC".to_vec()).with_id("base").with_payload_digest(),
|
||
Rendition::companion("wai.meta.hdr10plus", b"META".to_vec(), "wai.video.hevc").with_companion_of_id("base"),
|
||
]).to_bytes().unwrap()
|
||
}
|
||
|
||
/// The browser writers refuse a payload that is not the format SPEC §5 pins its
|
||
/// integer capability to, by the wire parser a sink runs: a tool's JSON form, a
|
||
/// `WIV1` cut short, and a keyframe-0 `WIV1` under `wai.video.int_hyper`, as the
|
||
/// capability or the fallback of a `WAI1` and as a `WAI2` component. The committed
|
||
/// payloads pass.
|
||
/// The writers read a manifest as every reader does (SPEC §2): a member named twice
|
||
/// in one object, at the top, in `target`, in `model_requirement`, nested, or in a
|
||
/// `WAI2` entry, is refused; the same manifest without the repeat is read.
|
||
#[test]
|
||
fn the_writers_refuse_a_member_named_twice() {
|
||
use wai::container::{Manifest, ManifestV2};
|
||
let ok = r#"{"wai":"1.0","media":"binary","intent":"improve","model_requirement":{"capability":"wai.image.png"},"conditioning":{"kind":"png"},"target":null}"#;
|
||
assert!(super::read_manifest::<Manifest>(ok).is_ok());
|
||
for twice in [
|
||
r#"{"wai":"1.0","media":"binary","intent":"improve","model_requirement":{"capability":"wai.image.png"},"conditioning":{"kind":"png"},"target":null,"zz":1,"zz":2}"#,
|
||
r#"{"wai":"1.0","media":"binary","intent":"improve","model_requirement":{"capability":"wai.image.png"},"conditioning":{"kind":"png"},"target":{"w":1,"w":2}}"#,
|
||
r#"{"wai":"1.0","media":"binary","intent":"improve","model_requirement":{"capability":"wai.image.png","q":1,"q":2},"conditioning":{"kind":"png"},"target":null}"#,
|
||
r#"{"wai":"1.0","media":"image","intent":"replicate","model_requirement":{"capability":"wai.image.png"},"conditioning":{"kind":"png"},"target":{"a":[{"b":1,"\u0062":1}]}}"#,
|
||
] {
|
||
let e = super::read_manifest::<Manifest>(twice).unwrap_err();
|
||
assert!(e.contains("twice"), "{twice}: {e}");
|
||
}
|
||
let v2 = r#"{"wai":"1.1","media":"video","intent":"improve","target":null,"renditions":[{"capability":"wai.image.png","kind":"png"}]}"#;
|
||
assert!(super::read_manifest::<ManifestV2>(v2).is_ok());
|
||
let e = super::read_manifest::<ManifestV2>(&v2.replace(r#""kind":"png""#, r#""kind":"png","zz":1,"zz":2"#)).unwrap_err();
|
||
assert!(e.contains("twice"), "{e}");
|
||
}
|
||
|
||
#[test]
|
||
fn the_writers_refuse_a_payload_that_is_not_the_pinned_format() {
|
||
use wai::container::{Conditioning, Manifest, ManifestV2, ModelRequirement};
|
||
let read = |rel: &str| std::fs::read(std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../wai-rs/tests/vectors").join(rel)).unwrap();
|
||
let (k0, k1) = (read("int_video/int_video_bitstream.wiv1"), read("int_video/video_nk_bitstream.wiv1"));
|
||
let v1 = |cap: &str, fallback: Option<&str>| {
|
||
let mut req = ModelRequirement::new(cap);
|
||
if let Some(f) = fallback {
|
||
req = req.with_fallback(f);
|
||
}
|
||
Manifest::new("video", "improve", req, Conditioning::new("bytes"))
|
||
};
|
||
// The refusal names the code: "... (<code>: <what failed>; ...)".
|
||
let refused = |r: Result<(), String>| r.err().and_then(|e| e.split_once(" to (").map(|(_, t)| t.split(':').next().unwrap_or_default().to_string()));
|
||
let (mot, hyp) = ("wai.video.int_motion", "wai.video.int_hyper");
|
||
assert_eq!(super::pack_refusal(&v1(mot, None), &k0), Ok(()));
|
||
assert_eq!(super::pack_refusal(&v1(hyp, None), &k1), Ok(()));
|
||
assert_eq!(refused(super::pack_refusal(&v1(mot, None), &read("int_video/int_video_bitstream.json"))).as_deref(), Some("bad_magic"));
|
||
assert_eq!(refused(super::pack_refusal(&v1(mot, None), &k0[..40])).as_deref(), Some("stream_length"));
|
||
assert_eq!(refused(super::pack_refusal(&v1(hyp, None), &k0)).as_deref(), Some("keyframe_capability_mismatch"));
|
||
assert_eq!(refused(super::pack_refusal(&v1("wai.video.av1", Some(mot)), &k1)).as_deref(), Some("keyframe_capability_mismatch"));
|
||
let m2: ManifestV2 = serde_json::from_value(serde_json::json!({
|
||
"wai": "1.1", "media": "video", "intent": "improve",
|
||
"renditions": [{"capability": mot, "kind": "bytes"}, {"capability": hyp, "kind": "bytes"}]
|
||
})).unwrap();
|
||
assert_eq!(super::pack_multi_refusal(&m2, &[k0.clone(), k1.clone()]), Ok(()));
|
||
let e = super::pack_multi_refusal(&m2, &[k0.clone(), k0.clone()]).unwrap_err();
|
||
assert!(e.starts_with("component 1:") && e.contains("keyframe_capability_mismatch"), "{e}");
|
||
// An encrypted envelope's payload is ciphertext, checked after decryption
|
||
// (SPEC §6.4): it is written.
|
||
let cipher: Vec<u8> = k0.iter().enumerate().map(|(i, b)| b ^ (i as u8).wrapping_mul(151).wrapping_add(7)).collect();
|
||
let enc = v1(mot, None).with_security(wai::container::Security::new("aes-256-gcm", 7));
|
||
assert_eq!(super::pack_refusal(&enc, &cipher), Ok(()));
|
||
let mut m2e = m2.clone();
|
||
m2e.extra.insert("security".into(), serde_json::json!({"encryption": "mls-sframe", "kid": 0}));
|
||
assert_eq!(super::pack_multi_refusal(&m2e, &[cipher.clone(), cipher.clone()]), Ok(()));
|
||
// A `security` member that is not a security object is not written: it would
|
||
// claim an encryption no sink can read.
|
||
for bad in [serde_json::json!(true), serde_json::json!("aes"), serde_json::json!([]), serde_json::json!({"encryption": 5})] {
|
||
m2e.extra.insert("security".into(), bad.clone());
|
||
let e = super::pack_multi_refusal(&m2e, &[cipher.clone(), cipher.clone()]).unwrap_err();
|
||
assert!(e.contains("is not a security object"), "{bad}: {e}");
|
||
}
|
||
}
|
||
|
||
/// A companion-only component is never presented as media: not picked, not
|
||
/// returned by `rendition`, and flagged `selectable: false` in the metadata.
|
||
#[test]
|
||
fn companion_only_component_is_never_presented() {
|
||
let m = core::parse_v2(&multi()).unwrap();
|
||
let all = ["wai.video.lcevc", "wai.video.hevc", "wai.meta.hdr10plus"];
|
||
let (index, skipped) = core::pick(&m, &all, &core::HeldSets::new()).unwrap();
|
||
assert_eq!(index, 1);
|
||
assert_eq!(skipped, [(0, "not_selectable".to_string())]);
|
||
assert!(core::rendition_payload(&m, 0).unwrap_err().contains("never presented as media"));
|
||
assert!(core::rendition_payload(&m, 2).is_err(), "a companion is not a rendition");
|
||
assert_eq!(core::rendition_payload(&m, 1).unwrap(), b"HEVC");
|
||
let meta = core::renditions_meta(&m);
|
||
let sel: Vec<bool> = meta.iter().map(|v| v["selectable"].as_bool().unwrap()).collect();
|
||
assert_eq!(sel, [false, true, false]);
|
||
// Only the mislabelled enhancement advertised: inert, not presented.
|
||
assert!(core::pick(&m, &["wai.video.lcevc"], &core::HeldSets::new()).is_err());
|
||
// A WAI1 whose capability is companion-only is never dispatched.
|
||
let v1 = wai::container::Wai::new(
|
||
wai::container::Manifest::new("video", "replicate", wai::container::ModelRequirement::new("wai.video.lcevc"),
|
||
wai::container::Conditioning::new("lcevc")), vec![1]);
|
||
assert!(core::dispatch_capability(&v1, &["wai.video.lcevc"], &core::HeldSets::new()).is_err());
|
||
}
|
||
|
||
/// `role`, `companion_of`, `companion_of_id`, `id`, `sha256` and `prior`
|
||
/// survive the parse and reach the metadata JS reads.
|
||
#[test]
|
||
fn role_companion_of_prior_survive_parse() {
|
||
let bytes = WaiMulti::new("image", "replicate", serde_json::Value::Null, vec![
|
||
Rendition::media("wai.neural.int_mlicpp", "int_mlicpp", b"P".to_vec())
|
||
.with_prior(Prior::new("m", "ab".repeat(32), "int")),
|
||
Rendition::media("wai.video.hevc", "hevc", b"HEVC".to_vec()).with_id("base").with_payload_digest(),
|
||
Rendition::companion("wai.meta.hdr10plus", b"META".to_vec(), "wai.video.hevc").with_companion_of_id("base"),
|
||
]).to_bytes().unwrap();
|
||
let m = core::parse_v2(&bytes).unwrap();
|
||
let meta = core::renditions_meta(&m);
|
||
assert_eq!(meta[0]["prior"]["sha256"], "ab".repeat(32));
|
||
assert_eq!(meta[1]["id"], "base");
|
||
assert_eq!(meta[1]["sha256"], wai::pin::sha256_hex(b"HEVC"));
|
||
assert_eq!(meta[2]["role"], "companion");
|
||
assert_eq!(meta[2]["companion_of"], "wai.video.hevc");
|
||
assert_eq!(meta[2]["companion_of_id"], "base");
|
||
assert_eq!(core::companions_of_index(&m, 1), [2]);
|
||
assert!(core::companions_of_index(&m, 0).is_empty());
|
||
}
|
||
|
||
/// Fetching an entry by index checks its `sha256` too: a tampered rendition or
|
||
/// companion is refused with `component_digest_mismatch`, not handed out.
|
||
#[test]
|
||
fn fetch_by_index_checks_the_component_digest() {
|
||
let mut m = core::parse_v2(&multi()).unwrap();
|
||
assert_eq!(core::companion_payload(&m, 2).unwrap(), b"META");
|
||
assert!(core::companion_payload(&m, 1).unwrap_err().contains("not a companion"));
|
||
let mut bytes = WaiMulti::new("video", "replicate", serde_json::Value::Null, vec![
|
||
Rendition::media("wai.video.hevc", "hevc", b"HEVC".to_vec()).with_id("base"),
|
||
Rendition::companion("wai.meta.hdr10plus", b"META".to_vec(), "wai.video.hevc")
|
||
.with_companion_of_id("base").with_payload_digest(),
|
||
]).to_bytes().unwrap();
|
||
let n = bytes.len();
|
||
bytes[n - 1] ^= 1; // the companion's last byte
|
||
let tampered = core::parse_v2(&bytes).unwrap();
|
||
assert!(core::companion_payload(&tampered, 1).unwrap_err().starts_with("component_digest_mismatch"));
|
||
assert!(core::companions_of_index(&tampered, 0).is_empty(), "a mismatching companion is not applied");
|
||
m.renditions[1][0] ^= 1; // the HEVC base, which states a digest
|
||
assert!(core::rendition_payload(&m, 1).unwrap_err().starts_with("component_digest_mismatch"));
|
||
}
|
||
|
||
/// `pick` verifies each entry's component digest and pin before it picks.
|
||
#[test]
|
||
fn pick_verifies_pin_and_component_digest() {
|
||
let model = b"one-file parameter set".to_vec();
|
||
let pin = Prior::new("m", wai::pin::sha256_hex(&model), "int");
|
||
let mut m = WaiMulti::new("image", "replicate", serde_json::Value::Null, vec![
|
||
Rendition::media("wai.neural.int_mlicpp", "int_mlicpp", b"PINNED".to_vec()).with_prior(pin.clone()),
|
||
Rendition::media("wai.image.png", "png", b"PNG".to_vec()).with_payload_digest(),
|
||
Rendition::media("wai.image.avif", "avif", b"AVIF".to_vec()),
|
||
]);
|
||
let adv = ["wai.neural.int_mlicpp", "wai.image.png", "wai.image.avif"];
|
||
let mut held = core::HeldSets::new();
|
||
// Not held: the pinned entry is skipped.
|
||
let (i, s) = core::pick(&m, &adv, &held).unwrap();
|
||
assert_eq!((i, s[0].1.split(':').next().unwrap()), (1, "prior_not_held"));
|
||
// Held: it is picked.
|
||
held.insert("wai.neural.int_mlicpp".into(), vec![core::Held::File(model.clone())]);
|
||
assert_eq!(core::pick(&m, &adv, &held).unwrap().0, 0);
|
||
assert_eq!(core::check_pin(&serde_json::to_string(&pin).unwrap(), "wai.neural.int_mlicpp", &held).unwrap(), "ok");
|
||
// A damaged PNG payload no longer matches its digest.
|
||
held.clear();
|
||
m.renditions[1][0] ^= 1;
|
||
let (i, s) = core::pick(&m, &adv, &held).unwrap();
|
||
assert_eq!(i, 2);
|
||
assert_eq!(s.iter().map(|(i, r)| (*i, r.split(':').next().unwrap().to_string())).collect::<Vec<_>>(),
|
||
[(0, "prior_not_held".to_string()), (1, "component_digest_mismatch".to_string())]);
|
||
}
|
||
}
|
||
|
||
/// The JS surface itself, run in a wasm32 host (`wasm-bindgen-test-runner`):
|
||
/// what a page calls, with JS values in and out.
|
||
#[cfg(all(test, target_arch = "wasm32"))]
|
||
mod wasm_tests {
|
||
use wasm_bindgen::prelude::*;
|
||
use wasm_bindgen_test::*;
|
||
use wai::container::{Prior, Rendition, WaiMulti};
|
||
|
||
fn held(cap: &str, file: &[u8]) -> JsValue {
|
||
let entry = js_sys::Object::new();
|
||
js_sys::Reflect::set(&entry, &"file".into(), &js_sys::Uint8Array::from(file).into()).unwrap();
|
||
let obj = js_sys::Object::new();
|
||
js_sys::Reflect::set(&obj, &cap.into(), &js_sys::Array::of1(&entry).into()).unwrap();
|
||
obj.into()
|
||
}
|
||
|
||
#[wasm_bindgen_test]
|
||
fn pick_rendition_and_companions_from_js() {
|
||
let model = b"one-file parameter set";
|
||
let bytes = WaiMulti::new("video", "replicate", serde_json::Value::Null, vec![
|
||
Rendition::media("wai.video.lcevc", "lcevc", b"ENH".to_vec()),
|
||
Rendition::media("wai.neural.int_mlicpp", "int_mlicpp", b"P".to_vec())
|
||
.with_prior(Prior::new("m", wai::pin::sha256_hex(model), "int")),
|
||
Rendition::media("wai.video.hevc", "hevc", b"HEVC".to_vec()).with_id("base"),
|
||
Rendition::companion("wai.meta.hdr10plus", b"META".to_vec(), "wai.video.hevc").with_companion_of_id("base"),
|
||
]).to_bytes().unwrap();
|
||
let m = super::parse_multi(&bytes).ok().unwrap();
|
||
let adv: Vec<String> = ["wai.video.lcevc", "wai.neural.int_mlicpp", "wai.video.hevc"].iter().map(|s| s.to_string()).collect();
|
||
// Holding no set: the pinned entry is skipped, the HEVC base is picked.
|
||
let p = m.pick(adv.clone(), JsValue::UNDEFINED).ok().unwrap();
|
||
assert_eq!(js_sys::Reflect::get(&p, &"index".into()).unwrap().as_f64(), Some(2.0));
|
||
assert_eq!(m.companions_of(2), vec![3]);
|
||
// Holding the set: the pinned entry is picked.
|
||
let p = m.pick(adv, held("wai.neural.int_mlicpp", model)).ok().unwrap();
|
||
assert_eq!(js_sys::Reflect::get(&p, &"index".into()).unwrap().as_f64(), Some(1.0));
|
||
// A companion-only component is never presented as media.
|
||
assert!(m.rendition(0).is_err());
|
||
assert!(m.rendition(3).is_err());
|
||
assert_eq!(m.rendition(2).ok().unwrap().payload().to_vec(), b"HEVC");
|
||
assert_eq!(m.companion(3).ok().unwrap().capability(), "wai.meta.hdr10plus");
|
||
let meta = m.renditions_meta().ok().unwrap();
|
||
let first = js_sys::Array::from(&meta).get(0);
|
||
assert_eq!(js_sys::Reflect::get(&first, &"selectable".into()).unwrap().as_bool(), Some(false));
|
||
let second = js_sys::Array::from(&meta).get(1);
|
||
let prior = js_sys::Reflect::get(&second, &"prior".into()).unwrap();
|
||
assert_eq!(js_sys::Reflect::get(&prior, &"sha256".into()).unwrap().as_string(), Some(wai::pin::sha256_hex(model)));
|
||
}
|
||
|
||
/// A rendition table whose lengths wrap a 32-bit sum is refused with a JS
|
||
/// error. usize is 32 bits here, which is where the wrap could happen.
|
||
#[wasm_bindgen_test]
|
||
fn wrapped_rendition_lengths_throw_rather_than_trap() {
|
||
let mut bytes = WaiMulti::new("image", "improve", serde_json::Value::Null, vec![
|
||
Rendition::media("wai.image.png", "png", vec![0; 8]),
|
||
Rendition::media("wai.image.avif", "avif", vec![0; 8]),
|
||
]).to_bytes().unwrap();
|
||
let table = bytes.len() - 16 - 16;
|
||
bytes[table + 4..table + 8].copy_from_slice(&0xFFFF_FFF0u32.to_le_bytes());
|
||
bytes[table + 8..table + 12].copy_from_slice(&0xFFFF_FFF0u32.to_le_bytes());
|
||
bytes[table + 12..table + 16].copy_from_slice(&0x20u32.to_le_bytes());
|
||
assert!(super::parse_multi(&bytes).is_err());
|
||
}
|
||
|
||
/// `companion(i)` and `rendition(i)` refuse an entry whose payload does not
|
||
/// match its `sha256`.
|
||
#[wasm_bindgen_test]
|
||
fn tampered_component_is_not_handed_to_js() {
|
||
let mut bytes = WaiMulti::new("video", "replicate", serde_json::Value::Null, vec![
|
||
Rendition::media("wai.video.hevc", "hevc", b"HEVC".to_vec()).with_id("base").with_payload_digest(),
|
||
Rendition::companion("wai.meta.hdr10plus", b"META".to_vec(), "wai.video.hevc")
|
||
.with_companion_of_id("base").with_payload_digest(),
|
||
]).to_bytes().unwrap();
|
||
let n = bytes.len();
|
||
bytes[n - 1] ^= 1; // the companion
|
||
bytes[n - 5] ^= 1; // the base
|
||
let m = super::parse_multi(&bytes).ok().unwrap();
|
||
assert!(m.companion(1).is_err());
|
||
assert!(m.rendition(0).is_err());
|
||
assert!(m.companions_of(0).is_empty());
|
||
}
|
||
|
||
#[wasm_bindgen_test]
|
||
fn verify_pin_and_wai1_dispatch_from_js() {
|
||
let model = b"one-file parameter set";
|
||
let pin = serde_json::to_string(&Prior::new("m", wai::pin::sha256_hex(model), "int")).unwrap();
|
||
assert_eq!(super::verify_pin(&pin, "wai.neural.int_mlicpp", held("wai.neural.int_mlicpp", model)).ok().unwrap(), "ok");
|
||
assert_eq!(super::verify_pin(&pin, "wai.neural.int_mlicpp", JsValue::NULL).ok().unwrap(), "prior_not_held");
|
||
assert_eq!(super::verify_pin(&pin, "wai.neural.bmshj2018", JsValue::NULL).ok().unwrap(), "form_required");
|
||
assert_eq!(super::sha256_hex(b"abc"), "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad");
|
||
let lcevc = super::pack(
|
||
r#"{"wai":"1.0","media":"video","intent":"replicate","model_requirement":{"capability":"wai.video.lcevc"},"conditioning":{"kind":"lcevc"},"target":null}"#,
|
||
b"E").ok().unwrap();
|
||
let e = super::parse(&lcevc).ok().unwrap();
|
||
assert_eq!(e.dispatch_capability(vec!["wai.video.lcevc".into()], JsValue::UNDEFINED).ok().unwrap(), None);
|
||
}
|
||
}
|