10c1597b7f
A 120-byte WIV1 (16 frames of 4729 x 4729 and an 8-byte stream) ran for about a minute and reached gigabytes before it was refused at its last symbol, and the JSON reader panicked on a side of 2^31. - Every integer decode (int_video, int_hyper_synth's checked conditional decode, int_audio, int_splat) stops at the first symbol during whose decode its stream runs out (stream_exhausted), before the value is used. The exhaustion flag is sticky, so a stream that never runs out decodes as before. - A frame, a clip or a latent is allocated only as its symbols are decoded; int_video holds frames as RGB8 and predicts from the last decoded frame, and reserves nothing from n_frames. - WIV1 caps the frame count at 2^20 and the motion-vector field at 2^26 elements, after the frame and clip caps (too_large); refusal fixtures wiv1_too_many_frames and wiv1_mv_field, checked by the generator and the Python verifier. - int_video::VideoJson reads each size as a usize it fits (absent otherwise), and caps every size before it pads a motion-vector list. - wai-web: decodeWiv1 keeps the decoder's frames (DecodedVideo.frame(i)) rather than concatenating the clip. - The Python verifier refuses at the first exhausted symbol, applies the new caps, and refuses a non-convertible JSON source explicitly instead of through assert. - integer-payloads.md §2, §3 and §7 state the early stop, the caps and the memory bound. Test: tests/int_payload_preflight.rs (counting allocator) refuses five starved payloads, the reviewer's two WIV1 among them, at the first exhausted symbol with no allocation sized from the header, and the two JSON headers without a panic or a header-sized allocation. Co-Authored-By: Claude Opus 5.5 <[email protected]>
96 lines
4.5 KiB
Rust
96 lines
4.5 KiB
Rust
//! In-browser **byte-exact integer video** decode — zero JS ML runtime.
|
|
//!
|
|
//! The same pure-i64 Rust that decodes the `wai.video.int_motion` /
|
|
//! `wai.video.int_hyper` inter-frame codec natively compiles to wasm32 and
|
|
//! produces the IDENTICAL frames here: integer block motion compensation +
|
|
//! integer-quantized residual, no float anywhere. So a publisher can pin each
|
|
//! frame's hash and the page re-checks them client-side — *frame-state-
|
|
//! equivalence*, a whole clip you can put in a receipt.
|
|
|
|
use wasm_bindgen::prelude::*;
|
|
|
|
/// A decoded clip: its frames, each `width · height · 3` bytes of RGB, as the
|
|
/// decoder produced them. `frame(i)` hands JS one frame; `rgb` concatenates them all
|
|
/// into one new buffer on each read, so a page that paints frame by frame holds the
|
|
/// clip once.
|
|
#[wasm_bindgen]
|
|
pub struct DecodedVideo {
|
|
frames: Vec<Vec<u8>>,
|
|
width: u32,
|
|
height: u32,
|
|
}
|
|
|
|
#[wasm_bindgen]
|
|
impl DecodedVideo {
|
|
/// All frames, concatenated into a new buffer: frame `f` is
|
|
/// `rgb[f·w·h·3 .. (f+1)·w·h·3]`.
|
|
#[wasm_bindgen(getter)]
|
|
pub fn rgb(&self) -> Box<[u8]> {
|
|
self.frames.concat().into_boxed_slice()
|
|
}
|
|
/// Frame `index` (`width · height · 3` bytes), or an empty buffer past the last.
|
|
pub fn frame(&self, index: u32) -> Box<[u8]> {
|
|
self.frames.get(index as usize).cloned().unwrap_or_default().into_boxed_slice()
|
|
}
|
|
#[wasm_bindgen(getter, js_name = nFrames)]
|
|
pub fn n_frames(&self) -> u32 {
|
|
self.frames.len() as u32
|
|
}
|
|
#[wasm_bindgen(getter)]
|
|
pub fn width(&self) -> u32 {
|
|
self.width
|
|
}
|
|
#[wasm_bindgen(getter)]
|
|
pub fn height(&self) -> u32 {
|
|
self.height
|
|
}
|
|
}
|
|
|
|
/// Decode a `WIV1` payload (`extensions/integer-payloads.md` §3) to RGB frames in
|
|
/// the browser, byte-identical to the native reference: `capability` is the
|
|
/// envelope's, `wai.video.int_motion` (keyframe 0, no parameters) or
|
|
/// `wai.video.int_hyper` (keyframe 1, whose nested `WIH1` keyframe decodes with the
|
|
/// conditional codec's `g_s.whm`, `h_s.whm` and `entropy.json`). A refusal's message
|
|
/// starts with its code (`bad_magic` for a JSON form). No float, no ML runtime.
|
|
#[wasm_bindgen(js_name = decodeWiv1)]
|
|
pub fn decode_wiv1(
|
|
capability: &str, payload: &[u8], g_s: Option<Box<[u8]>>, h_s: Option<Box<[u8]>>, entropy: Option<Box<[u8]>>,
|
|
) -> Result<DecodedVideo, JsError> {
|
|
let (g, h, e) = (g_s.unwrap_or_default(), h_s.unwrap_or_default(), entropy.unwrap_or_default());
|
|
let set = crate::int_payload::int_hyper_set(&g, &h, &e);
|
|
let set: &[(&str, &[u8])] = if capability == "wai.video.int_hyper" { &set } else { &[] };
|
|
match crate::int_payload::decode(capability, payload, set).map_err(|e| JsError::new(&e))? {
|
|
wai::int_payload::IntDecoded::Video { frames, width, height } => Ok(DecodedVideo { frames, width, height }),
|
|
_ => Err(JsError::new("unsupported_capability: not an integer video capability")),
|
|
}
|
|
}
|
|
|
|
/// Decode the tool's JSON bitstream (`int_video_bitstream.json`) to RGB frames. For
|
|
/// debugging and conversion only: the payload `wai.video.int_motion` carries is
|
|
/// `WIV1` ([`decode_wiv1`]), and a JSON form is not one. Byte-identical to the
|
|
/// native reference; `fnv1aHex` (the `generate` binding) hashes each frame.
|
|
#[wasm_bindgen(js_name = decodeIntVideo)]
|
|
pub fn decode_int_video(bitstream_json: &[u8]) -> Result<DecodedVideo, JsError> {
|
|
let bs: serde_json::Value =
|
|
serde_json::from_slice(bitstream_json).map_err(|e| JsError::new(&e.to_string()))?;
|
|
let clip = wai::int_video::decode(&bs).map_err(JsError::new)?;
|
|
Ok(flatten(clip))
|
|
}
|
|
|
|
/// Decode the **P-frames** of a JSON bitstream on top of an already-decoded I-frame
|
|
/// (`i_frame`, RGB HWC `u8`, e.g. from `decodeIntHyperBin`) — the NEURAL-KEYFRAME
|
|
/// path (`wai.video.int_hyper`): a learned `int_hyper` keyframe + integer
|
|
/// inter-frames, the whole clip byte-exact. For debugging and conversion only: the
|
|
/// payload is `WIV1` with its keyframe nested ([`decode_wiv1`]). No float, no ML
|
|
/// runtime.
|
|
#[wasm_bindgen(js_name = decodeIntVideoPframes)]
|
|
pub fn decode_int_video_pframes(i_frame: &[u8], w: u32, h: u32, bitstream_json: &[u8]) -> Result<DecodedVideo, JsError> {
|
|
let bs: serde_json::Value =
|
|
serde_json::from_slice(bitstream_json).map_err(|e| JsError::new(&e.to_string()))?;
|
|
let clip = wai::int_video::decode_pframes(i_frame, h as usize, w as usize, &bs).map_err(JsError::new)?;
|
|
Ok(flatten(clip))
|
|
}
|
|
|
|
fn flatten(clip: wai::int_video::DecodedClip) -> DecodedVideo {
|
|
DecodedVideo { width: clip.w as u32, height: clip.h as u32, frames: clip.frames }
|
|
}
|