265 lines
17 KiB
Markdown
265 lines
17 KiB
Markdown
# Security
|
||
|
||
This file says where WAI's reference implementation reads untrusted input, what it
|
||
bounds, and what it does not yet do. The rules a sink must follow are in
|
||
[`SPEC.md`](SPEC.md) and [`extensions/`](extensions/); this file points at them and
|
||
adds nothing normative.
|
||
|
||
## Reporting a vulnerability
|
||
|
||
Email **[email protected]**. Do not open a public issue or pull request
|
||
for a vulnerability that is not yet fixed.
|
||
|
||
Say which component is affected (a crate and file, a verifier, or a section of the
|
||
specification), the commit you tested, the input or steps that trigger it, what
|
||
happened and what you expected. An input file is the most useful thing to send.
|
||
|
||
In scope:
|
||
|
||
- the specification and its extensions, where a rule lets a conforming sink or
|
||
verifier be made to misbehave;
|
||
- the reference implementation: `wai-rs`, `wai-quantum`, `wai-web`, `wai-pack`,
|
||
`wai-relay`, `wai-verify` and the other crates under this directory;
|
||
- the conformance verifiers (`*-conformance/*.py`) and their corpora.
|
||
|
||
Out of scope: services anyone deploys with this code, which their operators answer for.
|
||
|
||
WAI is pre-1.0. Fixes go to `main` of the canonical repository,
|
||
`https://git.transaction.science/Transaction-Science/open-standards`; no earlier
|
||
release line is maintained.
|
||
|
||
## Manifest parsing
|
||
|
||
Every envelope a sink receives is untrusted. Its container and manifest are read by
|
||
`wai-rs/src/container.rs` (`Wai::from_bytes`, `WaiMulti::from_bytes`,
|
||
`manifest_bytes`) and, over HTTP, by `wai-rs/src/http_binding/head.rs`
|
||
(`parse_head`).
|
||
|
||
What the readers bound:
|
||
|
||
| what | bound | where it is specified |
|
||
|---|---|---|
|
||
| every length field (`man_len`, `payload_len`, each rendition's offset and length) | checked against the bytes in hand before anything is sliced; sums formed so they cannot wrap (`saturating_add`; the `WAI2` payload total in `u64`) | SPEC §2, §7.1 |
|
||
| manifest length | at most 134 217 728 bytes (2²⁷), sized to the largest manifest a conforming staged layer has. A longer one is refused from `man_len` alone, before it is read or, over HTTP, requested (`manifest_too_large`; `head_malformed` from `parse_head`) | SPEC §2, *Reader limits* |
|
||
| manifest values | at most 4 194 304 JSON values (2²²), the manifest object and every value in it at any depth; a member's name is not one. One more is refused while the values are counted, before anything is built from the manifest (`manifest_too_many_values`; `head_malformed` from `parse_head`) | SPEC §2, *Reader limits* |
|
||
| manifest nesting | at most 127 nested arrays and objects, the manifest object counting as one; 128 are refused | SPEC §2, *Reader limits* |
|
||
| member names | a member named twice in one object, at any depth, is refused | SPEC §2 |
|
||
| `WAI2` rendition table | at most 65 535 rows (`u16`), as many as the manifest's `renditions`, packed contiguously from offset 0 | SPEC §7.1 |
|
||
| an HTTP object | exactly one envelope; any other length is refused (`object_length_mismatch`) | http-delivery §5.1 |
|
||
| a head proof | at most 64 chaining values; `covered_len` is the object's length, or shorter and ending on a chunk boundary of the hashed input | http-delivery §6 |
|
||
|
||
A reader copies a payload only after its bounds are checked, so a payload never makes
|
||
it allocate more than it was given. A `WAI1` reader ignores bytes after the payload; an
|
||
HTTP object that carries any is refused ([http-delivery](extensions/http-delivery.md)
|
||
§5.1).
|
||
|
||
**What parsing a manifest takes.** A parsed value takes more memory than its text, and
|
||
many small values can take many times their length, so the length limit bounds what a
|
||
reader must read and not what a parse costs. The value limit bounds the rest. The
|
||
reference readers count a manifest's values before they build anything from it
|
||
(`check_manifest_text`) and refuse it as soon as the count passes the limit; they then
|
||
build each array and object at the size counted, and move the manifest's members out of
|
||
that one tree (`read_manifest`, `read_manifest_v2`). On a 64-bit target a read allocates
|
||
at most 128 bytes for each value and one for each byte of the manifest, and for a `WAI2`
|
||
one rendition entry of 768 bytes for each it lists: at most 640 MiB for a `WAI1`
|
||
manifest at both limits, whatever its shape, and 48 MiB more for a `WAI2`'s entries.
|
||
|
||
`tests/manifest_parse_memory.rs` measures this with an allocator that counts every byte
|
||
asked for: a manifest of small values at the length limit is refused for its values
|
||
before anything is built; one at the length limit within the value limit is read; a
|
||
manifest of each shape at the value limit is read and one value past it refused; and so
|
||
are the largest manifest a conforming staged layer has and a `WAI2` listing as many
|
||
renditions as its table holds. Each stays within the bound above.
|
||
`tests/manifest_reader_equivalence.rs` holds these readers to the result of the derived
|
||
reader they replace, for every envelope committed under `wai/` and for each with every
|
||
member removed or given a value of each JSON type.
|
||
|
||
The library tests in `container.rs` and `http_binding/tests.rs` check the manifest
|
||
limits at their edges: from its first eight bytes, every reader refuses a manifest one
|
||
byte past the length limit as too large and one at the limit only as truncated; a
|
||
2 MiB manifest is read and written back byte for byte; and 127 levels are read where
|
||
128 are refused. The writers (`Wai::to_bytes`, `WaiMulti::to_bytes`, and `pack` and
|
||
`packMulti` in `wai-web`) refuse a manifest past the length limit, and a payload whose
|
||
length does not fit its `u32` field. The Rust writers stop serialising as soon as the
|
||
manifest passes the limit; the test runs both writers at limits equal to an envelope's
|
||
manifest and payload lengths and one byte below them, since serialising 128 MiB takes a
|
||
debug build many seconds. `verify_pack.mjs` checks the browser writers at the real
|
||
limit.
|
||
|
||
The stdlib Python verifiers read envelopes too. `staged_verify.py`,
|
||
`claim_verify.py`, `pack_verify.py` and `http_verify.py` refuse a manifest past the
|
||
length limit, or past the value limit, as the reference does. `http_verify.py` and `wai_http_vectors` agree on
|
||
`parse_head` at each limit in the HTTP corpus; the other three each build an envelope in
|
||
memory with a manifest one byte past the length limit, and one value past the value
|
||
limit, and require each refused. Those that read
|
||
manifests through `staged_verify.strict_json` refuse nesting past 127; `pack_verify.py`
|
||
reads up to the interpreter's recursion limit and refuses past it as unparseable.
|
||
|
||
Payload formats that WAI defines carry their own caps, checked before the first symbol
|
||
is decoded: a tensor of at most 2²⁶ elements, a `WIV1` clip of at most 2³⁰ decoded
|
||
bytes and 2²⁰ frames, and a sink cost limit (`int_payload::check_cost`,
|
||
`cost_over_limit`) ([integer-payloads](extensions/integer-payloads.md) §7, §8).
|
||
`wai.neural.int_mlicpp` refuses a header whose shapes, or whose buffers, pass its cap
|
||
before it allocates them (`tests/int_mlicpp_preflight.rs`).
|
||
|
||
`wai.text.zstd` and `wai.text.xz` carry streams whose formats bound neither what a
|
||
payload decodes to nor the history its decoder must keep, so the specification sets
|
||
both (SPEC §2): at most 2²⁷ decoded bytes, and at most 2²⁷ bytes of window or
|
||
dictionary. The reference decoders (`codecs/text.rs`) write their output into a buffer
|
||
that is never sized past the length limit and refuse the payload when its output would
|
||
pass it (`text_decoded_too_long`); a Zstandard frame that declares a longer content
|
||
size is refused before anything is decoded. They give the decoding library the window
|
||
limit, so a stream that declares more is refused from its header, before the library
|
||
allocates it (`text_window_too_large`). The encoders refuse input past the length limit.
|
||
`tests/text_decode_bounds.rs` checks each limit at its edge, on streams made in the test
|
||
that are at it and one step past it; `tests/text_decode_memory.rs` measures, with a
|
||
counting allocator, that refusing a payload of a few kilobytes that would expand to
|
||
four times a limit takes no more output buffer than the limit. The C entry points
|
||
`wai_text_zstd_decode` and `wai_text_xz_decode` are these decoders.
|
||
|
||
The picture, audio and video payloads of the floor and the recommended set have reader
|
||
limits as well (SPEC §2): a picture at most 65 536 pixels wide and tall and 2²⁷ pixels
|
||
in all, audio of at most 2²⁷ samples counting every channel, and AV1 of at most 2²⁰
|
||
temporal units whose frames are pictures within those limits. The reference decoders
|
||
(`codecs/image.rs`, `codecs/audio.rs`, `codecs/video.rs`) refuse a picture from the
|
||
size its header declares, before they decode it (`image_too_large`): PNG and JPEG
|
||
through the `image` crate's header read, AVIF through libavif's own decoder limits,
|
||
which it checks against the `ispe` of every image item while it parses the container
|
||
and again against the AV1 sequence header, and JPEG XL through a pass of libjxl that
|
||
reads only the basic information. Each decodes only its own format. The FLAC decoder
|
||
refuses a `STREAMINFO` whose declared total passes the limit before it decodes, and
|
||
never sizes its output from the declaration past the limit; both audio decoders count
|
||
their output as it grows and refuse it past the limit (`audio_decoded_too_long`). The
|
||
AV1 decoder refuses a framing header past the limits (`video_too_large`), more temporal
|
||
units than the header declares, and a frame larger than it declares, and decodes every
|
||
layout and depth AV1 allows rather than assuming 8-bit 4:2:0. `av1_decode_each` hands
|
||
on one frame at a time, so its memory is one frame and dav1d's reference frames whatever
|
||
the payload's length; `av1_decode` and `wai_video_av1_decode`, which return every frame
|
||
at once, refuse a payload whose frames would take more than 4 GiB together, from its
|
||
header (`video_decoded_too_large`). The encoders refuse what a reader may refuse.
|
||
`tests/media_decode_bounds.rs` checks each limit at its edge and one past it, on real
|
||
payloads and on headers rewritten to declare more than their data holds.
|
||
|
||
## Native codec libraries and sandboxed decode
|
||
|
||
The default `codecs` feature binds C and C++ libraries for the floor and the
|
||
recommended set: libzstd and liblzma for `wai.text.zstd` and `wai.text.xz`, libjxl
|
||
for JPEG XL, libavif for AVIF decode (with dav1d inside it), libopus, dav1d for AV1
|
||
decode, and libFLAC for FLAC encode. PNG, JPEG and FLAC decode run in Rust (the `image` and `claxon` crates).
|
||
The `neural` feature binds ONNX Runtime for the float neural decoders. These libraries
|
||
parse payload bytes directly, and they are the usual place a memory-safety defect
|
||
in a media stack lives.
|
||
|
||
They run in the caller's process, with its privileges: the reference implementation
|
||
has no process or system-call sandbox. A sink that decodes payloads from origins it
|
||
does not trust through these libraries should run the decode in a separate process
|
||
with no file-system or network access, or use a decoder compiled to WebAssembly and
|
||
run it in a WebAssembly runtime.
|
||
|
||
What does not touch them:
|
||
|
||
- Most CI builds. One job builds `codecs`: `codec-wrappers` in `wai-conformance.yml`,
|
||
on the macOS runner, which runs the wrappers' fuzz smoke, the text and media decode
|
||
limits and the C entry points through the libraries. `wai-neural-gates.yml` and
|
||
`wai-neural-e2e.yml` build the `neural` feature, which links ONNX Runtime and
|
||
libzstd, and run the float neural decoders through them. No workflow enables
|
||
`semantic`, and no other workflow's build links any of these libraries.
|
||
- WAI's own decoders are Rust: the integer codecs, worlds, scores, films and the
|
||
generator. Outside the C ABI (`ffi.rs`, `ffi_codecs.rs`, `ffi_neural.rs`), the only
|
||
`unsafe` in `wai-rs`'s library code is in the AArch64 SIMD kernels of
|
||
`int_kernels.rs`, each block with the bounds argument beside it.
|
||
- `wai-web`, the browser crate, links none of the C libraries and runs inside the
|
||
browser's WebAssembly sandbox.
|
||
|
||
Model checking covers part of the integer core: Kani harnesses beside the code in
|
||
`rans.rs`, `int_entropy.rs`, `int_transform.rs`, `wai_video_policy.rs`,
|
||
`staged/predict.rs` and `staged/wir1_arith.rs`, hosted by `wai-verify` and run by
|
||
`.github/workflows/wai-formal-verification.yml`. `wai-verify` also compiles
|
||
`int_kernels.rs`, on its scalar path, because `int_transform.rs` calls it, but no
|
||
harness reaches it: its AArch64 SIMD kernels, the only `unsafe` named above, are not
|
||
model-checked. Equality tests hold them to the scalar reference kernels, element by
|
||
element beside each kernel and over every committed integer-decoder vector
|
||
(`tests/int_kernels_vectors.rs`), on every platform leg of `wai-conformance.yml`.
|
||
|
||
The extension formats are held to the reader limits they share with SPEC §2
|
||
(`wai::limits`): a score or haptic signal renders at most 2²⁷ samples counting every
|
||
channel, an audio scene's stereo floor the same, and a splat4d clip, a film or a reel's
|
||
shots together at most 2²⁰ frames. Each reader refuses a contract past them when it
|
||
reads it (`duration_too_long`), before anything is rendered, and refuses a film or reel
|
||
schedule whose last tick a `u64` does not hold (the tick arithmetic used to wrap).
|
||
`tests/extension_duration_limits.rs` checks each at the limit and one past it.
|
||
|
||
The reader is fuzzed with coverage guidance. `wai-rs/fuzz/` holds four libFuzzer
|
||
targets, built by cargo-fuzz with AddressSanitizer:
|
||
- `envelope`: version detection, the `WAI1` and `WAI2` readers and the listing form;
|
||
- `int_payload`: `int_payload::decode_int` against each parameter set the integer
|
||
conformance corpus commits;
|
||
- `media`: the picture, audio, video and text decoders of the floor and the recommended
|
||
set;
|
||
- `containers`: the extension containers (world, causal op, film, reel, score, haptic,
|
||
splat4d, audio scene).
|
||
|
||
They are seeded from the repository's own envelopes, integer payloads and containers,
|
||
and from a valid payload per decoder. `wai-fuzz.yml` runs each for 15 minutes every
|
||
night on the macOS runner and keeps the corpus between nights. A find fails the job
|
||
and keeps its input; `fuzz/run.sh` says how to turn one into a regression test. On every
|
||
change, `tests/fuzz_smoke.rs` still runs seeded random bytes through each wrapper and
|
||
`Wai::from_bytes`.
|
||
|
||
Known gaps:
|
||
|
||
- **The C libraries are not instrumented.** AddressSanitizer covers the Rust code. The
|
||
codec libraries run as the system builds them, so a memory error inside one is seen
|
||
only when it crashes the process or exceeds a limit.
|
||
- **Replaying a world costs CPU in proportion to the tick it is asked for.** A film or
|
||
reel samples its embedded world at the ticks its schedule names, and the reference
|
||
replay steps every tick from genesis. The schedule's frames are bounded and its last
|
||
tick must fit a `u64`, but no limit yet bounds how far a schedule may reach, so a
|
||
small film can ask a renderer for a long replay. Memory is bounded; time is not.
|
||
- **The float neural decoders are not fuzzed.** They need the `neural` feature and its
|
||
runtime. The integer decoders are fuzzed.
|
||
|
||
## Module authorisation
|
||
|
||
A module is code a sink obtains apart from an envelope and runs to decode a
|
||
capability or to rebuild a reconstruction ([SPEC](SPEC.md) §4, *Module
|
||
authorisation*). A sink runs one only if it holds it under
|
||
[jwp-receipts](extensions/jwp-receipts.md) §7.9: a grant, signed under a module
|
||
origin's key log, naming the SHA-256 of the module's bytes exactly as the sink
|
||
instantiates them, its form, the capabilities it may decode and a validity window,
|
||
with revocation through the same key log. A module the sink does not hold leaves its
|
||
capability unadvertised for that envelope, and the envelope's fallback still runs.
|
||
Parameter sets pinned by digest are checked before use and revoked the same way
|
||
(SPEC §3.1, checks 5–8).
|
||
|
||
The reference dispatcher runs no module obtained apart from an envelope. With a key
|
||
log (`key_log::SinkKeyLog`, feature `key_discovery`) it refuses a revoked parameter
|
||
set (`prior_revoked`).
|
||
|
||
Authorisation decides which code runs, not what that code can reach. The
|
||
environment a module runs in (its imports, a memory limit, a time limit, and the
|
||
determinism rules an exact replay needs) is not specified yet. Until it is, a sink
|
||
that runs modules confines them itself, for example in a WebAssembly runtime that
|
||
gives the module no file-system, network, clock or random-number imports.
|
||
|
||
## Fingerprinting: advertised parameter sets
|
||
|
||
The Sink Capability Descriptor ([http-delivery](extensions/http-delivery.md) §12.1)
|
||
can list `held_sets`: for each capability, the digests of the parameter sets the
|
||
sink holds. Which sets a sink holds depends on what it has fetched before, so the
|
||
list can identify a sink across the sites it is sent to, much as a cookie does. A
|
||
shared compression dictionary named by its hash raises the same concern (RFC 9842).
|
||
|
||
Privacy rules for the descriptor and for parameter-set fetches are not written yet.
|
||
They are to be added to http-delivery: the list kept per top-level site, clearable by
|
||
the user, disclosed only as far as a choice of rendition needs it, never sent across
|
||
sites, and off by default. Until they land, send `held_sets` only to the origin that
|
||
needs it to choose a rendition, and never across sites. No code in this tree builds
|
||
or sends the descriptor.
|
||
|
||
## Protected content
|
||
|
||
End-to-end encryption at the application layer (a room's sealed objects, SPEC §6 and [confidentiality](extensions/confidentiality.md)) keeps content
|
||
from relays and from anyone outside the group, not from a member's device: the
|
||
decrypted payload and the decoded output are in the application's memory. It is not a
|
||
protected-path robustness level ([SPEC](SPEC.md) §6.5).
|