eacf33dd6a
Merged head 17ff17d84f.
By meaning:
- codecs/mod.rs: is_companion_only lists the HDR descriptors, the gain map,
LCEVC and both DRED layers; the licence test's patent-pool list keeps
DRED's and the HDR companions' entries.
- tests/carriage.rs: base_of() (DRED's) gains B3b's gain-map case, an
image base (wai.image.png); the test reads the base from base_of() as
DRED left it.
- SPEC §7.1: the companion list names the HDR descriptors with ST 2094-50,
the gain map, LCEVC and the DRED layers; rule 2 keeps the gain map's
image class and DRED's base admission side by side.
- determinism-tiers §2: exact-upscale's paragraph (pinned integer
derivations keep the tier), then B3b's (HDR display adaptation, pinned
by exact-tonemap).
- wai-conformance.yml: paths unioned; the step that conflicted is rebuilt
as two whole steps, QoE aggregation's then exact tone mapping's.
- Shared lists kept both sides: CHANGELOG (B3b's entry on top), REGISTRY
rows in order, registry constants and wire identifiers, the
live-conformance index (tonemap/ row, its provenance sentence, its
verifier line). capabilities.json and v2.ttl taken from the PR and
regenerated next.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
766 lines
64 KiB
Markdown
766 lines
64 KiB
Markdown
# Media-Codec Registry — immersive audio, cinema, volumetric, haptics, HDR, broadcast and streaming
|
||
|
||
> Reference for the capabilities WAI dispatches to beyond the flat-media floor
|
||
> (PNG/FLAC/zstd) and modern menu (AVIF/JXL/Opus/AV1) and the neural bench.
|
||
> These cover object/scene audio, the theatrical mezzanine path, 3D-geometry
|
||
> compression, the standardized haptics codec, HDR dynamic metadata, and the
|
||
> broadcast and streaming video / audio bitstreams distribution chains carry.
|
||
>
|
||
> **Registration ≠ a bundled decoder.** As everywhere in WAI, registering a
|
||
> capability fixes the *capability string* and says "payload = the named
|
||
> standard's canonical bytes; the decoder is sink/library-supplied." The point
|
||
> is one envelope that can *name* any of these and fall back cleanly — not
|
||
> re-implementing them. The "ingest, not compete" posture: dispatch to the
|
||
> field's open reference where one exists, flag the proprietary/SEP-pool ones
|
||
> honestly, and never make a non-floor capability a default.
|
||
|
||
Royalty key: **RF** = royalty-free · **PROP** = proprietary license · **POOL**
|
||
= patent-pool / SEP royalties apply. Status checked against primary sources
|
||
**June 2026**, and **September 2026** for the broadcast and streaming rows. The
|
||
footnotes give each row's basis and residual caveats; two broadcast rows are
|
||
classifications rather than verified statuses — AVS3 (¹⁰: the fee schedule is not
|
||
published) and EVC Baseline (⁹: royalty-free by design, with no formal guarantee).
|
||
|
||
## License classification is enforceable, not just documented
|
||
|
||
Every registered capability carries an IP class in the registry —
|
||
`codecs::capability_license(cap) -> LicenseClass` ∈ {`RoyaltyFree`,
|
||
`PatentPool`, `Proprietary`, `NonCommercial`}, with `is_royalty_free(cap)` as
|
||
the predicate. It is a property of the **capability**, looked up in the
|
||
registry — deliberately **not** a per-file manifest claim, because a codec's
|
||
licence can't be honestly asserted in an envelope's own header. A deployer
|
||
audits a title or a whole catalogue in code: "encode only to royalty-free
|
||
capabilities", or "flag every pool/proprietary capability in this library."
|
||
That makes the per-title licensing choice a property of the format, checkable
|
||
in code. `wai capabilities` prints the class per row (the registry
|
||
reports the royalty-free count outright).
|
||
|
||
The classification is **exhaustive, with no default class**. Every registered
|
||
capability sits under an explicit arm of `capability_license`; one registered
|
||
without a class returns `None`, and `licence_classification_is_exhaustive_with_no_default`
|
||
fails. That test also pins the full non-royalty-free set, so adding or
|
||
reclassifying a pool / proprietary / non-commercial capability is a reviewed
|
||
diff. (Until September 2026 the function ended in a royalty-free fall-through,
|
||
so a newly registered pool codec would have been classified royalty-free by
|
||
omission.) `defaults_are_royalty_free_and_never_pool_or_proprietary` holds the
|
||
other half of the guarantee: the mandatory floor and the recommended set
|
||
(`codecs::mandatory_floor()` / `recommended_set()`, equal to the native sink
|
||
set) are all royalty-free, and no pool, proprietary or non-commercial
|
||
capability is among them.
|
||
|
||
## Replicate class: which reconstructions are byte-identical across decoders
|
||
|
||
`codecs::capability_replicate_class(cap)` answers a second registry question:
|
||
what does every conforming sink reconstruct? For a standard codec the answer is
|
||
what its named standard's decoder conformance fixes, not anything WAI adds; the
|
||
per-standard reading is in *Decoder conformance* below. Two rules keep
|
||
the answer honest:
|
||
|
||
- **Exhaustive, with no default.** Every registered capability sits under an
|
||
explicit arm; one registered without a class returns `None`, and the registry
|
||
tests reject it. Until September 2026 the function ended in a
|
||
`StandardDefined` fall-through, which gave AAC, Opus, JPEG and the other
|
||
tolerance-conformance codecs — and `wai.mesh.glb` and `wai.image.jpegai` — a
|
||
same-bytes guarantee nothing had established.
|
||
- **An exact class needs recorded evidence.** `StandardDefined` promises the same
|
||
bytes on every conforming sink, so a capability enters it only with a basis: a
|
||
conformance clause that requires identical output, or the WAI extension whose
|
||
`*-equivalence` pins the reconstruction. The registry tests pin one basis per
|
||
member (`STANDARD_DEFINED_EVIDENCE`), and a `StandardDefined` capability without
|
||
one fails `standard_defined_rests_on_recorded_evidence`. Without that evidence a
|
||
standard codec is `StandardUnclassified`, the conservative class, which
|
||
promises no byte identity.
|
||
|
||
`is_cross_decoder_exact(cap)` is true for the classes whose
|
||
`replicate_promise` is `ByteExact` (`StandardDefined`, `NeuralIntegerExact`),
|
||
except a capability whose output is computed from other envelopes' decoded
|
||
output (`composes_decoded_sources`: `wai.audio.scene`, whose mixdown is exact
|
||
given its sources' PCM). It is the one-call audit for "a hash over the decoded
|
||
output verifies at another conforming sink", answered from the capability
|
||
alone; for a scene, `is_cross_decoder_exact_with_sources` answers once its
|
||
sources' capabilities are known, and answers false when it cannot tell (no source
|
||
given, or a source outside `wai.audio.*`). It covers the output the standard or extension specifies — for a video
|
||
codec the cropped decoded pictures, before colour conversion — not a conversion a
|
||
sink applies afterwards.
|
||
|
||
**Measured** with `tools/wai_replicate_probe.sh` on an arm64 macOS laptop (18
|
||
logical cores, macOS 27.0) with ffmpeg 9.0.2, September 2026. Each format was encoded from a
|
||
synthetic source: 20 frames of 320×240 `testsrc2` video, a 1 s 440 Hz 48 kHz
|
||
mono tone, and a 320×240 grayscale still, so that no colour conversion enters
|
||
the comparison. Each was decoded by every independent decoder present, and
|
||
the raw outputs were compared byte for byte. VideoToolbox decodes were forced
|
||
onto hardware surfaces; a software-only format under the same flags fails
|
||
rather than falling back.
|
||
|
||
| capability | decoders | result |
|
||
|---|---|---|
|
||
| `wai.video.avc` (libx264, Annex B) | ffmpeg `h264` · VideoToolbox hardware | identical |
|
||
| `wai.video.hevc` (libx265, Annex B) | ffmpeg `hevc` · VideoToolbox hardware | identical |
|
||
| `wai.video.vp9` (libvpx, IVF) | ffmpeg `vp9` · libvpx · VideoToolbox hardware | identical |
|
||
| `wai.video.av1` (SVT-AV1, IVF) | dav1d · VideoToolbox hardware | identical |
|
||
| `wai.audio.flac` | ffmpeg · libFLAC · Core Audio, against the source PCM | identical |
|
||
| `wai.audio.aac` (ffmpeg `aac`, ADTS) | ffmpeg `aac` · ffmpeg `aac_fixed` · AudioToolbox | differ: 238 of 49 152 samples (max 1 LSB) float vs fixed; 8 629 (max 23 LSB) against AudioToolbox |
|
||
| `wai.audio.opus` (libopus, 64 kb/s) | ffmpeg `opus` · libopus | differ: 13 of 48 000 samples, max 1 LSB |
|
||
| `wai.image.jpeg` (cjpeg q75, grayscale) | libjpeg-turbo accurate-integer IDCT · its float IDCT · ffmpeg `mjpeg` | differ: 169 and 673 of 76 800 samples, max 1 |
|
||
| `wai.image.jp2`, irreversible 9/7 (OpenJPEG, 20:1) | OpenJPEG · ffmpeg `jpeg2000` | differ: 67 of 76 800 samples, max 1 |
|
||
| `wai.image.jp2`, reversible 5/3 | OpenJPEG · ffmpeg `jpeg2000`, against the source | identical (the lossless path) |
|
||
|
||
One stream per format on one host is a demonstration, not a survey, and it is not
|
||
the basis of any class: the classes rest on the conformance clauses in *Decoder
|
||
conformance* below. The measurement is a falsification check. An identical result on a
|
||
`StandardDefined` row is what its standard requires; a difference there would
|
||
contradict the class, and the probe exits non-zero (a perturbed hardware decode
|
||
run through the probe does exactly that). A `StandardTolerance` or
|
||
`StandardUnclassified` row may match on a given stream; the class permits it.
|
||
AV1 was measured identical here, but it is `StandardTolerance`: its
|
||
specification requires identical output only from streams that signal no film
|
||
grain. AAC is `StandardUnclassified`: its decoders differ here, and its
|
||
conformance document (ISO/IEC 14496-26) was not read, so the registry does not
|
||
say whether that difference is within its criterion.
|
||
|
||
## Immersive / object-based audio
|
||
|
||
| capability | standard | royalty | reference | tier |
|
||
|---|---|---|---|---|
|
||
| `wai.audio.iamf` | IAMF (AOMedia Immersive Audio Model & Formats) | **RF** | libiamf (Apache-2.0) | Lossless¹ |
|
||
| `wai.audio.ac4` | AC-4 (ETSI TS 103 190) | **PROP** | single licensor | Lossy |
|
||
| `wai.audio.ivas` | 3GPP IVAS (Immersive Voice & Audio Services, EVS-based) | **POOL**² | 3GPP | Lossy |
|
||
|
||
The floor (`wai.audio.{opus,flac}`) + the deterministic object mix
|
||
(`wai.audio.scene`) cover stereo and the bit-exact mixdown; these add the
|
||
standard *immersive carriers*. IAMF is open, royalty-free, from the Alliance
|
||
for Open Media, and codec-agnostic (it carries Opus / FLAC / LPCM for channel,
|
||
scene/ambisonic, and binaural audio), which fits "name the capability"
|
||
directly. AC-4 is registered as a licence-flagged slot so object-audio content
|
||
already encoded in it can ride inside a WAI envelope. IVAS is the
|
||
conversational/telepresence immersive codec.
|
||
|
||
¹ IAMF can carry lossless codecs (FLAC/LPCM) → lossless ceiling. **Verified
|
||
RF**: published under the AOMedia Patent License 1.0 ("no-charge, royalty-free,
|
||
irrevocable"); reference under BSD-3-Clause-Clear. (Deployed in streaming and
|
||
consumer devices in 2025.)
|
||
² **Verified POOL**: IVAS is in a voice-codec patent pool (launched Dec 2024,
|
||
alongside EVS) — **not** royalty-free.
|
||
|
||
## Cinema mastering / intra-frame mezzanine
|
||
|
||
| capability | standard | royalty | reference | tier |
|
||
|---|---|---|---|---|
|
||
| `wai.image.jp2` | JPEG 2000 still (ISO/IEC 15444-1; DCP via SMPTE ST 429 / MXF) | RF core³ | OpenJPEG (BSD-2) | Lossless |
|
||
| `wai.video.j2k` | Intra-frame JPEG 2000 video (each frame an independent codestream) | RF core³ | OpenJPEG | Lossless |
|
||
|
||
The theatrical interchange path: a DCP is JPEG 2000 in MXF. The proprietary
|
||
production mezzanine formats are out of scope for a royalty-free floor (a
|
||
licence-flagged slot could be added if a workflow needs one).
|
||
|
||
³ **Verified RF (Part 1)**: JPEG 2000 Part 1 (the core coding system) is
|
||
licensable fee-free under ITU-T patent policy 2.1 — 20+ organisations agreed to
|
||
license essential IP for the standard without royalties. Residual caveats: the
|
||
committee never gave a *formal* guarantee, some **extensions** may carry fees,
|
||
and a 2004 note acknowledged possible undeclared "submarine" patents. Safe for
|
||
the DCP/mezzanine Part-1 path; don't extend the RF claim to Part 2+.
|
||
|
||
## Volumetric / 3D geometry compression
|
||
|
||
| capability | standard | royalty | reference | tier |
|
||
|---|---|---|---|---|
|
||
| `wai.mesh.draco` | Draco (open-source library) mesh + point-cloud geometry | **RF** (Apache-2.0) | draco | Lossless⁴ |
|
||
| `wai.pc.gpcc` | MPEG G-PCC (ISO/IEC 23090-9), sparse / LiDAR clouds | **POOL**¹⁴ | mpeg-pcc-tmc13 | Lossy |
|
||
| `wai.pc.vpcc` | MPEG V-PCC (ISO/IEC 23090-5), dense / dynamic clouds | **POOL**¹⁴ | mpeg-pcc-tmc2 | Lossy |
|
||
| `wai.splat.sog` | Gaussian-splat compression — a **candidate**: no payload format pinned (registered for the Self-Organizing Gaussians family, SOG / SOGS) | POOL, provisional: classified conservatively, since the licensing of a format not yet pinned is not established | none cited | Lossy |
|
||
|
||
These compress the heavy asset clouds the existing capabilities already ship
|
||
(`wai.mesh.glb`, `wai.splat.spz`, `wai.splat.4d`). `wai.splat.sog` is a
|
||
**candidate**. The string was registered in June 2026 for the Self-Organizing
|
||
Gaussians family of splat formats before any payload format was pinned, and none
|
||
is pinned yet, so SPEC §5 (*Candidate capabilities*) makes it unemittable: an
|
||
encoder MUST NOT emit it and a sink treats it as unsupported. It pins when a
|
||
revision cites a published specification of the format and states its framing.
|
||
It was registered royalty-free, but no licence of a format specification or of
|
||
reference code was read, so that class was never established; it is now
|
||
classified patent-pool, conservatively and provisionally, until a revision pins
|
||
its payload format and the format's licensing is read. No
|
||
MPEG page or document read for this registry names Self-Organizing Gaussians, SOG
|
||
or SOGS (*MPEG's Gaussian-splat and point-cloud work*, below).
|
||
|
||
⁴ Draco mesh geometry has a lossless mode (exact-geometry ceiling, like
|
||
`wai.mesh.glb`); its quantization knobs are the encoder's choice.
|
||
|
||
¹⁴ No licensing programme for G-PCC or V-PCC was read. Both are MPEG standards
|
||
with patent declarations, and the class is a conservative classification, as
|
||
footnote ⁵ records for MIHS: not asserted royalty-free.
|
||
|
||
### MPEG's Gaussian-splat and point-cloud work (read 2026-09-30)
|
||
|
||
What MPEG's public pages and public output documents said on 2026-09-30, for the
|
||
three volumetric registrations above. The last column says what was read. The
|
||
meeting pages were read through a tool that returns an extraction of each page;
|
||
the output documents are the public files the Exploration 45 page links,
|
||
downloaded from mpeg.org and read in full. The draft call for proposals (ISO/IEC
|
||
JTC 1/SC 29/WG 4 N 836) and the amendment texts (WG 7 N 1453 and N 1454) are not
|
||
public (HTTP 404), and iso.org refused automated reads (HTTP 403), so no ISO
|
||
catalogue stage or edition is confirmed here.
|
||
|
||
| item | stage and what the source states | read |
|
||
|---|---|---|
|
||
| Exploration 45, *Gaussian splat coding* (edition 1, "Implicit Neural Visual Representation") | "ongoing"; documents run from MPEG 150 to MPEG 155; the page names no working group and no coding technology beyond "GSC" | https://www.mpeg.org/standards/Explorations/45/ |
|
||
| Who works on it | "Gaussian Splat Coding standardization is coordinated across WG 4 (Video Coding), WG 5 / JVET, and WG 7 (3D Graphics and Haptics)"; documents for the joint exploration at the next meeting "are to be managed by WG 5" | ISO/IEC JTC 1/SC 29/WG 4 N 837, *FAQ on Gaussian splat coding* (MPEG 154), https://www.mpeg.org/wp-content/uploads/mpeg_meetings/154_SantaEularia/w26333.zip |
|
||
| The call for proposals | N 837: "MPEG approved the first draft Call for Proposals (CfP) for Gaussian Splat Coding". N 422: "MPEG WGs are currently preparing a Call for Proposals (CfP) covering coding technology for static and dynamic objects and scenes represented as I-3DGS, where I-3DGS denotes per-frame 3DGS content in the original INRIA format"; test material is wanted "before October 15th 2026", for MPEG 156 | WG 4 N 837; WG 5 N 422, *Call for content for Gaussian splats coding* (MPEG 155), https://www.mpeg.org/wp-content/uploads/mpeg_meetings/155_Geneva/w26655.zip |
|
||
| I-3DGS and A-3DGS | I-3DGS "Refers to point clouds represented in the “INRIA” format"; A-3DGS "Encompasses alternative Gaussian splat representations" | WG 4 N 837; WG 2 N 531, *Gaussian splat coding use cases* (MPEG 155), https://www.mpeg.org/wp-content/uploads/mpeg_meetings/155_Geneva/w26708.zip |
|
||
| Anchors | a G-PCC anchor coded with the test model of "ISO/IEC 23090-9:2023"; a V-PCC anchor on "the V-PCC GS toolset and RecGS reconstruction profile of ISO/IEC 23090-5"; a video-based anchor that "uses packing of Gaussian splat positions and attributes to video planes that are encoded as separate videos using HEVC (HM)". An anchor "does not predetermine the architecture of a future GSC standard" (N 837) | WG 4 N 826 (MPEG 154), https://www.mpeg.org/wp-content/uploads/mpeg_meetings/154_SantaEularia/w26322.zip, and WG 4 N 751 (MPEG 152), https://www.mpeg.org/wp-content/uploads/mpeg_meetings/152_Geneva/w25782.zip, both titled *Common test conditions for Gaussian splat coding*; WG 4 N 837 |
|
||
| V-PCC amendment (ISO/IEC 23090-5) | Committee Draft Amendment at MPEG 153, promoted by "MPEG 3D Graphics and Haptics Coding (WG 7)", defining "a V-PCC profile specialized for Gaussian Splats"; Draft Amendment at MPEG 154 | https://www.mpeg.org/153rd-meeting-of-mpeg/; WG 4 N 837 |
|
||
| G-PCC amendment (ISO/IEC 23090-9) | "the first amendment to Geometry-based Point Cloud Compression, ISO/IEC 23090-9", at Committee Draft Amendment at MPEG 154 | https://www.mpeg.org/154th-meeting-of-mpeg/; WG 4 N 837 |
|
||
| Both amendments | "The AMD1 amendments to both standards introduce explicit support for Gaussian Splat parameters in their respective syntaxes and reconstruction processes." The V-PCC amendment adds a reconstruction profile because "Without a clearly specified reconstruction profile, different implementations could decode the same bitstream into different Gaussian Splat parameters" | WG 4 N 837 |
|
||
| NNC amendment (ISO/IEC 15938-17) | a Committee Draft of an amendment at MPEG 154, extending NNC to "parameters of 3D and 4D scene representations such as Gaussian splats" | https://www.mpeg.org/154th-meeting-of-mpeg/; [2030 landscape](2030-transport-landscape.md) §6.4 |
|
||
| Enhanced G-PCC, ISO/IEC 23090-38 | FDIS at MPEG 155; "a new generation of geometry-based point cloud compression specifically designed for dynamic point cloud sequences"; "While the original G-PCC standard (MPEG-I Part 9) primarily compresses point clouds on a frame-by-frame basis, the Enhanced G-PCC introduces a temporal coding framework". Neither page says whether a 23090-9 decoder decodes it | https://www.mpeg.org/155th-meeting-of-mpeg/; https://www.mpeg.org/standards/MPEG-I/38/ |
|
||
| MPEG-I Part 40, conformance and reference software for Enhanced G-PCC | Committee Draft at MPEG 152, for "the enhanced edition of the Geometry-based Point Cloud Compression (E-G-PCC) standard" | https://www.mpeg.org/152nd-meeting-of-mpeg/ |
|
||
| ISO/IEC 23090-41, G-PCC for solid objects | Committee Draft at MPEG 154; it "complements the original G-PCC standard, ISO/IEC 23090-9, which was primarily designed for sparse point clouds" | https://www.mpeg.org/154th-meeting-of-mpeg/ |
|
||
| ISO/IEC 23090-5, fourth edition | FDIS at MPEG 152, "allowing V3C to also serve as the structural foundation for V-DMC"; before it, "V3C could be instantiated for V-PCC (Video-based Point Cloud Compression) and MIV (MPEG Immersive Video)" | https://www.mpeg.org/152nd-meeting-of-mpeg/ |
|
||
|
||
*What this settles.* (a) No page or document above names Self-Organizing
|
||
Gaussians, SOG or SOGS, and MPEG's call concerns content in a representation
|
||
(I-3DGS), not a chosen coding technology: `wai.splat.sog` names no MPEG work item.
|
||
Whatever MPEG standardises for Gaussian splats takes a capability string of its
|
||
own when it is published; `wai.splat.sog` cannot be repurposed for it (SPEC §8).
|
||
(c) Enhanced G-PCC is a separate part with its own conformance part, and no source
|
||
read says a 23090-9 decoder decodes it, so it is not a `wai.pc.gpcc` payload
|
||
(SPEC §5, *Point clouds*); a MINOR revision may register it under its own string
|
||
once 23090-38 is published and its framing is pinned, and likewise 23090-41. What
|
||
`wai.pc.gpcc` and `wai.pc.vpcc` carry is open item (i).
|
||
|
||
## Standardized haptics
|
||
|
||
| capability | standard | royalty | reference | tier |
|
||
|---|---|---|---|---|
|
||
| `wai.haptic.mihs` | MPEG-I Haptics (ISO/IEC 23090-31, Jan 2025) | flag⁵ | MPEG RM | Lossy |
|
||
|
||
Complements the native `wai.haptic.signal` (the keyframe-envelope, Fx-floor,
|
||
sample-equivalence path). MIHS is the ISO standard codec: human-readable HJIF
|
||
(JSON) + a binary MIHS stream (~8–16 kbps). **ISO/IEC 23090-33 defines
|
||
bit-exact decoder verification + valid/invalid conformance datasets** — the
|
||
conformance-vector philosophy already in the standard, so a `*-equivalence`
|
||
conformance corpus is straightforward to mirror.
|
||
|
||
⁵ **Unconfirmed → classified POOL conservatively.** No public royalty-free
|
||
declaration was found; it is an MPEG/ISO standard, so the ISO patent-declaration
|
||
process applies and a pool is the likely outcome (as with the MPEG point
|
||
clouds). Treat as pool until the ITTF patent database / a pool announcement
|
||
confirms otherwise — do not assert royalty-free.
|
||
|
||
## HDR dynamic metadata (descriptors, not media payloads)
|
||
|
||
| capability | standard | royalty | tier |
|
||
|---|---|---|---|
|
||
| `wai.meta.hdr10plus` | HDR10+ dynamic metadata (SMPTE ST 2094-40) | **RF** (content) | Lossless |
|
||
| `wai.meta.dovi` | SMPTE ST 2094-10 dynamic metadata (RPU) | **PROP** | Lossless |
|
||
| `wai.meta.st2094_50` | SMPTE ST 2094-50:2026-08 dynamic metadata (Application #5) | **POOL** (not established) | Lossless |
|
||
| `wai.meta.gainmap` | ISO 21496-1:2025 gain map (metadata and gain map image) | **POOL** (not established) | Lossless |
|
||
|
||
**The two rows added 2026-10-22.** ST 2094-50:2026-08 was read in its published text: its
|
||
Foreword records that compliance may involve claimed IPR, that each holder assured SMPTE it will
|
||
license on reasonable terms "demonstrably free of discrimination", and that no royalty-free
|
||
commitment is stated; it is classified conservatively (pool), as NNC is. ISO 21496-1:2025 (first
|
||
edition, ISO/TC 42) was not read: the text is sold, and its public preview could not be fetched;
|
||
its patent declarations were not checked, so it too is classified conservatively. Its gain map
|
||
is a companion of an image payload, the first companion bound to an image rather than a video;
|
||
SPEC §7.1's rule 2 gained the image class for it. What [exact-tonemap](../extensions/exact-tonemap.md)
|
||
states of ISO 21496-1 (its binary metadata's fields, its application formula) is what ST
|
||
2094-50:2026-08's terms 3.1 to 3.11 say of it and what public, open-source readers of its binary
|
||
metadata implement; each such statement is marked in the extension.
|
||
|
||
**Architectural note.** Unlike the codecs above, HDR metadata is *not* a
|
||
standalone media payload — it is color / dynamic-range metadata that
|
||
accompanies a video payload (carried in SEI / ST 2094). It stresses the §2
|
||
invariant ("payload = the named codec's canonical bytes"). The clean design:
|
||
these are **companion descriptors** a manifest declares alongside a video
|
||
capability; the video bytes stay the video capability's, and the descriptor is
|
||
bound as a companion component (the multi-component / multi-rendition envelope's
|
||
job — v1.1). Static HDR10 (SMPTE ST 2086) and HLG (ARIB STD-B67) are baseline
|
||
characteristics already in-band in the video bitstream and need only a manifest
|
||
color-descriptor field, not a separate capability. **This is the one place to
|
||
resolve in SPEC before an integration partner's engineers ask.**
|
||
|
||
## Decoder conformance — what each named standard fixes
|
||
|
||
SPEC §7 classes every standard codec — the floor and modern menu, the broadcast
|
||
codecs, and the capabilities above and below — by what its named standard's
|
||
decoder conformance fixes,
|
||
because that is what `intent = replicate` over it promises: the same bytes on
|
||
every conforming sink (`StandardDefined`), a criterion around a reference
|
||
(`StandardTolerance`), or not yet classified (`StandardUnclassified`). Checked
|
||
2026-09-27. The last column says what was read; a row is only as strong as that.
|
||
A row reaches `StandardDefined` only on a clause that requires identical output;
|
||
a standard whose conformance text was not read stays `StandardUnclassified`.
|
||
|
||
| capability | class | what the standard fixes | read |
|
||
|---|---|---|---|
|
||
| `wai.image.png` | `StandardDefined` | lossless: "All PNG filters are strictly lossless", and the Paeth predictor's calculations "must be performed exactly". Reducing to a lesser sample depth is the decoder's choice: §10.4 gives a rounding formula and a right shift | RFC 2083, full text |
|
||
| `wai.audio.flac` | `StandardDefined` | lossless: decompressing "returns exactly the original data" (§3); STREAMINFO carries an MD5 of the unencoded audio | RFC 9639, definitions and STREAMINFO table |
|
||
| `wai.text.zstd`, `wai.text.xz` | `StandardDefined` | lossless byte formats | not re-read |
|
||
| `wai.video.avc` | `StandardDefined` | §C.4: test bitstreams go to the HRD and the decoder under test; "for each picture output by the HRD, the values of all samples that are output by the DUT for the corresponding picture shall be equal to the values of the samples output by the HRD". Film grain characteristics are an SEI message, outside the decoding process | Rec. ITU-T H.264 (V16) (06/2026), §C.4 |
|
||
| `wai.video.hevc`, `wai.video.vvc` | `StandardDefined` | §C.5.1: "for each such cropped decoded picture output by the DUT, the values of all samples that are output shall be equal to the values of the samples produced by the specified decoding process". Film grain characteristics (and, in H.266, other post-processing hints) are SEI messages, outside the decoding process | Rec. ITU-T H.265 (V11) (01/2026) and H.266 (V4) (01/2026), §C.5.1 |
|
||
| `wai.video.vp9` | `StandardDefined` | §8.1: "Decoders shall produce output frames that are identical in all respects and have the same output order as those produced by the decoding process specified herein" | VP9 Bitstream & Decoding Process Specification v0.6 (31 March 2016), §8.1 |
|
||
| `wai.audio.opus`, `wai.audio.opus.packet` | `StandardTolerance` | §6: conformance is "defined through the reference implementation"; a decoder's output "MUST also be within the thresholds specified by the opus_compare.c tool" against the reference on the test vectors, with "the same final range decoder state". §6.1: quality 0 is the passing threshold, calibrated to additive white noise at 48 dB SNR; 100 means identical to the reference. §1.1: "Various operations in the codec require bit-exact fixed-point behavior"; §4.2.7.9: the rest of SILK frame reconstruction "does not need to be bit-exact" | RFC 6716 §1.1, §4.2.7.9, §6–§6.1 |
|
||
| `wai.audio.opus.dred` | `StandardTolerance` | §4.1: "DRED acoustic feature decoding is strictly defined. The decoder implementation MUST use the same weights provided in this specification. The DNN weights can be further quantized and the exact implementation of the DNN arithmetic (including activation functions) can be approximated, provided that they comply with the following test"; the test compares decoded features with reference features, and its script passes at total ≤ 0.5, max ≤ 0.15 and pitch ≤ 0.02. §4.2: the vocoder is not normative. Measured (extensions/opus-dred.md §6): two builds of the draft's reference decoder on one host, and the build that wrote the test vectors, give three different feature sets from the same latents | draft-ietf-mlcodec-opus-dred-07 §2.2.2, §4.1, §4.2, §7.1 ([DRED-Compare-Script]) |
|
||
| `wai.audio.opus.dred.int` | `NeuralIntegerExact` | the same weights (§4.1 there), through the integer profile `dred-int/1`: every feature an exact integer function of the payload and the pinned weights file, the same on every conforming sink; within the draft's thresholds of its reference features | extensions/opus-dred.md §4, §6 |
|
||
| `wai.image.jpeg` | `StandardTolerance` | §7: a decoder shall "with appropriate accuracy, convert to reconstructed image data"; A.3.3 NOTE: the IDCT equations "contain terms which cannot be represented with perfect accuracy by any real implementation", and the accuracy of dequantisation + IDCT is specified in Part 2; §4.1: colour space "is outside the scope of this Specification". The lossless process (Annex H) is exact, but the capability's role is the DCT processes | ITU-T T.81 (1992), full text. Part 2 (ITU-T T.83 \| ISO/IEC 10918-2) not read: the download failed |
|
||
| `wai.image.jxl` | `StandardTolerance` | §4.1.1 NOTE: JPEG XL specifies infinite-precision arithmetic, and "this document therefore specifies tolerances for the decoded samples"; §4.2: a decoded image is similar to the reference iff every sample is within a peak-error threshold and each channel's RMSE within an RMSE threshold; §4.3: thresholds per test (`peak_error`, `rms_error`) | ISO/IEC 18181-3:2022, public preview (clauses 1–4.3) |
|
||
| `wai.image.jp2`, `wai.video.j2k` | `StandardTolerance` | decoder compliance compares decoded components with reference components and their error metrics with specified values (Annex B.2.4, B.2.5; "Tolerances", G.4.3; MSE among the abbreviations; a "9-7I precision guarantee", A.1.7); the reversible 5-3 path is lossless (definitions 3.41, 3.58) | ISO/IEC 15444-4:2024 \| ITU-T T.803, public preview: table of contents, terms and abbreviations only. The clauses that state the metrics and thresholds were not in the preview |
|
||
| `wai.video.av1`, `wai.video.av1.lossless`, `wai.image.avif` | `StandardTolerance` | §7.2: without film grain, decoders "shall produce output frames that are identical in all respects" to the decoding process's; with film grain, a decoder may instead produce identical frames before grain synthesis and output frames that "do not have perceptually significant differences" from the reference grain synthesis; "The definition of “perceptually significant differences” is beyond the scope of this specification and may be specified, for example, by a service provider as part of their accreditation program", so this criterion is not a stated bound. The film-grain syntax does not depend on lossless coding, so `av1.lossless` is in the same class. A text search of AVIF v1.2.0 found no film-grain restriction | AV1 Bitstream & Decoding Process Specification v1.0.0 with Errata 1, §7.2 and §5.9.30; AVIF v1.2.0, text search |
|
||
| `wai.video.av2` | `StandardTolerance` | §7.1: with `film_grain_params_present` equal to 0, output frames "identical in all respects" to the decoding process's; equal to 1, a conformant decoder satisfies one of two options — the exact reference film grain synthesis (§7.21.7), or identical intermediate frames (§7.21.2) and output frames that "do not have perceptually significant differences" from the reference grain synthesis, a term the specification leaves to others to define. Annex A.5 (decoder conformance) adds only level capability and the display schedule | AV2 Bitstream & Decoding Process Specification v1.0.0 (28 May 2026), §7.1 and Annex A.5 |
|
||
| `wai.audio.ivas` | `StandardTolerance` | §7.1: bit-exact conformance, output matching the reference test sequences, for every implementation; §7.2.1: an implementation "based on floating–point code (TS 26.258)" whose output is not bit-exact is tested instead by a non-bit-exact procedure — TS 26.444's for mono operation (§7.2.1.1), and for stereo and immersive operation a per-frame maximum-loudness-difference corridor derived from reference builds that "are not bit-exact with each other nor with the reference implementation" (§7.2.1.2, Annex A) | 3GPP TS 26.252 V19.2.0 (2026-06), §7 and Annex A. TS 26.444 not read |
|
||
| `wai.audio.iamf` | `StandardTolerance` | substreams are Opus, AAC-LC, FLAC or LPCM (`codec_id`); "specific renderers are out of scope" (with reference to §2.2) | IAMF working draft of 21 April 2025, text search |
|
||
| `wai.video.vp8` | `StandardUnclassified` | not checked. It shares VP9's framing (IVF), not its class: VP9's §8.1 says nothing about VP8 | RFC 6386 read for its framing only (§4, §9.1, §9.5); its decoder-conformance text was not read for this registration (2026-10-18) |
|
||
| `wai.video.evc`, `wai.video.evc.baseline`, `wai.video.avs3`, `wai.audio.mpegh`, `wai.audio.avs3` | `StandardUnclassified` | not checked | nothing read for this purpose: ISO/IEC 23094-1 and 23008-3 and their conformance parts are not freely available, and T/AI 109.2 / 109.3 were not obtained |
|
||
| `wai.video.lcevc` | `StandardUnclassified` | not checked; its decoding process includes a signalled dither step (clause 8.11, "Decoding process for dither filter"), so identical output across decoders is not presumed | ISO/IEC 23094-2:2021 public preview sample (front matter to clause 5.8): table of contents and clause 3 only. The conformance part (ISO/IEC 23094-3) not read |
|
||
| `wai.audio.aac` | `StandardUnclassified` | not checked. Three decoders on one host gave three different PCM streams from one ADTS stream (measured above), so deployed decoders do not agree byte for byte; whether ISO/IEC 14496-26 admits that, or makes some of them non-conforming, was not established | nothing read for this purpose |
|
||
| `wai.meta.st2094_50`, `wai.meta.gainmap` | `StandardUnclassified` | ST 2094-50:2026-08 states its tone map in real numbers (§6.2.5, Formulas (2) to (5)) and gives no decoder conformance over rendered pixels, so identical renditions across sinks are not presumed; the integer derivations of exact-tonemap are WAI's, not a property of the capability | ST 2094-50:2026-08, every clause and annex; ISO 21496-1 not read (2026-10-22) |
|
||
| `wai.audio.ac4`, `wai.mesh.draco`, `wai.pc.gpcc`, `wai.pc.vpcc`, `wai.splat.sog`, `wai.haptic.mihs`, `wai.meta.hdr10plus`, `wai.meta.dovi` | `StandardUnclassified` | not checked | nothing read for this purpose. (`wai.haptic.mihs`'s registry comment and SPEC §5 say ISO/IEC 23090-33 defines bit-exact decoder verification; that rests on a summary, not the standard text, and was not checked here); `wai.splat.sog` is a candidate with no pinned payload, so there is no conformance text for it to read |
|
||
| `wai.mesh.glb` | `StandardUnclassified` | the payload bytes pass through intact; what a sink decodes from them is not classified. A glTF binary carries more than raw buffers: glTF 2.0 images may be PNG or JPEG ("JPEG images MUST be compatible with JPEG File Interchange Format"), embedded through a `bufferView`, and geometry may be compressed through extensions (`KHR_draco_mesh_compression`; the vendor extension `EXT_meshopt_compression`). JPEG is `StandardTolerance` and Draco `StandardUnclassified` above, so a scene built from those can differ between conforming sinks; and glTF specifies an asset format, not a decoder conformance over rendered output (edge appearance, for one, "MAY be subject to implementation-specific techniques") | glTF 2.0.1 specification (Khronos registry HTML), text search; Khronos glTF extension registry README |
|
||
| `wai.splat.spz` | `StandardUnclassified` | the payload bytes pass through intact; SPZ stores quantised attributes, and its decode is not classified | not read |
|
||
|
||
Two limits apply to every row. The standard fixes its decoded output in its own
|
||
domain (sample depth, planes, colour representation); a conversion a sink applies
|
||
afterwards — PNG depth reduction, YUV→RGB, a JPEG colour-space choice, rendering an
|
||
IAMF mix — is not part of it, and the reference sink's `RGB8` / f32 buffers include
|
||
such conversions. And the class describes the standard, not the reference sink's
|
||
libraries: nothing here tests those libraries against their standard's conformance
|
||
suite.
|
||
|
||
JPEG AI (`wai.image.jpegai`) is not in a standard class: its registry class is
|
||
`NeuralEntropyExact` ([determinism-tiers](../extensions/determinism-tiers.md) §5),
|
||
from SPEC §7's neural rule, not from what its standard's conformance fixes. What
|
||
was read is an overview of the standard (arXiv 2510.13867, read 2026-09-27):
|
||
|
||
- the bit-exact conformance point is "only at the end of the entropy decoding
|
||
pipeline"; "Outputs from subsequent processes may diverge slightly from the
|
||
reference";
|
||
- one codestream may be decoded through "up to three different synthesis
|
||
transforms", restricted by the profile (Part 2) and possibly by the encoder;
|
||
bit-exact picture reconstruction was "deliberately excluded from the first
|
||
version";
|
||
- Part 4 (ISO/IEC 6048-4, conformance) "specifies the requirements for generating
|
||
a JPEG AI-compliant reconstruction" and "defines a testing suite for each
|
||
profile and level pair"; it "considers the possibility that a decoder is
|
||
conformant to the standard without having bit-exact reconstruction", which the
|
||
overview goes on to call "standard compliance with a certain leeway". It says
|
||
Part 4 was a draft at the time of writing.
|
||
|
||
Read together, these passages point to a criterion around a reference on the
|
||
reconstruction — the kind `StandardTolerance` records — but the overview does not
|
||
state one, nor its form or width.
|
||
A measurement of one decoder does not bear on this class. WAI's exact JPEG AI
|
||
decoder pins one reconstruction across machines under the conditions
|
||
[determinism-tiers](../extensions/determinism-tiers.md) states; that shows one
|
||
implementation is reproducible, and says nothing about what another conforming
|
||
decoder outputs, which is what the class records.
|
||
Part 4 itself was not read. A web search on 2026-09-27 returned the ISO catalogue
|
||
entry for ISO/IEC 6048-4 as under publication (stage 60.00); the catalogue page
|
||
refused a direct fetch (HTTP 403), so that status is the search result's, not
|
||
confirmed here. Keeping `wai.image.jpegai` at `NeuralEntropyExact`, where
|
||
`replicate` is refused, is therefore a conservative choice pending Part 4, not a
|
||
reading of the standard; if Part 4 states a criterion on the reconstruction, the
|
||
capability is a candidate for `StandardTolerance` (SPEC §7, *`wai.image.jpegai`
|
||
and the standard classes*). Only a Part 4 that required a bit-exact
|
||
reconstruction could support a same-bytes class, and none of the text read
|
||
suggests it does.
|
||
|
||
What would upgrade each conservative row: for a `StandardUnclassified` standard
|
||
codec, its conformance clause, read, requiring identical output (then
|
||
`StandardDefined`) or stating a criterion (then `StandardTolerance`). For
|
||
`wai.mesh.glb`, either a conformance document for glTF decoding, or a narrower
|
||
capability whose payload is restricted to uncompressed buffers with no embedded
|
||
images, whose reconstruction could then be defined as the parsed buffers — a new
|
||
capability string, since §8 fixes this one's payload.
|
||
|
||
## Broadcast and streaming video / audio
|
||
|
||
The elementary bitstreams terrestrial and satellite broadcast, streaming and
|
||
device playback carry. Registered so an envelope can *name* the bitstream a
|
||
distribution chain already holds; all sink-supplied, none a default.
|
||
|
||
| capability | standard | payload (canonical bytes) | royalty | reference / open decoder | tier |
|
||
|---|---|---|---|---|---|
|
||
| `wai.video.av2` | AV2 (AOMedia), spec v1.0.0, 28 May 2026 | AV2 Annex B length-delimited bitstream (`leb128` size before each OBU) | **RF**⁶ | AVM v1.0.0 | Lossy |
|
||
| `wai.video.vvc` | ITU-T H.266 \| ISO/IEC 23090-3 | H.266 Annex B byte stream | **POOL**⁷ | VTM | Lossy |
|
||
| `wai.video.hevc` | ITU-T H.265 \| ISO/IEC 23008-2 | H.265 Annex B byte stream | **POOL**⁷ | HM | Lossy |
|
||
| `wai.video.avc` | ITU-T H.264 \| ISO/IEC 14496-10 | H.264 Annex B byte stream | **POOL**⁷ | JM | Lossy |
|
||
| `wai.video.vp9` | VP9 Bitstream & Decoding Process Specification | IVF frame container, FourCC `VP90` | **RF**⁸ | libvpx | Lossy |
|
||
| `wai.video.vp8` | RFC 6386 | IVF frame container, FourCC `VP80` | **RF**¹⁵ | libvpx | Lossy |
|
||
| `wai.video.evc` | MPEG-5 Part 1, ISO/IEC 23094-1 (any profile) | Annex B raw bitstream file storage format | **POOL**⁹ | XEVD | Lossy |
|
||
| `wai.video.evc.baseline` | ISO/IEC 23094-1, Baseline profile only | as above, Baseline-conformant | RF by design⁹ | XEVD | Lossy |
|
||
| `wai.video.avs3` | AVS3-P2: T/AI 109.2-2021; IEEE 1857.10 | AVS3-P2 elementary bitstream | **POOL**¹⁰ | uavs3d (BSD-3-Clause; Baseline profile) | Lossy |
|
||
| `wai.video.lcevc` | MPEG-5 Part 2, ISO/IEC 23094-2 | LCEVC enhancement as the ISO/IEC 23094-2 Annex B byte stream (start-code-delimited NAL units) — **companion-only** | **POOL**¹¹ | LCEVCdec (BSD-3-Clause-Clear, patent licences excluded) | Lossy |
|
||
| `wai.audio.mpegh` | MPEG-H 3D Audio, ISO/IEC 23008-3 | MHAS packet stream | **POOL**¹² | mpeghdec | Lossy |
|
||
| `wai.audio.avs3` | AVS3-P3: T/AI 109.3-2023; GY/T 363-2023 ("Audio Vivid"); GB/T 46271-2025 | AVS3-P3 elementary bitstream | **POOL**¹⁰ | — | Lossy |
|
||
| `wai.audio.aac` | ISO/IEC 14496-3: AAC-LC, HE-AAC, HE-AAC v2 | ADTS stream | **POOL**¹³ | — | Lossy |
|
||
| `wai.audio.opus.packet` | RFC 6716 | one Opus packet (RFC 6716 §3), TOC byte first, no container | **RF**¹⁵ | libopus | Lossy |
|
||
| `wai.audio.opus.dred`, `wai.audio.opus.dred.int` | draft-ietf-mlcodec-opus-dred-07 | one DRED extension payload (draft §3) for the Opus packet it accompanies — **companion-only** | **POOL**¹⁶ | the draft's reference implementation (Appendix B there); for `.int`, wai-rs `dred_exact` (features only) | Lossy |
|
||
|
||
**Where these bitstreams are carried** (the reason for registering them, not a
|
||
ranking): Brazil's TV 3.0 / DTV+ — VVC video, LCEVC enhancement, MPEG-H 3D
|
||
Audio — launched by decree on 27 Aug 2025 and switched on in Rio de Janeiro, São
|
||
Paulo and Brasília in June 2026. DVB's video/audio coding specification (A001,
|
||
ETSI TS 101 154) added AVS3 video in 2022 and AVS3-P3 immersive audio in
|
||
November 2024 (BlueBook A001r22). China's GB/T 46271-2025 (3D audio coding,
|
||
distribution and presentation) was published 29 Aug 2025, in force 1 Mar 2026.
|
||
Japan's 4K/8K satellite broadcasting (on air since 1 Dec 2018) uses HEVC. AVC with AAC, and VP9, are
|
||
widely deployed in streaming and device playback. WAI registers LCEVC over
|
||
AVC, HEVC, VVC, EVC, VP9 and AV1, the base codecs a published description of
|
||
LCEVC gives as examples ("e.g., AVC, HEVC, VP9, AV1, EVC or VVC",
|
||
<https://en.wikipedia.org/wiki/LCEVC>); with all six registered, every LCEVC
|
||
base an envelope names is itself a registered capability. The list is WAI's
|
||
registration, not a limit of the standard: ISO/IEC 23094-2:2021 defines the
|
||
decoded base picture as the output of "a base decoder" (definition 3.17), says
|
||
the base decoder's IDR picture "is not specified in this document" (3.30, Note
|
||
1), and its normative references (clause 2: ISO/IEC 11578, ITU-T H.273 |
|
||
ISO/IEC 23091-2, ITU-T T.35) include no video codec; these clauses are in the
|
||
public preview sample cited below. AV2, AVS3 video, JPEG 2000 and the
|
||
WAI-native integer codecs are not registered bases; a later revision may add
|
||
one, which SPEC §8 treats as a widening.
|
||
|
||
**Payload framing is named per entry** because §8 fixes a capability's payload
|
||
format forever once registered, so each entry names one form. Where the
|
||
standard defines a self-delimiting elementary form the entry names it (EVC's
|
||
Annex B raw bitstream file storage format, MHAS for MPEG-H 3D Audio, ADTS for
|
||
AAC). H.264, H.265, H.266 and LCEVC each define a bitstream as either a NAL
|
||
unit stream or a byte stream (H.264 3.15, H.265 3.13, H.266 3.17, ISO/IEC
|
||
23094-2 3.2), and only the byte stream delimits its own NAL units, with the
|
||
three bytes `0x000001` "as a prefix to each NAL unit" (ISO/IEC 23094-2 3.51;
|
||
H.264 3.163, H.265 3.156 and H.266 3.138 say the same). So each of those pins
|
||
is that standard's Annex B byte stream, and a bare or length-prefixed NAL unit
|
||
stream is not the capability. For LCEVC this rests on ISO/IEC 23094-2:2021's
|
||
public preview sample, which runs from the front matter to clause 5.8; the
|
||
table of contents and clause 3 (definitions 3.1, 3.2, 3.6, 3.35, 3.36 and 3.51)
|
||
were read (<https://www.iso.org/standard/79143.html>, sample at
|
||
<https://cdn.standards.iteh.ai/samples/79143/fcb43bd468bf4d29825bc07103a32573/ISO-IEC-23094-2-2021.pdf>).
|
||
The table of contents lists "Annex B (normative) Byte stream format"; the
|
||
annex's body is not in the sample, so its syntax was not read. AV2 OBUs carry
|
||
no size of their own and the AV2 specification states that framing is outside
|
||
its scope *except* for the one method it defines, Annex B — so that is the pin.
|
||
VP9 frames are not self-delimiting and the VP9 specification defines no stream
|
||
framing, so the pin is IVF, the minimal frame container the VP9 reference
|
||
library (libvpx) reads and writes. VP8 (registered 2026-10-18) takes the same
|
||
pin with FourCC `VP80`: a VP8 frame does not delimit itself either, since its last
|
||
partition is "the remainder of the data not used by any of the previous
|
||
partitions" (RFC 6386 §9.5) and the decoder "is simply presented with a sequence
|
||
of compressed frames" (§4). RFC 6386 was read for this framing question only
|
||
(§4, §9), not for decoder conformance. The Opus packet form (registered the same
|
||
day) is one packet of RFC 6716 §3, its TOC byte first, with no container; the
|
||
payload's length is the packet's length (the undelimited form of §3, not the
|
||
self-delimiting framing of its Appendix B). It is the form a
|
||
live track carries, one packet per object, and it carries none of Ogg Opus's
|
||
header fields (pre-skip and output gain, RFC 7845 §5.1), so it is a string of
|
||
its own beside `wai.audio.opus`, the Ogg Opus stream. The two AVS3 rows name the elementary
|
||
bitstream of T/AI 109.2 (video) and T/AI 109.3 (audio). Neither text was
|
||
obtained, so whether either standard defines a second form of its bitstream was
|
||
not checked; if one does, the row must name the form before §8 fixes the
|
||
string (open item 5(h)).
|
||
|
||
**AV2 placement — registered, not recommended.** The recommended set is what
|
||
every conforming sink SHOULD implement, and WAI's value there is availability.
|
||
AV2 is royalty-free (⁶) and has a reference decoder, but as of September 2026
|
||
AVM is a reference model, a production software decoder is still in
|
||
development, and hardware decode in shipping devices is not confirmed. Asking
|
||
every sink to carry it
|
||
now would ask for a decoder most cannot deploy. So it is registered (a WAI file
|
||
can carry AV2 today, royalty-free) and SPEC §5 tells encoders to ship it in a
|
||
`WAI2` envelope with a `wai.video.av1` rendition of the same content. A v1.0
|
||
`fallback` of `wai.video.av1` would not do: dispatch step 5 hands the *same*
|
||
AV2 payload to the fallback's decoder, which cannot read it. Promotion is a
|
||
MINOR revision once deployable decode exists; the capability string and
|
||
framing stay fixed.
|
||
`defaults_are_royalty_free_and_never_pool_or_proprietary` asserts it is RF and
|
||
not (yet) a default.
|
||
|
||
**LCEVC and the companion mechanism — the fit, and where it stops.** LCEVC is an
|
||
enhancement layer: residuals added to the upscaled output of one specific base
|
||
encode. SPEC §7.1's companion components fit most of it:
|
||
|
||
- *Applied with, never instead of.* A companion is never returned by
|
||
`pick` / `primary`, so a sink with no LCEVC decoder presents the base — the
|
||
backward-compatible behaviour LCEVC is designed for (TV 3.0 uses it to carry a
|
||
UHD service on top of a usable HD one).
|
||
- *Intrinsic binding.* Base and enhancement ride in one envelope, not a sidecar.
|
||
- *One base.* ISO/IEC 14496-15's dual-track storage has the enhancement track
|
||
refer to exactly one base track; `companion_of` is the analogue.
|
||
|
||
Two gaps, one closed here and one not:
|
||
|
||
1. **Binding granularity (closed by a rule in both directions).**
|
||
`companion_of` names a *capability*, not a component. HDR metadata describes
|
||
content and can legitimately apply to several renditions of one codec, and
|
||
several descriptors can apply to one base; LCEVC residuals belong to one
|
||
encode and are added once. So an enhancement layer's `companion_of` MUST
|
||
name one of its base codecs — the capabilities that name the AVC, HEVC, VVC,
|
||
EVC, VP9 and AV1 bitstreams, EVC Baseline and lossless AV1 included — and
|
||
resolve to exactly one selectable rendition of it, and that base takes at
|
||
most one enhancement layer — a sink applies every companion of the rendition
|
||
it picks, so a second one would stack a second set of residuals.
|
||
`WaiMulti::check_component_roles` rejects a base outside that list (JPEG
|
||
2000, AV2, AVS3, the WAI-native integer codecs, anything not video), zero or
|
||
several renditions of the named base, a second enhancement on one base, and
|
||
a companion-only capability mislabelled as a rendition (which `pick` /
|
||
`primary` also never select). A component-id binding would lift the
|
||
one-base-rendition limit; it is not part of this change.
|
||
The v1.0 side has its own check: `Wai::check_component_roles` rejects a
|
||
companion-only capability as a `WAI1` `capability` or `fallback`, and the
|
||
`wai wrap` tool refuses to write either (with `--companion` it writes the
|
||
conforming `WAI2` form instead).
|
||
2. **Per-picture timing (open).** A timed container aligns base and enhancement
|
||
samples by timestamp; a `WAI2` payload is an untimed elementary stream. The
|
||
payload's framing is pinned (the Annex B byte stream, above), and ISO/IEC
|
||
23094-2 defines an access unit as the NAL units of one coded picture
|
||
"associated with a particular output time" (3.1), but this revision does
|
||
not pin how a separately carried enhancement's access units align picture
|
||
by picture with its base, so the **embedded** carriage — enhancement inside
|
||
the base bitstream as SEI / registered user data, carried under the base
|
||
capability, no separate component — is the unambiguous form until it does.
|
||
|
||
**Not registered this round, and why.**
|
||
|
||
- *E-AC-3 / AC-3.* Patent expiry is reported (AC-3 in 2017; the last E-AC-3
|
||
patent reportedly in 2019) but not confirmed here, and a licensing programme
|
||
still exists. Registering needs a verified class first.
|
||
- *Extended HE-AAC (USAC).* ADTS cannot carry it; it needs LOAS/LATM or
|
||
ISOBMFF framing, i.e. a separate capability with its own pinned payload.
|
||
- *HDR Vivid.* The HDR dynamic-metadata companion published alongside Audio
|
||
Vivid in the same national-standard set; licence not verified.
|
||
|
||
⁶ **Verified RF.** The AVM v1.0.0 reference software's `PATENTS` file is the
|
||
*Alliance for Open Media Patent License 1.0* ("no-charge, royalty-free,
|
||
irrevocable … patent license"); AOMedia's 9 June 2026 release states AV2 was
|
||
"developed under AOMedia's royalty-free patent policy". Residual caveat, the same
|
||
as for AV1 and VP9: a third-party licensing company announced (Dec 2025) an
|
||
intent to form a pool asserting AV2 patents, and on 25 Aug 2026 a second
|
||
licensing administrator launched an exploratory phase for a pool covering devices
|
||
and software implementing AV2 and AV1, saying it expects AV2 to be added to its
|
||
existing video-distribution pool licence (which covers VP9 and AV1) in 2027. WAI
|
||
classifies a format by its own licence and records such assertions here rather
|
||
than reclassifying on them. Sources: <https://av2.aomedia.org/>,
|
||
<https://github.com/AOMediaCodec/avm/blob/v1.0.0/PATENTS>,
|
||
<https://accessadvance.com/2026/08/25/access-advance-launches-exploration-phase-for-an-av1-av2-device-and-software-patent-pool-invites-participation-of-potential-licensors/>.
|
||
⁷ **Verified POOL.** VVC, HEVC and AVC essential patents are licensed for
|
||
royalties through patent pools (several for VVC and HEVC, the long-running
|
||
H.264 pool for AVC) plus bilateral licensors. None is royalty-free.
|
||
⁸ **RF.** VP9 is published with BSD-licensed source and a royalty-free patent
|
||
grant from its developer. Residual caveat as ⁶: a third-party pool asserts VP9
|
||
patents; recorded, not adopted as the class.
|
||
⁹ **EVC split by profile.** ISO/IEC 23094-1's Baseline profile uses only tools
|
||
more than 20 years old or offered royalty-free, and its proponents describe it
|
||
as royalty-free; the Main profile adds tools whose holders license them on FRAND
|
||
terms. So `wai.video.evc` (any profile — a decoder must handle Main) is POOL and
|
||
`wai.video.evc.baseline` is RF *by design*. Residual caveat, like JPEG 2000
|
||
Part 1 (³): no formal royalty-free guarantee exists for Baseline.
|
||
¹⁰ **Classified POOL.** The AVS patent pool issued calls for essential patents
|
||
for AVS3-P2 video and AVS3 audio on 11 Jan 2024, licensing on RAND terms with
|
||
"competitive licensing fees"; AVS licensing has historically charged devices
|
||
only. The fee schedule for AVS3 was not found published, but the pool is
|
||
royalty-bearing, not royalty-free. AVS3-P3 and Audio Vivid are treated as one
|
||
bitstream family (DVB and the open-source tooling name them together); the
|
||
bitstream identity of T/AI 109.3, GY/T 363 and GB/T 46271 was not checked
|
||
against the standard texts, which were not available.
|
||
¹¹ **Verified POOL (single SEP licensor).** LCEVC's essential patents are
|
||
licensed under programmes announced Aug 2025: royalties on consumer devices and
|
||
on video-distribution services, a free integration licence, and free-to-air
|
||
broadcast royalties waived. The open decoder library is BSD-3-Clause-Clear,
|
||
which expressly excludes patent licences.
|
||
¹² **Verified POOL.** MPEG-H 3D Audio essential patents are licensed through a
|
||
patent-pool programme.
|
||
¹³ **Verified POOL.** The AAC pool programme covers AAC-LC, HE-AAC, HE-AAC v2 and
|
||
Extended HE-AAC; a product shipping an AAC encoder or decoder pays a per-unit
|
||
royalty.
|
||
¹⁵ **RF, by the codec each one carries** (registered 2026-10-18). `wai.video.vp8`:
|
||
VP8's reference source is published under a BSD licence with a royalty-free
|
||
patent grant from its developer, the basis ⁸ records for VP9; whether a
|
||
third-party pool asserts VP8 patents, as one does VP9's (⁸), was not checked.
|
||
`wai.audio.opus.packet`: Opus, classified royalty-free as `wai.audio.opus` is;
|
||
the packet form adds no coding tool, only leaves out the Ogg container.
|
||
¹⁶ **POOL, conservatively** (registered 2026-10-22). One IPR disclosure bears on DRED: the IETF's
|
||
disclosure 6063, on the individual draft that became draft-ietf-mlcodec-opus-dred, states a
|
||
pending application and an intent to license royalty-free "if the technology covered by this
|
||
disclosure is included in a standard adopted by the IETF". The draft is not yet a standard, so
|
||
the royalty status is not established, and the registry's rule classes such a capability
|
||
`patent-pool`, never `royalty-free` (REGISTRY.md, *Licence classes*). It moves when the condition
|
||
is met.
|
||
|
||
## Implementation & demonstration status
|
||
|
||
WAI's job is **carriage + dispatch**; decoders are sink-supplied (even the
|
||
floor codecs are *wrapped* reference libraries, not re-implemented). So
|
||
"implemented and demonstrated" means, per capability:
|
||
|
||
- **Carriage proven for every registered class** — `wai wrap <cap> <payload>`
|
||
builds an envelope around the canonical bytes (over a class that does not back
|
||
an unpinned `replicate`, only with `--intent`, SPEC §7); `wai unwrap` returns them
|
||
**byte-identically**; the registry reports the tier + license. A
|
||
companion-only capability is carried as a `WAI2` companion instead
|
||
(`wai wrap <base-cap> <base-payload> --companion <cap> <payload>`, then
|
||
`wai unwrap --component 1`). The `tests/carriage.rs` conformance asserts this
|
||
for the *entire* registry (0 B … 64 KiB payloads), each capability in the
|
||
form SPEC §5 allows it, and runs in the no-system-deps CI matrix.
|
||
- **Real end-to-end decode demonstrated** for the open formats with an
|
||
installed/built reference tool — the full carry → dispatch → decode path on a
|
||
real file: **JPEG 2000** (`wai.image.jp2`, OpenJPEG — cinema), **IAMF**
|
||
(`wai.audio.iamf`, ffmpeg — immersive audio), **Draco** (`wai.mesh.draco`,
|
||
draco_decoder — 3D geometry), **G-PCC** (`wai.pc.gpcc`, MPEG `tmc3` built
|
||
from source — point clouds; patent-pool, so demonstrated-but-not-royalty-free),
|
||
and **IVAS** (`wai.audio.ivas`, 3GPP TS 26.258 `IVAS_cod`/`IVAS_dec` built
|
||
from source — immersive voice/audio: real stereo PCM → IVAS bitstream → full
|
||
PCM reconstruction, and the *carried* bitstream decodes bit-identically to a
|
||
direct decode; patent-pool, so demonstrated-but-not-royalty-free).
|
||
See `tools/wai_carriage_demo.sh`.
|
||
- **Real encode + structural decode demonstrated** — **V-PCC** (`wai.pc.vpcc`,
|
||
MPEG `tmc2` built from source with its bundled HM video codec): a real point
|
||
cloud is encoded end-to-end through the full V-PCC pipeline (patch projection
|
||
→ occupancy/geometry video → HM intra encode) into a conformant **V3C
|
||
bitstream**, which `PccAppParser` then structurally decodes into its V3C units
|
||
(`V3C_VPS` / `V3C_AD` / `V3C_OVD` / `V3C_GVD`), and which WAI carries
|
||
byte-exact. Full *pixel* reconstruction needs HM's standalone decoder, which
|
||
the V-PCC-patched HM source builds only as a namespaced library (the
|
||
`PccAppVideoDecoder` wrapper's job), not a standalone `TAppDecoder` — so the
|
||
reconstruct step stays sink-supplied while the encode + V3C-conformance +
|
||
carriage are real. Patent-pool, like G-PCC.
|
||
- **Decoder sink-supplied** (open reference exists, not bundled / not installed
|
||
here): `wai.video.j2k`, `wai.haptic.mihs`, `wai.meta.hdr10plus`.
|
||
- **Candidate, refused** (SPEC §5, *Candidate capabilities*): `wai.splat.sog` has
|
||
no pinned payload format, so the demo checks that `wai wrap` refuses it and
|
||
writes nothing.
|
||
- **Proprietary** (no open decoder, by design — carried, sink-supplied):
|
||
`wai.audio.ac4`, `wai.meta.dovi`.
|
||
- **Companion-only** (`wai.meta.hdr10plus`, `wai.meta.dovi`,
|
||
`wai.video.lcevc`): carried in the demo the one conforming way, as a `WAI2`
|
||
companion bound to a base rendition (a real AVC Annex B stream when libx264
|
||
is installed). Each row requires the companion back byte-exact, the role
|
||
check to pass, `wai wrap` to refuse the `WAI1` form, and the base to decode
|
||
to all 10 frames; the LCEVC row also requires a second enhancement on the
|
||
same base to be refused. The companion payloads are representative bytes.
|
||
- **Broadcast and streaming codecs.** Every row of that table is covered by
|
||
`tests/carriage.rs` (byte-exact wrap / unwrap, tier + licence present) and by
|
||
the registry tests in `codecs/mod.rs`; the LCEVC binding rules by the
|
||
`container.rs` unit tests. The crate gains no decoder and no C dependency:
|
||
every one is sink-supplied. **Real encode + decode demonstrated** for
|
||
`wai.video.avc`, `wai.video.hevc`, `wai.video.vp9` and `wai.audio.aac` in
|
||
`tools/wai_carriage_demo.sh` when ffmpeg has the encoders: the payload is
|
||
built in the framing SPEC §5 pins (Annex B byte stream, IVF, ADTS), carried
|
||
byte-exact, and decoded by ffmpeg with the matching raw demuxer *forced*. A
|
||
video row passes only if all 10 encoded frames come back; the AAC row only if
|
||
the decoded frame count equals the ADTS frame count of the encoded stream,
|
||
counted by walking its frame headers rather than by the decoder (a payload
|
||
cut to 600 B, or by its last byte, fails; random bytes under the forced
|
||
demuxers are rejected). **Carriage only** for `wai.video.vvc` (no VVC encoder
|
||
installed on the demo host), `wai.video.av2`, `wai.video.evc`,
|
||
`wai.video.evc.baseline`, `wai.video.avs3`, `wai.audio.mpegh`,
|
||
`wai.audio.avs3`, and `wai.video.vp8` and `wai.audio.opus.packet` (registered
|
||
2026-10-18; the demo script has no row for either). `wai.video.lcevc` is companion-only and is demonstrated as a
|
||
`WAI2` companion (above).
|
||
|
||
The reference sink decoders for the volumetric + immersive-audio rows (MPEG
|
||
`tmc13` for G-PCC, `tmc2` for V-PCC, 3GPP `IVAS_cod`/`IVAS_dec`) are fetched
|
||
and built at pinned versions by `tools/build_reference_decoders.sh`, which
|
||
prints the `PATH` / `TMC2_CFG` to export before `tools/wai_carriage_demo.sh`
|
||
— so every `real+decode` / `real+parse` row reproduces from a clean checkout.
|
||
|
||
`wai wrap` / `unwrap` is the universal carriage operation: the payload is opaque
|
||
to WAI, so the same envelope carries a JPEG 2000 codestream, an IAMF stream, a
|
||
Draco buffer, or a proprietary AC-4 frame identically — that opacity is the
|
||
point, and the reason WAI does not need to re-implement any of them.
|
||
|
||
## What stays the manifest's job (not new capabilities)
|
||
|
||
- **ADM / S-ADM** (ITU-R BS.2076) — the object-audio *authoring metadata model*
|
||
that feeds IAMF; agnostic, not a codec. A manifest/authoring concern.
|
||
- **Static HDR10 / HLG color descriptors** — a manifest color field (primaries,
|
||
transfer, mastering-display), not a payload.
|
||
|
||
## Spec follow-ups flagged by this round
|
||
|
||
1. **Companion-component binding — DONE.** The `WAI2` multi-component envelope
|
||
gains a component `role`: `"companion"` (with `companion_of`) marks a
|
||
descriptor applied *with* the chosen rendition. HDR dynamic metadata
|
||
(`wai.meta.hdr10plus` / `wai.meta.dovi`) binds to a video payload in the
|
||
**same envelope** — `pick`/`primary` skip companions, `companions_of(cap)`
|
||
returns the descriptors for the chosen component. Back-compat: `role` absent
|
||
⇒ rendition. SPEC §7.1 + a conformance test
|
||
(`companion_descriptors_bind_to_a_primary`). Immersive-audio metadata was
|
||
named here too, and that test first carried an IAMF stream as a companion;
|
||
no capability for immersive-audio mix metadata is registered, and IAMF
|
||
carries its own audio substreams, so it is standalone media. Since
|
||
September 2026 SPEC §7.1 states three rules a companion must meet — its
|
||
capability is companion-only (or unregistered), `companion_of` names a base
|
||
it applies to (a `video`-class capability for HDR metadata — `wai.video.*`,
|
||
the neural video codecs, the films — and a base codec for LCEVC), and
|
||
that base is present — and `check_component_roles` enforces each, with a
|
||
negative test per rule.
|
||
2. **License classification — DONE.** `codecs::capability_license()` /
|
||
`is_royalty_free()` classify every capability; `wai capabilities` reports
|
||
them; a coherence test enforces that no capability is unclassified. Built as
|
||
a **registry property**, not a manifest field (a per-file licence claim would
|
||
be spoofable; the codec's licence is intrinsic to the capability).
|
||
3. **Royalty footnotes — VERIFIED (June 2026).** IAMF (AOMedia PL 1.0), JPEG
|
||
2000 Part 1 (ITU-T 2.1), Draco (Apache-2.0), HDR10+ (content+device RF) →
|
||
confirmed royalty-free; IVAS (a voice-codec pool) → confirmed pool; G-PCC/V-PCC
|
||
→ pool, a conservative classification (¹⁴); AC-4 / ST 2094-10 dynamic metadata →
|
||
proprietary. Residual: JPEG 2000
|
||
extensions/submarine-patent caveat (³), MPEG-I Haptics pool status
|
||
unconfirmed (⁵).
|
||
4. **Broadcast and streaming codecs — REGISTERED (September 2026).** AV2, VVC,
|
||
HEVC, AVC, VP9, EVC (any profile / Baseline), AVS3 video, LCEVC, MPEG-H 3D
|
||
Audio, AVS3 audio / Audio Vivid and AAC, each naming one payload framing
|
||
(LCEVC's is the ISO/IEC 23094-2 Annex B byte stream; the AVS3 rows' are
|
||
unconfirmed against their standards, (h)) and a licence class with its
|
||
basis (⁶–¹³). LCEVC is companion-only, binds to exactly one rendition of
|
||
one of its base codecs, and a base takes at most one enhancement;
|
||
companion-only capabilities are refused as a v1.0 `capability`, and as a
|
||
`fallback` wherever the fallback would be used (§4 step 5). Licence
|
||
classification made exhaustive (no default class), and so was the replicate
|
||
classification, with an exact class admitted only on recorded evidence. AVC,
|
||
HEVC, VVC and VP9 are `StandardDefined` on their conformance clauses; AV2 is
|
||
`StandardTolerance`, as AV1; the other broadcast rows are
|
||
`StandardUnclassified` (see *Decoder conformance* above).
|
||
Residual: EVC Baseline's RF-by-design status (⁹), unpublished AVS3 fee
|
||
schedule and unverified AVS3-P3 / Audio Vivid bitstream identity (¹⁰),
|
||
third-party pool assertions against AV2 / VP9 (⁶, ⁸).
|
||
**Added 2026-10-18** for live ingest: `wai.video.vp8` (RFC 6386, IVF with
|
||
FourCC `VP80`, RF, `StandardUnclassified`) and `wai.audio.opus.packet` (one
|
||
Opus packet, RF, `StandardTolerance` as `wai.audio.opus`), both sink-supplied
|
||
and neither a default (¹⁵).
|
||
5. **Open.** (a) Per-picture alignment of a separately carried LCEVC
|
||
enhancement with its base. Its framing is pinned (the Annex B byte stream),
|
||
but how its access units map to the base's pictures outside a timed
|
||
container is not; until it is, the embedded carriage is the unambiguous
|
||
form. (b) A component-id `companion_of` binding, which would let one
|
||
envelope carry enhancements for several renditions of one base codec.
|
||
(c) The neural capabilities' model-weight licences were carried over, not
|
||
re-verified, when the classification became explicit — the EnCodec 32 kHz
|
||
checkpoint is distributed with a model collection whose weights are licensed
|
||
CC-BY-NC 4.0, which would make `wai.neural.encodec32` non-commercial if it
|
||
applies. (d) E-AC-3 / AC-3, Extended HE-AAC and HDR Vivid (see above).
|
||
(e) The `StandardUnclassified` rows (VP8, EVC, AVS3 video and audio, LCEVC,
|
||
MPEG-H 3D Audio, AAC, AC-4, G-PCC, V-PCC, Draco, SOG (which first needs a pinned payload), MIHS, the HDR
|
||
descriptors, glb, SPZ) are waiting on each standard's conformance text; any
|
||
whose clause requires identical decoded output moves to `StandardDefined`,
|
||
with that clause recorded, and any with a stated criterion to
|
||
`StandardTolerance`. (f) Closed: the spatial-audio extension now says a
|
||
mixdown is only as portable as its sources' decoded PCM, and
|
||
`is_cross_decoder_exact` no longer reports `wai.audio.scene` exact from the
|
||
capability alone (`is_cross_decoder_exact_with_sources` checks the sources). (g) Closed:
|
||
`wai.image.jpegai` is `NeuralEntropyExact` and does not back `replicate`
|
||
(SPEC §7); it is not cross-decoder-exact. A cross-platform measurement of
|
||
WAI's own decoder cannot change that, as *Decoder conformance* explains.
|
||
(h) AVS3 framing. `wai.video.avs3` and `wai.audio.avs3` name the elementary
|
||
bitstream of T/AI 109.2 and T/AI 109.3, which were not obtained. Before §8
|
||
fixes the strings, confirm from those texts that each defines one form of
|
||
its bitstream, or name the form the capability carries.
|
||
(i) What `wai.pc.gpcc` and `wai.pc.vpcc` carry. Under SPEC §8 (*Editions*), a
|
||
registration that names no edition carries the edition of its standard current
|
||
when it was registered, here 2026-06-17, and no later amendment, so the
|
||
Gaussian-splat amendments (V-PCC at Draft Amendment, G-PCC at Committee Draft
|
||
Amendment), a later edition, and V-DMC (a V3C application from the fourth
|
||
edition of ISO/IEC 23090-5) need strings of their own. Still open: which
|
||
edition of each part was current on 2026-06-17 (iso.org refused automated
|
||
reads; the common test conditions name a 2023 edition of 23090-9), and whether
|
||
`wai.pc.vpcc` carries the V3C bitstream of MIV.
|
||
(j) The video-based anchor of both common-test-conditions documents installs a
|
||
package named PLAS, which neither document describes. Whether it relates to the
|
||
family of formats `wai.splat.sog` was registered for was not established.
|