3ee1318540
wai-conformance: the probe unit tests on every leg and the feature built alone and for wasm32; a step running tests/format_probe.rs, the fetched LCEVC streams under wai_format_probe and format_probe_verify.py; steps for cmsd_read_verify.py and loc_verify.py; the new extensions in its paths. joule-code-conformance: the LOC corpus and the H.265 vector it reads in its paths. SPEC cites format-probes by its headed section. Co-Authored-By: Claude Opus 5.5 <[email protected]>
131 lines
7.3 KiB
Markdown
131 lines
7.3 KiB
Markdown
# WAI Extension: Projection Descriptor
|
||
|
||
> Status: Draft. Registers `wai.meta.projection`, a **companion descriptor** to a video
|
||
> rendition (SPEC §5, §7.1), as the HDR dynamic-metadata descriptors are: a 40-byte `WPJ1`
|
||
> record that says how the rendition's pictures map onto what a viewer sees (rectilinear,
|
||
> equirectangular, half-equirectangular or fisheye) and how its two views are arranged (one
|
||
> view, two views packed side by side or top and bottom, or two views in two layers of a
|
||
> multiview bitstream such as `wai.video.mvhevc`). The projection kinds follow ISO/IEC 23090-2
|
||
> (OMAF); the angle unit and ranges follow ITU-T H.274 (V4, 01/2026) §8.15; the packing
|
||
> arrangements are ITU-T H.273 (V4, 07/2024) §8.4's. The reference implementation is
|
||
> `format_probe::projection` in `wai-rs` (feature `format_probe`); its vectors are in
|
||
> [`format-probe-conformance/`](../format-probe-conformance/), with its second
|
||
> implementation, `format_probe_verify.py`.
|
||
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
|
||
>
|
||
> ISO/IEC 23090-2 is not openly readable and was not read. Its projection kinds are used here as
|
||
> the H.274 SEI messages that carry them in a bitstream state them (the equirectangular
|
||
> projection SEI message, §8.15.2, and the sphere rotation and region-wise packing messages,
|
||
> §8.15.4 and §8.15.5), and no clause of ISO/IEC 23090-2 is cited.
|
||
|
||
## 1. Scope
|
||
|
||
§1.1 A projection descriptor describes the pictures of one video rendition: the geometry a
|
||
renderer maps them onto and the arrangement of their views. It changes no sample and is applied
|
||
by the renderer, after decoding. A rendition without one is a rectilinear picture of one view.
|
||
|
||
§1.2 The descriptor is WAI's own record, so a rendition's projection is stated the same way
|
||
whatever the codec, and also where the codec's own signalling is absent: an AV1 payload, a
|
||
JPEG XS payload or an MV-HEVC payload whose bitstream carries no projection SEI message. Where a
|
||
bitstream carries such a message and the descriptor disagrees with it, the descriptor is what
|
||
WAI states, and a renderer **SHOULD** report the disagreement.
|
||
|
||
## 2. Terms
|
||
|
||
§2.1 Angles are in units of 2⁻¹⁶ degrees, as H.274 §8.15 states them: one degree is 65 536.
|
||
Azimuth increases eastward (to the viewer's left, seen from inside the sphere), elevation
|
||
northward, as for H.274's equirectangular projection (§8.15.2.2).
|
||
|
||
§2.2 **Projections.**
|
||
|
||
| code | name | the picture is |
|
||
|---|---|---|
|
||
| 0 | `rectilinear` | a flat picture with no sphere projection; its coverage, where stated, is its field of view |
|
||
| 1 | `equirectangular` | the equirectangular projection of the whole sphere, 360° of azimuth and 180° of elevation, as H.274 §8.15.2 maps it |
|
||
| 2 | `half-equirectangular` | the equirectangular projection of the front hemisphere: 180° of azimuth and 180° of elevation, centred on the coverage centre |
|
||
| 3 | `fisheye` | a circular fisheye image whose field of view is its coverage |
|
||
|
||
§2.3 **Stereo arrangements.**
|
||
|
||
| code | name | the two views are |
|
||
|---|---|---|
|
||
| 0 | `mono` | one view |
|
||
| 1 | `side-by-side` | the left and right halves of each picture, H.273 `VideoFramePackingType` 3 |
|
||
| 2 | `top-bottom` | the top and bottom halves of each picture, `VideoFramePackingType` 4 |
|
||
| 3 | `layers` | two layers of a multiview bitstream, each a view, named by their `nuh_layer_id` |
|
||
|
||
## 3. The record
|
||
|
||
§3.1 A `wai.meta.projection` payload is 40 bytes, multi-byte integers little-endian:
|
||
|
||
| offset | field | type | meaning |
|
||
|---|---|---|---|
|
||
| 0 | `magic` | 4 B | `WPJ1` |
|
||
| 4 | `version` | u8 | 1 |
|
||
| 5 | `projection` | u8 | §2.2 |
|
||
| 6 | `stereo` | u8 | §2.3 |
|
||
| 7 | `first_view` | u8 | 0: the left eye's view is the left half, the top half or `left_layer`'s; 1: the right eye's is |
|
||
| 8 | `azimuth_range` | u32 | coverage in azimuth, 2⁻¹⁶ degrees |
|
||
| 12 | `elevation_range` | u32 | coverage in elevation |
|
||
| 16 | `centre_azimuth` | i32 | the coverage's centre |
|
||
| 20 | `centre_elevation` | i32 | |
|
||
| 24 | `centre_tilt` | i32 | rotation about the centre's axis |
|
||
| 28 | `baseline_um` | u32 | the distance between the two cameras, micrometres; 0 where not stated |
|
||
| 32 | `left_layer` | u8 | with `stereo` 3, the left eye's layer; otherwise 0 |
|
||
| 33 | `right_layer` | u8 | with `stereo` 3, the right eye's layer; otherwise 0 |
|
||
| 34 | reserved | 6 B | zero |
|
||
|
||
§3.2 Its parameters: `projection` and `stereo` (the names of §2), `first_view` (`left` or
|
||
`right`), `coverage` (`[azimuth_range, elevation_range]`), `centre` (`[azimuth, elevation,
|
||
tilt]`), `baseline_um`, and `layers` (`[left_layer, right_layer]` with `stereo` 3, else `null`).
|
||
|
||
## 4. Checks
|
||
|
||
§4.1 A reader refuses, in this order:
|
||
|
||
1. fewer than 4 bytes or a magic other than `WPJ1`: `no_signature`;
|
||
2. a length other than 40: `length_mismatch`;
|
||
3. a `version` other than 1, a `projection` or `stereo` above 3, a `first_view` above 1, or a
|
||
reserved byte that is not zero: `reserved_value`;
|
||
4. fields that disagree, `descriptor_invalid`:
|
||
- coverage: for `equirectangular`, exactly 360° by 180°; for `half-equirectangular`, exactly
|
||
180° by 180°; for `fisheye`, each range above 0 and at most 360°; for `rectilinear`, both 0
|
||
(field of view not stated) or each above 0 and below 180°;
|
||
- centre: azimuth and tilt from −180° up to but not including 180°, elevation from −90° to 90°
|
||
(H.274 §8.15's ranges);
|
||
- views: `mono` with a `first_view`, `baseline_um`, `left_layer` or `right_layer` that is not
|
||
0; `side-by-side` or `top-bottom` with a layer that is not 0; `layers` with the two layers
|
||
equal or either above 62.
|
||
|
||
§4.2 The codes are those of [format-probes](format-probes.md) §12.
|
||
|
||
## 5. Binding to the rendition
|
||
|
||
§5.1 A projection descriptor is a `WAI2` companion (SPEC §7.1) whose `companion_of` names a
|
||
capability of the video media class, as an HDR descriptor's does. It is never a `WAI1`
|
||
`capability` or `fallback` and never a selectable rendition.
|
||
|
||
§5.2 A descriptor with `stereo` 3 applies only to a rendition of `wai.video.mvhevc` whose probe
|
||
(format-probes §4) reports both `left_layer` and `right_layer` among its `layers_present`;
|
||
otherwise `layer_absent`, and a sink **MUST NOT** apply it. A sink that presents a rendition with
|
||
such a descriptor presents the view of each layer to the eye the descriptor names.
|
||
|
||
§5.3 With `side-by-side` or `top-bottom`, each view is half the decoded picture's width or height;
|
||
a sink that presents one eye only presents the view `first_view` names as the left eye's when it
|
||
needs one view.
|
||
|
||
## 6. Mapping
|
||
|
||
§6.1 Writing a descriptor from a bitstream's own signalling: an H.274 equirectangular
|
||
projection SEI message gives `equirectangular` (or `half-equirectangular` where a region-wise
|
||
packing message packs only the front hemisphere); a sphere rotation message gives the centre; a
|
||
frame packing arrangement of type 3 or 4 gives `side-by-side` or `top-bottom`; an MV-HEVC
|
||
bitstream's two views (format-probes §4, `view_ids`) give `layers`. Writing the other way, a
|
||
descriptor gives the same messages. Fisheye has no H.274 message; its parameters beyond the
|
||
coverage (the image circle's centre and radius) are not in this revision of the record.
|
||
|
||
## 7. Conformance
|
||
|
||
§7.1 [`format-probe-conformance/`](../format-probe-conformance/) holds descriptors of every
|
||
projection and arrangement, descriptors refused with each code of §4, and bindings of descriptors
|
||
to an MV-HEVC rendition and an H.265 rendition with their outcomes (format-probes §13).
|