# WAI Extension: Projection Descriptor

> Status: Draft. Registers `wai.meta.projection`, a **companion descriptor** to a video
> rendition (SPEC §5, §7.1), as the HDR dynamic-metadata descriptors are: a 40-byte `WPJ1`
> record that says how the rendition's pictures map onto what a viewer sees (rectilinear,
> equirectangular, half-equirectangular or fisheye) and how its two views are arranged (one
> view, two views packed side by side or top and bottom, or two views in two layers of a
> multiview bitstream such as `wai.video.mvhevc`). The projection kinds follow ISO/IEC 23090-2
> (OMAF); the angle unit and ranges follow ITU-T H.274 (V4, 01/2026) §8.15; the packing
> arrangements are ITU-T H.273 (V4, 07/2024) §8.4's. The reference implementation is
> `format_probe::projection` in `wai-rs` (feature `format_probe`); its vectors are in
> [`format-probe-conformance/`](../format-probe-conformance/), with its second
> implementation, `format_probe_verify.py`.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
>
> ISO/IEC 23090-2 is not openly readable and was not read. Its projection kinds are used here as
> the H.274 SEI messages that carry them in a bitstream state them (the equirectangular
> projection SEI message, §8.15.2, and the sphere rotation and region-wise packing messages,
> §8.15.4 and §8.15.5), and no clause of ISO/IEC 23090-2 is cited.

## 1. Scope

§1.1 A projection descriptor describes the pictures of one video rendition: the geometry a
renderer maps them onto and the arrangement of their views. It changes no sample and is applied
by the renderer, after decoding. A rendition without one is a rectilinear picture of one view.

§1.2 The descriptor is WAI's own record, so a rendition's projection is stated the same way
whatever the codec, and also where the codec's own signalling is absent: an AV1 payload, a
JPEG XS payload or an MV-HEVC payload whose bitstream carries no projection SEI message. Where a
bitstream carries such a message and the descriptor disagrees with it, the descriptor is what
WAI states, and a renderer **SHOULD** report the disagreement.

## 2. Terms

§2.1 Angles are in units of 2⁻¹⁶ degrees, as H.274 §8.15 states them: one degree is 65 536.
Azimuth increases eastward (to the viewer's left, seen from inside the sphere), elevation
northward, as for H.274's equirectangular projection (§8.15.2.2).

§2.2 **Projections.**

| code | name | the picture is |
|---|---|---|
| 0 | `rectilinear` | a flat picture with no sphere projection; its coverage, where stated, is its field of view |
| 1 | `equirectangular` | the equirectangular projection of the whole sphere, 360° of azimuth and 180° of elevation, as H.274 §8.15.2 maps it |
| 2 | `half-equirectangular` | the equirectangular projection of the front hemisphere: 180° of azimuth and 180° of elevation, centred on the coverage centre |
| 3 | `fisheye` | a circular fisheye image whose field of view is its coverage |

§2.3 **Stereo arrangements.**

| code | name | the two views are |
|---|---|---|
| 0 | `mono` | one view |
| 1 | `side-by-side` | the left and right halves of each picture, H.273 `VideoFramePackingType` 3 |
| 2 | `top-bottom` | the top and bottom halves of each picture, `VideoFramePackingType` 4 |
| 3 | `layers` | two layers of a multiview bitstream, each a view, named by their `nuh_layer_id` |

## 3. The record

§3.1 A `wai.meta.projection` payload is 40 bytes, multi-byte integers little-endian:

| offset | field | type | meaning |
|---|---|---|---|
| 0 | `magic` | 4 B | `WPJ1` |
| 4 | `version` | u8 | 1 |
| 5 | `projection` | u8 | §2.2 |
| 6 | `stereo` | u8 | §2.3 |
| 7 | `first_view` | u8 | 0: the left eye's view is the left half, the top half or `left_layer`'s; 1: the right eye's is |
| 8 | `azimuth_range` | u32 | coverage in azimuth, 2⁻¹⁶ degrees |
| 12 | `elevation_range` | u32 | coverage in elevation |
| 16 | `centre_azimuth` | i32 | the coverage's centre |
| 20 | `centre_elevation` | i32 | |
| 24 | `centre_tilt` | i32 | rotation about the centre's axis |
| 28 | `baseline_um` | u32 | the distance between the two cameras, micrometres; 0 where not stated |
| 32 | `left_layer` | u8 | with `stereo` 3, the left eye's layer; otherwise 0 |
| 33 | `right_layer` | u8 | with `stereo` 3, the right eye's layer; otherwise 0 |
| 34 | reserved | 6 B | zero |

§3.2 Its parameters: `projection` and `stereo` (the names of §2), `first_view` (`left` or
`right`), `coverage` (`[azimuth_range, elevation_range]`), `centre` (`[azimuth, elevation,
tilt]`), `baseline_um`, and `layers` (`[left_layer, right_layer]` with `stereo` 3, else `null`).

## 4. Checks

§4.1 A reader refuses, in this order:

1. fewer than 4 bytes or a magic other than `WPJ1`: `no_signature`;
2. a length other than 40: `length_mismatch`;
3. a `version` other than 1, a `projection` or `stereo` above 3, a `first_view` above 1, or a
   reserved byte that is not zero: `reserved_value`;
4. fields that disagree, `descriptor_invalid`:
   - coverage: for `equirectangular`, exactly 360° by 180°; for `half-equirectangular`, exactly
     180° by 180°; for `fisheye`, each range above 0 and at most 360°; for `rectilinear`, both 0
     (field of view not stated) or each above 0 and below 180°;
   - centre: azimuth and tilt from −180° up to but not including 180°, elevation from −90° to 90°
     (H.274 §8.15's ranges);
   - views: `mono` with a `first_view`, `baseline_um`, `left_layer` or `right_layer` that is not
     0; `side-by-side` or `top-bottom` with a layer that is not 0; `layers` with the two layers
     equal or either above 62.

§4.2 The codes are those of [format-probes](format-probes.md) §12.

## 5. Binding to the rendition

§5.1 A projection descriptor is a `WAI2` companion (SPEC §7.1) whose `companion_of` names a
capability of the video media class, as an HDR descriptor's does. It is never a `WAI1`
`capability` or `fallback` and never a selectable rendition.

§5.2 A descriptor with `stereo` 3 applies only to a rendition of `wai.video.mvhevc` whose probe
(format-probes §4) reports both `left_layer` and `right_layer` among its `layers_present`;
otherwise `layer_absent`, and a sink **MUST NOT** apply it. A sink that presents a rendition with
such a descriptor presents the view of each layer to the eye the descriptor names.

§5.3 With `side-by-side` or `top-bottom`, each view is half the decoded picture's width or height;
a sink that presents one eye only presents the view `first_view` names as the left eye's when it
needs one view.

## 6. Mapping

§6.1 Writing a descriptor from a bitstream's own signalling: an H.274 equirectangular
projection SEI message gives `equirectangular` (or `half-equirectangular` where a region-wise
packing message packs only the front hemisphere); a sphere rotation message gives the centre; a
frame packing arrangement of type 3 or 4 gives `side-by-side` or `top-bottom`; an MV-HEVC
bitstream's two views (format-probes §4, `view_ids`) give `layers`. Writing the other way, a
descriptor gives the same messages. Fisheye has no H.274 message; its parameters beyond the
coverage (the image circle's centre and radius) are not in this revision of the record.

## 7. Conformance

§7.1 [`format-probe-conformance/`](../format-probe-conformance/) holds descriptors of every
projection and arrangement, descriptors refused with each code of §4, and bindings of descriptors
to an MV-HEVC rendition and an H.265 rendition with their outcomes (format-probes §13).