open-standards/wai/extensions/projection.md

WAI Extension: Projection Descriptor

Status: Draft. Registers wai.meta.projection, a companion descriptor to a video rendition (SPEC §5, §7.1), as the HDR dynamic-metadata descriptors are: a 40-byte WPJ1 record that says how the rendition's pictures map onto what a viewer sees (rectilinear, equirectangular, half-equirectangular or fisheye) and how its two views are arranged (one view, two views packed side by side or top and bottom, or two views in two layers of a multiview bitstream such as wai.video.mvhevc). The projection kinds follow ISO/IEC 23090-2 (OMAF); the angle unit and ranges follow ITU-T H.274 (V4, 01/2026) §8.15; the packing arrangements are ITU-T H.273 (V4, 07/2024) §8.4's. The reference implementation is format_probe::projection in wai-rs (feature format_probe); its vectors are in format-probe-conformance/, with its second implementation, format_probe_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

ISO/IEC 23090-2 is not openly readable and was not read. Its projection kinds are used here as the H.274 SEI messages that carry them in a bitstream state them (the equirectangular projection SEI message, §8.15.2, and the sphere rotation and region-wise packing messages, §8.15.4 and §8.15.5), and no clause of ISO/IEC 23090-2 is cited.

1. Scope

§1.1 A projection descriptor describes the pictures of one video rendition: the geometry a renderer maps them onto and the arrangement of their views. It changes no sample and is applied by the renderer, after decoding. A rendition without one is a rectilinear picture of one view.

§1.2 The descriptor is WAI's own record, so a rendition's projection is stated the same way whatever the codec, and also where the codec's own signalling is absent: an AV1 payload, a JPEG XS payload or an MV-HEVC payload whose bitstream carries no projection SEI message. Where a bitstream carries such a message and the descriptor disagrees with it, the descriptor is what WAI states, and a renderer SHOULD report the disagreement.

2. Terms

§2.1 Angles are in units of 2⁻¹⁶ degrees, as H.274 §8.15 states them: one degree is 65 536. Azimuth increases eastward (to the viewer's left, seen from inside the sphere), elevation northward, as for H.274's equirectangular projection (§8.15.2.2).

§2.2 Projections.

code	name	the picture is
0	rectilinear	a flat picture with no sphere projection; its coverage, where stated, is its field of view
1	equirectangular	the equirectangular projection of the whole sphere, 360° of azimuth and 180° of elevation, as H.274 §8.15.2 maps it
2	half-equirectangular	the equirectangular projection of the front hemisphere: 180° of azimuth and 180° of elevation, centred on the coverage centre
3	fisheye	a circular fisheye image whose field of view is its coverage

§2.3 Stereo arrangements.

code	name	the two views are
0	mono	one view
1	side-by-side	the left and right halves of each picture, H.273 VideoFramePackingType 3
2	top-bottom	the top and bottom halves of each picture, VideoFramePackingType 4
3	layers	two layers of a multiview bitstream, each a view, named by their nuh_layer_id

3. The record

§3.1 A wai.meta.projection payload is 40 bytes, multi-byte integers little-endian:

offset	field	type	meaning
0	magic	4 B	WPJ1
4	version	u8	1
5	projection	u8	§2.2
6	stereo	u8	§2.3
7	first_view	u8	0: the left eye's view is the left half, the top half or left_layer's; 1: the right eye's is
8	azimuth_range	u32	coverage in azimuth, 2⁻¹⁶ degrees
12	elevation_range	u32	coverage in elevation
16	centre_azimuth	i32	the coverage's centre
20	centre_elevation	i32
24	centre_tilt	i32	rotation about the centre's axis
28	baseline_um	u32	the distance between the two cameras, micrometres; 0 where not stated
32	left_layer	u8	with stereo 3, the left eye's layer; otherwise 0
33	right_layer	u8	with stereo 3, the right eye's layer; otherwise 0
34	reserved	6 B	zero

§3.2 Its parameters: projection and stereo (the names of §2), first_view (left or right), coverage ([azimuth_range, elevation_range]), centre ([azimuth, elevation, tilt]), baseline_um, and layers ([left_layer, right_layer] with stereo 3, else null).

4. Checks

§4.1 A reader refuses, in this order:

1. fewer than 4 bytes or a magic other than WPJ1: no_signature;
2. a length other than 40: length_mismatch;
3. a version other than 1, a projection or stereo above 3, a first_view above 1, or a reserved byte that is not zero: reserved_value;
4. fields that disagree, descriptor_invalid:
   • coverage: for equirectangular, exactly 360° by 180°; for half-equirectangular, exactly 180° by 180°; for fisheye, each range above 0 and at most 360°; for rectilinear, both 0 (field of view not stated) or each above 0 and below 180°;
   • centre: azimuth and tilt from −180° up to but not including 180°, elevation from −90° to 90° (H.274 §8.15's ranges);
   • views: mono with a first_view, baseline_um, left_layer or right_layer that is not 0; side-by-side or top-bottom with a layer that is not 0; layers with the two layers equal or either above 62.

§4.2 The codes are those of format-probes §12.

5. Binding to the rendition

§5.1 A projection descriptor is a WAI2 companion (SPEC §7.1) whose companion_of names a capability of the video media class, as an HDR descriptor's does. It is never a WAI1 capability or fallback and never a selectable rendition.

§5.2 A descriptor with stereo 3 applies only to a rendition of wai.video.mvhevc whose probe (format-probes §4) reports both left_layer and right_layer among its layers_present; otherwise layer_absent, and a sink MUST NOT apply it. A sink that presents a rendition with such a descriptor presents the view of each layer to the eye the descriptor names.

§5.3 With side-by-side or top-bottom, each view is half the decoded picture's width or height; a sink that presents one eye only presents the view first_view names as the left eye's when it needs one view.

6. Mapping

§6.1 Writing a descriptor from a bitstream's own signalling: an H.274 equirectangular projection SEI message gives equirectangular (or half-equirectangular where a region-wise packing message packs only the front hemisphere); a sphere rotation message gives the centre; a frame packing arrangement of type 3 or 4 gives side-by-side or top-bottom; an MV-HEVC bitstream's two views (format-probes §4, view_ids) give layers. Writing the other way, a descriptor gives the same messages. Fisheye has no H.274 message; its parameters beyond the coverage (the image circle's centre and radius) are not in this revision of the record.

7. Conformance

§7.1 format-probe-conformance/ holds descriptors of every projection and arrangement, descriptors refused with each code of §4, and bindings of descriptors to an MV-HEVC rendition and an H.265 rendition with their outcomes (format-probes §13).