Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions 10

wai.access.services: caption quality anyone can recompute, audio description, sign language and a completeness claim (ACC) #396

Merged
dcharlot merged 8 commits from wai/access-services into main 2026-10-21 21:38:39 -04:00
Conversation 0 Commits 8 Files changed 21 +5602 -2
21 changed files with 5602 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

23
.github/workflows/wai-conformance.yml vendored
View file

@ -83,6 +83,10 @@ on:
# tests/timed_text.rs requires this extension's §12 to list the codes the reference
# reports, and the registry's wire identifiers test reads its sections.
- 'wai/extensions/timed-text.md'
# tests/access_services.rs requires this extension's §10 to list the codes the reference
# reports, access_services_verify.py reads that table, and the registry's wire
# identifiers test reads its sections.
- 'wai/extensions/access-services.md'
# tests/live_metering.rs requires this extension's §9 to list the codes the reference
# reports, live_metering_verify.py reads that table, and the registry's wire identifiers
# test reads its sections.
@ -1023,6 +1027,25 @@ jobs:
(cd wai-rs && cargo test --no-default-features --features timed_text --lib timed_text::)
(cd wai-rs && cargo test --no-default-features --features timed_text --test timed_text)
python3 -I live-conformance/timed_text_verify.py live-conformance/captions
# wai.access.services (extensions/access-services.md): the reference module's unit tests
# (words, full and prefix alignment with their walk back, signed integers, gains) and
# tests/access_services.rs (vectors.json and each DAPT document regenerate byte for byte
# from their fixed seeds and no other file is there, wai-rs gives every vector and
# document its stated outcome and figures, every reason of §10 refuses a case, and §10
# lists exactly the reference's codes); then the corpus's second implementation,
# access_services_verify.py, written from the extension's text and sharing no code with
# wai-rs, which verifies events and captions with timed_text_verify.py and every record
# with WAI's strict Ed25519 rule, computes every figure and writes every DAPT document
# itself, parses each with xml.etree, reads the extension's reason table, and requires
# every mutated copy of the corpus to fail; standard library only, run isolated (-I).
- name: access services · wai-rs and access_services_verify.py
working-directory: wai
shell: bash
run: |
set -euo pipefail
(cd wai-rs && cargo test --no-default-features --features access_services --lib access_services::)
(cd wai-rs && cargo test --no-default-features --features access_services --test access_services)
python3 -I live-conformance/access_services_verify.py live-conformance/access
# wai.live.metering (extensions/live-metering.md): the reference module's unit tests
# (records and ledgers round-trip, the energy rules, the size bound at its limit, a
# total of two classes told from a wrong one, funds rounding, the CMCD event-timeline

50
wai/CHANGELOG.md
View file

@ -6,6 +6,56 @@ wire format may change and such changes are recorded here.
## v0.4.0 — current
### `wai.access.services`: caption quality anyone can recompute, audio description and sign language
- **The extension** ([access-services](extensions/access-services.md)) registers the record
`wai.access.services`. Captions are timed-text's, unchanged; its records are JSON texts in
the JCS form, each signed with Ed25519 under `wai:access` and named by its SHA-256.
- **Caption quality from the signed revisions.** A caption track's **latency** is each cue's
first display on the room clock less the capture of its first word: a sources record places
the words on a take, mapped to the room clock by the take's opening commitment (the take is
checked to derive from a `record_start` to the commitment's author), or on the room clock as
its signer states, and the class (`capture`, `stated`, `none`) says which. Its **stability**
is the revisions after first display and the words they changed; its **accuracy**, the word
errors of the first revision shown against a reference transcript or the reviewed final
revision, by a prefix alignment that does not count the words a partial has not reached.
Words fold ASCII case and edge punctuation, and the alignment walks back by one fixed rule,
so substitutions, deletions and insertions are the same everywhere. A quality record states
the figure and names its inputs by digest; a verifier recomputes it (`figure_mismatch`,
`figure_inputs_mismatch`). §9 maps each figure onto 47 CFR 79.1(j) and (k)(2), the two UK
regulator statements on live subtitling (2013, 2024) and EBU Tech 3370's authoring delay,
and says what is not computed (severity grading, speed, placement).
- **Audio description.** A description record holds the script (each event's interval, what
of the image it describes, its text), the ducking of the programme audio under each event,
and the voicing, on a separate track or by the sink with a pinned model, the script and the
voice each labelled human, machine or machine-reviewed. A voice must start within 35 ms
early and 45 ms late of the end of the duck and end no later than 45 ms after the duck
starts to rise, DAPT §5.5's bounds (`description_out_of_sync`). The record maps onto a DAPT
document (W3C Candidate Recommendation Draft of 2026-06-26), written by hand, with the
ducking as Mixing Instructions on each Script Event and the voice as an Audio Recording or
Synthesized Audio. A claim reports how much of each description is spoken over captioned
dialogue, a figure, not a refusal.
- **Sign language.** A signing record binds each object of an avatar pose stream, or of the
video of a signer, to the final caption revisions it translates, by digest
(`sign_revision_unknown`, `sign_revision_not_final`); a review signs off the translation.
Its figures are each segment's delay behind the speech and, for video, whether it meets the
resolution and frame rate EN 301 549 V4.1.1 clauses 6.5.2 and 6.5.3 state.
- **Completeness.** A claim declares a session's access tracks; a verifier checks each is
present, finished (every cue resolved, every description voiced, every cue signed), in its
language and within the tolerances the claim states, its reviews held and not rejected.
- **Reference and corpus.** The `access_services` module of `wai-rs` (feature
`access_services`) and `live-conformance/access/`, written from fixed seeds by
`wai_access_services_vectors`: a lesson with human and machine captions, a take, a
reference, quality records, a voiced description, a motion-captured signer and a claim; a
reviewed session with a sink-voiced description and a signer's video; a session with no
sources; and a negative control for every reason of §10, among them a misstated latency, a
broken revision chain, a signing bound to a superseded revision, descriptions spoken early
and late, and a claim over a track the verifier does not hold. Its second implementation,
`live-conformance/access_services_verify.py`, gives every case the same outcome and figures
and every DAPT document the same bytes, and every mutated copy of the corpus fails.
- **timed-text** §1 now points audio description, which it leaves out of scope, to this
extension.
### `wai.delivery.settlement`: delivery paid on what the party that received it signed
- **The extension** ([settlement](extensions/settlement.md)) registers the record

9
wai/REGISTRY.md
View file

@ -154,6 +154,7 @@ and a role are fixed once registered.
<!-- generated: wai registry export; do not edit by hand -->
| string | kind | status | media | role | licence | replicate | sets | registered in |
|---|---|---|---|---|---|---|---|---|
| `wai.access.services` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |
| `wai.ai.usage` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |
| `wai.asset.provenance` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |
| `wai.audio.aac` | payload | registered | audio | standalone | patent-pool | standard-unclassified | — | §5 Broadcast and streaming codecs |
@ -355,6 +356,7 @@ and a role are fixed once registered.
| `science.transaction.wai.multivariant` | data-id | defined | http-delivery §11.1 |
| `science.transaction.wai.stages` | data-id | defined | packaging §8 |
| `science.transaction.wai.timed-text` | data-id | defined | timed-text §10.5 |
| `tag:transaction.science,2026:wai/access` | scheme | defined | access-services §5.5 |
| `tag:transaction.science,2026:wai/class` | scheme | defined | packaging §8 |
| `tag:transaction.science,2026:wai/floor` | scheme | defined | http-delivery §11.1 |
| `tag:transaction.science,2026:wai/http-delivery` | scheme | defined | http-delivery §11.1 |
@ -364,10 +366,15 @@ and a role are fixed once registered.
| `urn:wai:event:1` | scheme | defined | live-events §8.4 |
| `urn:wai:event:1` | catalog | defined | live-events §8.1 |
| `wai` | catalog | defined | moq-streaming-format §3 |
| `wai-access-completeness` | document-kind | defined | access-services §7.1 |
| `wai-authorship-map` | document-kind | defined | contribution §7.3 |
| `wai-c2pa` | catalog | defined | live-provenance §10.2 |
| `wai-caption-quality` | document-kind | defined | access-services §4.7 |
| `wai-caption-reference` | document-kind | defined | access-services §4.3 |
| `wai-caption-sources` | document-kind | defined | access-services §4.2 |
| `wai-contribution` | document-kind | defined | contribution §6 |
| `wai-derivation` | document-kind | defined | deriving-edge §5.1 |
| `wai-description` | document-kind | defined | access-services §5.1 |
| `wai-detection` | document-kind | defined | synthesis §7.3 |
| `wai-key-status` | document-kind | defined | jwp-receipts §7.9 |
| `wai-label-policy` | document-kind | defined | synthesis §10.1 |
@ -381,6 +388,8 @@ and a role are fixed once registered.
| `wai-receipts` | catalog | defined | moq-streaming-format §4.1 |
| `wai-review` | document-kind | defined | synthesis §9.1 |
| `wai-session` | document-kind | defined | contribution §3.1 |
| `wai-signing` | document-kind | defined | access-services §6.1 |
| `wai-signing-review` | document-kind | defined | access-services §6.4 |
| `wai-step` | document-kind | defined | contribution §4.1 |
| `wai1` | codecs-prefix | defined | http-delivery §4.1 |
| `wai1` | fourcc | provisional | http-delivery Appendix B |

17
wai/SPEC.md
View file

@ -968,6 +968,23 @@ The event's origin and model digest are the disclosure label, carried through ev
IMSC1 attributes) or, where a target cannot hold it, in a sidecar:
[`extensions/timed-text.md`](extensions/timed-text.md).
And for the **access services** of a room session — `wai.access.services`: figures about a
caption track that any verifier recomputes from its signed revisions. Its latency is each cue's
first display on the room clock less the capture instant of the speech it carries, placed
through a take's opening commitment or as an aligner states it; its stability, the revisions
after first display and the words they changed; its accuracy, the word errors of the first
revision shown against a reviewed final revision or a signed reference transcript; all in
integer arithmetic over one fixed word alignment, so a quality figure a vendor states is checked
by recomputing it. An audio description track is a signed script of timed descriptions, the
ducking of the programme audio under each, and its voicing on a separate track or by the sink,
each labelled human or machine; its voice is held to the synchronisation bounds of the Dubbing
and Audio description Profiles of TTML2, and it maps onto a document of that profile. A sign
language track, an avatar pose stream or the video of a person signing, is bound by digest to
the final caption revisions it translates, with a reviewer's optional sign-off. A completeness
claim states that each access track a session declares was delivered complete and in sync,
against tolerances it names, and a verifier checks it against the records:
[`extensions/access-services.md`](extensions/access-services.md).
And for what a room session **cost** — `wai.live.metering`: every hop that carried the session
(a relay, a compositor, an ingest gateway) signs one per-hop session record, `WMH1`: per track
and direction the objects and bytes it carried and the objects it dropped by reason, with those

859
wai/extensions/access-services.md Normal file
View file

@ -0,0 +1,859 @@
# WAI Extension: Access Services
> Status: Draft. Registers the record `wai.access.services`: the access services of a room
> session, and figures about them that any verifier recomputes from signed records. Captions
> are [timed-text](timed-text.md)'s, unchanged. This extension adds a **quality figure** for a
> caption track: its **latency**, the room-clock instant a cue was first shown less the
> capture instant of the speech it carries; its **stability**, the revisions after first
> display and the words they changed; and its **accuracy**, the word errors of the first
> revision shown against a reviewed final revision or a signed reference transcript. Each is
> defined in integer arithmetic over the signed revisions, so two verifiers reach the same
> numbers, and a figure a vendor states is checked by recomputing it. **Audio description** is
> a signed description track, a script of timed descriptions with their mixing instructions
> (the programme audio ducked under each) and their voicing, a separate audio track or a voice
> the sink synthesizes, each labelled human or machine; it maps onto a document of the Dubbing
> and Audio description Profiles of TTML2. **Sign language** is a signing track, a
> `wai.avatar.pose` stream or the video of a person signing, whose objects are bound by digest
> to the caption revisions they translate, with a reviewer's optional sign-off. A
> **completeness claim** states, for a room session, that each access track it declares was
> delivered complete and in sync, measured from signed times against the tolerances the texts
> below state; a verifier checks it against the records. No new transport. The reference
> implementation is the `access_services` module of `wai-rs` (feature `access_services`); the
> corpus is [`live-conformance/access/`](../live-conformance/access/), and its second
> implementation is `live-conformance/access_services_verify.py`.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
>
> Texts read for this revision, each pinned to the version named: Directive (EU) 2019/882,
> Annex I, Section IV, point (b)(ii), and Article 31(2), from a reproduction of the text as
> adopted (the Official Journal could not be fetched for this revision); EN 301 549 V4.1.1
> (2026-09), clauses 4.2.4, 6.5.2, 6.5.3, 7.1.2, 7.1.5, 7.2.1 to 7.2.3, 7.3, 12.4 and 12.5;
> Web Content Accessibility Guidelines (WCAG) 2.2, Success Criteria 1.2.2 to 1.2.7; 47 CFR
> 79.1(j)(2), (j)(3) and (k)(2), and 47 CFR 79.3(a)(3), as in force on 2026-10-01; Dubbing and
> Audio description Profiles of TTML2 (DAPT), W3C Candidate Recommendation Draft of
> 2026-06-26 (`CRD-dapt-20260626`), §4.1 to §4.10, §5.2, §5.3, §5.5 to §5.7 and Appendix
> H.2.2; EBU Tech 3370, EBU-TT Part 3, version 1.0 (May 2017), §2.3.2 and §2.3.3; and two
> statements of the United Kingdom's communications regulator, *Measuring the quality of live
> subtitling* (16 October 2013), §1.3, §3.65 to §3.67 and footnote 21, and *Ensuring the
> quality of TV and on-demand services* (15 April 2024), §3.3, §5.17, §5.23, §5.25, §5.29,
> §6.2 and §6.14. A section number of one of those texts in this extension is that version's,
> and is written with the text's name; a bare section number is this extension's. DAPT is a
> Candidate Recommendation Draft, whose features may change before it is a Recommendation,
> and one that §5.5 uses is at risk there.
## 1. Scope
In scope: the records and their encoding (§2); words, and the alignment of two lists of words
(§3); the quality figure of a caption track: the sources that place its words on the room
clock, a reference transcript, latency, stability, accuracy, and the figure record (§4);
audio description: the description record, its script, mixing and voicing, its
synchronisation, its DAPT document, and what a player owes its listeners (§5); sign language:
the signing record, its binding to caption revisions, and a review (§6); the completeness
claim (§7); verification (§8); and what each figure measures in the texts above (§9).
Out of scope: a transport; recognising speech, writing descriptions, translating into a sign
language and synthesizing speech, which produce the content; grading the severity of an
error, which needs a person (§9); decoding a voicing track or a signing track, whose objects
this extension names by digest; spoken subtitles and extended audio description, which pause
or lengthen the programme; how a sink renders a description or a signer; and what any law
requires.
Relationship to the core spec: `wai.access.services` is a **record** ([SPEC](../SPEC.md)
§5): a signed record a WAI tool emits and verifies. It names no payload format and is not
dispatchable.
Relationship to [timed-text](timed-text.md): captions are its live captions, verified as its
§4.3 verifies them, and this extension changes nothing in them. The audio description that
timed-text §1 leaves out of its scope is this extension's §5.
Relationship to [avatars](avatars.md) and [capture-binding](capture-binding.md): a signing
track in pose form is a `wai.avatar.pose` world, whose op batches are its objects (avatars
§5); a source on a take is placed on the room clock by the take's clock mapping
(capture-binding §9.3).
Relationship to [synthesis](synthesis.md): the origins this extension's records state are a
caption's, which synthesis §1 maps onto its classes. A description voiced or written by a
model, and a signing a model generates, is synthesized media in its sense: class
`synthesized`, step `generate`, with the operation `speech-synthesis` for a voice, and
oversight `fully_autonomous` for origin `machine` or `human_validated` for origin
`machine-reviewed` (synthesis §3.1, §3.2). The publisher of such a track signs its origin
claim there; this extension's label is the same statement, carried where a description or a
signing is read. A review of §6.4 reviews a translation against caption revisions; a review
of one exact reconstruction of a generated track, which every exact sink then shows, is
synthesis §9.1's.
Duties this extension supports: the access services of Directive (EU) 2019/882, Annex I,
Section IV(b)(ii) ("subtitles for the deaf and hard of hearing, audio description, spoken
subtitles and sign language interpretation … fully transmitted with adequate quality for
accurate display, and synchronised with sound and video, while allowing for user control of
their display and use"); EN 301 549 V4.1.1 clauses 7.2.1 to 7.2.3, 7.3 and 12.4, beside the
caption clauses timed-text §11 answers; WCAG 2.2 Success Criteria 1.2.3, 1.2.5 and 1.2.6,
beside the caption criteria 1.2.2 and 1.2.4; and the caption quality standards of 47 CFR
79.1(j). WAI states what its records carry and what a verifier checks. It makes no statement
about any regime, and a deployment confirms what a regime asks of it against that regime's
text.
## 2. Records
Every record of this extension is a JSON object (RFC 8259), read as
[jwp-receipts](jwp-receipts.md) §3.1 step 1 reads a claim: RFC 8259's grammar, no byte-order
mark, no member named twice in one object, no `\u` escape that leaves a surrogate unpaired,
and at most 32 levels of nesting. Its values use the encodings of jwp-receipts §2.4.3: byte
strings in lowercase hexadecimal; integers unsigned, at most 2^53 − 1, in plain decimal; no
other number and no boolean. The types in this extension's tables are:
- `hex32`, `hex64`: 64 or 128 lowercase hexadecimal digits;
- `int`: such an integer;
- `sint`: a **signed integer**, written as a JSON string: `0`, or an optional `-` then a
digit 1 to 9 then any digits, whose value lies in −2^63 to 2^63 − 1;
- `string`: a JSON string; `tag`: a string that live-events §4.10 reads as a language tag
(checked by the rules of the record's section);
- a list of quoted values: a string equal to one of them;
- `object`, `array`: as the table under it says; `null`, written where a member may be null.
A record has exactly the members its table lists, at every level, none missing and none
added, each of its type, and a `kind` that names it. One that is not JSON, or does not have
that shape, is `record_malformed`.
Every record carries `signer_pubkey` (`hex32`, the Ed25519 key of RFC 8032 that signs it) and
`sig` (`hex64`):
```
sig = Ed25519(signer_key, "wai:access\x01" || JCS(record without "sig"))
record id = SHA-256(JCS(record))
```
JCS is RFC 8785, which on these values reduces to what jwp-receipts §2.4.3 states. The
signature is verified strictly ([SPEC](../SPEC.md) §7 *Signatures*): a key that fails step 1
or 2 is `bad_key`, and a signature that fails a later step `bad_signature`. A record is named
by its record id wherever another names it. A member `room` names the room by its id
(live-events §2); a record whose `room` is not the verifier's room is `wrong_room`.
Which signers a verifier trusts for each kind of record is its policy. This extension checks
what each record states against the events and the other records; a figure's weight comes
from its recomputation (§4.7), not from its signer.
## 3. Words and their alignment
### 3.1 Words
The **words** of a text are found in two steps:
1. The text is split at every U+0020 and U+000A, and empty pieces are dropped.
2. In each piece, every code point from U+0041 to U+005A is replaced by the one 0x20 above
it (ASCII letters to lower case), and code points of the set *P* are removed from its
start and its end, repeatedly, until it neither starts nor ends with one. A piece left
empty is dropped; the rest are the words, in order.
*P* is U+0021 `!`, U+0022 `"`, U+0027 `'`, U+0028 `(`, U+0029 `)`, U+002C `,`, U+002D `-`,
U+002E `.`, U+003A `:`, U+003B `;`, U+003F `?`, U+005B `[`, U+005D `]`, U+00A1 `¡`, U+00AB
`«`, U+00BB `»`, U+00BF `¿`, U+2013 `–`, U+2014 `—`, U+2018 `‘`, U+2019 `’`, U+201C `“`,
U+201D `”` and U+2026 `…`.
No other normalisation is made: letters outside ASCII keep their case, and two words are
equal when their code points are. The words so found ignore the case of ASCII letters and
the punctuation at a word's edges, so an error of capitalisation or of such punctuation is
not a word error here (§9).
### 3.2 Alignment
The alignment of a **hypothesis** *H* (words *h*₁ … *h*ₘ) to a **reference** *R* (words
*r*₁ … *r*ₙ) is the table *D* of integers, for 0 ≤ *i* ≤ *m* and 0 ≤ *j* ≤ *n*:
- *D*[0][*j*] = *j*, and *D*[*i*][0] = *i*;
- *D*[*i*][*j*] = the least of *D*[*i* − 1][*j* − 1] + *c*, *D*[*i* − 1][*j*] + 1 and
*D*[*i*][*j* − 1] + 1, where *c* is 0 when *h*ᵢ = *r*ⱼ and 1 otherwise.
Its **end** is a column *j*\*:
- a **full** alignment ends at *j*\* = *n*;
- a **prefix** alignment ends at the greatest *j* for which *D*[*m*][*j*] is the least
value in row *m*: the reference is read as far as the hypothesis reaches, and no further.
Its **errors** are *E* = *D*[*m*][*j*\*] and its **reference words** *N* = *j*\*. They are
split into substitutions, deletions and insertions by walking back from (*m*, *j*\*) to
(0, 0), taking at each step the first of these that holds:
1. *i* > 0, *j* > 0 and *D*[*i*][*j*] = *D*[*i* − 1][*j* − 1] + *c*: to (*i* − 1, *j* − 1),
a **substitution** when *c* is 1;
2. *i* > 0 and *D*[*i*][*j*] = *D*[*i* − 1][*j*] + 1: to (*i* − 1, *j*), an **insertion**
(a hypothesis word the reference does not have);
3. otherwise: to (*i*, *j* − 1), a **deletion** (a reference word the hypothesis does not
have).
The substitutions, deletions and insertions add up to *E*.
## 4. Caption quality
### 4.1 The caption track and its cues
A **caption track** is the caption events of one principal *P* in a room's stream, verified
as [timed-text](timed-text.md) §4.3 verifies them. Its **cues** are the chains of its
revisions (timed-text §2): each from its first revision *r*₀ to its last *r*ₖ. A cue is named
by the digest of *r*₀'s `caption` event (live-events §3.2).
*P*'s captions are **whole** when *P* has at least one cue, the verifier holds every `seq` of
*P* from 0 to the last it holds (no delivery gap of *P*, live-events §6.2), and, for every
cue, *r*ₖ is final and its annotation is not unknown (timed-text §4.3). They are
**resolved** when they are whole and every *r*ₖ has a `duration_ms` above 0, so that each cue
resolves (timed-text §4.5). A figure is computed only over whole captions.
The **events digest** of *P* is SHA-256 over the digests of *P*'s `caption` events the
verifier holds, 32 bytes each, in `seq` order.
### 4.2 Sources
A **sources** record places the words of each cue on the room clock: where the speech the cue
carries was captured.
| member | type | meaning |
|---|---|---|
| `kind` | `"wai-caption-sources"` | |
| `room` | `hex32` | the room |
| `principal` | `hex32` | *P*, whose cues it places |
| `sources` | array | one entry per cue it places, each an object below |
| `signer_pubkey`, `sig` | | §2 |
| member of an entry | type | meaning |
|---|---|---|
| `cue` | `hex32` | the cue, by the digest of its first revision |
| `basis` | `"take"`, `"room"` | what `start` and `end` are measured on |
| `take` | `hex32` or `null` | for `take`, the take id (capture-binding §3.2); `null` for `room` |
| `start`, `end` | `int` | the capture of the cue's first word and of its last: device-clock microseconds of the take, or room-clock microseconds |
Its signer is whoever aligned the words to the media: the captioner, or anyone holding the
media afterwards. The alignment of a word to the media is the signer's statement; the
media's own time is not. On the basis `take`, the instants are positions in the master of a
take, and the take's clock mapping places them on the room clock: the offset *o* of the
take's opening commitment (capture-binding §9.3), which the capturing device signed while it
captured. A **source instant** *d* maps to the room-clock instant *d* + *o*. On the basis
`room`, the instants are on the room clock as stated.
Rules, in order:
1. `room` is the verifier's: otherwise `wrong_room`.
2. No sources record of *P* comes before it in the order given: otherwise
`record_conflict`.
3. Each entry: `take` is a `hex32` exactly when `basis` is `take`, and `start` ≤ `end`;
no cue is named twice: otherwise `record_malformed`.
4. *P*'s captions are whole (§4.1): otherwise `stream_incomplete`.
5. Each entry's `cue` names a cue of *P*'s captions: otherwise `source_unknown`.
6. Each entry on the basis `take`: among the events the verifier accepted there is a
`take_commit` event whose payload capture-binding §5.2 reads, whose `take` is the entry's
take, and which is an opening commitment (`n` 0, `phase` 1, capture-binding §5.3); the
first such in stream order is the take's opening commitment, its author *A* and its
offset *o*. The take is **derived**: an accepted `control` event whose command is
`record_start` and whose `target` is *A* has a digest *s* for which SHA-256 of
`wai:capture-take`, 0x01, the room id, *A* and *s* is the take (capture-binding §3.2).
And `start` + *o* is not below 0. Otherwise `source_take_unknown`.
*P*'s **sources record** is the first sources record of *P* in the order given, and its
**reference record** the first reference record (§4.3); a later one of either is
`record_conflict`. Who may start a take, and which device captures for which principal, are
live-rooms' and capture-binding's checks; this extension uses the take's mapping as
capture-binding §9.3 defines it, and its bound, *J*, holds where a take verifier gives the
take `single-take` and `aligned` (capture-binding §9.4, §10).
### 4.3 The reference transcript
A **reference** record gives, for each cue of *P*, the text a reviewer holds correct.
| member | type | meaning |
|---|---|---|
| `kind` | `"wai-caption-reference"` | |
| `room` | `hex32` | the room |
| `principal` | `hex32` | *P* |
| `entries` | array | one per cue, each `{"cue": hex32, "text": string}` |
| `signer_pubkey`, `sig` | | §2 |
Rules, in order: `room` is the verifier's (`wrong_room`); no reference record of *P* comes
before it in the order given (`record_conflict`); no cue is named twice (`record_malformed`);
each text meets timed-text §3.2 rules 1 to 3 (`text_not_in_subset`); *P*'s captions are whole
(`stream_incomplete`); each entry names a cue of *P* (`reference_unknown`); and every cue of
*P* has an entry (`reference_incomplete`).
### 4.4 Latency
The latency of a cue that a sources entry places is *L* = `t`(*r*₀) − *S*, in microseconds,
where `t`(*r*₀) is the room-clock instant its first revision was shown from (timed-text
§4.1) and *S* is its entry's `start` on the room clock (§4.2). *L* is negative when the
caption was shown before the speech began.
Over the cues of *P* that the sources record places, *n* of them, with their latencies in
ascending order *L*₁ ≤ … ≤ *L*ₙ:
- `min` = *L*₁, `max` = *L*ₙ;
- `mean` = ⌊(*L*₁ + … + *L*ₙ) / *n*⌋, rounded toward negative infinity;
- `p50` and `p95`: the latency at rank ⌈50 × *n* / 100⌉ and ⌈95 × *n* / 100⌉, counting
from 1.
Each is `null` when *n* is 0. The **class** of the latency is `none` when *n* is 0;
otherwise `capture` when every entry is on the basis `take`, and `stated` when any is on the
basis `room`.
### 4.5 Stability
For one cue, its **revisions** are *k*, the revisions after the first, and its **words
changed** the sum, over each revision *r*ᵢ and the one that supersedes it *r*ᵢ₊₁, of the
substitutions and insertions of the full alignment of the words of *r*ᵢ (the hypothesis) to
the words of *r*ᵢ₊₁ (the reference): the words a viewer saw that the next revision did not
keep. A word added at the end, as a partial grows, is a deletion there and is not counted.
The track's figures are the sums over its cues of revisions and of words changed, and
`cues_corrected`, the number of cues whose words changed is above 0.
### 4.6 Accuracy
The reference of a cue is its entry's text in *P*'s reference record, when *P* has one;
otherwise the text of *r*ₖ, the cue's final revision. The **class** of the accuracy is:
- `transcript` when *P* has a reference record;
- `reviewed` when it has none and every cue's *r*ₖ has origin 1 (human) or 3
(machine-reviewed): a person wrote or reviewed the reference;
- `self` otherwise: the reference is the machine's own final text, and the figure measures
how far its first display was from its own last.
Two figures, each the sums over the cues of the counts of §3.2:
- `first`: the prefix alignment of the words of *r*₀ to the words of the reference. It counts
the errors in what the viewer first saw, against as much of the reference as that revision
reaches, so the words a partial has not reached yet are not errors.
- `final`: the full alignment of the words of *r*ₖ to the words of the reference, when the
class is `transcript`; `null` otherwise.
Each is `{"words", "errors", "substitutions", "deletions", "insertions"}`: *N*, *E* and its
three parts.
### 4.7 The figure
A **quality** record states the figures of *P*'s track.
| member | type | meaning |
|---|---|---|
| `kind` | `"wai-caption-quality"` | |
| `room` | `hex32` | the room |
| `principal` | `hex32` | *P* |
| `events` | `hex32` | *P*'s events digest (§4.1) |
| `sources` | `hex32` or `null` | the record id of *P*'s sources record, or `null` when there is none |
| `reference` | `hex32` or `null` | likewise, of its reference record |
| `cues` | `int` | the number of *P*'s cues |
| `latency` | object | `{"class", "n", "min", "max", "mean", "p50", "p95"}`: a string of §4.4, an `int`, and five `sint` or `null` |
| `stability` | object | `{"revisions", "words_changed", "cues_corrected"}`, each an `int` |
| `accuracy` | object | `{"class", "first", "final"}`: a string of §4.6, an object of §4.6, and one or `null` |
| `signer_pubkey`, `sig` | | §2 |
Rules, in order:
1. `room` is the verifier's: otherwise `wrong_room`.
2. *P*'s captions are whole (§4.1): otherwise `stream_incomplete`.
3. `events`, `sources` and `reference` are as the table says: otherwise
`figure_inputs_mismatch`.
4. Every other member but `kind`, `signer_pubkey` and `sig` equals the figure the verifier
computes from §4.4 to §4.6: otherwise `figure_mismatch`.
A figure that passes is the figure of exactly those revisions, placed by exactly those
sources and measured against exactly that reference. A verifier needs no record of its own to
compute a figure: the **figure** of a whole track is the object a quality record would carry,
without `kind`, `signer_pubkey` and `sig`.
## 5. Audio description
### 5.1 The description record
A **description** record is an audio description track: a script of timed descriptions of
what is seen, the mixing of each into the programme audio, and its voicing. Times are media
time in milliseconds, on the timeline of the media the descriptions accompany, as
timed-text §2 defines it.
| member | type | meaning |
|---|---|---|
| `kind` | `"wai-description"` | |
| `lang` | `tag` | the language of the descriptions |
| `script` | object | `{"origin", "model"}`: who wrote the descriptions |
| `voicing` | object | `{"form", "origin", "model", "track"}`: who or what speaks them |
| `events` | array | the descriptions, each an object below |
| `signer_pubkey`, `sig` | | §2 |
`origin` is `"human"`, `"machine"` or `"machine-reviewed"`, the names timed-text §10.5 gives
the origins of a caption, and `model` is a `hex32` naming the model that wrote or spoke the
text, or `null`. `form` is `"track"`, the voicing is a separate audio track named by `track`
(a `hex32`, the track's digest, live-rooms §6.2), or `"sink"`, the sink synthesizes the
voice from the text with the model `model`, and `track` is `null`.
| member of an event | type | meaning |
|---|---|---|
| `id` | `string` | its identifier |
| `begin`, `end` | `int` | the interval it describes, the description's time slot |
| `represents` | `"visual"`, `"visual.nonText"`, `"visual.text"`, `"visual.text.title"`, `"visual.text.credit"`, `"visual.text.location"`, `"visual.dialogue"` | what of the image it describes: DAPT's content descriptors of the visual image (DAPT §4.7 and Appendix H.2.2) |
| `text` | `string` | the description |
| `duck` | object | `{"gain", "ramp"}`, each an `int`: the programme audio's gain under the description in thousandths, and the milliseconds it takes to fall to it and to rise back |
| `voice` | object or `null` | `{"begin", "end"}`, each an `int`: where the voicing track speaks it; `null` when the form is `sink`, or when the track has not voiced it |
### 5.2 Rules
In order; the first that fails names the refusal:
1. `lang` reads as live-events §4.10 reads a tag: otherwise `description_malformed`.
2. `script.model` is a `hex32` exactly when `script.origin` is `machine` or
`machine-reviewed`; likewise `voicing.model` for `voicing.origin`; `voicing.track` is a
`hex32` exactly when `form` is `track`; and when `form` is `sink`, `voicing.origin` is not
`human`: otherwise `description_malformed`.
3. There is at least one event; each `id` is one of `A`–`Z`, `a`–`z` or `_`, then up to 63
of those, `0`–`9`, `.` and `-`; no `id` twice; each `begin` < `end` ≤ 359 999 999; each
event's `begin` is at or after the `end` of the event before it (one voice describes one
thing at a time): otherwise `description_malformed`.
4. Each `text` meets timed-text §3.2 rules 1 to 3: otherwise `text_not_in_subset`.
5. Each `duck`: `gain` ≤ 1000; `ramp` is 0 when `gain` is 1000 and above 0 otherwise; and
2 × `ramp` ≤ `end` − `begin`. Each `voice`: `null` when the form is `sink`, and otherwise
`null` or within its event, the event's `begin` ≤ `voice.begin` < `voice.end` ≤ the
event's `end`, since a Script Event's audio is truncated at its end (DAPT §4.9.1):
otherwise `description_malformed`.
6. The synchronisation of §5.3: otherwise `description_out_of_sync`.
### 5.3 Synchronisation
The programme audio falls to the event's gain from `begin` to `begin` + `ramp`, and rises
back from `end` − `ramp` to `end`. The description is **spoken** over the interval between,
from *a* = `begin` + `ramp` to *z* = `end` − `ramp`. Of a voiced event (its `voice` not `null`):
- `voice.begin` ≥ *a* − 35 and `voice.begin` ≤ *a* + 45;
- `voice.end` ≤ *z* + 45.
These are DAPT §5.5's bounds on a presentation processor: audio presentation changes no more
than 35 ms before the time a document specifies and no more than 45 ms after it, which DAPT
derives from EBU R 37's bounds on audio relative to image. A voice that starts before the
programme audio has fallen, or runs on after it has risen, is out of those bounds. EN 301 549
V4.1.1 clause 7.2.2 requires the synchronisation to be preserved and states no figure.
The **offset** of a voiced event is `voice.begin` − *a*, a signed number of milliseconds;
an unvoiced event has none.
### 5.4 Description over dialogue
A description is spoken in the pauses of the dialogue (47 CFR 79.3(a)(3): "into natural
pauses between the program's dialogue"), though the regulator's 2024 statement accepts
describing over dialogue where what is seen matters more (§6.14 there). So this is a figure,
not a refusal. Against the dialogue of one or more caption tracks, whose resolved cues
(timed-text §4.5) cover the union *U* of their intervals, each from its `start` up to, not
including, its `end`, an event's **overlap** is the milliseconds of its spoken interval that lie
in *U*: from `voice.begin` up to `voice.end` when it is voiced, and from *a* up to *z*
otherwise. The figure is the sum of the overlaps and the
number of events whose overlap is above 0.
### 5.5 The DAPT document
A description record maps onto one DAPT document, given, for the form `track`, the voicing
track's URI `U` and media type `T`, each one to 2048 of the ASCII characters `!` to `~`
other than `"`, `&`, `<` and `>` (otherwise `parameter_malformed`). It is written as UTF-8
with U+000A line ends:
```
<?xml version="1.0" encoding="UTF-8"?>
<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" xmlns:tta="http://www.w3.org/ns/ttml#audio" xmlns:daptm="http://www.w3.org/ns/ttml/profile/dapt#metadata" xmlns:wai="tag:transaction.science,2026:wai/access" xml:lang="LANG" ttp:contentProfiles="http://www.w3.org/ns/ttml/profile/dapt1.0/content" daptm:langSrc="zxx" daptm:scriptRepresents="REPS" daptm:scriptType="TYPE" wai:script="SO" wai:voice="VO">
<body>
EVENTS
</body>
</tt>
```
- `LANG` is `lang`. `REPS` is each `represents` value of the events once, in the order each
first appears, joined by U+0020: every event's value is then one of them (DAPT §4.1.1).
`TYPE` is `asRecorded` for the form `track` and `preRecording` for the form `sink`.
- `SO` and `VO` are the script's and the voicing's origins. After `wai:script="SO"` comes
` wai:script-model="M"` when the script has a model, and after `wai:voice="VO"`,
` wai:voice-model="M"` when the voicing has one, `M` the digest in lowercase hexadecimal.
- `EVENTS` is, for each event in order:
```
<div xml:id="ID" begin="B" end="E" daptm:represents="R">
<animate begin="00:00:00.000" end="RAMP" tta:gain="1;G" fill="freeze"/>
<animate begin="FALL" end="DUR" tta:gain="G;1"/>
<p>CONTENT</p>
</div>
```
`B` and `E` are `begin` and `end` written `HH:MM:SS.mmm` (timed-text §5.1); `RAMP`, `FALL`
and `DUR` are `ramp`, `end` − `begin` − `ramp` and `end` − `begin`, written the same way;
`G` is the gain: `0` for 0, and otherwise `0.` and its three decimal digits with trailing
zeros removed. The two `animate` lines are written only when the gain is below 1000.
- `CONTENT` is the text, its lines separated by `<br/>` and `&`, `<` and `>` written `&amp;`,
`&lt;` and `&gt;`:
- for the form `track` and a voiced event,
`<span begin="VB" end="VE"><audio src="U" type="T" clipBegin="CB" clipEnd="CE"/>TEXT</span>`,
with `VB` = `voice.begin` − `begin`, `VE` = `voice.end` − `begin`, `CB` = `voice.begin`
and `CE` = `voice.end`, each written `HH:MM:SS.mmm`. The voicing track is on the
programme's timeline (an audio description stem), so the event's audio is the interval
of it that the event's voice states;
- for the form `track` and an event not yet voiced, the text alone;
- for the form `sink`, `<span tta:speak="normal">TEXT</span>`.
The document's features are DAPT's: a Script Event per description, a `div` with `xml:id`,
`begin`, `end` and `daptm:represents` (DAPT §4.3); its Mixing Instructions as `animate`
children of the Script Event, which adjust the programme audio during it (DAPT §4.3, §4.10);
an Audio Recording by `src` and `type` with `clipBegin` and `clipEnd` (DAPT §4.9.1); and
Synthesized Audio by `tta:speak` (DAPT §4.9.2). `ttp:contentProfiles` names the DAPT content
profile, and no `ttp:profile` is written (DAPT §5.6). The document's time base is media time,
times are clock times without frames (DAPT §5.7), and `daptm:langSrc` is `zxx`, content
without an inherent language (DAPT §4.5). Two choices are this extension's:
- `src` on `audio`, for an external resource, is at risk in the Candidate Recommendation
(its Issue 218). If it does not stay, a later revision of this mapping names the voicing
track another way DAPT §4.9.1 keeps.
- For the form `sink`, DAPT says a pre-recording script SHOULD NOT contain Audio objects, and
an as-recorded script SHOULD NOT contain Synthesized Audio. There is no recording: the
sink's voice is the recording. The document is the script it speaks, marked as one, with
`tta:speak` saying that it is to be spoken.
The attributes in the namespace `tag:transaction.science,2026:wai/access` are foreign
vocabulary, which DAPT §5.2 has a processor keep or prune. They carry the origin labels of
§5.6, which DAPT has no field for.
### 5.6 Obligations of a WAI player
These are requirements on a WAI player that plays audio description or shows sign language.
Each cites the provision it answers by identifier; they are engineering requirements, and
meeting them is not a statement about any law.
1. **Description can be played.** A player of video with synchronised audio MUST let the
listener select an available description track and play it to the default audio output
(EN 301 549 V4.1.1 clause 7.2.1; WCAG 2.2 Success Criteria 1.2.3 and 1.2.5, which ask
content to provide it).
2. **In sync.** A player that mixes a description MUST apply its mixing instructions and play
each voice at the times its event states, within §5.3's bounds (EN 301 549 V4.1.1 clause
7.2.2).
3. **Preserved.** A player or converter that transmits, converts or records a description MUST
keep every event's timing, text, mixing instructions, voicing and labels (EN 301 549 V4.1.1
clause 7.2.3): §5.5's document keeps them all.
4. **One operation.** A player whose purpose is video MUST offer turning description on and
off as a single operation, at the same level as the volume control (EN 301 549 V4.1.1
clause 7.3), as timed-text §11 item 5 has it for captions.
5. **Labelled.** A player MUST show, with a description whose voicing is `machine` or
`machine-reviewed`, that the voice is synthesized, in a way the listener perceives without
turning on another feature, as timed-text §11 item 7 has it for a machine caption, and
SHOULD let the listener learn whether the script was machine-written. The script's origin
and the voicing's are the description's labels, as a caption's origin is its label
(timed-text §10.1); a synthesized voice's origin, with the model's digest, is the statement
its origin claim makes of it (synthesis §3.1), and a player shows its label as synthesis
§10 has it show a synthesized object's.
6. **Signing under the viewer's control.** A player that shows a signing track MUST let the
viewer choose to show it, and SHOULD let the viewer choose its size and position (Directive
(EU) 2019/882 Annex I Section IV(b)(ii), "user control of their display"); a pose stream
leaves both to the sink (§6.1). It MUST show, with a signing track whose origin is
`machine` or `machine-reviewed`, that the signing is generated.
## 6. Sign language
### 6.1 The signing record
A **signing** record is a sign language track: a `wai.avatar.pose` stream (avatars §1), or a
video track of a person signing. Its objects are bound, by digest, to the caption revisions
they translate.
| member | type | meaning |
|---|---|---|
| `kind` | `"wai-signing"` | |
| `room` | `hex32` | the room |
| `lang` | `tag` | the sign language |
| `source` | `hex32` | the principal of the caption track it translates |
| `form` | `"pose"`, `"video"` | an avatar pose stream, or a video of a signer |
| `track` | `hex32` | the track's digest (live-rooms §6.2) |
| `origin`, `model` | | as §5.1: `human` for a person signing (filmed, or motion-captured into the pose stream), `machine` for signing a model generated |
| `video` | object or `null` | for `video`, `{"width", "height", "rate_num", "rate_den"}`, each an `int`: its picture size and frame rate; `null` for `pose` |
| `segments` | array | each an object below |
| `signer_pubkey`, `sig` | | §2 |
| member of a segment | type | meaning |
|---|---|---|
| `object` | `hex32` | the content hash of an object of the track (jwp-receipts §2.1): an op batch of the pose world, or a video object |
| `start`, `end` | `int` | room-clock microseconds the signing of the object spans |
| `revisions` | array of `hex32` | the caption revisions it translates, by the digest of each `caption` event |
Pose-equivalence (avatars §4) makes the motion every sink shows the motion in the bound
objects, while the appearance is the sink's: a viewer may choose the signer's size, place
and appearance, which Directive (EU) 2019/882 Annex I Section IV(b)(ii) calls user control of
display, and the binding still holds.
### 6.2 Rules
In order:
1. `room` is the verifier's: otherwise `wrong_room`.
2. `lang` reads as a tag; `model` is a `hex32` exactly when `origin` is `machine` or
`machine-reviewed`; `video` is `null` exactly for `pose`, and its four numbers are above 0;
there is at least one segment, each with `start` < `end` and at least one revision, each
segment's `start` at or after the `end` of the segment before it; and no revision is
named twice in the record: otherwise `signing_malformed`.
3. `source`'s captions are whole (§4.1): otherwise `stream_incomplete`.
4. Each named revision is a `caption` event of `source` that the verifier holds: otherwise
`sign_revision_unknown`.
5. Each named revision is the last of its cue and final, its annotation not unknown: otherwise
`sign_revision_not_final`. Signing translates the text a cue resolves to; a partial that a
later revision changed is not what the captions say.
### 6.3 Figures
- The **offset** of a segment is its `start` less *S* (§4.4) of the earliest cue, by its
first revision's `t`, among those whose final revisions it names, when `source`'s sources
record places that cue; it has none otherwise. It is a signed number of microseconds: the
delay of the signing behind the speech it interprets.
- For the form `video`, the **video floor** is `meets` when the picture is at least 320 by 240
in either orientation (the smaller side at least 240 and the larger at least 320) and the
frame rate at least 20 a second (`rate_num` ≥ 20 × `rate_den`), and `below` otherwise: the
resolution and frame rate EN 301 549 V4.1.1 clauses 6.5.2 and 6.5.3 require of real-time
video communication, which its clause 4.2.4 NOTE 3 names for sign language that accompanies
audiovisual content. For the form `pose` it is `null`: the sink renders the signer at the
size it chooses.
### 6.4 Review
A person who reviews the translation signs a **review**:
| member | type | meaning |
|---|---|---|
| `kind` | `"wai-signing-review"` | |
| `signing` | `hex32` | the record id of the signing record it reviews |
| `scope` | `"translation"` | what was reviewed: the signing against the caption revisions it is bound to |
| `verdict` | `"approved"`, `"approved-with-notes"`, `"rejected"` | |
| `notes` | `hex32` or `null` | the digest of the reviewer's notes; `null` exactly for `approved` |
| `signer_pubkey`, `sig` | | §2 |
Rules: `notes` is `null` exactly when the verdict is `approved` (`record_malformed`), and
`signing` is the record id of a signing record the verifier holds (`review_subject_unknown`).
The review binds the objects by their digests, so, for a pose stream, every viewer whose sink
is pose-equivalent sees the motion the reviewer approved. Its vocabulary is synthesis §9.1's
review with the scope `translation`; that review binds one reconstruction of a synthesized
object, and this one binds a signing record to the caption revisions it translates.
## 7. The completeness claim
### 7.1 The record
A **completeness** claim states, for a room session, which access tracks it carried, and that
each was delivered complete and in sync.
| member | type | meaning |
|---|---|---|
| `kind` | `"wai-access-completeness"` | |
| `room` | `hex32` | the room |
| `from`, `to` | `int` | the session, from `from` up to, not including, `to`, in room-clock microseconds |
| `pt0` | `int` | the media time of the room's `t0`, in milliseconds (timed-text §2) |
| `tracks` | array | the access tracks it declares, each an object below |
| `signer_pubkey`, `sig` | | §2 |
| service | its members besides `service` |
|---|---|
| `"captions"` | `principal` (`hex32`), `lang` (`tag`), `latency_mean_max` and `latency_max` (each `int` or `null`, microseconds) |
| `"audio-description"` | `record` (`hex32`, a description record's id), `lang` (`tag`) |
| `"sign-language"` | `record` (`hex32`, a signing record's id), `lang` (`tag`), `review` (`hex32` or `null`, a review's id), `offset_max` (`int` or `null`, microseconds) |
The tracks a claim declares are the availability of access services an electronic programme
guide states (EN 301 549 V4.1.1 clauses 12.4 and 12.5), made checkable.
A cue of *P* is **in the session** when its first revision's `t` is at or after `from` and
before `to`. The
session's media interval is from `pt0` + ⌊`from` / 1000⌋ to `pt0` + ⌊`to` / 1000⌋.
### 7.2 Checking the claim
1. `room` is the verifier's: otherwise `wrong_room`.
2. `from` < `to`; at least one track; each track's `lang` reads; no principal is named by two
caption tracks, and no record by two tracks: otherwise `claim_malformed`.
Then each track in order, the first failure naming the refusal:
**Captions**, of principal *P*:
1. *P* has a cue in the session: otherwise `track_missing`.
2. *P*'s captions are resolved (§4.1): otherwise `track_incomplete`. A delivery gap is an
event not delivered, and a cue that does not resolve was not finished.
3. Every revision of every cue of *P* in the session has the track's `lang`: otherwise
`track_mismatch`.
4. When `latency_mean_max` or `latency_max` is not `null`: *P*'s sources record places every
cue of *P* in the session, otherwise `sync_unmeasured`; and over those cues the mean
latency (§4.4) is at most `latency_mean_max`, and each latency at most `latency_max`, where
each is given: otherwise `out_of_sync`.
**Audio description**, of record *D*:
1. *D* is a description record the verifier holds: otherwise `track_missing`.
2. Its `lang` is the track's, and each of its events lies in the session's media interval:
otherwise `track_mismatch`.
3. For the form `track`, every event is voiced: otherwise `track_incomplete`. (Each voice is
already in sync, §5.3.)
**Sign language**, of record *G*:
1. *G* is a signing record the verifier holds: otherwise `track_missing`.
2. Its `lang` is the track's, and its `source` is the principal of a caption track of this
claim: otherwise `track_mismatch`.
3. Every cue of `source` in the session has its final revision named by a segment of *G*:
otherwise `track_incomplete`.
4. When `review` is not `null`: it is the id of a review the verifier holds (otherwise
`review_missing`), which names *G* (otherwise `review_mismatch`) and whose verdict is not
`rejected` (otherwise `review_rejected`).
5. When `offset_max` is not `null`: every segment of *G* has an offset (§6.3), otherwise
`sync_unmeasured`; and each offset's absolute value is at most `offset_max`, otherwise
`out_of_sync`.
A claim that passes is **held**. Its **dialogue overlap** is, for each audio description
track, §5.4's figure against the resolved cues of the claim's caption tracks, their media
times as timed-text §4.5 gives them with the claim's `pt0`.
### 7.3 The tolerances
| track | measured from | tolerance | stated by |
|---|---|---|---|
| captions, live | each cue's first display and its source (§4.4) | a mean of at most 4.5 s across live programming: a recommendation, not a limit per programme | the regulator's 2024 statement, §5.17 and §5.25; its 2013 statement (§3.65 to §3.67) recorded the earlier guidance of at most 3 s |
| captions, shown by a player | a cue's time | within 100 ms of it, 20 ms as good practice | EN 301 549 V4.1.1 clause 7.1.2: the player's, timed-text §11 item 2 |
| audio description | each voice and its event (§5.3) | 35 ms early to 45 ms late | DAPT §5.5, from EBU R 37 |
| sign language | each segment and its source (§6.3) | none stated | no text read for this revision gives a figure; EN 301 549 V4.1.1 clause 4.2.4 NOTE 3 asks only that it be "well synchronized with the spoken content" |
A claim states the caption and sign tolerances it is held to (`latency_mean_max`,
`latency_max`, `offset_max`); the description tolerance is fixed by §5.3. The captions'
mean is computed over the session, while the 2024 statement's is over a provider's live
programming taken together; a claim that states 4 500 000 holds one session to it.
## 8. Verification
### 8.1 Inputs
A verifier is given the room id; the records of the room's event stream, in stream order
(live-events §6); and the records of this extension, as JSON texts, in the order given.
### 8.2 Steps
1. **Events.** Each event record is offered, in stream order, to a verifier of captions
([timed-text](timed-text.md) §4.3), which verifies it as live-events §6 does and the
captions in it as timed-text §4.3 does. A refusal is the outcome, at `events` and the
record's index, with that extension's reason. At the end of the stream the captions are
finished (timed-text §4.3), and the gaps, which are not refusals, are noted.
2. **Reading.** Each record is read (§2), its signature verified and its record id computed,
in the order given: a failure is the outcome, at `record` and the record's index.
3. **Rules.** Then each record is checked by its section's rules, kind by kind in this
order, and the records of one kind in the order given: sources (§4.2), references
(§4.3), descriptions (§5.2), signings (§6.2), reviews (§6.4), quality records (§4.7) and
claims (§7.2). So a record that another names has been checked before it. The first
failure is the outcome, at `record` and the failing record's index.
4. **Figures.** With no refusal, the outcome is **accept**, with: the figure (§4.7) of every
principal whose captions are whole, in the order of each one's first `caption` event in
the stream; for
each description record, its number of events, the number voiced and each event's offset
(§5.3); for each signing record, its number of segments and of revisions named, each
segment's offset and its video floor (§6.3); and for each claim, its dialogue overlap
(§7.2).
## 9. What the figures measure
| figure | what it measures | in the texts read |
|---|---|---|
| latency: per cue, its mean, its range (`min`, `max`) and two ranks | first display on the room clock, less the capture of the first word the cue carries | the 2013 statement's "average latency of the subtitling (the delay between speech and live subtitling), and the range of latencies" (§1.3(b)); the 2024 statement's mean (§5.17, §5.25); 47 CFR 79.1(k)(2)(vi), "lag between the spoken word supplied by the program origination point and when captions are received at the same program origination point", which the room clock's `t`, stamped as the caption leaves its author, measures; 79.1(j)(2)(ii) and (j)(3)(ii); EBU Tech 3370 §2.3.2's authoring delay, from the subtitler hearing the audio to the tool's output, and its §2.3.3 `ebuttm:authoringDelay`, which a producer may state and this figure measures |
| stability: revisions after first display, words changed, cues corrected | what the viewer saw that a later revision changed | the 2024 statement's research finding that subtitles corrected live on air distract (§5.23) |
| accuracy: words, errors, and their kinds | word errors against a reference a person wrote or reviewed | 47 CFR 79.1(k)(2)(iv): accuracy is the total words less the errors, over the total words, which is (*N* − *E*) / *N* here, and (k)(2)(v)'s kinds of error, of which this figure counts incorrect and missing words; 79.1(j)(2)(i); the 2013 statement's "number and type of errors" (§1.3(c)); the 2024 statement sets no minimum accuracy (§5.29) |
| dialogue overlap | description spoken over captioned dialogue | 47 CFR 79.3(a)(3); the 2024 statement §6.2 and §6.14 |
| completeness | every declared track present, every cue finished, every event voiced and every cue signed | Directive (EU) 2019/882 Annex I Section IV(b)(ii), "fully transmitted … synchronised"; 47 CFR 79.1(j)(2)(iii), captioning "from the beginning to the end of the program", which this claim checks over the cues the session holds, not over the speech it does not hold |
Not computed here:
- the severity of an error, which the error-classification model the 2013 statement's
measurement used (its footnote 21) grades minor, standard or serious: that grading needs a
person. The alignment gives the errors and the reference words a grader starts from;
- errors of capitalisation and of punctuation at a word's edge, which §3.1 removes, and which
47 CFR 79.1(j)(2)(i) counts;
- the speed of the captions, and their placement (79.1(j)(2)(iv));
- the quality of a description, of a voice or of a signing, which a person judges, and
whose judgement a review records.
## 10. Reasons
| reason | raised at |
|---|---|
| `record_malformed` | §2, §4.2, §4.3, §6.4 |
| `bad_key` | §2 |
| `bad_signature` | §2 |
| `wrong_room` | §2, §4.2, §4.3, §4.7, §6.2, §7.2 |
| `record_conflict` | §4.2, §4.3 |
| `source_unknown` | §4.2 |
| `source_take_unknown` | §4.2 |
| `text_not_in_subset` | §4.3, §5.2 |
| `reference_unknown` | §4.3 |
| `reference_incomplete` | §4.3 |
| `stream_incomplete` | §4.2, §4.3, §4.7, §6.2 |
| `figure_inputs_mismatch` | §4.7 |
| `figure_mismatch` | §4.7 |
| `description_malformed` | §5.2 |
| `description_out_of_sync` | §5.2, §5.3 |
| `parameter_malformed` | §5.5 |
| `signing_malformed` | §6.2 |
| `sign_revision_unknown` | §6.2 |
| `sign_revision_not_final` | §6.2 |
| `review_subject_unknown` | §6.4 |
| `claim_malformed` | §7.2 |
| `track_missing` | §7.2 |
| `track_incomplete` | §7.2 |
| `track_mismatch` | §7.2 |
| `sync_unmeasured` | §7.2 |
| `out_of_sync` | §7.2 |
| `review_missing` | §7.2 |
| `review_mismatch` | §7.2 |
| `review_rejected` | §7.2 |
An event record that [live-events](live-events.md) or [timed-text](timed-text.md) refuses
keeps that extension's reason.
## 11. Conformance
[`live-conformance/access/`](../live-conformance/access/) holds the corpus, written from fixed
seeds by `wai_access_services_vectors gen`: `vectors.json`, and each description record's DAPT
document as a file of its own, `NAME.dapt.ttml`. Each vector is a room's event stream and the
records of this extension, with the outcome §8 gives them and, when accepted, every figure. It
has:
- **accepted sessions**: a lesson with a human captioner, a machine captioner whose partials
are revised to final, a capture take placing the machine captions' words, a reference
transcript, a quality record for each track, a description voiced on a separate track, a
motion-captured signer bound to the human captions with an approved review, and a claim over
all of them held to the 2024 statement's mean; a session whose machine captions a person
reviews, with a description the sink voices by a pinned model, the video of a signer, and a
claim without latency bounds; and a session with no sources, whose latency class is `none`;
- **negative controls**, each a copy of an accepted session with one defect, and each refused
with its reason: a latency figure misstated, a broken revision chain, a sign track bound to a
revision a later one superseded and to a revision the stream does not hold, a description
spoken late and one spoken early, a claim over a track the verifier does not hold, and one
for every other reason of §10.
A conforming verifier gives every vector the outcome, the reason, the place and every figure
the corpus states, and writes every DAPT document byte for byte. `wai_access_services_vectors
verify` and the test `tests/access_services.rs` check that every file regenerates byte for
byte, that no other file is there, that `wai-rs` gives every vector its outcome, that every
reason of §10 refuses a vector, and that §10 lists exactly the reference's codes.
`live-conformance/access_services_verify.py` is the second implementation, written from this
text: it shares no code with `wai-rs`; it verifies events and captions with the timed-text
corpus's second implementation (`timed_text_verify.py`) and signatures with WAI's strict rule
(`claim-conformance/_ed25519.py`); it computes every figure, writes every DAPT document and
parses each with an XML parser; and it requires every mutated copy of the corpus to fail.
## 12. Security and privacy considerations
- Every figure is recomputed from signed records. A figure states nothing a verifier takes on
trust; a quality record's signer vouches only that it computed the figure, and a verifier
that recomputes it needs no trust in the signer.
- A source places words on the room clock as its signer states. On a take, the media's time is
the device's signed commitment and cannot be moved; where a word lies in the media is the
aligner's statement, attributable to it by its signature, and checkable by anyone holding the
master. A source on the basis `room` places the words by the signer's word alone, and the
latency's class says so (`stated`).
- A take's opening commitment is the device's, but a principal that is not a capturing device
can sign one too. This extension requires the take to be derived from a `record_start`
addressed to the commitment's author; that the author was permitted to capture, and for
whom, is what a take verifier and a room verifier check (capture-binding §9, live-rooms §9).
- `t` is its author's statement of the room clock (live-events §11). A captioner who stamps
its captions early lowers its latency, and the stamp is signed: the room's relays see when
each event arrived, and a later extension's per-hop receipts witness it.
- A reference transcript is its reviewer's. The accuracy's class names what the reference is;
a verifier's policy decides whose transcripts it accepts.
- Labels are their producers' statements, as timed-text §14 says of captions. A description
or a signing labelled human that a model produced misleads its audience, and the signature
makes that attributable.
- A signing track's binding names caption revisions by digest. It shows what the signing
translates; it does not show that the translation is correct, which a review states.
- A sources record ties a cue to a position in a recording, and a signing record ties a track
to a person who signs. Both persist with the session's records, as the events do.

6
wai/extensions/timed-text.md
View file

@ -48,8 +48,10 @@ disclosure label in every target (§10); and what a WAI player owes its viewers
Out of scope: a transport; speech recognition and translation, which produce the text; the
fragmented MP4 packaging of an IMSC1 rendition (HLS `draft-pantos-hls-rfc8216bis-22` §3.1.5
and ISO/IEC 14496-30 carry the document §6 writes); the paint-on style of 608, colours,
underline and flash; audio description; and a sink's own rendering of text, beyond §4.4
and §11.
underline and flash; audio description, which [access-services](access-services.md) §5
defines, with sign language and figures of a caption track's latency, stability and accuracy
computed from its revisions (§4 there); and a sink's own rendering of text, beyond §4.4 and
§11.
Relationship to the core spec: `wai.text.timed` is a **record** ([SPEC](../SPEC.md) §5).
Its unit is the signed caption revision, and the documents this extension writes are

2
wai/live-conformance/README.md
View file

@ -22,6 +22,7 @@ and the independent verifier of each, beside it.
| [`usage/`](usage/) | [ai-usage](../extensions/ai-usage.md) (`wai.ai.usage`) | `wai_ai_usage_vectors gen` | `ai_usage_verify.py` |
| [`contribution/`](contribution/) | [contribution](../extensions/contribution.md) (`wai.media.contribution`) | `wai_contribution_vectors gen` | `contribution_verify.py` |
| [`settlement/`](settlement/) | [settlement](../extensions/settlement.md) (`wai.delivery.settlement`) | `wai_settlement_vectors gen` | `settlement_verify.py` |
| [`access/`](access/) | [access-services](../extensions/access-services.md) (`wai.access.services`) | `wai_access_services_vectors gen` | `access_services_verify.py` |
**Provenance.** Generated here: each directory's corpus is written by its row's generator
(`gen`, in `wai/wai-rs`), from records authored in code and from fixed seeds, and each
@ -44,6 +45,7 @@ python3 capture_take_verify.py takes
python3 live_compose_verify.py compose
python3 live_record_verify.py recordings
python3 timed_text_verify.py captions
python3 access_services_verify.py access
python3 live_metering_verify.py metering
python3 energy_steering_verify.py steering
python3 live_moderation_verify.py moderation

46
wai/live-conformance/access/README.md Normal file
View file

@ -0,0 +1,46 @@
# live-conformance/access — access services
The corpus of [`extensions/access-services.md`](../../extensions/access-services.md) §11: room
sessions whose captions are `WEV1` events with their `WTX1` annotations, and the records of the
extension beside them: sources, reference transcripts, quality figures, audio descriptions,
signings, reviews and completeness claims, each a JSON text signed under `wai:access`.
**Access-services equivalence.** For each vector, a conforming verifier of the corpus's room,
offered the events of the vector's `stream` in the order given and then its `records` (§8.1),
reaches the outcome the vector states (§8.2): `refuse`, with `at` (`events` or `record`), the
`index` of the record and the `reason`; or `accept`, with the `figures`: each whole caption
track's figure (§4.7), each description's events, voiced events and offsets (§5.3), each
signing's segments, revisions named, offsets and video floor (§6.3), and each claim's dialogue
overlap (§7.2). Signed integers are written as strings. For each entry of `documents`, the
description record it names maps, with the voicing track's `uri` and `type`, onto the DAPT
document in `file` byte for byte (§5.5), or is refused with `reason`. A verifier that reaches
another outcome, place, reason, figure or byte does not conform.
**Provenance.** Generated here: `wai_access_services_vectors gen` writes `vectors.json` and
each DAPT document from events and records authored in code, from fixed seeds. Each key,
model, track and object digest is SHA-256 of a fixed label and an index, the room id
likewise, every time is a constant, and Ed25519 signing is deterministic (RFC 8032), so every
file is the same bytes on every machine. The captions, descriptions and transcripts are
written for the corpus, in English. The objects a signing names are digests only: no pose
stream, video or audio is in the corpus. The generator asserts, before it writes, that each
vector reaches the outcome it was written to have and that every reason of §10 refuses a
case. No recorded data.
## Layout
- `vectors.json`: `room`, `pt0_ms`, `principals` (the event authors `captioner`, `asr`,
`host` and `device`, and the record signers), `streams` (each a list of `WEV1` records in
lowercase hexadecimal: `lesson`, `reviewed`, and three defective copies of the lesson),
`vectors` (each with `name`, `note`, `stream`, `records` and its outcome) and `documents`.
- `NAME.dapt.ttml`: the DAPT document of vector `NAME`'s description.
## Checking
```
cargo run --no-default-features --features access_services --bin wai_access_services_vectors -- verify ../live-conformance/access
python3 -I access_services_verify.py access
```
The first runs from `wai-rs/`, the second from `live-conformance/`. The second is the
corpus's second implementation, written from the extension's text; it shares no code with
`wai-rs`.

13
wai/live-conformance/access/lesson.dapt.ttml Normal file
View file

@ -0,0 +1,13 @@
<?xml version="1.0" encoding="UTF-8"?>
<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" xmlns:tta="http://www.w3.org/ns/ttml#audio" xmlns:daptm="http://www.w3.org/ns/ttml/profile/dapt#metadata" xmlns:wai="tag:transaction.science,2026:wai/access" xml:lang="en" ttp:contentProfiles="http://www.w3.org/ns/ttml/profile/dapt1.0/content" daptm:langSrc="zxx" daptm:scriptRepresents="visual.nonText visual.text" daptm:scriptType="asRecorded" wai:script="human" wai:voice="machine" wai:voice-model="1c1172ff98d71af4286f2a67fa50ccf58dedcfd642a1974ebbcc1cbaff53848f">
<body>
<div xml:id="d1" begin="00:00:10.500" end="00:00:11.900" daptm:represents="visual.nonText">
<animate begin="00:00:00.000" end="00:00:00.200" tta:gain="1;0.4" fill="freeze"/>
<animate begin="00:00:01.200" end="00:00:01.400" tta:gain="0.4;1"/>
<p><span begin="00:00:00.220" end="00:00:01.190"><audio src="https://media.example/lesson/description.mp4" type="audio/mp4" clipBegin="00:00:10.720" clipEnd="00:00:11.690"/>A teacher stands beside a tide chart.</span></p>
</div>
<div xml:id="d2" begin="00:00:15.200" end="00:00:16.000" daptm:represents="visual.text">
<p><span begin="00:00:00.010" end="00:00:00.790"><audio src="https://media.example/lesson/description.mp4" type="audio/mp4" clipBegin="00:00:15.210" clipEnd="00:00:15.990"/>On the board: HIGH TIDE 06:10.</span></p>
</div>
</body>
</tt>

13
wai/live-conformance/access/reviewed.dapt.ttml Normal file
View file

@ -0,0 +1,13 @@
<?xml version="1.0" encoding="UTF-8"?>
<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" xmlns:tta="http://www.w3.org/ns/ttml#audio" xmlns:daptm="http://www.w3.org/ns/ttml/profile/dapt#metadata" xmlns:wai="tag:transaction.science,2026:wai/access" xml:lang="en" ttp:contentProfiles="http://www.w3.org/ns/ttml/profile/dapt1.0/content" daptm:langSrc="zxx" daptm:scriptRepresents="visual.nonText visual.text.title" daptm:scriptType="preRecording" wai:script="machine" wai:script-model="cd01c4b4a40ca82e51f554a0d17cba629c8e828c960c3f3f396b232419bcedb2" wai:voice="machine-reviewed" wai:voice-model="78735f2962641be6afd75736236889f0fb4eab577c9b29d4c6e0c1de32ee7879">
<body>
<div xml:id="r1" begin="00:00:11.000" end="00:00:12.200" daptm:represents="visual.nonText">
<animate begin="00:00:00.000" end="00:00:00.150" tta:gain="1;0.5" fill="freeze"/>
<animate begin="00:00:01.050" end="00:00:01.200" tta:gain="0.5;1"/>
<p><span tta:speak="normal">Divers swim over a reef.</span></p>
</div>
<div xml:id="r2" begin="00:00:14.000" end="00:00:15.000" daptm:represents="visual.text.title">
<p><span tta:speak="normal">Title:<br/>Reef Tour &amp; Co.</span></p>
</div>
</body>
</tt>

1116
wai/live-conformance/access/vectors.json Normal file
View file

File diff suppressed because it is too large Load diff

990
wai/live-conformance/access_services_verify.py Normal file
View file

@ -0,0 +1,990 @@
#!/usr/bin/env python3
"""Independent verifier of the access-services corpus (extensions/access-services.md).
Written from the extension's text. It shares no code with wai-rs, which generates the
corpus, and never runs it. It verifies every event and the captions in it with
timed_text_verify.py (this directory), the timed-text corpus's second implementation, which
verifies events with live_events_verify.py; checks every record's signature with
../claim-conformance/_ed25519.py, WAI's strict Ed25519 rule written from RFC 8032; and writes
everything else from the extension's sections:
- the records (§2): a strict JSON reading, each record's shape from the tables, its JCS
form, its signature and its record id;
- words and their alignment (§3), with the walk back of §3.2;
- caption quality (§4): the tracks and cues, sources placed through a take's opening
commitment, reference transcripts, latency, stability and accuracy, and the check of a
stated figure;
- audio description (§5): the rules, the synchronisation bounds, the overlap with dialogue,
and the DAPT document, written byte for byte and parsed with xml.etree;
- sign language (§6): the rules, the binding to final revisions, offsets and the video
floor, and reviews;
- the completeness claim (§7) and verification (§8).
For every vector it requires the outcome, place, reason and figures the corpus states; for
every document, its bytes or its refusal; that every reason of §10 (read from the extension)
refuses a case; and that every mutated copy of the corpus fails.
python3 -I access_services_verify.py <live-conformance/access>
Exit status 0 only if every case verifies. Python 3.8 or later; standard library only.
"""
import copy
import hashlib
import json
import os
import re
import sys
import xml.etree.ElementTree as ET
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
sys.path.insert(0, os.path.join(HERE, "..", "claim-conformance"))
import timed_text_verify as TT # noqa: E402
import live_events_verify as LEV # noqa: E402
import _ed25519 as ED # noqa: E402
EXTENSION = os.path.join(HERE, "..", "extensions", "access-services.md")
SIGN_DOMAIN = b"wai:access\x01"
MAX_SAFE = (1 << 53) - 1
MAX_MS = 359999999
NS_WAI = "tag:transaction.science,2026:wai/access"
NS_TT = "http://www.w3.org/ns/ttml"
NS_TTA = "http://www.w3.org/ns/ttml#audio"
NS_TTP = "http://www.w3.org/ns/ttml#parameter"
NS_DAPTM = "http://www.w3.org/ns/ttml/profile/dapt#metadata"
NS_XML = "http://www.w3.org/XML/1998/namespace"
DAPT_CONTENT = "http://www.w3.org/ns/ttml/profile/dapt1.0/content"
CAPTION, CONTROL, TAKE_COMMIT = 0x0A, 0x09, 0x0C
class Refused(Exception):
def __init__(self, reason):
super().__init__(reason)
self.reason = reason
def reasons_of_extension():
"""The codes of §10's table, in order."""
text = open(EXTENSION, encoding="utf-8").read()
at = text.index("\n## 10. Reasons")
end = text.find("\n## ", at + 1)
body = text[at:end if end >= 0 else len(text)]
return [l.split("`")[1] for l in body.split("\n") if l.startswith("| `")]
# ---------------------------------------------------------------- §2 records
class _Bad:
"""A JSON number this extension's records may not hold."""
BAD = _Bad()
def _no_dupes(pairs):
out = {}
for k, v in pairs:
if k in out:
raise Refused("record_malformed")
out[k] = v
return out
def _clean(v, depth):
if depth > 32:
raise Refused("record_malformed")
if isinstance(v, str):
if any(0xD800 <= ord(c) <= 0xDFFF for c in v):
raise Refused("record_malformed")
elif isinstance(v, list):
for x in v:
_clean(x, depth + 1)
elif isinstance(v, dict):
for k, x in v.items():
_clean(k, depth)
_clean(x, depth + 1)
def read_json(text):
"""jwp-receipts §3.1 step 1's reading: no BOM, no repeated member, no lone surrogate, at
most 32 levels. A number with a sign, a fraction or an exponent is kept as BAD."""
if text.startswith(""):
raise Refused("record_malformed")
try:
v = json.loads(text, object_pairs_hook=_no_dupes,
parse_int=lambda s: BAD if s.startswith("-") else int(s),
parse_float=lambda s: BAD,
parse_constant=lambda s: BAD)
except Refused:
raise
except ValueError:
raise Refused("record_malformed")
_clean(v, 1)
return v
HEX = re.compile(r"[0-9a-f]*")
SINT = re.compile(r"0|-?[1-9][0-9]*")
def is_hex(v, n):
return isinstance(v, str) and len(v) == 2 * n and HEX.fullmatch(v) is not None
def is_int(v):
return type(v) is int and 0 <= v <= MAX_SAFE
def is_sint(v):
return isinstance(v, str) and SINT.fullmatch(v) is not None and -(1 << 63) <= int(v) < (1 << 63)
# A type: "hex32", "hex64", "int", "sint", "str", ("one", [...]), ("obj", table),
# ("arr", type), ("null", type).
ORIGINS = ["human", "machine", "machine-reviewed"]
REPRESENTS = ["visual", "visual.nonText", "visual.text", "visual.text.title", "visual.text.credit",
"visual.text.location", "visual.dialogue"]
SIG = [("signer_pubkey", "hex32"), ("sig", "hex64")]
SOURCES = [("kind", ("one", ["wai-caption-sources"])), ("room", "hex32"), ("principal", "hex32"),
("sources", ("arr", ("obj", [("cue", "hex32"), ("basis", ("one", ["take", "room"])), ("take", ("null", "hex32")),
("start", "int"), ("end", "int")])))] + SIG
REFERENCE = [("kind", ("one", ["wai-caption-reference"])), ("room", "hex32"), ("principal", "hex32"),
("entries", ("arr", ("obj", [("cue", "hex32"), ("text", "str")])))] + SIG
COUNTS = ("obj", [("words", "int"), ("errors", "int"), ("substitutions", "int"), ("deletions", "int"), ("insertions", "int")])
QUALITY = [("kind", ("one", ["wai-caption-quality"])), ("room", "hex32"), ("principal", "hex32"), ("events", "hex32"),
("sources", ("null", "hex32")), ("reference", ("null", "hex32")), ("cues", "int"),
("latency", ("obj", [("class", ("one", ["none", "capture", "stated"])), ("n", "int")] +
[(k, ("null", "sint")) for k in ("min", "max", "mean", "p50", "p95")])),
("stability", ("obj", [("revisions", "int"), ("words_changed", "int"), ("cues_corrected", "int")])),
("accuracy", ("obj", [("class", ("one", ["transcript", "reviewed", "self"])), ("first", COUNTS), ("final", ("null", COUNTS))]))] + SIG
DESCRIPTION = [("kind", ("one", ["wai-description"])), ("lang", "str"),
("script", ("obj", [("origin", ("one", ORIGINS)), ("model", ("null", "hex32"))])),
("voicing", ("obj", [("form", ("one", ["track", "sink"])), ("origin", ("one", ORIGINS)),
("model", ("null", "hex32")), ("track", ("null", "hex32"))])),
("events", ("arr", ("obj", [("id", "str"), ("begin", "int"), ("end", "int"), ("represents", ("one", REPRESENTS)),
("text", "str"), ("duck", ("obj", [("gain", "int"), ("ramp", "int")])),
("voice", ("null", ("obj", [("begin", "int"), ("end", "int")])))])))] + SIG
SIGNING = [("kind", ("one", ["wai-signing"])), ("room", "hex32"), ("lang", "str"), ("source", "hex32"),
("form", ("one", ["pose", "video"])), ("track", "hex32"), ("origin", ("one", ORIGINS)), ("model", ("null", "hex32")),
("video", ("null", ("obj", [("width", "int"), ("height", "int"), ("rate_num", "int"), ("rate_den", "int")]))),
("segments", ("arr", ("obj", [("object", "hex32"), ("start", "int"), ("end", "int"), ("revisions", ("arr", "hex32"))])))] + SIG
REVIEW = [("kind", ("one", ["wai-signing-review"])), ("signing", "hex32"), ("scope", ("one", ["translation"])),
("verdict", ("one", ["approved", "approved-with-notes", "rejected"])), ("notes", ("null", "hex32"))] + SIG
TRACKS = {
"captions": [("service", ("one", ["captions"])), ("principal", "hex32"), ("lang", "str"),
("latency_mean_max", ("null", "int")), ("latency_max", ("null", "int"))],
"audio-description": [("service", ("one", ["audio-description"])), ("record", "hex32"), ("lang", "str")],
"sign-language": [("service", ("one", ["sign-language"])), ("record", "hex32"), ("lang", "str"),
("review", ("null", "hex32")), ("offset_max", ("null", "int"))],
}
CLAIM = [("kind", ("one", ["wai-access-completeness"])), ("room", "hex32"), ("from", "int"), ("to", "int"), ("pt0", "int"),
("tracks", "tracks")] + SIG
TABLES = {"wai-caption-sources": SOURCES, "wai-caption-reference": REFERENCE, "wai-caption-quality": QUALITY,
"wai-description": DESCRIPTION, "wai-signing": SIGNING, "wai-signing-review": REVIEW, "wai-access-completeness": CLAIM}
def has_type(v, t):
if v is BAD:
return False
if t == "hex32":
return is_hex(v, 32)
if t == "hex64":
return is_hex(v, 64)
if t == "int":
return is_int(v)
if t == "sint":
return is_sint(v)
if t == "str":
return isinstance(v, str)
if t == "tracks":
return isinstance(v, list) and all(isinstance(x, dict) and x.get("service") in TRACKS and fits(x, TRACKS[x["service"]]) for x in v)
tag, arg = t
if tag == "one":
return isinstance(v, str) and v in arg
if tag == "obj":
return fits(v, arg)
if tag == "arr":
return isinstance(v, list) and all(has_type(x, arg) for x in v)
if tag == "null":
return v is None or has_type(v, arg)
raise AssertionError(t)
def fits(v, table):
return isinstance(v, dict) and len(v) == len(table) and all(k in v and has_type(v[k], t) for k, t in table)
def jcs(v):
"""RFC 8785 over strings, unsigned integers, null, arrays and objects with ASCII member
names, which is what these records hold."""
return json.dumps(v, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def read_record(text):
"""§2: (kind, value, record id)."""
v = read_json(text)
if not isinstance(v, dict) or v.get("kind") not in TABLES or not fits(v, TABLES[v["kind"]]):
raise Refused("record_malformed")
body = {k: x for k, x in v.items() if k != "sig"}
bad = ED.verify(bytes.fromhex(v["signer_pubkey"]), SIGN_DOMAIN + jcs(body), bytes.fromhex(v["sig"]))
if bad is not None:
raise Refused(bad)
return v["kind"], v, hashlib.sha256(jcs(v)).digest()
# ---------------------------------------------------------------- §3 words and alignment
EDGE = "!\"'(),-.:;?[]¡«»¿–—‘’“”…"
UPPER = {c: c + 0x20 for c in range(0x41, 0x5B)}
def words(text):
out = []
for piece in re.split("[ \n]", text):
if piece == "":
continue
w = piece.translate(UPPER).strip(EDGE)
if w:
out.append(w)
return out
def align(h, r, prefix):
"""§3.2: (words, errors, substitutions, deletions, insertions)."""
m, n = len(h), len(r)
d = [[0] * (n + 1) for _ in range(m + 1)]
for j in range(n + 1):
d[0][j] = j
for i in range(m + 1):
d[i][0] = i
for i in range(1, m + 1):
for j in range(1, n + 1):
c = 0 if h[i - 1] == r[j - 1] else 1
d[i][j] = min(d[i - 1][j - 1] + c, d[i - 1][j] + 1, d[i][j - 1] + 1)
if prefix:
least = min(d[m])
end = max(j for j in range(n + 1) if d[m][j] == least)
else:
end = n
s = de = ins = 0
i, j = m, end
while i > 0 or j > 0:
if i > 0 and j > 0:
c = 0 if h[i - 1] == r[j - 1] else 1
if d[i][j] == d[i - 1][j - 1] + c:
s += c
i, j = i - 1, j - 1
continue
if i > 0 and d[i][j] == d[i - 1][j] + 1:
ins += 1
i -= 1
else:
de += 1
j -= 1
assert s + de + ins == d[m][end]
return [end, d[m][end], s, de, ins]
def counts_json(c):
return {"words": c[0], "errors": c[1], "substitutions": c[2], "deletions": c[3], "insertions": c[4]}
def add(a, b):
return [x + y for x, y in zip(a, b)]
# ---------------------------------------------------------------- §4 the caption tracks
class Stream:
def __init__(self, room, cap):
self.room, self.revs, self.accepted, self.gaps = room, cap.revs, cap.ev.accepted, cap.ev.gaps
def principals(self):
out = []
for r in self.revs:
if r["event"]["principal"] not in out:
out.append(r["event"]["principal"])
return out
def cues(self, p):
"""§4.1: each cue as the positions of its revisions, first to last."""
out = []
for i, r in enumerate(self.revs):
if r["event"]["principal"] != p or r["sup"] is not None or r["sup_digest"] is not None:
continue
chain = [i]
while self.revs[chain[-1]]["by"] is not None:
chain.append(self.revs[chain[-1]]["by"])
out.append(chain)
return out
def head(self, c):
return self.revs[c[0]]["event"]["digest"]
def last(self, c):
return self.revs[c[-1]]
def whole(self, p):
cues = self.cues(p)
return (len(cues) > 0 and not any(g[0] == p for g in self.gaps)
and all(self.last(c)["final"] and not self.last(c)["unknown"] for c in cues))
def resolved(self, p):
return self.whole(p) and all(self.last(c)["duration"] > 0 for c in self.cues(p))
def events_digest(self, p):
mine = sorted((r for r in self.revs if r["event"]["principal"] == p), key=lambda r: r["event"]["seq"])
return hashlib.sha256(b"".join(r["event"]["digest"] for r in mine)).digest()
def take_offset(self, take):
"""§4.2 rule 6: the offset of the take's opening commitment, when the take is derived."""
opening = None
for _, ev, p in self.accepted:
if ev["kind"] != TAKE_COMMIT:
continue
c = read_commit(p["bytes"])
if c is not None and c["take"] == take and c["n"] == 0 and c["phase"] == 1:
opening = (ev["principal"], c["offset"])
break
if opening is None:
return None
author, offset = opening
for _, ev, p in self.accepted:
if ev["kind"] == CONTROL and p["command"] == 1 and p["target"] == author:
if hashlib.sha256(b"wai:capture-take\x01" + self.room + author + ev["digest"]).digest() == take:
return offset
return None
def intervals(self, p, pt0):
out = []
for c in self.cues(p):
h, l = self.revs[c[0]], self.last(c)
out.append((pt0 + h["event"]["t"] // 1000, pt0 + l["event"]["t"] // 1000 + l["duration"]))
return out
def read_commit(b):
"""capture-binding §5.1 and §5.2: a take_commit payload, or None when it does not read."""
if len(b) < 126:
return None
take, n, phase = b[0:32], int.from_bytes(b[32:36], "little"), b[36]
first, size = int.from_bytes(b[69:77], "little"), int.from_bytes(b[77:85], "little")
root, offset, k = b[85:117], int.from_bytes(b[117:125], "little", signed=True), b[125]
if phase not in (1, 2, 3) or size < first or k > 64 or len(b) != 126 + 32 * k or (size == 0 and root != bytes(32)):
return None
return {"take": take, "n": n, "phase": phase, "offset": offset}
def floor_div(a, n):
return a // n
class Run:
def __init__(self, stream, records):
self.s, self.records = stream, records # records: [(kind, value, id)]
self.sources = {} # principal -> (id, {cue: (start, on_take)})
self.references = {} # principal -> (id, {cue: text})
def room_ok(self, v):
if bytes.fromhex(v["room"]) != self.s.room:
raise Refused("wrong_room")
def latencies(self, p, cues):
src = self.sources.get(p)
if src is None:
return []
out = []
for c in cues:
placed = src[1].get(self.s.head(c))
if placed is not None:
out.append((self.s.revs[c[0]]["event"]["t"] - placed[0], placed[1]))
return out
def figure(self, p):
s = self.s
cues = s.cues(p)
lat = sorted(x[0] for x in self.latencies(p, cues))
n = len(lat)
if n == 0:
latency = {"class": "none", "n": 0, "min": None, "max": None, "mean": None, "p50": None, "p95": None}
else:
def rank(q):
return lat[-(-q * n // 100) - 1]
on_take = all(x[1] for x in self.latencies(p, cues))
latency = {"class": "capture" if on_take else "stated", "n": n, "min": str(lat[0]), "max": str(lat[-1]),
"mean": str(floor_div(sum(lat), n)), "p50": str(rank(50)), "p95": str(rank(95))}
revisions = changed_total = corrected = 0
for c in cues:
revisions += len(c) - 1
changed = 0
for a, b in zip(c, c[1:]):
al = align(words(s.revs[a]["text"]), words(s.revs[b]["text"]), False)
changed += al[2] + al[4]
changed_total += changed
corrected += 1 if changed > 0 else 0
ref = self.references.get(p)
if ref is not None:
cls = "transcript"
elif all(s.last(c)["origin"] in ("human", "machine-reviewed") for c in cues):
cls = "reviewed"
else:
cls = "self"
first, final = [0] * 5, [0] * 5
for c in cues:
rt = ref[1][s.head(c)] if ref is not None else s.last(c)["text"]
rw = words(rt)
first = add(first, align(words(s.revs[c[0]]["text"]), rw, True))
if ref is not None:
final = add(final, align(words(s.last(c)["text"]), rw, False))
src = self.sources.get(p)
return {"room": s.room.hex(), "principal": p.hex(), "events": s.events_digest(p).hex(),
"sources": None if src is None else src[0].hex(), "reference": None if ref is None else ref[0].hex(),
"cues": len(cues), "latency": latency,
"stability": {"revisions": revisions, "words_changed": changed_total, "cues_corrected": corrected},
"accuracy": {"class": cls, "first": counts_json(first), "final": counts_json(final) if ref is not None else None}}
def heads(self, p):
return [self.s.head(c) for c in self.s.cues(p)]
def earlier(self, at, kind, p):
return any(k == kind and bytes.fromhex(v["principal"]) == p for k, v, _ in self.records[:at])
# §4.2
def sources_rec(self, at):
_, v, rid = self.records[at]
self.room_ok(v)
p = bytes.fromhex(v["principal"])
if self.earlier(at, "wai-caption-sources", p):
raise Refused("record_conflict")
seen = set()
for e in v["sources"]:
if (e["basis"] == "take") != (e["take"] is not None) or e["start"] > e["end"] or e["cue"] in seen:
raise Refused("record_malformed")
seen.add(e["cue"])
if not self.s.whole(p):
raise Refused("stream_incomplete")
heads = self.heads(p)
if any(bytes.fromhex(e["cue"]) not in heads for e in v["sources"]):
raise Refused("source_unknown")
starts = {}
for e in v["sources"]:
if e["basis"] == "take":
o = self.s.take_offset(bytes.fromhex(e["take"]))
if o is None or e["start"] + o < 0:
raise Refused("source_take_unknown")
starts[bytes.fromhex(e["cue"])] = (e["start"] + o, True)
else:
starts[bytes.fromhex(e["cue"])] = (e["start"], False)
self.sources.setdefault(p, (rid, starts))
# §4.3
def reference_rec(self, at):
_, v, rid = self.records[at]
self.room_ok(v)
p = bytes.fromhex(v["principal"])
if self.earlier(at, "wai-caption-reference", p):
raise Refused("record_conflict")
texts = {}
for e in v["entries"]:
if e["cue"] in texts:
raise Refused("record_malformed")
texts[e["cue"]] = e["text"]
if not all(TT.text_ok(t) for t in texts.values()):
raise Refused("text_not_in_subset")
if not self.s.whole(p):
raise Refused("stream_incomplete")
heads = self.heads(p)
if any(bytes.fromhex(k) not in heads for k in texts):
raise Refused("reference_unknown")
if any(h.hex() not in texts for h in heads):
raise Refused("reference_incomplete")
self.references.setdefault(p, (rid, {bytes.fromhex(k): t for k, t in texts.items()}))
# §4.7
def quality_rec(self, at):
_, v, _ = self.records[at]
self.room_ok(v)
p = bytes.fromhex(v["principal"])
if not self.s.whole(p):
raise Refused("stream_incomplete")
f = self.figure(p)
if any(v[k] != f[k] for k in ("events", "sources", "reference")):
raise Refused("figure_inputs_mismatch")
if any(v[k] != x for k, x in f.items()):
raise Refused("figure_mismatch")
# §6.2
def signing_rec(self, at):
_, v, _ = self.records[at]
self.room_ok(v)
machine = v["origin"] != "human"
ok = TT.lang_ok(v["lang"]) and machine == (v["model"] is not None) and (v["form"] == "video") == (v["video"] is not None)
if v["video"] is not None:
ok = ok and all(v["video"][k] > 0 for k in ("width", "height", "rate_num", "rate_den"))
segs = v["segments"]
ok = ok and len(segs) > 0
named = []
for i, sg in enumerate(segs):
ok = ok and sg["start"] < sg["end"] and len(sg["revisions"]) > 0 and (i == 0 or sg["start"] >= segs[i - 1]["end"])
named += sg["revisions"]
ok = ok and len(set(named)) == len(named)
if not ok:
raise Refused("signing_malformed")
src = bytes.fromhex(v["source"])
if not self.s.whole(src):
raise Refused("stream_incomplete")
mine = {r["event"]["digest"]: r for r in self.s.revs if r["event"]["principal"] == src}
if any(bytes.fromhex(d) not in mine for d in named):
raise Refused("sign_revision_unknown")
for d in named:
r = mine[bytes.fromhex(d)]
if r["by"] is not None or not r["final"] or r["unknown"]:
raise Refused("sign_revision_not_final")
def segment_offset(self, src, sg):
cues = [c for c in self.s.cues(src) if self.s.last(c)["event"]["digest"].hex() in sg["revisions"]]
if not cues:
return None
earliest = min(cues, key=lambda c: (self.s.revs[c[0]]["event"]["t"], c[0]))
placed = self.sources.get(src, (None, {}))[1].get(self.s.head(earliest))
return None if placed is None else sg["start"] - placed[0]
# §6.4
def review_rec(self, at):
_, v, _ = self.records[at]
if (v["verdict"] == "approved") != (v["notes"] is None):
raise Refused("record_malformed")
if not any(k == "wai-signing" and rid.hex() == v["signing"] for k, _, rid in self.records):
raise Refused("review_subject_unknown")
def find(self, kind, rid):
return next((v for k, v, i in self.records if k == kind and i.hex() == rid), None)
# §7.2
def claim_rec(self, at):
_, v, _ = self.records[at]
self.room_ok(v)
frm, to, pt0, tracks = v["from"], v["to"], v["pt0"], v["tracks"]
ok = frm < to and len(tracks) > 0
principals, recs = [], []
for t in tracks:
ok = ok and TT.lang_ok(t["lang"])
if t["service"] == "captions":
ok = ok and t["principal"] not in principals
principals.append(t["principal"])
else:
ok = ok and t["record"] not in recs
recs.append(t["record"])
if not ok:
raise Refused("claim_malformed")
def session(p):
return [c for c in self.s.cues(p) if frm <= self.s.revs[c[0]]["event"]["t"] < to]
m_from, m_to = pt0 + frm // 1000, pt0 + to // 1000
for t in tracks:
if t["service"] == "captions":
p = bytes.fromhex(t["principal"])
cues = session(p)
if not cues:
raise Refused("track_missing")
if not self.s.resolved(p):
raise Refused("track_incomplete")
if any(self.s.revs[r]["lang"] != t["lang"] for c in cues for r in c):
raise Refused("track_mismatch")
if t["latency_mean_max"] is not None or t["latency_max"] is not None:
lat = [x[0] for x in self.latencies(p, cues)]
if len(lat) != len(cues):
raise Refused("sync_unmeasured")
if (t["latency_mean_max"] is not None and floor_div(sum(lat), len(lat)) > t["latency_mean_max"]) or \
(t["latency_max"] is not None and any(x > t["latency_max"] for x in lat)):
raise Refused("out_of_sync")
elif t["service"] == "audio-description":
d = self.find("wai-description", t["record"])
if d is None:
raise Refused("track_missing")
if d["lang"] != t["lang"] or any(e["begin"] < m_from or e["end"] > m_to for e in d["events"]):
raise Refused("track_mismatch")
if d["voicing"]["form"] == "track" and any(e["voice"] is None for e in d["events"]):
raise Refused("track_incomplete")
else:
g = self.find("wai-signing", t["record"])
if g is None:
raise Refused("track_missing")
if g["lang"] != t["lang"] or g["source"] not in principals:
raise Refused("track_mismatch")
src = bytes.fromhex(g["source"])
named = {d for sg in g["segments"] for d in sg["revisions"]}
if any(self.s.last(c)["event"]["digest"].hex() not in named for c in session(src)):
raise Refused("track_incomplete")
if t["review"] is not None:
r = self.find("wai-signing-review", t["review"])
if r is None:
raise Refused("review_missing")
if r["signing"] != t["record"]:
raise Refused("review_mismatch")
if r["verdict"] == "rejected":
raise Refused("review_rejected")
if t["offset_max"] is not None:
offs = [self.segment_offset(src, sg) for sg in g["segments"]]
if any(o is None for o in offs):
raise Refused("sync_unmeasured")
if any(abs(o) > t["offset_max"] for o in offs):
raise Refused("out_of_sync")
def overlap(self, v, rid):
dialogue = []
for t in v["tracks"]:
if t["service"] == "captions":
dialogue += self.s.intervals(bytes.fromhex(t["principal"]), v["pt0"])
union = []
for a, b in sorted(dialogue):
if union and a <= union[-1][1]:
union[-1][1] = max(union[-1][1], b)
else:
union.append([a, b])
out = []
for i, t in enumerate(v["tracks"]):
if t["service"] != "audio-description":
continue
d = self.find("wai-description", t["record"])
ms = n = 0
for e in d["events"]:
if e["voice"] is not None:
a, b = e["voice"]["begin"], e["voice"]["end"]
else:
a, b = e["begin"] + e["duck"]["ramp"], e["end"] - e["duck"]["ramp"]
o = sum(max(0, min(b, y) - max(a, x)) for x, y in union)
ms += o
n += 1 if o > 0 else 0
out.append({"track": i, "ms": ms, "events": n})
return {"record": rid.hex(), "overlap": out}
# ---------------------------------------------------------------- §5 audio description
ID = re.compile(r"[A-Za-z_][A-Za-z0-9_.\-]{0,63}")
def description_rules(v):
"""§5.2, in order."""
m = "description_malformed"
if not TT.lang_ok(v["lang"]):
raise Refused(m)
sc, vo = v["script"], v["voicing"]
sink = vo["form"] == "sink"
if (sc["origin"] != "human") != (sc["model"] is not None) or (vo["origin"] != "human") != (vo["model"] is not None) \
or sink == (vo["track"] is not None) or (sink and vo["origin"] == "human"):
raise Refused(m)
evs = v["events"]
if not evs:
raise Refused(m)
ids = set()
for i, e in enumerate(evs):
if ID.fullmatch(e["id"]) is None or e["id"] in ids or not e["begin"] < e["end"] <= MAX_MS or \
(i > 0 and e["begin"] < evs[i - 1]["end"]):
raise Refused(m)
ids.add(e["id"])
if not all(TT.text_ok(e["text"]) for e in evs):
raise Refused("text_not_in_subset")
for e in evs:
g, r = e["duck"]["gain"], e["duck"]["ramp"]
if not (g <= 1000 and (g == 1000) == (r == 0) and 2 * r <= e["end"] - e["begin"]):
raise Refused(m)
vc = e["voice"]
if vc is not None and (sink or not e["begin"] <= vc["begin"] < vc["end"] <= e["end"]):
raise Refused(m)
for e in evs:
vc = e["voice"]
if vc is None:
continue
a, z = e["begin"] + e["duck"]["ramp"], e["end"] - e["duck"]["ramp"]
if not (a - 35 <= vc["begin"] <= a + 45 and vc["end"] <= z + 45):
raise Refused("description_out_of_sync")
def description_figures(v, rid):
offs = [None if e["voice"] is None else str(e["voice"]["begin"] - (e["begin"] + e["duck"]["ramp"])) for e in v["events"]]
return {"record": rid.hex(), "events": len(v["events"]), "voiced": sum(1 for e in v["events"] if e["voice"] is not None),
"offsets": offs}
PARAM = re.compile(r"[!#-%'-;=?-~]{1,2048}")
def hhmmss(ms):
return "%02d:%02d:%02d.%03d" % (ms // 3600000, ms // 60000 % 60, ms // 1000 % 60, ms % 1000)
def gain(g):
if g == 0:
return "0"
if g == 1000:
return "1"
return "0." + ("%03d" % g).rstrip("0")
def esc(s):
return s.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;").replace("\n", "<br/>")
def dapt(v, uri, typ):
"""§5.5's document."""
track = v["voicing"]["form"] == "track"
if track and not (PARAM.fullmatch(uri or "") and PARAM.fullmatch(typ or "")):
raise Refused("parameter_malformed")
reps = []
for e in v["events"]:
if e["represents"] not in reps:
reps.append(e["represents"])
def label(o, name):
s = ' wai:%s="%s"' % (name, o["origin"])
if o["model"] is not None:
s += ' wai:%s-model="%s"' % (name, o["model"])
return s
out = ['<?xml version="1.0" encoding="UTF-8"?>',
'<tt xmlns="%s" xmlns:ttp="%s" xmlns:tta="%s" xmlns:daptm="%s" xmlns:wai="%s" xml:lang="%s" ttp:contentProfiles="%s" '
'daptm:langSrc="zxx" daptm:scriptRepresents="%s" daptm:scriptType="%s"%s%s>'
% (NS_TT, NS_TTP, NS_TTA, NS_DAPTM, NS_WAI, v["lang"], DAPT_CONTENT, " ".join(reps),
"asRecorded" if track else "preRecording", label(v["script"], "script"), label(v["voicing"], "voice")),
"<body>"]
for e in v["events"]:
b, en, g, r = e["begin"], e["end"], e["duck"]["gain"], e["duck"]["ramp"]
out.append('<div xml:id="%s" begin="%s" end="%s" daptm:represents="%s">' % (e["id"], hhmmss(b), hhmmss(en), e["represents"]))
if g < 1000:
out.append('<animate begin="00:00:00.000" end="%s" tta:gain="1;%s" fill="freeze"/>' % (hhmmss(r), gain(g)))
out.append('<animate begin="%s" end="%s" tta:gain="%s;1"/>' % (hhmmss(en - b - r), hhmmss(en - b), gain(g)))
text = esc(e["text"])
if track and e["voice"] is not None:
vb, ve = e["voice"]["begin"], e["voice"]["end"]
content = '<span begin="%s" end="%s"><audio src="%s" type="%s" clipBegin="%s" clipEnd="%s"/>%s</span>' % (
hhmmss(vb - b), hhmmss(ve - b), uri, typ, hhmmss(vb), hhmmss(ve), text)
elif track:
content = text
else:
content = '<span tta:speak="normal">%s</span>' % text
out.append("<p>%s</p>" % content)
out.append("</div>")
out += ["</body>", "</tt>"]
return "\n".join(out) + "\n"
def dapt_structure(doc, v):
"""Parse the document and check what DAPT §4 and §5 ask of it."""
root = ET.fromstring(doc.encode("utf-8"))
q = lambda ns, n: "{%s}%s" % (ns, n) # noqa: E731
fails = []
if root.tag != q(NS_TT, "tt"):
fails.append("the root is not tt")
if root.get(q(NS_TTP, "contentProfiles")) != DAPT_CONTENT or root.get(q(NS_TTP, "profile")) is not None:
fails.append("the profile is not signalled as DAPT §5.6 asks")
if root.get(q(NS_DAPTM, "scriptType")) not in ("preRecording", "asRecorded") or not root.get(q(NS_DAPTM, "scriptRepresents")):
fails.append("no script type or script represents")
if root.get(q(NS_XML, "lang")) != v["lang"]:
fails.append("xml:lang is not the record's")
reps = root.get(q(NS_DAPTM, "scriptRepresents"), "").split(" ")
divs = root.findall(q(NS_TT, "body") + "/" + q(NS_TT, "div"))
if len(divs) != len(v["events"]):
fails.append("not one Script Event per description")
for div, e in zip(divs, v["events"]):
if div.get(q(NS_XML, "id")) != e["id"] or div.get(q(NS_DAPTM, "represents")) not in reps:
fails.append("Script Event %s: its id or represents" % e["id"])
anim = div.findall(q(NS_TT, "animate"))
if len(anim) != (2 if e["duck"]["gain"] < 1000 else 0):
fails.append("Script Event %s: its mixing instructions" % e["id"])
ps = div.findall(q(NS_TT, "p"))
if len(ps) != 1:
fails.append("Script Event %s: not one Text" % e["id"])
continue
if "".join(ps[0].itertext()) != e["text"].replace("\n", ""):
fails.append("Script Event %s: its text" % e["id"])
audio = ps[0].findall(".//" + q(NS_TT, "audio"))
spoken = [s for s in ps[0].iter(q(NS_TT, "span")) if s.get(q(NS_TTA, "speak")) is not None]
want_audio = v["voicing"]["form"] == "track" and e["voice"] is not None
if (len(audio) == 1) != want_audio or (len(spoken) == 1) != (v["voicing"]["form"] == "sink"):
fails.append("Script Event %s: its audio" % e["id"])
return fails
# ---------------------------------------------------------------- §8 verification
KIND_ORDER = ["wai-caption-sources", "wai-caption-reference", "wai-description", "wai-signing", "wai-signing-review",
"wai-caption-quality", "wai-access-completeness"]
def verify(room, events, texts):
out, cap = TT.verify_events(room, events)
if out["outcome"] == "refuse":
return {"outcome": "refuse", "at": "events", "index": out["index"], "reason": out["reason"]}
stream = Stream(room, cap)
records = []
for i, t in enumerate(texts):
try:
records.append(read_record(t))
except Refused as e:
return {"outcome": "refuse", "at": "record", "index": i, "reason": e.reason}
run = Run(stream, records)
checks = {"wai-caption-sources": run.sources_rec, "wai-caption-reference": run.reference_rec,
"wai-description": lambda i: description_rules(records[i][1]), "wai-signing": run.signing_rec,
"wai-signing-review": run.review_rec, "wai-caption-quality": run.quality_rec,
"wai-access-completeness": run.claim_rec}
for kind in KIND_ORDER:
for i, (k, _, _) in enumerate(records):
if k != kind:
continue
try:
checks[kind](i)
except Refused as e:
return {"outcome": "refuse", "at": "record", "index": i, "reason": e.reason}
signings = []
for k, v, rid in records:
if k != "wai-signing":
continue
src = bytes.fromhex(v["source"])
offs = [run.segment_offset(src, sg) for sg in v["segments"]]
floor = None
if v["video"] is not None:
w, h = v["video"]["width"], v["video"]["height"]
floor = "meets" if min(w, h) >= 240 and max(w, h) >= 320 and v["video"]["rate_num"] >= 20 * v["video"]["rate_den"] else "below"
signings.append({"record": rid.hex(), "segments": len(v["segments"]), "revisions": sum(len(s["revisions"]) for s in v["segments"]),
"offsets": [None if o is None else str(o) for o in offs], "video_floor": floor})
figures = {
"captions": [run.figure(p) for p in stream.principals() if stream.whole(p)],
"descriptions": [description_figures(v, rid) for k, v, rid in records if k == "wai-description"],
"signings": signings,
"claims": [run.overlap(v, rid) for k, v, rid in records if k == "wai-access-completeness"],
}
return {"outcome": "accept", "figures": figures}
# ---------------------------------------------------------------- the corpus
def check(doc, files, quiet=False, only=None):
"""Every vector and document, or only those `only` names (a mutation's), with the corpus's
coverage of §10 checked when all are."""
fails, seen = [], set()
room = bytes.fromhex(doc["room"])
streams = {n: [bytes.fromhex(h) for h in evs] for n, evs in doc["streams"].items()}
texts_of = {}
for v in doc["vectors"]:
name = v["name"]
texts_of[name] = v["records"]
if only is not None and name not in only:
continue
before = len(fails)
try:
got = verify(room, streams[v["stream"]], v["records"])
except Exception as e: # a verifier that cannot read a case fails it
fails.append("%s: does not verify: %s: %s" % (name, type(e).__name__, e))
continue
for k in ("outcome", "at", "index", "reason", "figures"):
if got.get(k) != v.get(k):
fails.append("%s: %s is %s, the corpus states %s" % (name, k, json.dumps(got.get(k))[:300], json.dumps(v.get(k))[:300]))
if got.get("reason"):
seen.add(got["reason"])
if not quiet:
print("%s %s: %s" % ("PASS" if len(fails) == before else "FAIL", name, got["outcome"] + (" " + got["reason"] if got.get("reason") else "")))
for d in doc["documents"]:
if only is not None and d.get("file", d["vector"]) not in only:
continue
before = len(fails)
try:
_, v, _ = read_record(texts_of[d["vector"]][d["record"]])
doc_text, why = dapt(v, d["uri"], d["type"]), None
except Refused as e:
doc_text, why = None, e.reason
if why is not None:
seen.add(why)
if d.get("reason") != why:
fails.append("document of %s: refused %s, the corpus states %s" % (d["vector"], why, d.get("reason")))
elif doc_text is not None:
if files.get(d["file"]) != doc_text:
fails.append("document of %s: %s differs from the text's" % (d["vector"], d["file"]))
else:
fails += ["document %s: %s" % (d["file"], f) for f in dapt_structure(doc_text, v)]
if not quiet:
print("%s document of %s: %s" % ("PASS" if len(fails) == before else "FAIL", d["vector"], d.get("file") or why))
missing = [r for r in reasons_of_extension() if r not in seen]
if missing and only is None:
fails.append("nothing is refused %s" % ", ".join(missing))
return fails
def mutations(doc, files):
"""Copies of the corpus, each with one defect a conforming verifier must catch."""
out = []
def mutate(label, f, only):
d, fs = copy.deepcopy(doc), dict(files)
f(d, fs)
out.append((label, d, fs, only))
def flip_sig(text):
v = json.loads(text)
v["sig"] = v["sig"][:-1] + ("1" if v["sig"][-1] == "0" else "0")
return json.dumps(v, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
for i, v in enumerate(doc["vectors"]):
if v["outcome"] == "accept":
def stated(d, fs, i=i):
d["vectors"][i]["figures"]["captions"][0]["latency"]["n"] += 1
mutate("%s: a stated figure changed" % v["name"], stated, {v["name"]})
for k in range(len(v["records"])):
def sig(d, fs, i=i, k=k):
d["vectors"][i]["records"][k] = flip_sig(d["vectors"][i]["records"][k])
mutate("%s: record %d's signature flipped" % (v["name"], k), sig, {v["name"]})
for k in range(len(doc["streams"][v["stream"]])):
def ev(d, fs, i=i, k=k):
name = "mutated"
evs = list(d["streams"][d["vectors"][i]["stream"]])
b = bytearray(bytes.fromhex(evs[k]))
b[-1] ^= 0x01
evs[k] = b.hex()
d["streams"][name] = evs
d["vectors"][i]["stream"] = name
mutate("%s: event %d altered" % (v["name"], k), ev, {v["name"]})
else:
def place(d, fs, i=i):
d["vectors"][i]["index"] += 1
mutate("%s: the place of its refusal moved" % v["name"], place, {v["name"]})
def reason(d, fs, i=i):
d["vectors"][i]["reason"] = "record_malformed" if d["vectors"][i]["reason"] != "record_malformed" else "bad_key"
mutate("%s: its reason changed" % v["name"], reason, {v["name"]})
for d0 in doc["documents"]:
if "file" in d0:
def doc_byte(d, fs, f=d0["file"]):
fs[f] = fs[f].replace("<p>", "<p> ", 1)
mutate("%s: a byte added" % d0["file"], doc_byte, {d0["file"]})
return out
def main(corpus_dir):
files = {}
for n in sorted(os.listdir(corpus_dir)):
if n != "README.md":
with open(os.path.join(corpus_dir, n), encoding="utf-8") as f:
files[n] = f.read()
doc = json.loads(files["vectors.json"])
fails = check(doc, files)
for f in fails:
print("FAIL " + f)
caught = 0
muts = mutations(doc, files)
for label, d, fs, only in muts:
if check(d, fs, quiet=True, only=only):
caught += 1
else:
fails.append("a mutated copy verifies: " + label)
print("FAIL a mutated copy verifies: " + label)
print("%s every mutated copy of the corpus fails (%d of %d)" % ("PASS" if caught == len(muts) else "FAIL", caught, len(muts)))
print("%s %s" % ("PASS" if not fails else "FAIL", "the corpus verifies" if not fails else "%d failures" % len(fails)))
return 0 if not fails else 1
if __name__ == "__main__":
if len(sys.argv) != 2:
sys.exit("usage: access_services_verify.py <live-conformance/access>")
sys.exit(main(sys.argv[1]))

70
wai/registry/capabilities.json
View file

@ -1,6 +1,20 @@
{
"schema": "https://git.transaction.science/Transaction-Science/open-standards/src/branch/main/wai/registry/capabilities.schema.json",
"capabilities": [
{
"capability": "wai.access.services",
"kind": "record",
"status": "registered",
"media": null,
"role": null,
"tier": null,
"licence": null,
"replicate": null,
"sets": [],
"spec_section": "§5 Interactive, temporal & computed content",
"edition": null,
"registered": "2026-10-21"
},
{
"capability": "wai.ai.usage",
"kind": "record",
@ -2333,6 +2347,13 @@
"section": "§10.5",
"status": "defined"
},
{
"name": "tag:transaction.science,2026:wai/access",
"kind": "scheme",
"document": "extensions/access-services.md",
"section": "§5.5",
"status": "defined"
},
{
"name": "tag:transaction.science,2026:wai/class",
"kind": "scheme",
@ -2396,6 +2417,13 @@
"section": "§3",
"status": "defined"
},
{
"name": "wai-access-completeness",
"kind": "document-kind",
"document": "extensions/access-services.md",
"section": "§7.1",
"status": "defined"
},
{
"name": "wai-authorship-map",
"kind": "document-kind",
@ -2410,6 +2438,27 @@
"section": "§10.2",
"status": "defined"
},
{
"name": "wai-caption-quality",
"kind": "document-kind",
"document": "extensions/access-services.md",
"section": "§4.7",
"status": "defined"
},
{
"name": "wai-caption-reference",
"kind": "document-kind",
"document": "extensions/access-services.md",
"section": "§4.3",
"status": "defined"
},
{
"name": "wai-caption-sources",
"kind": "document-kind",
"document": "extensions/access-services.md",
"section": "§4.2",
"status": "defined"
},
{
"name": "wai-contribution",
"kind": "document-kind",
@ -2424,6 +2473,13 @@
"section": "§5.1",
"status": "defined"
},
{
"name": "wai-description",
"kind": "document-kind",
"document": "extensions/access-services.md",
"section": "§5.1",
"status": "defined"
},
{
"name": "wai-detection",
"kind": "document-kind",
@ -2515,6 +2571,20 @@
"section": "§3.1",
"status": "defined"
},
{
"name": "wai-signing",
"kind": "document-kind",
"document": "extensions/access-services.md",
"section": "§6.1",
"status": "defined"
},
{
"name": "wai-signing-review",
"kind": "document-kind",
"document": "extensions/access-services.md",
"section": "§6.4",
"status": "defined"
},
{
"name": "wai-step",
"kind": "document-kind",

7
wai/vocab/v2.ttl
View file

@ -103,6 +103,13 @@ wai:pinTier a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:rang
# ─── Registered strings ──────────────────────────────────────────────
wai:wai.access.services a wai:Capability ;
rdfs:label "wai.access.services" ;
wai:kind wai:kind-record ;
wai:status wai:status-registered ;
wai:specSection "§5 Interactive, temporal & computed content" ;
wai:registeredOn "2026-10-21"^^xsd:date .
wai:wai.ai.usage a wai:Capability ;
rdfs:label "wai.ai.usage" ;
wai:kind wai:kind-record ;

18
wai/wai-rs/Cargo.toml
View file

@ -331,6 +331,14 @@ settlement = ["live_metering", "jwp_hash"]
# machine-generation disclosure carried through each. No new dependency: the XML is
# written by hand.
timed_text = ["live_events"]
# Access services (extensions/access-services): `wai.access.services` — a caption track's
# latency, stability and accuracy computed from its signed revisions (sources placed on the
# room clock through a take's opening commitment, a reference transcript, word alignment in
# integer arithmetic) and the check of a stated figure; audio description (script, ducking,
# voicing, synchronisation to DAPT §5.5's bounds) and its DAPT document, written by hand;
# sign language as a pose or video track bound to final caption revisions, with a review; and
# the completeness claim of a room session. SHA-256, Ed25519 and JCS only; no new dependency.
access_services = ["timed_text", "capture_take"]
# Object-based spatial audio (extensions/spatial-audio): the deterministic
# Fx mixdown engine. Reuses the world `Fx` floor + blake3; no codec/ML deps
# (sources arrive as decoded mono PCM — decoding the wai.audio.* assets is
@ -831,6 +839,11 @@ name = "wai_timed_text_vectors"
path = "src/bin/wai_timed_text_vectors.rs"
required-features = ["timed_text"]
[[bin]]
name = "wai_access_services_vectors"
path = "src/bin/wai_access_services_vectors.rs"
required-features = ["access_services"]
[[bin]]
name = "wai_world_meter"
path = "src/bin/wai_world_meter.rs"
@ -1561,6 +1574,11 @@ name = "timed_text"
path = "tests/timed_text.rs"
required-features = ["timed_text"]
[[test]]
name = "access_services"
path = "tests/access_services.rs"
required-features = ["access_services"]
[[test]]
name = "world_replay"
path = "tests/world_replay.rs"

1483
wai/wai-rs/src/access_services.rs Normal file
View file

File diff suppressed because it is too large Load diff

718
wai/wai-rs/src/access_services/corpus.rs Normal file
View file

@ -0,0 +1,718 @@
//! The conformance corpus of extensions/access-services.md §11: `live-conformance/access/`.
//!
//! [`generate`] writes it from fixed seeds: every key, model, track and object digest is
//! SHA-256 of a fixed label and an index, the room id likewise, every time is a constant, and
//! Ed25519 signing is deterministic (RFC 8032), so every file is the same bytes on every
//! machine. Each vector is written with the outcome it is meant to have, and the generator
//! asserts, before it writes them, that this module gives it that outcome and that every
//! reason of §10 refuses a vector.
//!
//! [`verify`] checks a corpus directory's files against this module.
//! `live-conformance/access_services_verify.py` checks the same files from the extension's
//! text, sharing no code with this module.
use super::*;
use crate::capture_take::{phase, Commit};
use crate::live_events::{kind, Caption, Event};
use crate::timed_text::{origin, Anchor, Annotation};
use std::collections::BTreeMap;
/// The media time of the room's `t0`, in milliseconds.
pub const PT0_MS: u64 = 10_000;
/// The offset of the lesson's take: a device-clock instant *d* is the room-clock *d* + this.
pub const TAKE_OFFSET: i64 = -499_000_000;
const CAPTIONER: u8 = 1;
const ASR: u8 = 2;
const HOST: u8 = 3;
const DEVICE: u8 = 4;
const ALIGNER: u8 = 5;
const REVIEWER: u8 = 6;
const DESCRIBER: u8 = 7;
const INTERPRETER: u8 = 8;
const VENDOR: u8 = 9;
const AUTHORITY: u8 = 10;
const NAMES: [(&str, u8); 10] = [
("captioner", CAPTIONER),
("asr", ASR),
("host", HOST),
("device", DEVICE),
("aligner", ALIGNER),
("reviewer", REVIEWER),
("describer", DESCRIBER),
("interpreter", INTERPRETER),
("vendor", VENDOR),
("authority", AUTHORITY),
];
const VOICING_URI: &str = "https://media.example/lesson/description.mp4";
const VOICING_TYPE: &str = "audio/mp4";
fn label(l: &[u8], i: &[u8]) -> [u8; 32] {
sha(&[b"wai:access/corpus/", l, b"\x00", i])
}
fn key(i: u8) -> SigningKey {
SigningKey::from_bytes(&label(b"key", &[i]))
}
fn pk(i: u8) -> [u8; 32] {
key(i).verifying_key().to_bytes()
}
/// The room every vector's events belong to.
pub fn room() -> [u8; 32] {
label(b"room", &[1])
}
fn model(i: u8) -> [u8; 32] {
label(b"model", &[i])
}
fn track(i: u8) -> [u8; 32] {
label(b"track", &[i])
}
fn object(a: u8, b: u8) -> [u8; 32] {
label(b"object", &[a, b])
}
/// A digest that names nothing the corpus holds.
fn nothing(i: u8) -> [u8; 32] {
label(b"nothing", &[i])
}
fn h(b: &[u8; 32]) -> Value {
Value::String(hex(b))
}
/// A principal emitting its chain.
struct Author {
key: SigningKey,
seq: u64,
prev: [u8; 32],
}
impl Author {
fn new(i: u8) -> Self {
Author { key: key(i), seq: 0, prev: [0; 32] }
}
fn raw(&mut self, out: &mut Vec<Vec<u8>>, t: u64, k: u8, payload: Vec<u8>) -> [u8; 32] {
let ev = Event::sign(&self.key, room(), self.seq, self.prev, t, None, k, payload);
self.seq += 1;
self.prev = ev.digest();
out.push(ev.to_bytes());
self.prev
}
#[allow(clippy::too_many_arguments)]
fn caption(&mut self, out: &mut Vec<Vec<u8>>, t: u64, d: u32, o: u8, m: Option<[u8; 32]>, text: &str) -> [u8; 32] {
let c = Caption { duration_ms: d, origin: o, model: m, lang: "en".into(), text: text.into() };
self.raw(out, t, kind::CAPTION, Payload::Caption(c).encode())
}
fn annotate(&mut self, out: &mut Vec<Vec<u8>>, t: u64, supersedes: [u8; 32], final_: bool, stable: &str) -> [u8; 32] {
let a = Annotation { supersedes, final_, stable_cp: stable.chars().count() as u16, anchor: Anchor::BottomCenter, italic: vec![] };
self.raw(out, t, kind::META, a.to_bytes())
}
}
/// The digests of a stream's events that records name.
type Digests = BTreeMap<&'static str, [u8; 32]>;
/// A defect of the lesson's stream.
#[derive(Clone, Copy, PartialEq, Eq)]
enum StreamDefect {
None,
/// The captioner's second event names a `prev` that is not its first's.
ChainBroken,
/// An annotation of the machine captions supersedes a revision no one wrote.
SupersedeUnknown,
/// The machine captions' second cue's partial and its annotation are not delivered.
Gap,
}
/// The lesson: a human captioner, machine captions revised to final, and a take whose
/// opening commitment places the machine captions' words.
fn lesson_stream(defect: StreamDefect) -> (Vec<Vec<u8>>, Digests) {
let s = 1_000_000u64;
let mut out = Vec::new();
let mut d = Digests::new();
let (mut cap, mut asr, mut host, mut dev) = (Author::new(CAPTIONER), Author::new(ASR), Author::new(HOST), Author::new(DEVICE));
let m1 = Some(model(1));
let start = host.raw(&mut out, s, kind::CONTROL, Payload::Control { target: pk(DEVICE), command: Command::RecordStart { count_in_ms: 0 } }.encode());
let take = take_id(&room(), &pk(DEVICE), &start);
d.insert("take", take);
let commit =
Commit { take, n: 0, phase: phase::OPEN, prev: [0; 32], first: 0, size: 0, root: [0; 32], offset: TAKE_OFFSET, proof: vec![] };
dev.raw(&mut out, s, kind::TAKE_COMMIT, Payload::TakeCommit(commit.encode()).encode());
d.insert("cap1", cap.caption(&mut out, 2 * s, 2500, origin::HUMAN, None, "Good morning, everyone."));
let a0 = asr.caption(&mut out, 3_200_000, 0, origin::MACHINE, m1, "the moon pulls");
d.insert("asrA0", a0);
asr.annotate(&mut out, 3_200_000, [0; 32], false, "the moon ");
let a1 = asr.caption(&mut out, 3_900_000, 0, origin::MACHINE, m1, "the moon pulls the see");
d.insert("asrA1", a1);
let sup = if defect == StreamDefect::SupersedeUnknown { nothing(6) } else { a0 };
asr.annotate(&mut out, 3_900_000, sup, false, "the moon pulls ");
let a2 = asr.caption(&mut out, 4_600_000, 2000, origin::MACHINE, m1, "the moon pulls the sea.");
d.insert("asrA2", a2);
asr.annotate(&mut out, 4_600_000, a1, true, "the moon pulls the sea.");
if defect == StreamDefect::ChainBroken {
cap.prev = nothing(9);
}
d.insert("cap2", cap.caption(&mut out, 5 * s, 3000, origin::HUMAN, None, "Today we look at tides."));
let before_gap = out.len();
let b0 = asr.caption(&mut out, 6_500_000, 0, origin::MACHINE, m1, "high tied");
d.insert("asrB0", b0);
asr.annotate(&mut out, 6_500_000, [0; 32], false, "high ");
let b1 = asr.caption(&mut out, 7_300_000, 2500, origin::MACHINE, m1, "high tide comes twice day.");
d.insert("asrB1", b1);
asr.annotate(&mut out, 7_300_000, b0, true, "high tide comes twice day.");
d.insert("cap3", cap.caption(&mut out, 9 * s, 2000, origin::HUMAN, None, "[Waves crash]"));
if defect == StreamDefect::Gap {
out.drain(before_gap..before_gap + 2);
}
(out, d)
}
/// A session whose machine captions a person reviews before they are final.
fn reviewed_stream() -> (Vec<Vec<u8>>, Digests) {
let mut out = Vec::new();
let mut d = Digests::new();
let mut asr = Author::new(ASR);
let m1 = Some(model(1));
let a0 = asr.caption(&mut out, 1_500_000, 0, origin::MACHINE, m1, "welcome to the");
d.insert("A0", a0);
asr.annotate(&mut out, 1_500_000, [0; 32], false, "welcome ");
d.insert("A1", asr.caption(&mut out, 2_400_000, 3000, origin::REVIEWED, m1, "welcome to the reef tour."));
asr.annotate(&mut out, 2_400_000, a0, true, "welcome to the reef tour.");
d.insert("B0", asr.caption(&mut out, 6_000_000, 2500, origin::REVIEWED, m1, "Corals are animals."));
(out, d)
}
fn body(v: Value) -> Map<String, Value> {
v.as_object().expect("a record body is an object").clone()
}
fn signed(signer: u8, v: Value) -> Value {
sign(&key(signer), body(v))
}
/// A defect of a lesson's records, each the control of one rule.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum Defect {
None,
LatencyMisstated,
DescriptionLate,
DescriptionEarly,
DescriptionRunsOn,
SignUnknownRevision,
ClaimMissingTrack,
DuplicateMember,
NegativeNumber,
FlippedSignature,
WeakKey,
WrongRoom,
SecondSources,
SourceNotAHead,
SourceUnknownTake,
ReferenceSpaces,
ReferenceUnknownCue,
ReferenceMissingCue,
QualityWithoutReference,
DescriptionOverlap,
SigningEmptySegment,
ReviewUnknownSigning,
ClaimEmptySession,
DescriptionUnvoiced,
ClaimWrongLanguage,
LatencyAboveBound,
ReviewNotGiven,
ReviewOfAnother,
ReviewRejected,
CaptionsOfNoOne,
SignOffsetAboveBound,
/// Only the captioner's quality record and a claim without latency bounds.
Unplaced,
}
/// The lesson's records, with one defect: (records, the index of the description record).
fn lesson_records(events: &[Vec<u8>], d: &Digests, defect: Defect) -> (Vec<String>, usize) {
let r = hex(&room());
let mut records: Vec<Value> = Vec::new();
let mut texts: Vec<Option<String>> = Vec::new();
let push = |records: &mut Vec<Value>, texts: &mut Vec<Option<String>>, v: Value| {
records.push(v);
texts.push(None);
};
if defect == Defect::Unplaced {
let q = quality(events, &[], CAPTIONER);
let claim = signed(
AUTHORITY,
json!({"kind": kinds::CLAIM, "room": r, "from": 0, "to": 12_000_000, "pt0": PT0_MS, "tracks": [
{"service": "captions", "principal": h(&pk(CAPTIONER)), "lang": "en", "latency_mean_max": null, "latency_max": null}]}),
);
return (vec![text(&q), text(&claim)], usize::MAX);
}
// 0: the captioner's sources, on the room clock.
let src_room = if defect == Defect::WrongRoom { hex(&nothing(1)) } else { r.clone() };
let mut cap_sources = json!({"kind": kinds::SOURCES, "room": src_room, "principal": h(&pk(CAPTIONER)), "sources": [
{"cue": h(&d["cap1"]), "basis": "room", "take": null, "start": 1_400_000, "end": 1_900_000},
{"cue": h(&d["cap2"]), "basis": "room", "take": null, "start": 3_900_000, "end": 4_800_000},
{"cue": h(&d["cap3"]), "basis": "room", "take": null, "start": 8_500_000, "end": 8_900_000}]});
if defect == Defect::NegativeNumber {
cap_sources["sources"][0]["start"] = json!(0);
}
push(&mut records, &mut texts, signed(ALIGNER, cap_sources));
if defect == Defect::NegativeNumber {
let t = text(&records[0]).replacen("\"start\":0", "\"start\":-1", 1);
texts[0] = Some(t);
}
// 1: the machine captions' sources, on the take.
let base = (-TAKE_OFFSET) as u64;
let a_cue = if defect == Defect::SourceNotAHead { d["asrA1"] } else { d["asrA0"] };
let take = if defect == Defect::SourceUnknownTake { nothing(2) } else { d["take"] };
let asr_sources = json!({"kind": kinds::SOURCES, "room": r, "principal": h(&pk(ASR)), "sources": [
{"cue": h(&a_cue), "basis": "take", "take": h(&take), "start": base + 2_400_000, "end": base + 4_400_000},
{"cue": h(&d["asrB0"]), "basis": "take", "take": h(&take), "start": base + 5_600_000, "end": base + 7_000_000}]});
push(&mut records, &mut texts, signed(ALIGNER, asr_sources.clone()));
if defect == Defect::SecondSources {
push(&mut records, &mut texts, signed(REVIEWER, asr_sources));
}
// 2: the reference transcript of the machine captions.
let mut entries = vec![
json!({"cue": h(&d["asrA0"]), "text": if defect == Defect::ReferenceSpaces { "The moon pulls the sea." } else { "The moon pulls the sea." }}),
];
if defect != Defect::ReferenceMissingCue {
entries.push(json!({"cue": h(&d["asrB0"]), "text": "High tide comes twice a day."}));
}
if defect == Defect::ReferenceUnknownCue {
entries.push(json!({"cue": h(&nothing(3)), "text": "Spring tides."}));
}
push(
&mut records,
&mut texts,
signed(REVIEWER, json!({"kind": kinds::REFERENCE, "room": r, "principal": h(&pk(ASR)), "entries": entries})),
);
// 3: the description, voiced on a track by a synthesized voice.
let mut d1 = json!({"id": "d1", "begin": 10_500, "end": 11_900, "represents": "visual.nonText",
"text": "A teacher stands beside a tide chart.", "duck": {"gain": 400, "ramp": 200}, "voice": {"begin": 10_720, "end": 11_690}});
match defect {
Defect::DescriptionLate => d1["voice"]["begin"] = json!(10_746),
Defect::DescriptionEarly => d1["voice"]["begin"] = json!(10_664),
Defect::DescriptionRunsOn => d1["voice"]["end"] = json!(11_746),
_ => {}
}
let mut d2 = json!({"id": "d2", "begin": 15_200, "end": 16_000, "represents": "visual.text",
"text": "On the board: HIGH TIDE 06:10.", "duck": {"gain": 1000, "ramp": 0}, "voice": {"begin": 15_210, "end": 15_990}});
if defect == Defect::DescriptionOverlap {
d2["begin"] = json!(11_800);
d2["voice"] = Value::Null;
}
if defect == Defect::DescriptionUnvoiced {
d2["voice"] = Value::Null;
}
let description = signed(
DESCRIBER,
json!({"kind": kinds::DESCRIPTION, "lang": "en", "script": {"origin": "human", "model": null},
"voicing": {"form": "track", "origin": "machine", "model": h(&model(2)), "track": h(&track(1))}, "events": [d1, d2]}),
);
let description_at = records.len();
if defect != Defect::ClaimMissingTrack {
push(&mut records, &mut texts, description.clone());
}
let description_id = record_id(&description);
// 4: the signing, motion-captured from a person, bound to the human captions.
let mut seg1 = json!({"object": h(&object(1, 1)), "start": 2_600_000, "end": 4_800_000, "revisions": [h(&d["cap1"])]});
if defect == Defect::SignUnknownRevision {
seg1["revisions"] = json!([h(&nothing(4))]);
}
if defect == Defect::SigningEmptySegment {
seg1["end"] = json!(2_600_000);
}
let signing_body = json!({"kind": kinds::SIGNING, "room": r, "lang": "bfi", "source": h(&pk(CAPTIONER)), "form": "pose",
"track": h(&track(2)), "origin": "human", "model": null, "video": null, "segments": [seg1,
{"object": h(&object(1, 2)), "start": 5_700_000, "end": 8_800_000, "revisions": [h(&d["cap2"])]},
{"object": h(&object(1, 3)), "start": 9_300_000, "end": 11_000_000, "revisions": [h(&d["cap3"])]}]});
let signing = signed(INTERPRETER, signing_body.clone());
push(&mut records, &mut texts, signing.clone());
let signing_id = record_id(&signing);
// 5: the review of the signing.
let mut reviewed = signing_id;
if defect == Defect::ReviewOfAnother {
let mut other = signing_body;
other["track"] = h(&track(3));
let other = signed(INTERPRETER, other);
reviewed = record_id(&other);
push(&mut records, &mut texts, other);
}
if defect == Defect::ReviewUnknownSigning {
reviewed = nothing(5);
}
let (verdict, notes) = if defect == Defect::ReviewRejected { ("rejected", h(&nothing(7))) } else { ("approved", Value::Null) };
let review = signed(REVIEWER, json!({"kind": kinds::REVIEW, "signing": h(&reviewed), "scope": "translation", "verdict": verdict, "notes": notes}));
let review_at = records.len();
if defect != Defect::ReviewNotGiven {
push(&mut records, &mut texts, review.clone());
}
let review_id = record_id(&review);
match defect {
Defect::DuplicateMember => {
let t = text(&review);
texts[review_at] = Some(t.replacen("\"verdict\":\"approved\"", "\"verdict\":\"approved\",\"verdict\":\"rejected\"", 1));
}
Defect::FlippedSignature => {
let mut v = review.clone();
let mut s = v["sig"].as_str().expect("a signature").to_string();
let last = if s.ends_with('0') { "1" } else { "0" };
s.replace_range(s.len() - 1.., last);
v["sig"] = Value::String(s);
texts[review_at] = Some(text(&v));
}
Defect::WeakKey => {
let mut v = review.clone();
v["signer_pubkey"] = h(&crate::sig::IDENTITY);
texts[review_at] = Some(text(&v));
}
_ => {}
}
// 6, 7: the quality records of both caption tracks, computed from what comes before.
// A defect the checks before the quality records refuse leaves them unread: they are the
// lesson's own then.
let so_far: Vec<String> = records.iter().map(text).collect();
let (cap_q, mut asr_q) = match super::verify(room(), events, &so_far) {
Outcome::Accept(_) => (quality(events, &so_far, CAPTIONER), quality(events, &so_far, ASR)),
Outcome::Refuse { .. } => {
let base = lesson_records(events, d, Defect::None).0;
(read(&base[6]).expect("a record").value, read(&base[7]).expect("a record").value)
}
};
push(&mut records, &mut texts, cap_q);
if defect == Defect::LatencyMisstated {
asr_q["latency"]["mean"] = json!("850001");
asr_q = signed(VENDOR, strip(asr_q));
}
if defect == Defect::QualityWithoutReference {
asr_q["reference"] = Value::Null;
asr_q = signed(VENDOR, strip(asr_q));
}
push(&mut records, &mut texts, asr_q);
// 8: the claim.
let (from, to) = if defect == Defect::ClaimEmptySession { (12_000_000, 12_000_000) } else { (0, 12_000_000) };
let asr_max = if defect == Defect::LatencyAboveBound { json!(850_000) } else { json!(3_000_000) };
let cap_principal = if defect == Defect::CaptionsOfNoOne { pk(HOST) } else { pk(CAPTIONER) };
let ad_lang = if defect == Defect::ClaimWrongLanguage { "fr" } else { "en" };
let offset_max = if defect == Defect::SignOffsetAboveBound { 1_500_000 } else { 4_000_000 };
let claim = signed(
AUTHORITY,
json!({"kind": kinds::CLAIM, "room": r, "from": from, "to": to, "pt0": PT0_MS, "tracks": [
{"service": "captions", "principal": h(&cap_principal), "lang": "en", "latency_mean_max": 4_500_000, "latency_max": null},
{"service": "captions", "principal": h(&pk(ASR)), "lang": "en", "latency_mean_max": 4_500_000, "latency_max": asr_max},
{"service": "audio-description", "record": h(&description_id), "lang": ad_lang},
{"service": "sign-language", "record": h(&signing_id), "lang": "bfi", "review": h(&review_id), "offset_max": offset_max}]}),
);
push(&mut records, &mut texts, claim);
let out = records.iter().zip(texts).map(|(v, t)| t.unwrap_or_else(|| text(v))).collect();
(out, description_at)
}
/// A record's body: without `signer_pubkey` and `sig`.
fn strip(v: Value) -> Value {
let mut m = body(v);
m.remove("signer_pubkey");
m.remove("sig");
Value::Object(m)
}
/// The quality record of `p`'s track, stating the figure this module computes over `events`
/// and `records`.
fn quality(events: &[Vec<u8>], records: &[String], p: u8) -> Value {
let Outcome::Accept(f) = super::verify(room(), events, records) else { panic!("the records before a quality record verify") };
let fig = f["captions"].as_array().expect("figures").iter().find(|x| x["principal"] == h(&pk(p))).expect("a figure").clone();
let mut m = body(fig);
m.insert("kind".into(), Value::String(kinds::QUALITY.into()));
signed(VENDOR, Value::Object(m))
}
/// A defect of the reviewed session's records.
#[derive(Clone, Copy, PartialEq, Eq)]
enum ReviewedDefect {
None,
SignPartial,
LatencyBoundUnmeasured,
}
/// The reviewed session's records: a description the sink voices, the video of a signer, a
/// quality record and a claim without latency bounds.
fn reviewed_records(events: &[Vec<u8>], d: &Digests, defect: ReviewedDefect) -> Vec<String> {
let r = hex(&room());
let description = signed(
DESCRIBER,
json!({"kind": kinds::DESCRIPTION, "lang": "en", "script": {"origin": "machine", "model": h(&model(3))},
"voicing": {"form": "sink", "origin": "machine-reviewed", "model": h(&model(4)), "track": null}, "events": [
{"id": "r1", "begin": 11_000, "end": 12_200, "represents": "visual.nonText", "text": "Divers swim over a reef.",
"duck": {"gain": 500, "ramp": 150}, "voice": null},
{"id": "r2", "begin": 14_000, "end": 15_000, "represents": "visual.text.title", "text": "Title:\nReef Tour & Co.",
"duck": {"gain": 1000, "ramp": 0}, "voice": null}]}),
);
let first = if defect == ReviewedDefect::SignPartial { d["A0"] } else { d["A1"] };
let signing = signed(
INTERPRETER,
json!({"kind": kinds::SIGNING, "room": r, "lang": "ase", "source": h(&pk(ASR)), "form": "video", "track": h(&track(4)),
"origin": "machine", "model": h(&model(5)), "video": {"width": 640, "height": 360, "rate_num": 25, "rate_den": 1},
"segments": [
{"object": h(&object(2, 1)), "start": 3_000_000, "end": 5_000_000, "revisions": [h(&first)]},
{"object": h(&object(2, 2)), "start": 6_500_000, "end": 8_200_000, "revisions": [h(&d["B0"])]}]}),
);
let mut records = vec![text(&description), text(&signing)];
let q = quality(events, &[], ASR);
records.push(text(&q));
let bound = if defect == ReviewedDefect::LatencyBoundUnmeasured { json!(4_500_000) } else { Value::Null };
let claim = signed(
AUTHORITY,
json!({"kind": kinds::CLAIM, "room": r, "from": 0, "to": 10_000_000, "pt0": PT0_MS, "tracks": [
{"service": "captions", "principal": h(&pk(ASR)), "lang": "en", "latency_mean_max": bound, "latency_max": null},
{"service": "audio-description", "record": h(&record_id(&description)), "lang": "en"},
{"service": "sign-language", "record": h(&record_id(&signing)), "lang": "ase", "review": null, "offset_max": null}]}),
);
records.push(text(&claim));
records
}
struct Vector {
name: &'static str,
note: &'static str,
stream: &'static str,
records: Vec<String>,
/// The reason it is written to be refused with, or `None` for accept.
want: Option<&'static str>,
}
fn vectors(streams: &BTreeMap<&'static str, (Vec<Vec<u8>>, Digests)>) -> (Vec<Vector>, usize, usize) {
let lesson = &streams["lesson"];
let reviewed = &streams["reviewed"];
let mut out = Vec::new();
let (lesson_records_base, lesson_desc) = lesson_records(&lesson.0, &lesson.1, Defect::None);
out.push(Vector {
name: "lesson",
note: "a human captioner, machine captions revised to final and placed by a take, a reference transcript, a quality record for each track, a description voiced on a track by a synthesized voice, a motion-captured signer bound to the human captions with an approved review, and a claim held to the 2024 statement's mean",
stream: "lesson",
records: lesson_records_base,
want: None,
});
out.push(Vector {
name: "reviewed",
note: "machine captions a person reviews before they are final, a description the sink voices with a pinned model, the video of a generated signer, and a claim without latency bounds",
stream: "reviewed",
records: reviewed_records(&reviewed.0, &reviewed.1, ReviewedDefect::None),
want: None,
});
out.push(Vector {
name: "unplaced",
note: "no sources: the latency's class is none, and a claim with no latency bound holds",
stream: "lesson",
records: lesson_records(&lesson.0, &lesson.1, Defect::Unplaced).0,
want: None,
});
let negatives: [(&'static str, &'static str, Defect, &'static str); 30] = [
("latency_misstated", "the machine captions' quality record states a mean latency one microsecond above the recomputed one", Defect::LatencyMisstated, "figure_mismatch"),
("description_late", "a description's voice starts 46 ms after the programme audio has fallen", Defect::DescriptionLate, "description_out_of_sync"),
("description_early", "a description's voice starts 36 ms before the programme audio has fallen", Defect::DescriptionEarly, "description_out_of_sync"),
("description_runs_on", "a description's voice ends 46 ms after the programme audio has risen", Defect::DescriptionRunsOn, "description_out_of_sync"),
("sign_unknown_revision", "the signing names a caption revision the stream does not hold", Defect::SignUnknownRevision, "sign_revision_unknown"),
("claim_missing_track", "the claim declares a description the verifier does not hold", Defect::ClaimMissingTrack, "track_missing"),
("duplicate_member", "the review names its verdict twice", Defect::DuplicateMember, "record_malformed"),
("negative_number", "a source's start is a negative number", Defect::NegativeNumber, "record_malformed"),
("flipped_signature", "a bit of the review's signature is flipped", Defect::FlippedSignature, "bad_signature"),
("weak_key", "the review's key is the identity point", Defect::WeakKey, "bad_key"),
("wrong_room", "the captioner's sources name another room", Defect::WrongRoom, "wrong_room"),
("second_sources", "a second sources record for the machine captions", Defect::SecondSources, "record_conflict"),
("source_not_a_head", "a source names a later revision, not a cue's first", Defect::SourceNotAHead, "source_unknown"),
("source_unknown_take", "a source names a take with no opening commitment", Defect::SourceUnknownTake, "source_take_unknown"),
("reference_spaces", "a reference text holds two spaces in a row", Defect::ReferenceSpaces, "text_not_in_subset"),
("reference_unknown_cue", "the reference names a cue the stream does not hold", Defect::ReferenceUnknownCue, "reference_unknown"),
("reference_missing_cue", "the reference leaves a cue out", Defect::ReferenceMissingCue, "reference_incomplete"),
("quality_without_reference", "the quality record names no reference while the track has one", Defect::QualityWithoutReference, "figure_inputs_mismatch"),
("description_overlap", "a description begins before the one before it ends", Defect::DescriptionOverlap, "description_malformed"),
("signing_empty_segment", "a signing segment ends where it starts", Defect::SigningEmptySegment, "signing_malformed"),
("review_unknown_signing", "the review names a signing record the verifier does not hold", Defect::ReviewUnknownSigning, "review_subject_unknown"),
("claim_empty_session", "the claim's session ends where it starts", Defect::ClaimEmptySession, "claim_malformed"),
("description_unvoiced", "a description on a voicing track has an event the track does not voice", Defect::DescriptionUnvoiced, "track_incomplete"),
("claim_wrong_language", "the claim declares the description in another language", Defect::ClaimWrongLanguage, "track_mismatch"),
("latency_above_bound", "the claim holds the machine captions to a latency of at most 850 ms; one cue's is 900 ms", Defect::LatencyAboveBound, "out_of_sync"),
("review_not_given", "the claim names a review the verifier does not hold", Defect::ReviewNotGiven, "review_missing"),
("review_of_another", "the claim's review approves another signing record", Defect::ReviewOfAnother, "review_mismatch"),
("review_rejected", "the claim's review rejects the signing", Defect::ReviewRejected, "review_rejected"),
("captions_of_no_one", "the claim declares captions by a principal with none", Defect::CaptionsOfNoOne, "track_missing"),
("sign_offset_above_bound", "the claim holds the signing to 1.5 s behind the speech; one segment is 1.8 s behind", Defect::SignOffsetAboveBound, "out_of_sync"),
];
for (name, note, defect, want) in negatives {
out.push(Vector { name, note, stream: "lesson", records: lesson_records(&lesson.0, &lesson.1, defect).0, want: Some(want) });
}
let base = lesson_records(&lesson.0, &lesson.1, Defect::None).0;
out.push(Vector { name: "chain_broken", note: "the captioner's second caption names a prev that is not its first's", stream: "chain_broken", records: base.clone(), want: Some("chain_broken") });
out.push(Vector { name: "supersede_unknown", note: "a machine revision supersedes a revision no one wrote", stream: "supersede_unknown", records: base.clone(), want: Some("supersede_unknown") });
out.push(Vector { name: "undelivered_revisions", note: "the machine captions' second cue's partial and its annotation are not delivered, so the track is not whole", stream: "gap", records: base, want: Some("stream_incomplete") });
out.push(Vector { name: "sign_partial", note: "the signing is bound to a partial a later revision superseded", stream: "reviewed", records: reviewed_records(&reviewed.0, &reviewed.1, ReviewedDefect::SignPartial), want: Some("sign_revision_not_final") });
out.push(Vector { name: "latency_unmeasured", note: "the claim bounds the latency of captions no sources record places", stream: "reviewed", records: reviewed_records(&reviewed.0, &reviewed.1, ReviewedDefect::LatencyBoundUnmeasured), want: Some("sync_unmeasured") });
(out, lesson_desc, 0)
}
fn streams() -> BTreeMap<&'static str, (Vec<Vec<u8>>, Digests)> {
let mut m = BTreeMap::new();
m.insert("lesson", lesson_stream(StreamDefect::None));
m.insert("reviewed", reviewed_stream());
m.insert("chain_broken", lesson_stream(StreamDefect::ChainBroken));
m.insert("supersede_unknown", lesson_stream(StreamDefect::SupersedeUnknown));
m.insert("gap", lesson_stream(StreamDefect::Gap));
m
}
fn outcome_json(o: &Outcome) -> Value {
match o {
Outcome::Accept(f) => json!({"outcome": "accept", "figures": f}),
Outcome::Refuse { at, index, reason } => json!({"outcome": "refuse", "at": at, "index": index, "reason": reason}),
}
}
/// The corpus files, as (path relative to the corpus directory, contents), in order;
/// `vectors.json` first.
pub fn generate() -> Vec<(String, String)> {
let streams = streams();
let (vectors, lesson_desc, reviewed_desc) = vectors(&streams);
let mut files: Vec<(String, String)> = Vec::new();
let mut seen = BTreeSet::new();
let mut vs = Vec::new();
for v in &vectors {
let o = super::verify(room(), &streams[v.stream].0, &v.records);
match (&o, v.want) {
(Outcome::Accept(_), None) => {}
(Outcome::Refuse { reason, .. }, Some(w)) if *reason == w => {
seen.insert(w);
}
_ => panic!("{}: written to be {:?}, this module gives {o:?}", v.name, v.want),
}
let mut j = json!({"name": v.name, "note": v.note, "stream": v.stream, "records": v.records});
for (k, x) in body(outcome_json(&o)) {
j[k] = x;
}
vs.push(j);
}
let mut documents = Vec::new();
for (name, at, params) in [("lesson", lesson_desc, Some((VOICING_URI, VOICING_TYPE))), ("reviewed", reviewed_desc, None)] {
let v = vectors.iter().find(|v| v.name == name).expect("an accepted vector");
let r = read(&v.records[at]).expect("a description record");
let (uri, ty) = params.unwrap_or(("", ""));
let doc = dapt(&r.value, uri, ty).expect("a DAPT document");
let file = format!("{name}.dapt.ttml");
let (uri, ty) = match params {
Some((u, t)) => (json!(u), json!(t)),
None => (Value::Null, Value::Null),
};
documents.push(json!({"vector": name, "record": at, "uri": uri, "type": ty, "file": file}));
files.push((file, doc));
}
for (uri, ty) in [("https://media.example/a\"b", VOICING_TYPE), (VOICING_URI, "")] {
let v = &vectors[0];
let r = read(&v.records[lesson_desc]).expect("a description record");
let reason = dapt(&r.value, uri, ty).expect_err("a parameter refused").code();
seen.insert(reason);
documents.push(json!({"vector": "lesson", "record": lesson_desc, "uri": uri, "type": ty, "reason": reason}));
}
for r in Reason::ALL {
assert!(seen.contains(r.code()), "no vector is refused {}", r.code());
}
let doc = json!({
"corpus": "wai.access.services",
"extension": "extensions/access-services.md",
"room": hex(&room()),
"pt0_ms": PT0_MS,
"principals": NAMES.iter().map(|(n, i)| (n.to_string(), json!(hex(&pk(*i))))).collect::<Map<String, Value>>(),
"streams": streams.iter().map(|(n, (e, _))| (n.to_string(), json!(e.iter().map(|b| hex(b)).collect::<Vec<_>>()))).collect::<Map<String, Value>>(),
"vectors": vs,
"documents": documents,
});
let mut out = vec![("vectors.json".to_string(), serde_json::to_string_pretty(&doc).expect("the corpus serialises") + "\n")];
out.extend(files);
let fails = verify_files(&out.iter().cloned().collect());
assert!(fails.is_empty(), "the generated corpus does not verify:\n{}", fails.join("\n"));
out
}
fn unhex_bytes(s: &str) -> Option<Vec<u8>> {
if !s.len().is_multiple_of(2) {
return None;
}
(0..s.len()).step_by(2).map(|i| u8::from_str_radix(s.get(i..i + 2)?, 16).ok()).collect()
}
/// Check a corpus directory's files (name → contents) against this module: every vector's
/// outcome, place, reason and figures, and every DAPT document. Returns the failures.
pub fn verify_files(files: &BTreeMap<String, String>) -> Vec<String> {
let mut fails = Vec::new();
let Some(doc) = files.get("vectors.json") else { return vec!["no vectors.json".into()] };
let root: Value = match serde_json::from_str(doc) {
Ok(v) => v,
Err(e) => return vec![format!("vectors.json does not parse: {e}")],
};
let room_id = unhex::<32>(root["room"].as_str().unwrap_or("")).unwrap_or([0; 32]);
let mut streams: BTreeMap<String, Vec<Vec<u8>>> = BTreeMap::new();
for (n, e) in root["streams"].as_object().into_iter().flatten() {
let evs: Option<Vec<Vec<u8>>> = e.as_array().into_iter().flatten().map(|x| x.as_str().and_then(unhex_bytes)).collect();
match evs {
Some(evs) => {
streams.insert(n.clone(), evs);
}
None => fails.push(format!("stream {n} does not read")),
}
}
let mut records_of: BTreeMap<String, Vec<String>> = BTreeMap::new();
for v in root["vectors"].as_array().into_iter().flatten() {
let name = v["name"].as_str().unwrap_or("?").to_string();
let Some(events) = v["stream"].as_str().and_then(|s| streams.get(s)) else {
fails.push(format!("{name}: names no stream the corpus holds"));
continue;
};
let records: Vec<String> = v["records"].as_array().into_iter().flatten().filter_map(|r| r.as_str().map(String::from)).collect();
let got = outcome_json(&super::verify(room_id, events, &records));
for k in ["outcome", "at", "index", "reason", "figures"] {
let (g, w) = (got.get(k).unwrap_or(&Value::Null), v.get(k).unwrap_or(&Value::Null));
if g != w {
fails.push(format!("{name}: {k}: {w} is stated, this module gives {g}"));
}
}
records_of.insert(name, records);
}
for d in root["documents"].as_array().into_iter().flatten() {
let vname = d["vector"].as_str().unwrap_or("?");
let at = d["record"].as_u64().unwrap_or(u64::MAX) as usize;
let Some(text) = records_of.get(vname).and_then(|r| r.get(at)) else {
fails.push(format!("document of {vname}: names no record"));
continue;
};
let rec = match read(text) {
Ok(r) if r.kind == kinds::DESCRIPTION => r,
_ => {
fails.push(format!("document of {vname}: record {at} is not a description"));
continue;
}
};
let got = dapt(&rec.value, d["uri"].as_str().unwrap_or(""), d["type"].as_str().unwrap_or(""));
match (got, d["file"].as_str(), d["reason"].as_str()) {
(Ok(doc), Some(f), None) => {
if files.get(f) != Some(&doc) {
fails.push(format!("document of {vname}: {f} differs from this module's"));
}
}
(Err(e), None, Some(r)) if e.code() == r => {}
(g, f, r) => fails.push(format!("document of {vname}: the corpus states {f:?} / {r:?}, this module gives {:?}", g.map(|_| "a document"))),
}
}
fails
}
/// The corpus check of a directory's files: [`verify_files`].
pub fn verify(files: &BTreeMap<String, String>) -> Vec<String> {
verify_files(files)
}

75
wai/wai-rs/src/bin/wai_access_services_vectors.rs Normal file
View file

@ -0,0 +1,75 @@
//! Conformance corpus for access services (`live-conformance/access/`, extensions/access-services.md §11).
//!
//! wai_access_services_vectors gen <corpus-dir> write vectors.json and every expected output
//! (.dapt.ttml), from fixed seeds
//! wai_access_services_vectors verify <corpus-dir> every file regenerates byte for byte, no other
//! file but README.md is there, and every case
//! gets the outcome it states
//!
//! The generator and this verifier are one implementation. The corpus's second
//! implementation is `live-conformance/access_services_verify.py`, written from the extension's
//! text and sharing no code with this binary.
use std::collections::BTreeMap;
use std::path::Path;
use wai::access_services::corpus;
fn main() {
let a: Vec<String> = std::env::args().collect();
match (a.get(1).map(String::as_str), a.get(2)) {
(Some("gen"), Some(d)) => {
if let Err(e) = std::fs::create_dir_all(d) {
eprintln!("create {d}: {e}");
std::process::exit(1);
}
for (name, body) in corpus::generate() {
let p = Path::new(d).join(&name);
if let Err(e) = std::fs::write(&p, body) {
eprintln!("write {}: {e}", p.display());
std::process::exit(1);
}
}
println!("wrote {d}");
}
(Some("verify"), Some(d)) => std::process::exit(verify(Path::new(d))),
_ => {
eprintln!("usage: wai_access_services_vectors gen|verify <corpus-dir>");
std::process::exit(2);
}
}
}
fn verify(dir: &Path) -> i32 {
let mut files = BTreeMap::new();
let entries = match std::fs::read_dir(dir) {
Ok(e) => e,
Err(e) => {
eprintln!("read {}: {e}", dir.display());
return 1;
}
};
for e in entries.flatten() {
let name = e.file_name().to_string_lossy().into_owned();
if name == "README.md" {
continue;
}
match std::fs::read_to_string(e.path()) {
Ok(s) => {
files.insert(name, s);
}
Err(err) => {
eprintln!("read {name}: {err}");
return 1;
}
}
}
let generated: BTreeMap<String, String> = corpus::generate().into_iter().collect();
let same = files == generated;
println!("{} every file regenerates byte for byte from its seeds, and no other file is there", if same { "PASS" } else { "FAIL" });
let fails = corpus::verify(&files);
for f in &fails {
println!("FAIL {f}");
}
println!("{} every case gets the outcome the corpus states", if fails.is_empty() { "PASS" } else { "FAIL" });
(!same || !fails.is_empty()) as i32
}

12
wai/wai-rs/src/codecs/registry.rs
View file

@ -227,6 +227,7 @@ pub const CAP_MEDIA_SYNTHESIS: &str = "wai.media.synthesis";
pub const CAP_AI_USAGE: &str = "wai.ai.usage";
pub const CAP_MEDIA_CONTRIBUTION: &str = "wai.media.contribution";
pub const CAP_DELIVERY_SETTLEMENT: &str = "wai.delivery.settlement";
pub const CAP_ACCESS_SERVICES: &str = "wai.access.services";
/// One row per string SPEC §5 registers that is a payload, companion or record:
/// `(string, SPEC section, edition, registered)`.
@ -322,6 +323,7 @@ const ROWS: &[(&str, &str, Option<&str>, &str)] = &[
(CAP_AI_USAGE, S_INTERACTIVE, None, "2026-10-21"),
(CAP_MEDIA_CONTRIBUTION, S_INTERACTIVE, None, "2026-10-21"),
(CAP_DELIVERY_SETTLEMENT, S_INTERACTIVE, None, "2026-10-21"),
(CAP_ACCESS_SERVICES, S_INTERACTIVE, None, "2026-10-21"),
(CAP_AUDIO_IAMF, S_IMMERSIVE, None, "2026-06-17"),
(CAP_AUDIO_AC4, S_IMMERSIVE, None, "2026-06-17"),
(CAP_AUDIO_IVAS, S_IMMERSIVE, None, "2026-06-17"),
@ -380,6 +382,7 @@ const RECORDS: &[&str] = &[
CAP_AI_USAGE,
CAP_MEDIA_CONTRIBUTION,
CAP_DELIVERY_SETTLEMENT,
CAP_ACCESS_SERVICES,
];
/// A named identifier SPEC uses that has the shape of a capability string but is
@ -542,6 +545,7 @@ const DERIVING_EDGE: &str = "extensions/deriving-edge.md";
const SYNTHESIS: &str = "extensions/synthesis.md";
const CONTRIBUTION: &str = "extensions/contribution.md";
const SETTLEMENT: &str = "extensions/settlement.md";
const ACCESS_SERVICES: &str = "extensions/access-services.md";
const fn wire(name: &'static str, kind: WireKind, document: &'static str, section: &'static str,
status: WireStatus) -> WireIdentifier {
@ -658,6 +662,14 @@ pub const WIRE_IDENTIFIERS: &[WireIdentifier] = &[
wire("WDA1", WireKind::Magic, SETTLEMENT, "§3.1", WireStatus::Defined),
wire("WBR1", WireKind::Magic, SETTLEMENT, "§5.2", WireStatus::Defined),
wire("WSS1", WireKind::Magic, SETTLEMENT, "§7.1", WireStatus::Defined),
wire("wai-caption-sources", WireKind::DocumentKind, ACCESS_SERVICES, "§4.2", WireStatus::Defined),
wire("wai-caption-reference", WireKind::DocumentKind, ACCESS_SERVICES, "§4.3", WireStatus::Defined),
wire("wai-caption-quality", WireKind::DocumentKind, ACCESS_SERVICES, "§4.7", WireStatus::Defined),
wire("wai-description", WireKind::DocumentKind, ACCESS_SERVICES, "§5.1", WireStatus::Defined),
wire("tag:transaction.science,2026:wai/access", WireKind::Scheme, ACCESS_SERVICES, "§5.5", WireStatus::Defined),
wire("wai-signing", WireKind::DocumentKind, ACCESS_SERVICES, "§6.1", WireStatus::Defined),
wire("wai-signing-review", WireKind::DocumentKind, ACCESS_SERVICES, "§6.4", WireStatus::Defined),
wire("wai-access-completeness", WireKind::DocumentKind, ACCESS_SERVICES, "§7.1", WireStatus::Defined),
];
/// A string with the shape of a capability that the reference implementation uses

8
wai/wai-rs/src/lib.rs
View file

@ -544,6 +544,14 @@ pub mod settlement;
/// cbindgen:ignore
#[cfg(feature = "timed_text")]
pub mod timed_text;
// Access services (extensions/access-services.md): `wai.access.services` — caption quality
// (latency, stability, accuracy) recomputed from the signed revisions, audio description with
// its mixing, voicing and DAPT document, sign language bound to caption revisions by digest,
// and the completeness claim of a room session. Rust-only, like timed_text: its constants
// must not become macros in include/wai.h.
/// cbindgen:ignore
#[cfg(feature = "access_services")]
pub mod access_services;
// Typed receipt classes (extensions/jwp-receipts.md §2.4.3–§2.4.12, §2.6, §3.1–§3.2):
// energy, lawful-basis, log-inclusion, soft-binding, binding-recovery, measurement,

69
wai/wai-rs/tests/access_services.rs Normal file
View file

@ -0,0 +1,69 @@
//! `wai.access.services` (extensions/access-services.md): the committed corpus
//! `live-conformance/access/` regenerates byte for byte from its fixed seeds, holds no
//! other file, every vector and document in it gets the outcome it states, every reason of
//! §10 refuses a case, and the extension's §10 lists exactly the reasons the reference
//! reports.
#![cfg(feature = "access_services")]
use std::collections::{BTreeMap, BTreeSet};
use std::path::PathBuf;
use wai::access_services::{corpus, Reason};
fn wai_dir() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("..")
}
fn committed() -> BTreeMap<String, String> {
let dir = wai_dir().join("live-conformance/access");
std::fs::read_dir(&dir)
.expect("the committed corpus")
.flatten()
.map(|e| (e.file_name().to_string_lossy().into_owned(), e.path()))
.filter(|(n, _)| n != "README.md")
.map(|(n, p)| (n, std::fs::read_to_string(p).expect("a text file")))
.collect()
}
#[test]
fn the_committed_corpus_regenerates_from_its_seeds() {
let files = committed();
let generated: BTreeMap<String, String> = corpus::generate().into_iter().collect();
let extra: Vec<&String> = files.keys().filter(|k| !generated.contains_key(*k)).collect();
let missing: Vec<&String> = generated.keys().filter(|k| !files.contains_key(*k)).collect();
let differ: Vec<&String> = generated.iter().filter(|(k, v)| files.get(*k).is_some_and(|f| f != *v)).map(|(k, _)| k).collect();
assert!(extra.is_empty() && missing.is_empty() && differ.is_empty(), "extra {extra:?}, missing {missing:?}, differ {differ:?}: run `wai_access_services_vectors gen`");
}
#[test]
fn every_case_gets_its_stated_outcome() {
let fails = corpus::verify(&committed());
assert!(fails.is_empty(), "{}", fails.join("\n"));
}
/// Every reason of §10 refuses a vector or a document of the committed corpus.
#[test]
fn every_reason_refuses_a_case() {
let files = committed();
let root: serde_json::Value = serde_json::from_str(&files["vectors.json"]).expect("vectors.json");
let mut seen = BTreeSet::new();
for v in root["vectors"].as_array().into_iter().flatten().chain(root["documents"].as_array().into_iter().flatten()) {
if let Some(r) = v["reason"].as_str() {
seen.insert(r.to_string());
}
}
for r in Reason::ALL {
assert!(seen.contains(r.code()), "no case of the corpus is refused {}", r.code());
}
}
/// §10 of the extension lists exactly the codes `Reason` has, each once, in order.
#[test]
fn the_extension_lists_the_reasons() {
let text = std::fs::read_to_string(wai_dir().join("extensions/access-services.md")).expect("the extension");
let at = text.find("\n## 10. Reasons").expect("a §10");
let end = text[at + 1..].find("\n## ").map_or(text.len(), |e| at + 1 + e);
let listed: Vec<String> =
text[at..end].lines().filter(|l| l.starts_with("| `")).filter_map(|l| l.split('`').nth(1).map(String::from)).collect();
let want: Vec<String> = Reason::ALL.iter().map(|r| r.code().to_string()).collect();
assert_eq!(listed, want, "§10's table and Reason::ALL differ");
}