wai: timed text and live captions, wai.text.timed (L8) #303

Merged

dcharlot merged 28 commits from wai/timed-text into main 2026-10-18 00:37:23 -04:00

Conversation 0 Commits 28 Files changed 73 +19424 -46

dcharlot commented 2026-10-17 11:26:51 -04:00

Owner

Copy link

L8: timed text and live captions (wai.text.timed)

This PR adds a spec, a reference implementation, a conformance corpus and an independent verifier. It does not change live-events.

What it adds

• wai/extensions/timed-text.md registers the record wai.text.timed. It contains:

  • §3 The cue model. start and end in ms, BCP 47 language, origin, model digest, one of six anchors, text, and italic ranges. The text is limited to the subset that WebVTT and the IMSC1 Text Profile both hold exactly. Text outside that subset is refused, never changed. Examples: a tab, two spaces in a row, a space at the edge of a line, a C0 or C1 control, U+2028. Italic ranges have one canonical spelling. A table lists what each target drops and where that information goes instead.
  • §4 Live captions.
    • A caption is a live-events caption event, unchanged.
    • Each revision of a machine caption is a new caption event, followed by a meta event whose payload is the WTX1 annotation. The annotation holds: the digest of the revision it supersedes, partial or final, stable_cp, the anchor, and the italic ranges.
    • Checks, in a fixed order: annotation_malformed, annotation_misplaced, supersede_unknown, supersede_final, supersede_forked, supersede_mismatch, stability_broken, text_not_in_subset.
    • §4.4 says what a sink shows at instant x. §4.5 gives the cues a gapless stream resolves to (refusals: not_final, duration_unknown).
    • §4.6 adds two items to the events track's join set: the open partial, and each caption's annotation. With them, a subscriber that joins mid-stream shows what one from the start shows.
  • §5 WebVTT. Exact document bytes, and segments with X-TIMESTAMP-MAP. The map is re-anchored in every segment, so a 33-bit wrap stays exact. There is also a reader of the subset.
  • §6 IMSC1 Text Profile. Exact document bytes, and a sound check against the hypothetical render model (§6.2). §6.3 lists the structure a verifier checks.
  • §7 The MoQ caption timelines.
    • The WebVTT timeline follows draft-wilaw-moq-webvtt-msf-00; the IMSC1 timeline (cue mode) follows draft-law-moq-imsc1-msf-00. Records are written byte for byte.
    • Object 0 of each group repeats every cue still showing.
    • Catalog entries carry a custom wai_machine_generated field.
  • §8 CTA-608 on field 1, CC1:
    • basic, special and extended characters, with substitutions and fallbacks;
    • rows and columns by preamble address code and tab offset;
    • italics by preamble address code at column 0, otherwise by mid-row code;
    • odd parity, and every two-byte code sent twice;
    • pop-on with backward-filled loads, End of Caption and Erase Displayed Memory;
    • roll-up with 2, 3 or 4 rows and Carriage Return;
    • a decoder for round trips.
  • §9 CTA-708. Specified only (see below).
  • §10 The disclosure label in every target:
    • the HLS characteristic public.machine-generated on SUBTITLES and CLOSED-CAPTIONS;
    • C2PA 2.4 c2pa.ai-disclosure, one assertion per model, plus the IPTC digital source type;
    • a WebVTT class span, <c.wai-machine.wai-model-HEX>;
    • IMSC1 wai:origin and wai:model attributes;
    • members in the timeline records and a field in the catalog;
    • a sidecar, published in HLS by EXT-X-SESSION-DATA, for targets that cannot carry the label. A target with no carrier must still publish the sidecar; the label is never dropped silently.
  • §11 Player obligations. Captions can be shown, are on time, are preserved, are under the viewer's control, are discoverable (one operation, at the level of the volume control), persist across items and sessions, and are labelled. Each cites EN 301 549 V4.1.1 clauses 7.1.1–7.1.4 and 7.3, or WCAG 2.2 SC 1.2.2 and 1.2.4. These are engineering requirements, not legal advice.
  • §12 lists 24 refusal reasons.

• wai/wai-rs/src/timed_text.rs and timed_text/cc608.rs, with the generator in timed_text/corpus.rs:

  • feature timed_text = ["live_events"];
  • marked cbindgen:ignore;
  • no new dependency (the XML is written by hand);
  • 13 unit tests.

• wai_timed_text_vectors (gen / verify) and tests/timed_text.rs, declared with required-features. The test checks that:

  • every committed file regenerates byte for byte, and no other file is there;
  • every case gets its stated outcome;
  • every 608 code path is covered;
  • §12 lists exactly Reason::ALL.

• wai/live-conformance/captions/: vectors.json plus 39 expected outputs (.vtt, .segK.vtt, .ttml, .608.hex), and a README with the provenance line. It holds:

  • 14 event vectors: a human-captioned lesson, a machine-captioned lesson with partials revised to final, a mid-stream join (on the events track and on both timelines), and 11 refusals, including the live-events refusal malformed_payload for a machine caption with no model digest;
  • 17 cue vectors: anchors and styles, the disclosure label through every target, 608 special and extended characters, roll-up 2/3/4 and a second slot clock, and one refusal per remaining reason;
  • 7 read cases and 4 decode cases.

  The 608 files contain every code path: indent and italics preamble address codes, TO1, TO2 and TO3, both mid-row codes, special characters, extended characters from both tables, RCL, ENM, EOC, EDM, CR, RU2, RU3 and RU4. Every reason in §12 refuses at least one case.

• wai/live-conformance/timed_text_verify.py uses the standard library only and runs with -I.

  • It is written from the spec text and verifies events with live_events_verify.py.
  • It rewrites WebVTT, segments, IMSC1, both timelines, 608 (encoder and decoder) and the disclosure, and checks every byte.
  • It parses each IMSC1 document with xml.etree against §6.3 and runs the HRM check.
  • It agreed with the Rust on every byte the first time it ran. It also requires 20 mutated copies of the corpus to fail; all 20 do.

• CI and registry:

  • a new step, "timed text · wai-rs and timed_text_verify.py", in independent-verifiers;
  • extensions/timed-text.md added to both path lists;
  • SPEC §5: the records list and an "And for the captions…" paragraph;
  • registry.rs: CAP_TEXT_TIMED as a record, and wire identifiers WTX1, wai_origin, wai_model, wai_machine_generated, the two tag: URIs, and the DATA-ID;
  • wai registry export re-run;
  • a CHANGELOG entry;
  • int_kernels_vectors reason updated to "live events, rooms and takes, and captions: …".

Implemented, and specified only

• Implemented, with corpus:
  • the cue model;
  • WTX1 revisions and the event checks;
  • what a sink shows, and stream resolution;
  • the events-track join set;
  • WebVTT documents, segments and the reader;
  • IMSC1 documents with the HRM check;
  • both MoQ caption timelines and their catalog entries;
  • 608 pop-on and roll-up 2/3/4, encoder and decoder;
  • HLS EXT-X-MEDIA and EXT-X-SESSION-DATA, the C2PA statement and digital source type, and the sidecar.
• Specified only: CTA-708 (§9). The text gives the service-block semantics: service 1, one window per anchor with relative positioning, pop-on through display and hide, roll-up, italics through pen attributes, G0/G1/G2 characters, and the label in the sidecar. It gives no byte values. CTA-708-E could not be read, so an exact implementation was not possible in this slice.
• Not implemented: fMP4 packaging of IMSC1, which is out of scope.

Where the HLS machine-generated characteristic is defined

public.machine-generated is defined in draft-pantos-hls-rfc8216bis-22 §4.4.6.1 (EXT-X-MEDIA, CHARACTERISTICS): "Any Rendition MAY include the "public.machine-generated" characteristic to indicate the Rendition was authored or translated programmatically." The spec cites the IETF draft directly, so no vendor document is needed. Revision -22 is the latest; datatracker shows rev 22. The draft defines it per rendition, so per-cue granularity comes from the in-segment forms and the sidecar.

Also verified at the source:

• C2PA 2.4 §18.28 c2pa.ai-disclosure, its modelType (Table 12), and the §18.28.3 pairing of digital source type with humanOversightLevel;
• the IPTC vocabulary, release dated 2024-10-23.

Sources and what is unchecked

• Pinned versions:
  • WebVTT CRD 2026-05-20;
  • IMSC 1.3 REC 2026-05-21, and IMSC 1.0.1 REC 2018-04-24 (for its profile signalling, §6.9);
  • IMSC-HRM REC 2024-04-25;
  • draft-wilaw-moq-webvtt-msf-00, draft-law-moq-imsc1-msf-00, draft-ietf-moq-msf-01;
  • draft-pantos-hls-rfc8216bis-22;
  • C2PA 2.4;
  • EN 301 549 V4.1.1 (2026-09).
• CTA-608-E and CTA-708-E could not be read. Their publisher's store needs an account even for the free download.
  • The 608 basic and special characters, mid-row codes, miscellaneous control codes, preamble address codes, parity and redundancy rule come from 47 CFR 79.101 as in force on 2021-06-01. That regulation reproduces the decoder rules, and its text has one evident typo in the preamble address code table (row 13, yellow underline, printed 68).
  • The extended character table (0x12 and 0x13) is not in that regulation. The spec says so and asks a reviewer with CTA-608-E to check it.
  • 0x12 0x29 is left unassigned: the encoder never sends it, and the decoder refuses it.
  • The fallback characters are this extension's choice, stated in §8.4.
• draft-wilaw-moq-webvtt-msf-00 cites MSF sections (§11, §5.3.x) that do not match draft-ietf-moq-msf-01 (§8, §5.2.x). The spec notes this and cites -01's own numbers.

Design choices to review

1. Registered as a record, not a payload capability. The unit is the signed caption revision, and the documents are mappings of it, as live-events maps an event onto a tag. A dispatchable timed-text payload capability would need a media class, tiers and a decoder in dispatch; that is left for a later revision. The graphics-vector §5.7 "timed-text component" is a §5/§6 document. The id is wai.text.timed under the wai.text. prefix, even though it is a record.
2. live-events needs no change. Its caption payload has no supersede field, and appending one would break its reader. So the annotation rides in meta, and is bound to its caption by being the next seq with the same t. It is not bound by a field. Revisions supersede by caption digest. The spec notes that a later live-events revision may fold these fields into caption.
3. t of a revision is when it is shown. duration_ms counts from t. A resolved cue runs from its first revision's t to its final's t plus duration. Partials never reach documents, timelines or 608, because none of those can replace a cue in place. Live HLS segments use the cues final at cut time.
4. When a revision's annotation is unknown. This happens on a gap right after a caption. The caption is shown as an unannotated one. A supersede reference to a revision the verifier does not hold is refused only when the principal's chain is whole; otherwise it is left unresolved, which is how a joiner verifies.
5. One language per document. A WebVTT file carries no language in itself; the rendition and the sidecar carry it.
6. Machine-reviewed is read as C2PA's human_validated, which pairs with compositeWithTrainedAlgorithmicMedia. A mix of origins maps to compositeSynthetic.
7. modelIdentifier is tag:transaction.science,2026:wai/model/<hex>, because live-events does not name the model digest's algorithm, so an ni: URI cannot be formed.
8. The HRM check is sound but conservative. It uses the slowest Ren (0.6) and GCpy (3) for every glyph, empties the glyph cache at empty ISDs, and works in integer units of 1/2700 s. It may refuse a document that conforms; it never passes one that does not. This avoids needing Unicode Script tables.
9. IMSC profile signalling (revised in review). tt carries ttp:profile="http://www.w3.org/ns/ttml/profile/imsc1/text", which IMSC 1.0.1 §6.9 says SHOULD be present. 1.0.1 is the IMSC1 the HLS draft names, and these documents are neither EBU-TT-D nor SDP-US, so that edition's exceptions do not apply. IMSC 1.3's feature table permits #profile.
   • The two ebuttm:conformsToStandard elements stay (the imsc1 and imsc1.3 text designators). IMSC 1.3 §8.10.2 specifies that form, and §6.3.2's override reads it.
   • No Image Profile designator is ever signalled, as 1.0.1 §6.9's SHALL NOT requires, and no ttp:profile element is written.
   • ttp:contentProfiles is still absent, following IMSC 1.3 §8.10.1's note for documents meant for TTML1-based profiles.
   • The Rust and Python writers were each changed on their own. imsc_structure in the verifier now checks the attribute, the absence of the element, and the absence of any Image designator. A new mutated copy, with ttp:profile removed, fails.
   • Regions: top 10–45%, bottom 55–90%, so they never overlap. Default cell resolution.
10. "Cues so far" at a group start means cues not yet ended, the way live-events §8.1 reads MSF independence. Older cues stay reachable by FETCH. Timeline groups are placed by publication time, which is the final revision's t.
11. 608 limits:
    • italics can start only at a row's start or after a space, because a mid-row code shows as a space; otherwise style_not_expressible;
    • roll-up refuses a mix of top and bottom anchors, because the window would move;
    • EDM is sent only when the next cue starts at least 2 slots after this one ends;
    • the slot clock's rate is a parameter. The corpus uses 30000/1001, plus 25/1 to show the frame-rate dependence. How a carrier maps slots into frames is out of scope.
12. IMSC1 cue-mode wai_origin / wai_model are members the IMSC1 MoQ draft does not define. The catalog field and the sidecar back them up.
13. CHARACTERISTICS on a CLOSED-CAPTIONS tag. The HLS draft says that tag names a channel rather than a rendition; the spec says so and carries the characteristic there anyway.
14. §11 "persistent" is a MUST here. In EN 301 549 clause 7.3 NOTE 3 it is only best practice.

rb runs

All ran on the build host through ~/kani-logs/remote/rb /Users/dcharlot/data-share/vibe-coding/_wt-timed-text -- '…'. The set was run first at 053bb285, and again at head 06f006bf after the ttp:profile change. Each exit status is 0 at both heads:

• cargo test --no-default-features --features timed_text --lib timed_text:: (13 passed)
• cargo test --no-default-features --features timed_text --test timed_text (4 passed)
• cargo run -q --no-default-features --features timed_text --bin wai_timed_text_vectors -- verify ../live-conformance/captions
• touch cbindgen.toml, then git diff --exit-code --stat include/wai.h. The header is unchanged after build.rs re-ran.
• cargo test --no-default-features --lib codecs::registry (14 passed)
• cargo run -q --no-default-features --bin wai -- registry export --check
• cargo test --no-default-features --features live_events --lib live_events:: and --test live_events
• cargo test --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int,generate,score,staged,stage_measure,legacy_int --test int_kernels_vectors (8 passed). These are the features of the cargo-class corpora job.
• python3 -I live-conformance/timed_text_verify.py live-conformance/captions (all vectors pass; 20 of 20 mutated copies fail)
• python3 -I live-conformance/live_events_verify.py live-conformance/events

The corpus and the registry export were generated with rb --pull … wai_timed_text_vectors gen and wai registry export. The corpus was regenerated at 06f006bf; only the six .ttml files changed.

Clippy: cargo clippy --no-default-features --features timed_text --lib --tests --bins -- -D warnings reports nothing in this PR's files after the lint fixes. It still exits 101, on 36 findings in modules this PR does not touch (int_payload/wire.rs, container.rs, quality.rs, codecs/glossary_check.rs, world/*, sig.rs, pin.rs, hls_render.rs).

Local checks, all clean: check-tests-in-ci.py, check-spec-refs.py, check-wai-section-refs.py, check-no-vanity-metrics.sh, check-comparative-prose.sh, check-orphan-sources.py.

Merge notes

#296 (wai/live-compose) and #299 (wai/live-record) also touch SPEC §5, REGISTRY.md, the CHANGELOG, wai-conformance.yml, registry.rs, the registry export and the live-conformance exclusion line in tests/int_kernels_vectors.rs. This PR's edits there are additive (new lines, new list items). The exclusion line is a single line: whichever PR lands second should merge the reason to read "live events, rooms and takes, compositions, and captions". After rebasing, the export should be regenerated (wai registry export).

🤖 Generated with Claude Code

## L8: timed text and live captions (`wai.text.timed`) This PR adds a spec, a reference implementation, a conformance corpus and an independent verifier. It does not change live-events. ### What it adds - **`wai/extensions/timed-text.md`** registers the record `wai.text.timed`. It contains: - **§3 The cue model.** start and end in ms, BCP 47 language, origin, model digest, one of six anchors, text, and italic ranges. The text is limited to the subset that WebVTT and the IMSC1 Text Profile both hold exactly. Text outside that subset is refused, never changed. Examples: a tab, two spaces in a row, a space at the edge of a line, a C0 or C1 control, U+2028. Italic ranges have one canonical spelling. A table lists what each target drops and where that information goes instead. - **§4 Live captions.** - A caption is a live-events `caption` event, unchanged. - Each revision of a machine caption is a new `caption` event, followed by a `meta` event whose payload is the `WTX1` annotation. The annotation holds: the digest of the revision it supersedes, partial or final, `stable_cp`, the anchor, and the italic ranges. - Checks, in a fixed order: `annotation_malformed`, `annotation_misplaced`, `supersede_unknown`, `supersede_final`, `supersede_forked`, `supersede_mismatch`, `stability_broken`, `text_not_in_subset`. - §4.4 says what a sink shows at instant *x*. §4.5 gives the cues a gapless stream resolves to (refusals: `not_final`, `duration_unknown`). - §4.6 adds two items to the events track's join set: the open partial, and each caption's annotation. With them, a subscriber that joins mid-stream shows what one from the start shows. - **§5 WebVTT.** Exact document bytes, and segments with `X-TIMESTAMP-MAP`. The map is re-anchored in every segment, so a 33-bit wrap stays exact. There is also a reader of the subset. - **§6 IMSC1 Text Profile.** Exact document bytes, and a sound check against the hypothetical render model (§6.2). §6.3 lists the structure a verifier checks. - **§7 The MoQ caption timelines.** - The WebVTT timeline follows `draft-wilaw-moq-webvtt-msf-00`; the IMSC1 timeline (cue mode) follows `draft-law-moq-imsc1-msf-00`. Records are written byte for byte. - Object 0 of each group repeats every cue still showing. - Catalog entries carry a custom `wai_machine_generated` field. - **§8 CTA-608** on field 1, CC1: - basic, special and extended characters, with substitutions and fallbacks; - rows and columns by preamble address code and tab offset; - italics by preamble address code at column 0, otherwise by mid-row code; - odd parity, and every two-byte code sent twice; - pop-on with backward-filled loads, End of Caption and Erase Displayed Memory; - roll-up with 2, 3 or 4 rows and Carriage Return; - a decoder for round trips. - **§9 CTA-708.** Specified only (see below). - **§10 The disclosure label in every target:** - the HLS characteristic `public.machine-generated` on `SUBTITLES` and `CLOSED-CAPTIONS`; - C2PA 2.4 `c2pa.ai-disclosure`, one assertion per model, plus the IPTC digital source type; - a WebVTT class span, `<c.wai-machine.wai-model-HEX>`; - IMSC1 `wai:origin` and `wai:model` attributes; - members in the timeline records and a field in the catalog; - a sidecar, published in HLS by `EXT-X-SESSION-DATA`, for targets that cannot carry the label. A target with no carrier must still publish the sidecar; the label is never dropped silently. - **§11 Player obligations.** Captions can be shown, are on time, are preserved, are under the viewer's control, are discoverable (one operation, at the level of the volume control), persist across items and sessions, and are labelled. Each cites EN 301 549 V4.1.1 clauses 7.1.1–7.1.4 and 7.3, or WCAG 2.2 SC 1.2.2 and 1.2.4. These are engineering requirements, not legal advice. - **§12** lists 24 refusal reasons. - **`wai/wai-rs/src/timed_text.rs`** and **`timed_text/cc608.rs`**, with the generator in `timed_text/corpus.rs`: - feature `timed_text = ["live_events"]`; - marked `cbindgen:ignore`; - no new dependency (the XML is written by hand); - 13 unit tests. - **`wai_timed_text_vectors`** (`gen` / `verify`) and **`tests/timed_text.rs`**, declared with `required-features`. The test checks that: - every committed file regenerates byte for byte, and no other file is there; - every case gets its stated outcome; - every 608 code path is covered; - §12 lists exactly `Reason::ALL`. - **`wai/live-conformance/captions/`**: `vectors.json` plus 39 expected outputs (`.vtt`, `.segK.vtt`, `.ttml`, `.608.hex`), and a README with the provenance line. It holds: - 14 event vectors: a human-captioned lesson, a machine-captioned lesson with partials revised to final, a mid-stream join (on the events track and on both timelines), and 11 refusals, including the live-events refusal `malformed_payload` for a machine caption with no model digest; - 17 cue vectors: anchors and styles, the disclosure label through every target, 608 special and extended characters, roll-up 2/3/4 and a second slot clock, and one refusal per remaining reason; - 7 read cases and 4 decode cases. The 608 files contain every code path: indent and italics preamble address codes, TO1, TO2 and TO3, both mid-row codes, special characters, extended characters from both tables, RCL, ENM, EOC, EDM, CR, RU2, RU3 and RU4. Every reason in §12 refuses at least one case. - **`wai/live-conformance/timed_text_verify.py`** uses the standard library only and runs with `-I`. - It is written from the spec text and verifies events with `live_events_verify.py`. - It rewrites WebVTT, segments, IMSC1, both timelines, 608 (encoder and decoder) and the disclosure, and checks every byte. - It parses each IMSC1 document with `xml.etree` against §6.3 and runs the HRM check. - It agreed with the Rust on every byte the first time it ran. It also requires 20 mutated copies of the corpus to fail; all 20 do. - **CI and registry:** - a new step, "timed text · wai-rs and timed_text_verify.py", in independent-verifiers; - `extensions/timed-text.md` added to both path lists; - SPEC §5: the records list and an "And for the captions…" paragraph; - `registry.rs`: `CAP_TEXT_TIMED` as a record, and wire identifiers `WTX1`, `wai_origin`, `wai_model`, `wai_machine_generated`, the two `tag:` URIs, and the `DATA-ID`; - `wai registry export` re-run; - a CHANGELOG entry; - `int_kernels_vectors` reason updated to "live events, rooms and takes, and captions: …". ### Implemented, and specified only - **Implemented, with corpus:** - the cue model; - WTX1 revisions and the event checks; - what a sink shows, and stream resolution; - the events-track join set; - WebVTT documents, segments and the reader; - IMSC1 documents with the HRM check; - both MoQ caption timelines and their catalog entries; - 608 pop-on and roll-up 2/3/4, encoder and decoder; - HLS `EXT-X-MEDIA` and `EXT-X-SESSION-DATA`, the C2PA statement and digital source type, and the sidecar. - **Specified only: CTA-708 (§9).** The text gives the service-block semantics: service 1, one window per anchor with relative positioning, pop-on through display and hide, roll-up, italics through pen attributes, G0/G1/G2 characters, and the label in the sidecar. It gives no byte values. CTA-708-E could not be read, so an exact implementation was not possible in this slice. - **Not implemented:** fMP4 packaging of IMSC1, which is out of scope. ### Where the HLS machine-generated characteristic is defined `public.machine-generated` is defined in **`draft-pantos-hls-rfc8216bis-22` §4.4.6.1** (EXT-X-MEDIA, `CHARACTERISTICS`): *"Any Rendition MAY include the "public.machine-generated" characteristic to indicate the Rendition was authored or translated programmatically."* The spec cites the IETF draft directly, so no vendor document is needed. Revision -22 is the latest; datatracker shows rev 22. The draft defines it per rendition, so per-cue granularity comes from the in-segment forms and the sidecar. Also verified at the source: - C2PA 2.4 §18.28 `c2pa.ai-disclosure`, its `modelType` (Table 12), and the §18.28.3 pairing of digital source type with `humanOversightLevel`; - the IPTC vocabulary, release dated 2024-10-23. ### Sources and what is unchecked - **Pinned versions:** - WebVTT CRD 2026-05-20; - IMSC 1.3 REC 2026-05-21, and IMSC 1.0.1 REC 2018-04-24 (for its profile signalling, §6.9); - IMSC-HRM REC 2024-04-25; - `draft-wilaw-moq-webvtt-msf-00`, `draft-law-moq-imsc1-msf-00`, `draft-ietf-moq-msf-01`; - `draft-pantos-hls-rfc8216bis-22`; - C2PA 2.4; - EN 301 549 V4.1.1 (2026-09). - **CTA-608-E and CTA-708-E could not be read.** Their publisher's store needs an account even for the free download. - The 608 basic and special characters, mid-row codes, miscellaneous control codes, preamble address codes, parity and redundancy rule come from **47 CFR 79.101** as in force on 2021-06-01. That regulation reproduces the decoder rules, and its text has one evident typo in the preamble address code table (row 13, yellow underline, printed 68). - The **extended character table (0x12 and 0x13) is not in that regulation.** The spec says so and asks a reviewer with CTA-608-E to check it. - 0x12 0x29 is left unassigned: the encoder never sends it, and the decoder refuses it. - The fallback characters are this extension's choice, stated in §8.4. - `draft-wilaw-moq-webvtt-msf-00` cites MSF sections (§11, §5.3.x) that do not match `draft-ietf-moq-msf-01` (§8, §5.2.x). The spec notes this and cites -01's own numbers. ### Design choices to review 1. **Registered as a record, not a payload capability.** The unit is the signed caption revision, and the documents are mappings of it, as live-events maps an event onto a tag. A dispatchable timed-text payload capability would need a media class, tiers and a decoder in dispatch; that is left for a later revision. The graphics-vector §5.7 "timed-text component" is a §5/§6 document. The id is `wai.text.timed` under the `wai.text.` prefix, even though it is a record. 2. **live-events needs no change.** Its `caption` payload has no supersede field, and appending one would break its reader. So the annotation rides in `meta`, and is bound to its caption by being the next seq with the same `t`. It is not bound by a field. Revisions supersede by caption digest. The spec notes that a later live-events revision may fold these fields into `caption`. 3. **`t` of a revision is when it is shown.** `duration_ms` counts from `t`. A resolved cue runs from its first revision's `t` to its final's `t` plus duration. Partials never reach documents, timelines or 608, because none of those can replace a cue in place. Live HLS segments use the cues final at cut time. 4. **When a revision's annotation is unknown.** This happens on a gap right after a caption. The caption is shown as an unannotated one. A supersede reference to a revision the verifier does not hold is refused only when the principal's chain is whole; otherwise it is left unresolved, which is how a joiner verifies. 5. **One language per document.** A WebVTT file carries no language in itself; the rendition and the sidecar carry it. 6. **Machine-reviewed is read as C2PA's `human_validated`**, which pairs with `compositeWithTrainedAlgorithmicMedia`. A mix of origins maps to `compositeSynthetic`. 7. **`modelIdentifier` is `tag:transaction.science,2026:wai/model/<hex>`**, because live-events does not name the model digest's algorithm, so an `ni:` URI cannot be formed. 8. **The HRM check is sound but conservative.** It uses the slowest `Ren` (0.6) and `GCpy` (3) for every glyph, empties the glyph cache at empty ISDs, and works in integer units of 1/2700 s. It may refuse a document that conforms; it never passes one that does not. This avoids needing Unicode Script tables. 9. **IMSC profile signalling (revised in review).** `tt` carries `ttp:profile="http://www.w3.org/ns/ttml/profile/imsc1/text"`, which IMSC 1.0.1 §6.9 says SHOULD be present. 1.0.1 is the IMSC1 the HLS draft names, and these documents are neither EBU-TT-D nor SDP-US, so that edition's exceptions do not apply. IMSC 1.3's feature table permits `#profile`. - The two `ebuttm:conformsToStandard` elements stay (the imsc1 and imsc1.3 text designators). IMSC 1.3 §8.10.2 specifies that form, and §6.3.2's override reads it. - No Image Profile designator is ever signalled, as 1.0.1 §6.9's SHALL NOT requires, and no `ttp:profile` element is written. - `ttp:contentProfiles` is still absent, following IMSC 1.3 §8.10.1's note for documents meant for TTML1-based profiles. - The Rust and Python writers were each changed on their own. `imsc_structure` in the verifier now checks the attribute, the absence of the element, and the absence of any Image designator. A new mutated copy, with `ttp:profile` removed, fails. - Regions: top 10–45%, bottom 55–90%, so they never overlap. Default cell resolution. 10. **"Cues so far" at a group start means cues not yet ended**, the way live-events §8.1 reads MSF independence. Older cues stay reachable by FETCH. Timeline groups are placed by publication time, which is the final revision's `t`. 11. **608 limits:** - italics can start only at a row's start or after a space, because a mid-row code shows as a space; otherwise `style_not_expressible`; - roll-up refuses a mix of top and bottom anchors, because the window would move; - EDM is sent only when the next cue starts at least 2 slots after this one ends; - the slot clock's rate is a parameter. The corpus uses 30000/1001, plus 25/1 to show the frame-rate dependence. How a carrier maps slots into frames is out of scope. 12. **IMSC1 cue-mode `wai_origin` / `wai_model`** are members the IMSC1 MoQ draft does not define. The catalog field and the sidecar back them up. 13. **`CHARACTERISTICS` on a `CLOSED-CAPTIONS` tag.** The HLS draft says that tag names a channel rather than a rendition; the spec says so and carries the characteristic there anyway. 14. **§11 "persistent" is a MUST here.** In EN 301 549 clause 7.3 NOTE 3 it is only best practice. ### `rb` runs All ran on the build host through `~/kani-logs/remote/rb /Users/dcharlot/data-share/vibe-coding/_wt-timed-text -- '…'`. The set was run first at `053bb285`, and again at head `06f006bf` after the `ttp:profile` change. Each exit status is 0 at both heads: - `cargo test --no-default-features --features timed_text --lib timed_text::` (13 passed) - `cargo test --no-default-features --features timed_text --test timed_text` (4 passed) - `cargo run -q --no-default-features --features timed_text --bin wai_timed_text_vectors -- verify ../live-conformance/captions` - `touch cbindgen.toml`, then `git diff --exit-code --stat include/wai.h`. The header is unchanged after build.rs re-ran. - `cargo test --no-default-features --lib codecs::registry` (14 passed) - `cargo run -q --no-default-features --bin wai -- registry export --check` - `cargo test --no-default-features --features live_events --lib live_events::` and `--test live_events` - `cargo test --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int,generate,score,staged,stage_measure,legacy_int --test int_kernels_vectors` (8 passed). These are the features of the cargo-class corpora job. - `python3 -I live-conformance/timed_text_verify.py live-conformance/captions` (all vectors pass; 20 of 20 mutated copies fail) - `python3 -I live-conformance/live_events_verify.py live-conformance/events` The corpus and the registry export were generated with `rb --pull … wai_timed_text_vectors gen` and `wai registry export`. The corpus was regenerated at `06f006bf`; only the six `.ttml` files changed. Clippy: `cargo clippy --no-default-features --features timed_text --lib --tests --bins -- -D warnings` reports nothing in this PR's files after the lint fixes. It still exits 101, on 36 findings in modules this PR does not touch (`int_payload/wire.rs`, `container.rs`, `quality.rs`, `codecs/glossary_check.rs`, `world/*`, `sig.rs`, `pin.rs`, `hls_render.rs`). Local checks, all clean: `check-tests-in-ci.py`, `check-spec-refs.py`, `check-wai-section-refs.py`, `check-no-vanity-metrics.sh`, `check-comparative-prose.sh`, `check-orphan-sources.py`. ### Merge notes #296 (`wai/live-compose`) and #299 (`wai/live-record`) also touch SPEC §5, REGISTRY.md, the CHANGELOG, `wai-conformance.yml`, `registry.rs`, the registry export and the `live-conformance` exclusion line in `tests/int_kernels_vectors.rs`. This PR's edits there are additive (new lines, new list items). The exclusion line is a single line: whichever PR lands second should merge the reason to read "live events, rooms and takes, compositions, and captions". After rebasing, the export should be regenerated (`wai registry export`). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 8 commits 2026-10-17 11:26:51 -04:00

wai: timed_text, the cue model, WTX1 revisions, WebVTT, IMSC1, timelines, 608 and the disclosure label (WIP) ... ca13bffd68

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: timed_text corpus generator, vectors binary and test (WIP) ... 6bc4480db8

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: live-conformance/captions from fixed seeds (WIP) ... 8a11294aa9

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: timed-text.md, the extension text (WIP) ... 4b0f7902ab

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: registry, SPEC §5, CHANGELOG, CI step and timed_text_verify.py for wai.text.timed (WIP) ... 28f5d04800

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: registry export for wai.text.timed; disclosure_mix gains a reviewed-only model ... 2afd087b92

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: timed_text, clippy's collapsible-if, multiple-of and type lints ... 5322493a43

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: timed-text.md, the model digest's carriage, 708's characters and the source-type section said exactly ... 053bb285d6

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot added 2 commits 2026-10-17 11:33:56 -04:00

wai: IMSC1 documents carry ttp:profile on tt (IMSC 1.0.1 §6.9) beside conformsToStandard ... 7954b98e77

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: live-conformance/captions regenerated: every IMSC1 document carries ttp:profile ... 06f006bff9

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot referenced this pull request from a commit 2026-10-17 16:47:07 -04:00

Merge wai/live-record (#299, with main) into wai/timed-text; captions beside compose and recording

dcharlot added 18 commits 2026-10-17 16:47:07 -04:00

wai: live_record — recording manifest, recording verification, clip records and range proofs (WIP) ... 4af0b3f5dd

output_tree gains audit_path, range_proof and verify_range: RFC 6962's PATH taken over a run
of leaves, whose one-leaf case is the audit path node for node, tested against RFC 9162
§2.1.3.2's check.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: live_record corpus generator (WIP) ... b50389ecf5

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: live-conformance/recordings from fixed seeds, and tests/live_record.rs ... 123e2f4139

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: extensions/live-recording.md, and the recordings corpus README ... 8652abc0ec

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: live_record_verify.py, the recordings corpus's second implementation ... 9d16b7dbd2

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: a recording manifest states where the package is kept, held to the room's region pin ... 3ad589b1b8

live-rooms §3.7 said the recording extension checks a recording's storage against the pin;
WRM1 now carries region and the verifier refuses one outside it (store_outside_region).
live-events §8.6, live-rooms §3.7 and capture-binding §1 point at live-recording.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: wai.live.recording in the registry, SPEC §5, CI and CHANGELOG; the corpus regenerated with region ... 4560dac423

The registry lists the record and its magics WRM1 (§4.1) and WCL1 (§5.1); the
independent-verifiers job runs the module's tests, tests/live_record.rs and
live_record_verify.py; the workflow's path lists name the extension; int_kernels_vectors'
live-conformance exclusion names recordings and clips. live-conformance/recordings is
regenerated from its seeds with the manifest's region and the outside_region vector.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: registry export for wai.live.recording ... 4c356c1f59

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: live-recording names each composition receipt by its digest; two wording fixes ... f7d4c65d2e

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: recordings corpus — a session with a composite component, and its digest mismatch ... dd2710ef14

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: live_record — four clippy findings ... 1ac7b15ab8

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: live-conformance/recordings regenerated with the composite vectors ... 7e620720a8

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: a recording's signer holds the record power at every instant of its span ... 901a97f1ca

A principal other than the room authority may sign WRM1 only under a grant with the record
power in effect over the whole span: effective at or before its start, with no leave,
grant_exchange or expiry taking the power away before its end. A refusal keeps the code
recording_signer_not_permitted and states the first instant of the span without the power,
at ["signer", t]. Since a recording holds the room's chains whole, the span now binds the
takes the package holds (each start and closing commitment within it) and the takes
complete counts, not every context record, so a recorder that gained the power partway
seals a span that starts at its grant. New vectors: late_recorder_span (stands),
signer_power_late, signer_removed, signer_power_exchanged, signer_grant_expired and
signer_grant_revoked (refused at the room: a key-log revocation is not bounded by a time).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai: live-conformance/recordings regenerated with the record-power vectors ... 99029a6d33

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Merge origin/main into wai/live-record; live-compose and live-recording side by side ... 9c891740f9

Main gained wai.live.compose (#296), which edits the same lines: SPEC §5's live-records
list and its paragraph, REGISTRY's list, wire-identifier table and magics, the registry's
capability constants, statuses and wire rows, Cargo features, binaries and tests, the
module list, the live-conformance README, the int_kernels_vectors reason and the
independent-verifiers job. Both are kept, compose before recording; the two CI steps stay
separate. capabilities.json is regenerated by wai registry export.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai-rs: the live-record binary and test keep their own [[bin]] and [[test]] tables ... 01cb76a4e4

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai registry export: wai.live.compose and wai.live.recording both listed ... 4ece596222

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Merge wai/live-record (#299, with main) into wai/timed-text; captions beside compose and recording ... e2555d66e6

#303 is stacked on #299: this branch now carries main (wai.live.compose, #296) and
wai.live.recording. The same lines conflict as in #299's merge: SPEC §5's live-records
list and paragraphs, REGISTRY's list, wire table and magics, the registry's capability
constants, statuses and wire rows, Cargo features, binaries and tests, the module list,
the live-conformance README, the int_kernels_vectors reason and the independent-verifiers
job. All are kept, compose, recording, then timed text; the three CI steps stay separate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit 8c3f3f406f into main 2026-10-18 00:37:23 -04:00

dcharlot referenced this pull request from a commit 2026-10-18 00:37:23 -04:00

Merge pull request 'wai: timed text and live captions, wai.text.timed (L8)' (#303) from wai/timed-text into main

dcharlot referenced this pull request 2026-10-18 00:37:34 -04:00

wai: wai.live.recording — a room session as one package, and provable clips (L7a) #299

dcharlot referenced this pull request from a commit 2026-10-18 00:40:02 -04:00

Merge origin/main into wai/d2c-m8; CHANGELOG keeps every entry

dcharlot referenced this pull request 2026-10-18 07:41:25 -04:00

joule-moq-relay: a MOQT draft-22 relay over QUIC, WebTransport and a WebSocket fallback #293

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!303

No description provided.