open-standards/wai/wai-sink-swift/README.md

wai-sink-swift — the native sink for the platform player, in Swift

The native sink of extensions/native-sink.md for phones and tablets, televisions and set-top boxes, and desktops whose native player an application reaches from Swift. The player is not changed and decodes on the platform's media engine; the sink sits on the path of its requests and, for one playback session:

• verifies a media playlist against its playlist receipt and the chain of its live revisions (http-delivery §8.6), and every object whose name states its content hash against that hash, and withholds from the player whatever does not verify;
• signs, under a fresh key per session, stream-audit's acknowledgements of every object received, a presentation claim for each marked ad, and the session record, and posts the bundle to the owner's collector (wai-audit serve), with its Event-mode CMCD while it plays;
• ends a session nobody can see, or whose viewer has gone idle, and stops the player's requests.

Everything that decides is libwai's C interface (native-sink §9), the same on every native platform; this package is the platform's half: the loopback relay, the player, and the signals of who can see it.

Layout

path	what
Package.swift	the package: library WaiSink, binary target WaiFFI (the C interface): a release's framework by URL and checksum, or the one built here
Sources/WaiSink/Sink.swift	WaiSink: one session, every call of the C interface, JSON in and out
Sources/WaiSink/Relay.swift	WaiRelay: the loopback server the player is given (native-sink §4.4)
Sources/WaiSink/PlayerSession.swift	WaiPlayerSession: the platform player through the relay, observed every 200 ms (§5, §6)
Sources/WaiSink/Attention.swift	WaiViewAttention: whether the application is shown, the player view's area in view, the viewer's input (§6.2); WaiFixedAttention for hosts that decide it themselves
Sources/WaiSink/Collector.swift	WaiCollector: v1/sessions and v1/cmcd (stream-audit §12)
Sources/WaiSink/IntDecoder.swift	WaiIntDecoder: WAI's integer capabilities decoded in the application's process, on the device's GPU where it passes the start-up exactness check, with the report naming the unit (native-sink §13)
scripts/build-xcframework.sh	builds build/WaiFFI.xcframework from ../wai-rs
scripts/package-release.sh	the framework zipped, and the copy of the package that names the zip by URL and checksum, for a release
scripts/check-release-package.sh	that copy resolved, built and run as an application uses it, and refused with a wrong checksum
scripts/test-simulator.sh	runs the tests hosted in TestHost/ in a fresh simulator it creates, boots and deletes, every step under a hard limit
Tests/WaiSinkTests	the corpus through the wrapper; the relay over the reference package's floor; the player end to end; the integer decode on the GPU and the CPU (IntDecodeTests)
TestHost/	a minimal application the same tests run inside on a phone or tablet, where no tool-hosted test bundle runs: an Xcode project that depends on this package by its path; consumers never see it

Use

import WaiSink

let cfg = WaiSink.Config(channel: "news-east", owner: "<the collector's key, 64 hex>")
let session = try WaiPlayerSession(url: URL(string: "https://media.example/ch/7/v.m3u8")!,
                                   config: cfg,
                                   attention: WaiViewAttention(view: playerView),
                                   collector: WaiCollector(base: URL(string: "https://collector.example/")!))
playerLayer.player = session.player     // the host shows the player as it always has
session.onEnd = { bundle, ended in /* posted already; kept here if the host wants it */ }
session.start()

For a stream the owner's packager seals (playlist receipts, segments named by their SHA-256, packaging §7), set requireSigned: true (or contentAddressed: true), and nothing the publisher did not sign reaches the decoder. For a stream another service delivers, leave both off: it is acknowledged and evidenced, never refused for what its names look like.

Decoding WAI's integer capabilities on the GPU

A host that plays a WAI capability itself, not through the platform's player, decodes it with WaiIntDecoder (native-sink §13): wai.neural.int_hyper, wai.neural.int_synth, wai.neural.int_mlicpp, wai.video.int_hyper, wai.video.int_cond, wai.video.int_motion, wai.audio.int_codec and wai.splat.int_codec, as whole payloads or WAI1 envelopes.

let decoder = try WaiIntDecoder()                 // finds the GPU and checks it, once per process
print(try decoder.gpu())                          // state "ready", the adapter, the translator
let out = try decoder.decode(capability: "wai.neural.int_hyper", payload: wih1,
                             set: ["entropy.json": e, "g_s.whm": gs, "h_s.whm": hs])
out.bytes        // RGB8 rows, the same bytes on every device and on the CPU
out.unit         // "gpu", or "cpu" where the GPU is not used, with the reason in out.dispatch

The GPU runs the integer operator set's WGSL kernels through libwai's Metal backend, integer arithmetic only, every value in a width the graph's bounds certificate proves it fits. It runs nothing until it has passed the start-up exactness check: WGSL's integer semantics, the arithmetic probe, graphs at every width and at the edge of the narrow kernels' bound, and a whole decode, each giving the CPU's integers. A GPU that differs anywhere is not used, and every decode's report says so (dispatch_unit_inexact); the decode then runs on the CPU and gives the same bytes. WaiIntDecoder(gpu: .off) decodes on the CPU alone. The library links Metal, QuartzCore and CoreGraphics for it; Package.swift declares them.

Build and test

On the build host (Xcode with the phone, television and desktop SDKs; Rust with the targets scripts/build-xcframework.sh names):

cd wai/wai-sink-swift
scripts/build-xcframework.sh                 # every slice; WAI_SINK_SLICES="macos ios-sim" for some
swift test                                   # the desktop: corpus, relay, player end to end, integer decode
scripts/test-simulator.sh ios                # the same tests, hosted in TestHost, in a fresh phone simulator
scripts/test-simulator.sh tvos               # and in a fresh television simulator

On a phone or tablet, the tests run hosted in TestHost/ (signed with the team configured in Xcode):

xcodebuild build-for-testing -project TestHost/WaiSinkTestHost.xcodeproj -scheme WaiSinkTestHost \
  -destination 'generic/platform=iOS' -allowProvisioningUpdates DEVELOPMENT_TEAM=<team>
xcodebuild test-without-building -xctestrun <the .xctestrun it wrote> -destination 'platform=iOS,id=<device>'

The player tests make their presentation at run time with the platform's own encoder (AVC, ISO/IEC 14496-10, in fragmented MP4, a segment a second), serve it from an origin on loopback, and play it through the relay with a video output attached, since no layer is on screen in a test.

Distribution

A release attaches WaiFFI-<version>.xcframework.zip, the framework with every slice, and wai-sink-swift-<version>.zip, this package with its Package.swift naming that zip by its download URL and its SwiftPM checksum (.binaryTarget(name: "WaiFFI", url: …, checksum: …), the archive's SHA-256, as swift package compute-checksum gives it). scripts/package-release.sh writes both. An application adds the package from that folder, as a local package or a copy in a repository of its own; SwiftPM fetches the framework from the URL and refuses it unless its checksum is the one named.

Before the release attaches them, scripts/check-release-package.sh resolves, builds and runs an application that depends on the package, the framework served from the release's own files through a mirror of its URL, and requires the package with one digit of its checksum changed to be refused; once they are attached, it does both again from the URL itself. In this repository, Package.swift names build/WaiFFI.xcframework while scripts/build-xcframework.sh has built it there, so the tests run against the libwai of their checkout.

The library holds no codec: the platform's player decodes, and its codecs are the platform's, licensed with it. The integer decoder is WAI's own (wai-rs), with wgpu (MIT or Apache-2.0) for the GPU.

Energy

The platform's player decodes in a process of the platform's, on its fixed-function decoder, which no counter an application can read covers. The session therefore states the sink decode and the display as the unmetered marker (native-sink §7), never the application's own counter as either. A decode through WaiIntDecoder runs in the application's process: on the CPU a counter of the process covers it; on the GPU a CPU counter does not, and the sink decode stays the marker unless the host holds a figure that covers the GPU's work (native-sink §13.4).