open-standards/wai/extensions/score.md

WAI Extension: Score (Music as Instructions)

Status: Draft. Phase 6 — the deepest audio lever. Ship the composition, not the recording: a score of note events the sink synthesizes. Conformance is mixdown-equivalence, the music counterpart of spatial audio. Reference impl: the score feature of wai-rs; corpus: score-conformance/; live sink: wai.transaction.science/score. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

1. Scope and model

A wai.audio.score object carries no recorded audio — only note events. Where wai.audio.scene mixes recorded sound objects, a score is reconstructed from instructions: a deterministic reference synth turns the notes into a canonical PCM mixdown of one channel or two, and a sink with a richer instrument (soundfont, neural) renders the same score for presentation. A note may carry a voice (§2.4): its place between the two channels, curved envelope segments, a low-pass filter that follows its envelope, a pitch glide, a noise seed and echoes. It is lever 3 (instructions-at-the-sink) for music — a minute of music is kilobytes of notes, not megabytes of samples.

In scope: the container, the note model, the deterministic reference synthesis, the mixdown-equivalence conformance criterion, the receipt profile. Out of scope: a transport; the sink's instrument (soundfonts, neural synths — presentation, not conformance); MIDI-file import (an authoring concern).

Relationship to the core spec: mirrors spatial audio. The reconstruct runs in the wai.det.fixed64 floor; media takes the value "audio"; wai.audio.score is authoritative.

2. Container (WSCR)

"WSCR" | u16 n_sections | section table (u8 kind, u32 off, u32 len) | sections

kind	section	required
0x01	contract (canonical JSON)	REQUIRED
0x02	note events	REQUIRED
0x03	voices, one per note (§2.4)	OPTIONAL

off counts from the end of the section table; the reference writer writes the sections in this order, each padded with zero bytes to a multiple of 8. A score with a voices section is voiced. A reader skips a section of a kind the table does not name: it does not enter the mixdown. A score of the first revision's two sections renders as it always has (§3.2).

2.1 Contract

{ "channels": 1, "duration": 34000, "numeric": "wai.det.fixed64",
  "sample_rate": 22050 }

duration is the rendered mixdown length in samples of each channel (sample frames). channels is 1 or 2; a reader refuses any other count. A mix of two channels is the left and the right, interleaved (§3.4).

Reader limit. A reader MUST NOT refuse a score for its length while duration × channels is at most 134 217 728 (2²⁷, the audio limit of SPEC §2), and MAY refuse one past it (duration_too_long), when it reads the contract, before it renders anything. A conforming encoder writes no score past it.

2.2 Notes

n_notes u32, then per note (65 bytes): start u64 | dur u64 | phase_inc i64 Fx | gain i64 Fx | waveform u8 | attack u64 | decay u64 | sustain i64 Fx | release u64.

phase_inc is the per-sample phase advance freq / sample_rate, authored off-line from the pitch — so the floor never evaluates an exponential to turn a note into a frequency (the same authoring discipline as the avatar quaternion). waveform selects a reference timbre (§3.1):

code	timbre
0	saw, piecewise-linear
1	square, piecewise-linear
2	triangle, piecewise-linear
3	25 %-pulse, piecewise-linear
4	saw, band-limited (each step corrected, PolyBLEP)
5	square, band-limited
6	25 %-pulse, band-limited
7	sine (a refined parabola)
8	noise (a generator's value held between wraps of the phase)

A note of another code is silent. The band-limited timbres are the piecewise-linear ones with the step at each discontinuity spread over the samples beside it, so a high note does not fold its harmonics back below the Nyquist frequency.

2.3 Authoring form (informative)

wai score build writes a WSCR from a JSON document that holds every field as an integer, so the committed document is the definition and no platform's floating-point pow can change a pitch:

{ "sample_rate": 22050, "channels": 1, "duration": 34000,
  "numeric": "wai.det.fixed64",
  "notes": [ { "start": 0, "dur": 3500, "phase_inc": 50960238,
               "gain": 1717986918, "waveform": 0, "attack": 220,
               "decay": 600, "sustain": 2791728742, "release": 1500 } ] }

A voiced score of two channels at 48 000 Hz: one band-limited saw (A4, phase_inc = round(440 / 48 000 × 2³²)), panned half left, its decay and release curved, through two low-pass sections that open with its envelope, with three echoes a quarter second apart crossing from right to left:

{ "sample_rate": 48000, "channels": 2, "duration": 72000,
  "numeric": "wai.det.fixed64",
  "notes": [ { "start": 0, "dur": 24000, "phase_inc": 39370534,
               "gain": 1288490189, "waveform": 4, "attack": 480,
               "decay": 24000, "sustain": 2147483648, "release": 9600,
               "voice": { "pan": -2147483648, "decay_curve": 4,
                          "release_curve": 4, "lp_poles": 2,
                          "lp_a": 214748365, "lp_env": 1717986918,
                          "echo_taps": 3, "echo_delay": 12000,
                          "echo_gain": 1717986918,
                          "echo_pan": 3006477107 } } ] }

phase_inc, gain, sustain and a voice's pan, lp_a, lp_env, glide_to, echo_gain and echo_pan are the raw Q32.32 bits (a pan negative for the left). numeric may be left out. voice is optional on each note, and each of its members is optional, a member left out taking the default voice's value (§2.4); a score in which any note has a voice is voiced, and a note of it with none takes the default voice. The tool refuses a number with a fraction or an exponent, a member it does not name, a member given twice (readers differ on which of the two values they keep), a channel count other than 1 or 2, a duration × channels past 2²⁶, a phase_inc or a glide_to of a whole cycle or more, a gain or sustain above 1.0, a waveform above 8, a voice member out of the range §2.4 gives, a pan or an echo_pan other than 0 in one channel, a glide with no glide_to, an echo of no delay, and a note whose release, or last echo, would end after duration, so the mixdown always holds every note whole. score-conformance/notes/ holds each vector in this form, and wai score build turns it back into the committed WSCR byte for byte.

2.4 Voices

A voices section holds one 73-byte record per note, in the notes' order, and nothing else: its length is n_notes × 73 (it has no count of its own). Each record:

pan i64 Fx | attack_curve u8 | decay_curve u8 | release_curve u8 | lp_poles u8 |
lp_a i64 Fx | lp_env i64 Fx | glide_to i64 Fx | glide u64 | seed u32 |
echo_taps u8 | echo_delay u64 | echo_gain i64 Fx | echo_pan i64 Fx

field	meaning	range
pan	the note's place in a two-channel mix, −1.0 left to 1.0 right	−1.0 to 1.0; 0 in one channel
attack_curve, decay_curve, release_curve	the exponent of each envelope segment (§3.3); 1 is a line	1 to 8
lp_poles	one-pole low-pass sections in series; 0 is none	0 to 4
lp_a	each section's coefficient at an envelope of 0	0 to 1.0
lp_env	what the coefficient gains per unit of envelope	0 to 1.0
glide_to	the phase increment the note glides to	0 to below 1.0
glide	the glide's length in samples; 0 is none (glide_to is then not read)	any
seed	the noise generator's seed (waveform 8)	any
echo_taps	how many echoes of the note	0 to 8
echo_delay	samples from one echo to the next	1 to 2²⁷ when echo_taps > 0
echo_gain	each echo's level against the one before it	0 to 1.0
echo_pan	the pan of the odd echoes; the even ones sit at its negation	−1.0 to 1.0; 0 in one channel

A reader refuses a voices section of another length, and a record with a field out of its range. The default voice is pan 0, the three curves 1, lp_poles 0, lp_a 1.0, lp_env 0, glide_to 0, glide 0, seed 0, echo_taps 0, echo_delay 0, echo_gain 0 and echo_pan 0. A voiced note sounds from start to its end, start + dur + release + echo_taps × echo_delay (each sum exact); a note of a score with no voices to start + dur + release.

3. Reference synthesis (trig-free)

Every value is a wai.det.fixed64 Q32.32 number: + and − saturate, × and ÷ are the profile's (computed in 128 bits, rounded toward negative infinity, then saturated), a negation of the least value is the greatest, and √ is the profile's (the floor square root of the raw bits times 2³², and 0 for a value of 0 or below). An expression is evaluated as written, × and ÷ before + and − and otherwise left to right, each operation rounded as it is made; 1 is 1.0, ½ is 2³¹ raw, ¼ is 2³⁰ raw, and an integer n of samples used as a value is n · 2³² raw. The whole path is + − × ÷ √ in the Fx floor — bit-identical on every machine.

3.1 Oscillators

A note's phase p starts at 0. At each sample the oscillator gives a value from p and the sample's increment dt (the note's phase_inc, or its glide's, §3.3); then p ← p + dt, and where p ≥ 1, p ← p − 1 once (the phase wraps).

• 0 saw: 2·p − 1. 1 square: 1 for p < ½, else −1. 2 triangle: 4·p − 1 for p < ½, else 3 − 4·p. 3 pulse: 1 for p < ¼, else −1.
• The step correction B(t, dt): 0 for dt ≤ 0; for t < dt, with x = t ÷ dt, x + x − x·x − 1; for t > 1 − dt, with x = (t − 1) ÷ dt, x·x + x + x + 1; otherwise 0. With W(q) = q − 1 for q ≥ 1 and q otherwise: 4 saw: 2·p − 1 − B(p, dt); 5 square: s + B(p, dt) − B(W(p + ½), dt), s the square's value; 6 pulse: s + B(p, dt) − B(W(p + ½ + ¼), dt), s the pulse's value.
• 7 sine: u = 2·p − 1, y = 4·u·(1 − |u|), y ← P·(y·|y| − y) + y with P = 966 367 642 raw (0.225), and the value is −y (sin 2πp to within a thousandth).
• 8 noise: the value held, from a 32-bit state s, set to the voice's seed (0 in a score with no voices) when the note starts, and advanced once then and again each time the phase wraps: s ← (1 664 525 · s + 1 013 904 223) mod 2³², the held value then ((s >> 8) − 2²³) · 2⁹ raw, in [−1, 1). A phase_inc just below 1 draws a new value every sample after the first.

3.2 A score with no voices

Each note's envelope at t samples into it is the linear ADSR: for attack > 0 and t < attack, t ÷ attack; else for t < attack + decay, 1 − (1 − sustain) × (t − attack) ÷ decay (or sustain when decay is 0); else for t < dur, sustain; else for release > 0 and t < dur + release, sustain × (1 − (t − dur) ÷ release); else 0. The note's sample is osc × env × gain, for t from 0 while start + t is below both start + dur + release and duration. In one channel it is added to the mix; in two, each channel adds it times √((1 − 0) / 2) (the centre, §3.3's pan of 0).

3.3 A voiced score

For each note and its voice, with F(t, n) = ⌊t · 2³² / n⌋ raw (exact, for t < n) and v^k = v multiplied by v k − 1 times in turn:

1. Glide. With glide = 0, dt = phase_inc. Otherwise dt = phase_inc + ⌊(glide_to − phase_inc) · min(t, glide) / glide⌋ raw, the difference of the raw values times min(t, glide), divided by glide toward negative infinity: the increment moves in a line to glide_to and holds there.
2. Envelope. The held level H(t): for attack > 0 and t < attack, 1 − (1 − F(t, attack))^attack_curve; else, with u = t − attack, for decay > 0 and u < decay, sustain + (1 − sustain) × (1 − F(u, decay))^decay_curve; else sustain. The envelope E(t) is H(t) for t < dur; for release > 0 and t − dur < release, H(dur) × (1 − F(t − dur, release))^release_curve; else 0. The release starts from the level the note had at note-off, so a note let go during its attack or decay does not jump.
3. Low-pass. With lp_poles = k > 0, the coefficient is a = lp_a + lp_env × E(t), held to [0, 1]; each section i of 1..k, its state 0 when the note starts, makes y_i ← y_i + a × (x − y_i) from its input x (the oscillator's value for the first, the section before's y for the rest), and the last section's y is the filtered value. With lp_poles 0 the oscillator's value passes.
4. Gain. x = filtered × E(t) × gain.
5. Pan. The gains of a pan q are (L, R) = (√((1 − q) ⁄ 2), √((1 + q) ⁄ 2)), each ⁄ 2 the raw value shifted right one bit (floor): L² + R² is 1 at every pan, to within the rounding, and the centre is 3 037 000 499 raw (√½) on each side. In two channels the note adds x × L to the left and x × R to the right at frame start + t, by its pan; in one channel it adds x.
6. Echoes. Echo j of 1..echo_taps has the level g_j = g_{j−1} × echo_gain, g_0 = 1, and the pan echo_pan for odd j, −echo_pan for even j. After the note's own sample, each echo in order adds x × g_j at frame start + t + j × echo_delay, by its pan (in one channel, x × g_j), while that frame is below duration.

t runs from 0 while start + t is below both start + dur + release and duration; each sample the oscillator's value is taken, then the phase advances by that sample's dt (§3.1).

3.4 Mixdown

The mix is duration × channels values, each 0 to start, two channels interleaved left, right (frame f's left is value 2f, its right 2f + 1). The notes are rendered in their order; within a note, sample by sample as above, each addition saturating as it is made. Each value is then quantized to i16 (round(v · 32767), half away from zero, v first clamped to [−1, 1]) in integer space.

4. Conformance — mixdown-equivalence

Given the same wai.audio.score, a conforming sink MUST reconstruct the identical canonical i16 PCM via the reference synth, hence the identical BLAKE3 — on every machine, no tolerance parameter.

mixdown_hash = BLAKE3("wai:score-mixdown\x01" || i16le PCM)
score_hash   = BLAKE3("wai:score\x01" || contract || notes [|| voices])   // identity

The PCM is the mix of §3.4, two channels interleaved. contract, notes and voices are the sections' bytes; a score with no voices section hashes as the first revision's did. A conformance vector also states pcm_sha256, the SHA-256 of the same i16le PCM (the audio buffer of SPEC §7 Digests), so a verifier that holds only SHA-256 checks the samples. wai score render prints both, with samples the frames and channels the contract's, and, with --pcm, writes the samples.

A sink MAY play a soundfont/neural instrument instead; that is presentation and does not affect conformance, exactly as the audio-scene HRTF render or the worlds renderer does not.

In a package (packaging §5, Components beside the lead), a score is cut at the lead's piece starts: each piece a WSCR of the notes that sound in it (to their end, §2.4, echoes included), its starts relative to the piece and their voices with them, whose mixdown is the whole's for the piece's frames; its entry states the SHA-256 of that mixdown, and its floor is the mixdown as FLAC in the score's channels.

5. Receipt (JWP profile)

score_hash is the identity; the group's objects are the mix PCM blocks (4096 frames each, every channel of them; the last may be short), Merkle-bound; one Ed25519 signature (verified as SPEC §7 Signatures requires) over score_hash + root + exact work (= duration × notes) + carried joules_micro; parent_receipt_hash chains — the worlds/audio receipt shape reused. The figure's acquisition class rides beside it as an optional signed label (energy-measurement §2.8): absent, the receipt keeps its legacy bytes and the figure is unlabelled; present, it is inside the signature. The reference meter labels every figure it measures OnChipCounter, with its declared uncertainty.

Appendix

Mixdown-equivalence is spatial audio's criterion with the recorded objects replaced by note events: both render a deterministic i16 mix in the Fx floor and hash it. The score is the most extreme instructions-at-the-sink case in audio — there is no recording anywhere in the payload, only the composition. Sight, hearing, touch, motion, state, captured-content provenance — and now the composition itself — are one floor, one verification.