task-fidelity: name the contract for media a model consumes, not a hash #83

Merged

dcharlot merged 1 commit from wai/task-fidelity into main 2026-09-10 12:20:55 -04:00

Conversation 0 Commits 1 Files changed 2 +134

2026-09-10

Transaction Science

task-fidelity: name the contract for media a model consumes, not a hash ...

`determinism-tiers.md` answers one question — does the sink reproduce the same
bytes? A growing class of coded media does not answer that question at all, and
WAI can register these capabilities today without being able to say what
conformance means for them. `ReplicateClass::Behavioral` is a disclaimer, not a
criterion; "task-equivalent but not bit-equal" is unstateable, so an
implementation either overclaims a reproducibility tier it does not meet or says
nothing.

Three independent arrivals, none of them ours:

  * ISO/IEC 23888-2 (VCM, DIS, FDIS planned late 2026) codes video AND
    descriptors, optimised for machine-task performance after decoding.
  * ISO/IEC 23888-4 (FCM, CD) codes intermediate feature tensors from inside a
    network — no pixels are reconstructed at any point, so there is no medium to
    hash.
  * A vendor markets compression as safe for machine consumption when the
    behavioural change is statistically indistinguishable from the variance the
    model already tolerates — with, by its own account, no shared framework for
    the claim.

Deliberately a SIBLING file, not a third tier. The two tiers in
`determinism-tiers.md` are reproducibility guarantees a sink can check from the
bytes it received. A task claim cannot be checked that way — it needs a model, a
corpus and a test run. Listing them together would blur the one property that
file keeps clean.

The claim is a five-tuple and is meaningless with any element missing: model,
test, tolerance τ, observed δ, evidence class. `model` and `test` MUST be content
addresses — "tested on a standard detector" is unfalsifiable, a hash is not — and
a tolerance without an observation is a target, not a claim.

Evidence is graded the way `energy-measurement.md` grades acquisition, on the
same principle that what separates the rungs is provenance and scope, not
accuracy: Measured / Transferred / Asserted, no promotion in transit, and a
verifier that cannot resolve model and test MUST treat the claim as Asserted
whatever it says.

§4 states the limitation rather than hiding it: this is a graded assertion, not a
verification, and a relying party that reads it as the latter has misread it.

§5 names FCM's split budget without pretending to solve it. The cut point N makes
the origin/sink work partition a wire parameter, which breaks `joules_micro` as
one u64 for one producer. A cut-point-aware receipt needs joules either side plus
a network-identity hash making the halves provably compose — and that third piece
is a carriage problem WAI has not solved. Until then such a claim MUST state the
cut point and MUST NOT report a single joule figure as if one producer did all
the work.

No new wire format, no new receipt mechanism. Second item in the landscape survey
(#80) ingest queue, and the second of the two with no external dependency.

Co-Authored-By: Claude Opus 5 <[email protected]>

00dcf1f1aa

• View at this point in history

2026-09-10 10:24:58 -04:00