task-fidelity: name the contract for media a model consumes, not a hash #83
Loading…
Reference in a new issue
No description provided.
Delete branch "wai/task-fidelity"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
determinism-tiers.mdanswers one question: does the sink reproduce the same bytes? A growing class of coded media does not answer that question at all — and WAI can register these capabilities today without being able to say what conformance means for them.ReplicateClass::Behavioralis a disclaimer, not a criterion. "Task-equivalent but not bit-equal" is unstateable, so an implementation either overclaims a reproducibility tier it does not meet, or says nothing.Three arrivals, none of them ours
Why a sibling file, not a third tier
The two tiers in
determinism-tiers.mdare reproducibility guarantees: a sink can check them from the bytes it received. A task claim cannot be checked that way — it needs a model, a corpus, and a test run. Listing them together would blur the one property that file currently keeps clean.The claim
A five-tuple, meaningless with any element missing:
model,test,tolerance(τ),observed(δ),evidence.modelandtestMUST be content addresses. "Tested on a standard detector" is unfalsifiable; a hash is not.Evidence grading
Modelled on
energy-measurement.md§2, on the same principle that what separates the rungs is provenance and scope, not accuracy:Measured/Transferred/Asserted, no promotion in transit, and a verifier that cannot resolvemodelandtestMUST treat the claim asAssertedwhatever it says.§4 states the limitation rather than hiding it: this is a graded assertion, not a verification, and a relying party reading it as the latter has misread it.
FCM's split budget — named, not solved
The cut point N makes the origin/sink work partition a wire parameter, which breaks
joules_microas oneu64for one producer. A cut-point-aware receipt needs joules either side plus a network-identity hash making the halves provably compose — and that third piece is a carriage problem WAI has not solved. Until then such a claim MUST state the cut point and MUST NOT report a single joule figure as if one producer did all the work.No new wire format, no new receipt mechanism. Second item in the #80 ingest queue, and the second of only two with no external dependency (the first was #81).
🤖 Generated with Claude Code
`determinism-tiers.md` answers one question — does the sink reproduce the same bytes? A growing class of coded media does not answer that question at all, and WAI can register these capabilities today without being able to say what conformance means for them. `ReplicateClass::Behavioral` is a disclaimer, not a criterion; "task-equivalent but not bit-equal" is unstateable, so an implementation either overclaims a reproducibility tier it does not meet or says nothing. Three independent arrivals, none of them ours: * ISO/IEC 23888-2 (VCM, DIS, FDIS planned late 2026) codes video AND descriptors, optimised for machine-task performance after decoding. * ISO/IEC 23888-4 (FCM, CD) codes intermediate feature tensors from inside a network — no pixels are reconstructed at any point, so there is no medium to hash. * A vendor markets compression as safe for machine consumption when the behavioural change is statistically indistinguishable from the variance the model already tolerates — with, by its own account, no shared framework for the claim. Deliberately a SIBLING file, not a third tier. The two tiers in `determinism-tiers.md` are reproducibility guarantees a sink can check from the bytes it received. A task claim cannot be checked that way — it needs a model, a corpus and a test run. Listing them together would blur the one property that file keeps clean. The claim is a five-tuple and is meaningless with any element missing: model, test, tolerance τ, observed δ, evidence class. `model` and `test` MUST be content addresses — "tested on a standard detector" is unfalsifiable, a hash is not — and a tolerance without an observation is a target, not a claim. Evidence is graded the way `energy-measurement.md` grades acquisition, on the same principle that what separates the rungs is provenance and scope, not accuracy: Measured / Transferred / Asserted, no promotion in transit, and a verifier that cannot resolve model and test MUST treat the claim as Asserted whatever it says. §4 states the limitation rather than hiding it: this is a graded assertion, not a verification, and a relying party that reads it as the latter has misread it. §5 names FCM's split budget without pretending to solve it. The cut point N makes the origin/sink work partition a wire parameter, which breaks `joules_micro` as one u64 for one producer. A cut-point-aware receipt needs joules either side plus a network-identity hash making the halves provably compose — and that third piece is a carriage problem WAI has not solved. Until then such a claim MUST state the cut point and MUST NOT report a single joule figure as if one producer did all the work. No new wire format, no new receipt mechanism. Second item in the landscape survey (#80) ingest queue, and the second of the two with no external dependency. Co-Authored-By: Claude Opus 5 <[email protected]>4f7360a71900dcf1f1aa