wai-quantum: a study of the co-design, every measure the engine's #512
Loading…
Reference in a new issue
No description provided.
Delete branch "wai/quantum-study-2"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Replays #509 unchanged (same tree, same base) with a fresh commit, because its CI failed on the runners, not on the change: 19 wai-quantum jobs on the Framework
linuxrunner hit a cache whose crate sources a nightly prune had deleted (fixed on the host), one arm64 job ran out of disk while three jobs ran, and one timing test failed in joule-moq-relay, which this change does not touch.The measurement harness the co-design report puts under every feature. Each feature sits behind a flag and is compared against a control, in a study whose protocol is sealed before any data. Every measure is the engine's, computed again from what the participant made; none is a model's judgement.
Engine (
engine::study)study::FLAGS), the control, and the analysis (resamples, seed, level). It is checked, then sealed with the researcher's key before any data.CLI and MCP
wai-quantum study catalog | new | assign | measure | cohort, with--outto seal the report.study_measureandstudy_cohort. Probes are not offered to agents, because a probe is done without the assistant.Studio: the Study pane
Checked on the build host, at low priority:
engine::: 119 pass, 4 of them new;The CLI sealed a real protocol and listed its catalog. The public-names check is clean.
Two e2e fixes the new pane made necessary:
🤖 Generated with Claude Code
The measurement harness the co-design report puts under every feature: each feature behind a flag, compared against a control, in a study whose protocol is sealed before any data. - engine::study: - Protocol: the study, hypotheses, primary measure, conditions (each a setting of FLAGS), control and analysis (resamples, seed, level), checked and sealed with the researcher's key before any data; - assign: the protocol's digest and the participant's key hashed together, so the condition is fixed and anyone can check it; - seal_record / open_record: the participant's record sealed with their key, refused if changed, for another study, sealed by another key, or in a condition its key is not assigned; - measure: the report's eight, each computed again from the record: quality of the edits proved equal (and its trend), diversity (Vendi), novelty, selection skill against chance, calibration (mean distance, slope), unassisted probe scores (four probes, each a workbench specification), ownership (five items, 1 to 7) beside the authorship share, and convergence; - cohort: per condition the means, each condition's difference from the control on the primary measure with a seeded bootstrap interval, Cronbach's alpha, the Vendi score of the final designs, one record per participant; call and call_sealing for hosts. - The command line: study catalog | new | assign | measure | cohort (--out seals the report). - MCP: study_measure and study_cohort. The probes are not offered to agents: a probe is done without the assistant. - The Studio: the Study pane (Features, each switchable off and held where a study's condition names it; Load a protocol, shown before Take part; a record kept once asked: choices in Alternatives and the Workbench with the engine's best, How much is this yours?, probe tasks with the assistant paused; My record; Export my record, sealed with the browser's key). A condition holds only the features it names; leaving restores the reader's own. - Bindings: QuantumEngine::study in the page (it seals with the page's key), study in the desktop app. - CI: wai-web's native engine tests count 14. - The e2e brings the editor's first lines into view before reading its gutter, and counts the taste profile from the choices it already holds. Co-Authored-By: Claude Opus 5.5 <[email protected]>