smart-byte: the pack is placeholders, and the spec mandates a suite that does not exist #99

Open

dcharlot wants to merge 3 commits from ci/smart-byte-pack into main

pull from: ci/smart-byte-pack

merge into: Transaction-Science:main

Transaction-Science:main

Transaction-Science:ci/macos-energy-widen

Transaction-Science:ci/neural-e2e-guard

Transaction-Science:ci/tier2-packs

Transaction-Science:wai-webcodecs-negotiation

Transaction-Science:ci/macos-energy-backend

Transaction-Science:feat/wai-video-workflow

Transaction-Science:ci/joule-code-pack

Transaction-Science:ci/jcp-pack

Transaction-Science:joulecontract/run-negative-vectors

Transaction-Science:ci/jouleclaw-arm64-debuginfo

Transaction-Science:ci/run-conformance-verifiers

Transaction-Science:mesh/cost-calibration

Transaction-Science:diagnose/grounding-backing

Transaction-Science:wai/energy-operating-point

Transaction-Science:wai/2030-transport-landscape

Transaction-Science:wai/constraints-and-unidirectional

Transaction-Science:wai/determinism-tier-covers-emitted-medium

Transaction-Science:wai/moq-streaming-format

Transaction-Science:wai/mpeg-ai-part6-mapping

Transaction-Science:wai/prior-carriage-and-derivations

Transaction-Science:wai/receipt-classes-and-key-discovery

Transaction-Science:wai/task-fidelity

Transaction-Science:wai/transparency-emission

Transaction-Science:wai/c2pa-emitter

Transaction-Science:compliance/eu-scale-pack

Transaction-Science:compliance/composite-grade-v2

Transaction-Science:jcp/erasure-tombstone-v2

Transaction-Science:jcp/avoided-energy

Transaction-Science:jcp/erasure-tombstone

Transaction-Science:jcp/environment-binding

Transaction-Science:proof/stub-on-the-wire

Transaction-Science:compliance/composite-grade

Transaction-Science:mesh/context-cost-profile

Transaction-Science:ar-1-on-main

Transaction-Science:jcp/mcp-meta-receipt

Transaction-Science:wai/energy-binding

Transaction-Science:jcp/energy-coverage-relanded

Transaction-Science:jcp/energy-coverage

Transaction-Science:wai/video-byte-equality

Transaction-Science:eoc/multi-tenant-allocation

Transaction-Science:compliance/banding-function

Transaction-Science:ci-standards-workspaces

Transaction-Science:sandbox/honest-tier

Transaction-Science:proof/joule-ceiling

Transaction-Science:chore/rust-1.98-and-deps

Transaction-Science:wai-jpegai-full-wasm

Transaction-Science:wai-3d-landscape

Transaction-Science:wai-binding-real-decode

Transaction-Science:wai-determinism-tiers

Transaction-Science:wai-jpegai-wasm-demo

Transaction-Science:wai-jpegai-integrate

Transaction-Science:fix-byte-exact-includes

Transaction-Science:ci-windows-shell

Transaction-Science:jpegai-dequant-derive

Transaction-Science:ci-rust-setup

Transaction-Science:fl2-land

Transaction-Science:ci/cross-workspace-check

Transaction-Science:ci-enable

Transaction-Science:jcp-training-receipt

Transaction-Science:joule-code/repair-grant-literals

Transaction-Science:jcp-gateway-flow-relay

Transaction-Science:jcp-receipt-flow-seal

Transaction-Science:jcp-runtime-flow-gate

Transaction-Science:jcp-flow-witness

Transaction-Science:jcp-flow-lattice

Transaction-Science:rust-toolchain-1.97.1

Transaction-Science:score-binary-on-320fa1e

Transaction-Science:sandbox/cred-injection-2

Transaction-Science:sandbox/credential-injection

Transaction-Science:corpus-embedder-s5

Transaction-Science:jcp/cite

Transaction-Science:receipts-corpus-connector

Transaction-Science:jcp/grant-caveats

Transaction-Science:feat/jpegai-derive-inputs

Transaction-Science:feat/jpegai-weight-tools

Transaction-Science:feat/jpegai-icci-validation

Transaction-Science:jcp/x402-example-settlement

Transaction-Science:jcp/x402-bridge

Transaction-Science:jcp/receipt-payload-hash

Transaction-Science:fix/jpegai-pih-flags

Transaction-Science:feat/jpegai-e2e-chain

Transaction-Science:feat/jpegai-e2e-z

Transaction-Science:feat/jpegai-ton

Transaction-Science:jcp/dual-ceilings

Transaction-Science:feat/jpegai-pih

Transaction-Science:feat/jpegai-bitstream

Transaction-Science:feat/jpegai-icci-net2

Transaction-Science:feat/jpegai-icci-net

Transaction-Science:feat/jpegai-dwt

Transaction-Science:feat/jpegai-color

Transaction-Science:score-binary-rerank

Transaction-Science:feat/jpegai-efl

Transaction-Science:feat/jpegai-lef

Transaction-Science:feat/jpegai-efn

Transaction-Science:feat/jpegai-lsbs

Transaction-Science:feat/jpegai-reconstruct

Transaction-Science:jcp/orchestrate-profile-mapping

Transaction-Science:jpegai-sigma-index

Transaction-Science:hyperscale-floorfix

Transaction-Science:feat/jpegai-pipeline-dequant

Transaction-Science:feat/jpegai-e2e-synthesis

Transaction-Science:recovery/jpegai-decoders

Transaction-Science:wai/jpegai-synthesis-executor

Transaction-Science:jouletag-standard

Transaction-Science:wai/jpegai-gate-fixes

Transaction-Science:wai/int-transformer-executors

Transaction-Science:wai/int-transformer-kernels

Transaction-Science:energy/rapl-nvml-hardware-fixes

Transaction-Science:fix/omni-build-and-gguf-tokenizer

Transaction-Science:jouletable-standard

Transaction-Science:jouleclaw/gguf-decoders

Transaction-Science:wai/confidentiality-freshness

Transaction-Science:joulehook-standard

Transaction-Science:honest-counter-provenance

Transaction-Science:amd-kds-adapter

Transaction-Science:evidence-derived-grounding

Transaction-Science:mapping-physically-rooted-clarify

Transaction-Science:attested-efficiency-demo

Transaction-Science:efficiency-energy-attestation

Transaction-Science:mapping-attested-kernel

Transaction-Science:energy-counter-attestation

Transaction-Science:efficiency-surface

Transaction-Science:efficiency-spec

Transaction-Science:harness-econ

Transaction-Science:federation-quic-on-main

Conversation 0 Commits 3 Files changed 5 +314 -3

dcharlot commented 2026-09-19 05:36:05 -04:00

Owner

Copy link

smart-byte/conformance/v1 is published for third parties to certify against.
Its gate — .github/workflows/conformance.yml, named "smart-byte conformance
gate" — runs jq over it: valid JSON, top-level array, name and
description present, names unique within a file.

Those are well-formedness checks. No value in the pack has ever been compared
against the implementation, and smart-byte-rs has no conformance crate that
reads it.

This is documented, not hidden

conformance/v1/regeneration.md is candid that the expected_* fields are
placeholders "not yet cryptographically computed". Nobody concealed anything.

What changed is its precondition. The doc says to regenerate "when
smart-byte-rs lands", describing it as a "private repo, to be open-sourced
alongside spec v1.0 freeze". smart-byte-rs has landed — 22 crates, in the
standards-build matrix. The mechanical pass it describes is now possible, and
overdue. Its named entry point, cargo run --bin gen-conformance-vectors, does
not exist.

What the pack actually contains

Established by running it, not reading it.

✅ REAL — and now asserted properly. Every positive vector's
signing_key_seed derives exactly the verifying_key it publishes. Genuine
Ed25519. sig-004-wrong-key's deliberate mismatch is asserted too, so the
negative case cannot quietly become a positive one.

⚠️ NOT REAL — now pinned so it is visible.

vector	claim	reality
sig-001	"sign a known envelope SAID"	it is RFC 8032's signature over the empty message — proven by verifying against b""
sig-002/003	signatures over their SAIDs	another RFC 8032 signature, over neither their SAID nor empty
all 5 SAID vectors	envelope SAIDs	literal <PLACEHOLDER_SAID>; expected_said is one repeated character; canonical_cbor_hex is the same 7-byte truncated prefix
canon-002, canon-003	canonical CBOR	expected_cbor_hex of odd length (109 and 37 chars) — undecodable
said-004 vs said-005	"internally consistent"	different envelopes, same expected SAID — the pack asserts a collision

Second commit: the spec is in worse shape than the pack

From asking why the vectors could not simply be regenerated.

1. spec/identity_and_key_rotation.md says implementers "MUST pass every
   vector" in a pack whose every expected_said is a placeholder. The
   requirement is unsatisfiable as written — a conforming implementation
   cannot be conforming.
2. The described suite is not the published one. The spec says it covers the
   empty-map case, the one-field case, the worked example, several
   envelope-shaped objects, and "one adversarial case where a non-canonical CBOR
   encoding produces a different digest (to verify that implementations reject
   non-canonical input rather than silently re-encoding)". The file holds five
   vectors, all envelope-shaped. That adversarial case is the only negative
   test named, it is the security-relevant one, and it is absent — an
   implementation that silently re-encodes non-canonical input passes this pack
   completely.
3. A broken normative reference, now fixed. The spec pointed three times at
   ../conformance/said_vectors.json, which does not exist; the file is at
   ../conformance/v1/said_vectors.json. Line 266 states "all paths are relative
   to this file", so both the breakage and the intent were unambiguous. This is
   the one thing here that is corrected rather than pinned — it is a pointer
   fix, not a normative change.

Falsified

Every pinning test was falsified by simulating the fix, which is the only
way to prove a pin will fire when the thing it pins is repaired:

simulated fix	result
substitute a real-looking SAID for the placeholder	✅ fails
give said-005 a distinct expected_said	✅ fails
make canon-002's hex even-length	✅ fails
corrupt a verifying_key	✅ fails
replace sig-001's signature	✅ fails
add the adversarial vector the spec mandates	✅ fails
soften the spec's MUST	✅ fails

Pack and spec restored afterwards and verified byte-identical to main. Workspace
622 passed, 0 failures; clippy -D warnings clean on smart-byte-core;
check-no-vanity-metrics.sh passes.

What this does not do

No vector is touched. Regenerating a published pack — and deciding whether it
should gain the missing cases — is a normative change to a public artifact and a
maintainer's call. This moves the state out of a doc nobody opens and into test
output, and gives a signal the moment someone regenerates, correctly or not.

The follow-up, if you want it: write the missing gen-conformance-vectors
against the now-landed smart-byte-rs, add the adversarial non-canonical case,
and do the mechanical pass. That is a separate, normative PR.

🤖 Generated with Claude Code

`smart-byte/conformance/v1` is published for third parties to certify against. Its gate — `.github/workflows/conformance.yml`, named **"smart-byte conformance gate"** — runs `jq` over it: valid JSON, top-level array, `name` and `description` present, names unique within a file. Those are well-formedness checks. **No value in the pack has ever been compared against the implementation**, and `smart-byte-rs` has no conformance crate that reads it. ### This is documented, not hidden `conformance/v1/regeneration.md` is candid that the `expected_*` fields are placeholders "not yet cryptographically computed". Nobody concealed anything. **What changed is its precondition.** The doc says to regenerate "when `smart-byte-rs` lands", describing it as a *"private repo, to be open-sourced alongside spec v1.0 freeze"*. `smart-byte-rs` **has** landed — 22 crates, in the `standards-build` matrix. The mechanical pass it describes is now possible, and overdue. Its named entry point, `cargo run --bin gen-conformance-vectors`, does not exist. ### What the pack actually contains Established by running it, not reading it. **✅ REAL — and now asserted properly.** Every positive vector's `signing_key_seed` derives exactly the `verifying_key` it publishes. Genuine Ed25519. `sig-004-wrong-key`'s deliberate mismatch is asserted too, so the negative case cannot quietly become a positive one. **⚠️ NOT REAL — now pinned so it is visible.** | vector | claim | reality | |---|---|---| | `sig-001` | "sign a known envelope SAID" | it is RFC 8032's signature over the **empty message** — proven by verifying against `b""` | | `sig-002`/`003` | signatures over their SAIDs | another RFC 8032 signature, over neither their SAID nor empty | | all 5 SAID vectors | envelope SAIDs | literal `<PLACEHOLDER_SAID>`; `expected_said` is one repeated character; `canonical_cbor_hex` is the same 7-byte truncated prefix | | `canon-002`, `canon-003` | canonical CBOR | `expected_cbor_hex` of **odd** length (109 and 37 chars) — **undecodable** | | `said-004` vs `said-005` | "internally consistent" | **different envelopes, same expected SAID** — the pack asserts a collision | ### Second commit: the spec is in worse shape than the pack From asking why the vectors could not simply be regenerated. 1. **`spec/identity_and_key_rotation.md` says implementers "MUST pass every vector"** in a pack whose every `expected_said` is a placeholder. The requirement is **unsatisfiable as written** — a conforming implementation cannot be conforming. 2. **The described suite is not the published one.** The spec says it covers the empty-map case, the one-field case, the worked example, several envelope-shaped objects, and *"one adversarial case where a non-canonical CBOR encoding produces a different digest (to verify that implementations reject non-canonical input rather than silently re-encoding)"*. The file holds five vectors, all envelope-shaped. **That adversarial case is the only negative test named, it is the security-relevant one, and it is absent** — an implementation that silently re-encodes non-canonical input passes this pack completely. 3. **A broken normative reference, now fixed.** The spec pointed three times at `../conformance/said_vectors.json`, which does not exist; the file is at `../conformance/v1/said_vectors.json`. Line 266 states "all paths are relative to this file", so both the breakage and the intent were unambiguous. This is the one thing here that is *corrected* rather than pinned — it is a pointer fix, not a normative change. ### Falsified Every pinning test was falsified by **simulating the fix**, which is the only way to prove a pin will fire when the thing it pins is repaired: | simulated fix | result | |---|---| | substitute a real-looking SAID for the placeholder | ✅ fails | | give `said-005` a distinct `expected_said` | ✅ fails | | make `canon-002`'s hex even-length | ✅ fails | | corrupt a `verifying_key` | ✅ fails | | replace `sig-001`'s signature | ✅ fails | | **add the adversarial vector the spec mandates** | ✅ fails | | **soften the spec's MUST** | ✅ fails | Pack and spec restored afterwards and verified byte-identical to main. Workspace **622 passed, 0 failures**; `clippy -D warnings` clean on `smart-byte-core`; `check-no-vanity-metrics.sh` passes. ### What this does not do **No vector is touched.** Regenerating a published pack — and deciding whether it should gain the missing cases — is a normative change to a public artifact and a maintainer's call. This moves the state out of a doc nobody opens and into test output, and gives a signal the moment someone regenerates, correctly or not. The follow-up, if you want it: write the missing `gen-conformance-vectors` against the now-landed `smart-byte-rs`, add the adversarial non-canonical case, and do the mechanical pass. That is a separate, normative PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 1 commit 2026-09-19 05:36:05 -04:00

smart-byte: verify what the pack really contains, and pin what it does not ... d92626d086

`smart-byte/conformance/v1` is published for third parties to certify against.
Its gate — `.github/workflows/conformance.yml`, named "smart-byte conformance
gate" — runs `jq` over it: valid JSON, top-level array, `name` and `description`
present, names unique. Those are well-formedness checks. No value in the pack
has ever been compared against the implementation, and `smart-byte-rs` has no
conformance crate that reads it.

`conformance/v1/regeneration.md` is candid that the `expected_*` fields are
placeholders "not yet cryptographically computed". So this is a known state,
openly documented, and not a defect anyone hid. What has changed is its stated
precondition: the doc says regenerate "when `smart-byte-rs` lands", describing
it as a "private repo, to be open-sourced alongside spec v1.0 freeze".
`smart-byte-rs` HAS landed — 22 crates, in the `standards-build` matrix — so
the one mechanical pass that doc describes is now possible, and overdue. Its
named entry point, `cargo run --bin gen-conformance-vectors`, does not exist.

What the pack actually contains, established by running it rather than reading
it:

REAL — every positive vector's `signing_key_seed` derives exactly the
`verifying_key` it publishes. That is asserted properly here, along with
`sig-004-wrong-key`'s deliberate mismatch, so the negative case cannot quietly
become a positive one.

NOT REAL — the `expected_*` values:
  - `sig-001` says it signs "a known envelope SAID". It does not. The value is
    RFC 8032's signature over the EMPTY message, proven by verifying it against
    `b""`. `sig-002`/`sig-003` carry another RFC 8032 signature, over neither
    their SAID nor the empty message.
  - Every SAID envelope carries the literal `<PLACEHOLDER_SAID>`; every
    `expected_said` is one repeated character rather than a digest; every
    `canonical_cbor_hex` is the same 7-byte truncated prefix.
  - `canon-002` and `canon-003` publish `expected_cbor_hex` of ODD length (109
    and 37 characters), which cannot be decoded to bytes at all. A third party
    fails before reaching any comparison.
  - The doc's own consistency claim does not hold in the direction that matters:
    `said-004` and `said-005` are different envelopes published with the SAME
    expected SAID, so even as filler the pack asserts a collision.

The pinning tests are written to FAIL once the pack is regenerated, which is
the intended signal to replace each with a real assertion — the same pattern
`map-conformance` uses for its JSON round-trip defect. Each was falsified by
simulating the fix: substituting a real-looking SAID, giving `said-005` a
distinct value, making `canon-002`'s hex even-length, corrupting a verifying
key, and replacing `sig-001`'s signature. All five fail as intended, and the
pack was restored and verified byte-identical to main.

Regenerating a published pack is a normative change to a public artifact and a
maintainer's call, so no vector is touched here. This only makes the state
visible in test output instead of in a doc nobody opens.

Co-Authored-By: Claude Opus 5 <[email protected]>

dcharlot force-pushed ci/smart-byte-pack from d92626d086 to 23ceabf08e 2026-09-21 15:20:17 -04:00 Compare

dcharlot changed title from smart-byte: verify what the pack really contains, and pin what it does not to smart-byte: the pack is placeholders, and the spec mandates a suite that does not exist 2026-09-21 15:20:28 -04:00

dcharlot force-pushed ci/smart-byte-pack from 23ceabf08e to 0acb0cc57f 2026-09-22 01:19:27 -04:00 Compare

dcharlot force-pushed ci/smart-byte-pack from 0acb0cc57f to 5391e895d5 2026-09-26 05:10:27 -04:00 Compare

All checks were successful

Hide checks

smart-byte conformance gate / Validate conformance JSON files (pull_request) Successful in 3s

Details

smart-byte conformance gate / Spec changes require vector-pack updates (pull_request) Successful in 3s

Details

smart-byte conformance gate / VERSIONS.md and README.md must exist (pull_request) Successful in 3s

Details

no-vanity-metrics / Scan committed Markdown for vanity metrics (pull_request) Successful in 3s

Details

standards workspaces build + test / jouleclaw-rs (pull_request) Successful in 1m47s

Details

standards workspaces build + test / eoc-rs (pull_request) Successful in 1m51s

Details

standards workspaces build + test / aac-rs (pull_request) Successful in 35s

Details

standards workspaces build + test / arl-rs (pull_request) Successful in 2m45s

Details

standards workspaces build + test / joule-code-rs (pull_request) Successful in 38s

Details

standards workspaces build + test / jcp-rs (pull_request) Successful in 1m18s

Details

standards workspaces build + test / joulecontract-rs (pull_request) Successful in 33s

Details

standards workspaces build + test / map-rs (pull_request) Successful in 30s

Details

standards workspaces build + test / mesh-rs (pull_request) Successful in 25s

Details

standards workspaces build + test / proof-rs (pull_request) Successful in 1m11s

Details

standards workspaces build + test / joule-ui-rs (pull_request) Successful in 2m47s

Details

standards workspaces build + test / openpay (pull_request) Successful in 1m32s

Details

standards workspaces build + test / sandbox-rs (pull_request) Successful in 2m10s

Details

standards workspaces build + test / smart-byte-rs (pull_request) Successful in 2m33s

Details

standards workspaces build + test / jouledesk-rs (pull_request) Successful in 3m50s

Details

standards workspaces build + test / joulehook-rs (pull_request) Successful in 3m2s

Details

standards workspaces build + test / sense-rs (pull_request) Successful in 3m48s

Details

standards workspaces build + test / macOS energy backend (jcp-energy) (pull_request) Successful in 5m7s

Details

This pull request can be merged automatically.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin ci/smart-byte-pack:ci/smart-byte-pack

git switch ci/smart-byte-pack

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main

git merge --no-ff ci/smart-byte-pack

git switch ci/smart-byte-pack

git rebase main

git switch main

git merge --ff-only ci/smart-byte-pack

git switch ci/smart-byte-pack

git rebase main

git switch main

git merge --no-ff ci/smart-byte-pack

git switch main

git merge --squash ci/smart-byte-pack

git switch main

git merge --ff-only ci/smart-byte-pack

git switch main

git merge ci/smart-byte-pack

git push origin main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!99

No description provided.