wai: bearer-receipts §13, reconciliations and holdings records over a return path of a few bytes #637

Merged

dcharlot merged 2 commits from wai/compact-return-profile into main 2026-11-06 13:44:54 -05:00

Conversation 0 Commits 2 Files changed 16 +5738 -2

dcharlot commented 2026-11-05 04:25:43 -05:00

Owner

Copy link

What this is

The compact return profile (bearer-receipts §13) is a binary form of a reconciliation (§5) and of a holdings record (holdings §4), for a return path whose messages hold few bytes, with fragments for messages longer than a frame.

Merge order. This branch sits on #632's commit, 946f3beda, which moves changelog entries into wai/changes/ and the conformance workflow's lists into step files. Merge #632 first; until it merges, this pull request's diff also shows #632's change.

What it carries. One commit, f0dc0b8d6: the change of wai/compact-return-profile (98192fe63..9a8353be8) in #632's layout. Its message lists the source commits. The branch's changelog entry is a file of wai/changes/. Its edits to wai-conformance.yml are a new step file, .github/wai-conformance/verifiers/compact-return-profile.sh, at order 535. It names the features bearer_receipts,holdings, which also puts that list into the features-alone check, and the path wai/extensions/bearer-receipts.md, with the reason the branch gave for it. Every other file is as the source branch has it.

What it defines

References, not copies. A compact record names by reference what the receiver already holds:

• the group, by 8 bytes of SHA-256 over its link hash;
• the bearer instance among that group's, by 4 bytes over its bearer_id;
• the signer, by 8 bytes over its key.

It writes integers in LEB128 and issued_at as seconds. A reference that resolves to no value, or to more than one, is refused, never resolved by choosing. A whole reconciliation carries no bitmap, and its root is the group's. A partial one carries its bitmap, and its root is derived from the object receipts the receiver holds.

Exact both ways. A compact record expands, against the receiver's context, to the record's own bytes: a claim's canonical form, or a holdings record's WHD1. Those bytes are then verified by their own text, under the signature the compact record carried, so the compact form states nothing of its own. A record has one compact form against a context. A record with none is refused compact_not_encodable and travels in its own form. That covers:

• an issued_at with a fraction or an offset;
• a group or instance the receiver does not hold;
• a leaf_count that is not the group's;
• a root its objects do not give;
• a hybrid claim.

Nothing is sized by a claim's own leaf_count before it is found to be the group's.

Fragments. A message is the compact record and a 4-byte check, the first bytes of SHA-256 over the record, cut into frames under a 3-byte header: the sender's number for the message, the index and the count. Reassembly places fragments by index, in any order of arrival, and holds a fragment delivered twice, byte for byte, once. It refuses a fragment that is malformed, foreign (another number or count), repeated (other data at its index) or missing. By the check, it also refuses one reordered or mixed in under the same header.

DAP reports are never fragmented. A task's floor is the length of its longest report. A path whose frame is below it carries none of the task's reports; a sink holds them, or gives them to a node with a path that carries them, which uploads them unchanged. The floors of telemetry-aggregation's measures 1 to 4 are stated and checked against the reports of its corpus.

Beside it. jwp-receipts §2.7 and holdings §1 point to the profile.

The reference and the independent verifier

The reference. The compact_return module of wai-rs (features bearer_receipts and holdings). Its unit tests cover a uint's one spelling, an instant's, the whole bitmap, a message reassembled in any order with each defect refused, and the bounds. tests/compact_return.rs checks that:

• vectors.json regenerates byte for byte from its fixed seeds, and no other file is there;
• wai-rs gives every case its stated outcome, with every round trip holding both ways;
• §13.9 lists exactly the reference's codes;
• §13.8's floors are the lengths of the aggregate corpus's reports, read with the reference's DAP reader.

The corpus. live-conformance/compact-return/ holds:

• round trips of reconciliations in each signer form and over each bearer class, and of holdings records;
• expansions: every form a reader takes, a changed byte refused by the record's own signature, and a control for each reader rule of both records;
• references unknown and ambiguous, an instance held for other groups only, and a holder's key under two signer_ids;
• records with no compact form;
• messages cut into frames of several sizes, and reassemblies with each defect.

The independent verifier. live-conformance/compact_return_verify.py is written from the extension's text and shares no code with wai-rs. It uses other corpora's second implementations: jwp_verify.py for groups and objects, claim_verify.py for claims, bearer_verify.py for §5.3, holdings_verify.py for holdings records and telemetry_aggregate_verify.py for DAP reports. It writes the references, the compact records, their expansion and writing, the fragments and reassembly itself. It checks the floors against telemetry-aggregation's table and corpus. It requires every mutated copy of the corpus to fail, and drops each of its own checks in turn.

How it was verified

This commit, f0dc0b8d6.

• Locally (git and Python checks only): scripts/wai-conformance-lists.py selftest and check, scripts/wai-conformance-lists-selftest.sh, scripts/check-changelog-fragment.py against main, scripts/changelog-assemble.py --check and scripts/changelog-fragments-selftest.sh all pass. Rebasing from the conversion's first base, #632's earlier tip 14f1abb7c, onto 946f3beda left every file's change the same.
• On a Linux x86_64 build host: in wai/wai-rs, cargo check --locked --no-default-features --tests with bearer_receipts,holdings, and with bearer_receipts,holdings,telemetry_aggregate. Both pass.
• With the other converted branches. This branch merges without conflict with each of the other branches converted to #632's layout (by git merge-tree). Merging all of them in forward, reverse and a mixed order gives one tree. On the build host, that tree passes these runs: cargo check --tests under the derived WAI_FEATURES and under WAI_PQ_FEATURES; cargo check --tests of quickstart,app; the catalogue's library tests; and the wai binary's unit tests.

The module's tests, the corpus's regeneration and the Python verifier did not run on this commit here; their results come from this pull request's CI, in the new step.

On the CI hosts. Results on the CI runners come from this pull request's own CI run. None of the runs above is a CI result.

🤖 Generated with Claude Code

## What this is The compact return profile (bearer-receipts §13) is a binary form of a reconciliation (§5) and of a holdings record (holdings §4), for a return path whose messages hold few bytes, with fragments for messages longer than a frame. **Merge order.** This branch sits on #632's commit, 946f3beda, which moves changelog entries into `wai/changes/` and the conformance workflow's lists into step files. Merge #632 first; until it merges, this pull request's diff also shows #632's change. **What it carries.** One commit, f0dc0b8d6: the change of `wai/compact-return-profile` (98192fe63..9a8353be8) in #632's layout. Its message lists the source commits. The branch's changelog entry is a file of `wai/changes/`. Its edits to `wai-conformance.yml` are a new step file, `.github/wai-conformance/verifiers/compact-return-profile.sh`, at order 535. It names the features `bearer_receipts,holdings`, which also puts that list into the features-alone check, and the path `wai/extensions/bearer-receipts.md`, with the reason the branch gave for it. Every other file is as the source branch has it. ## What it defines **References, not copies.** A compact record names by reference what the receiver already holds: - the group, by 8 bytes of SHA-256 over its link hash; - the bearer instance among that group's, by 4 bytes over its `bearer_id`; - the signer, by 8 bytes over its key. It writes integers in LEB128 and `issued_at` as seconds. A reference that resolves to no value, or to more than one, is refused, never resolved by choosing. A whole reconciliation carries no bitmap, and its root is the group's. A partial one carries its bitmap, and its root is derived from the object receipts the receiver holds. **Exact both ways.** A compact record expands, against the receiver's context, to the record's own bytes: a claim's canonical form, or a holdings record's `WHD1`. Those bytes are then verified by their own text, under the signature the compact record carried, so the compact form states nothing of its own. A record has one compact form against a context. A record with none is refused `compact_not_encodable` and travels in its own form. That covers: - an `issued_at` with a fraction or an offset; - a group or instance the receiver does not hold; - a `leaf_count` that is not the group's; - a root its objects do not give; - a hybrid claim. Nothing is sized by a claim's own `leaf_count` before it is found to be the group's. **Fragments.** A message is the compact record and a 4-byte check, the first bytes of SHA-256 over the record, cut into frames under a 3-byte header: the sender's number for the message, the index and the count. Reassembly places fragments by index, in any order of arrival, and holds a fragment delivered twice, byte for byte, once. It refuses a fragment that is malformed, foreign (another number or count), repeated (other data at its index) or missing. By the check, it also refuses one reordered or mixed in under the same header. **DAP reports are never fragmented.** A task's floor is the length of its longest report. A path whose frame is below it carries none of the task's reports; a sink holds them, or gives them to a node with a path that carries them, which uploads them unchanged. The floors of telemetry-aggregation's measures 1 to 4 are stated and checked against the reports of its corpus. **Beside it.** jwp-receipts §2.7 and holdings §1 point to the profile. ## The reference and the independent verifier **The reference.** The `compact_return` module of `wai-rs` (features `bearer_receipts` and `holdings`). Its unit tests cover a uint's one spelling, an instant's, the whole bitmap, a message reassembled in any order with each defect refused, and the bounds. `tests/compact_return.rs` checks that: - `vectors.json` regenerates byte for byte from its fixed seeds, and no other file is there; - `wai-rs` gives every case its stated outcome, with every round trip holding both ways; - §13.9 lists exactly the reference's codes; - §13.8's floors are the lengths of the aggregate corpus's reports, read with the reference's DAP reader. **The corpus.** `live-conformance/compact-return/` holds: - round trips of reconciliations in each signer form and over each bearer class, and of holdings records; - expansions: every form a reader takes, a changed byte refused by the record's own signature, and a control for each reader rule of both records; - references unknown and ambiguous, an instance held for other groups only, and a holder's key under two `signer_id`s; - records with no compact form; - messages cut into frames of several sizes, and reassemblies with each defect. **The independent verifier.** `live-conformance/compact_return_verify.py` is written from the extension's text and shares no code with `wai-rs`. It uses other corpora's second implementations: `jwp_verify.py` for groups and objects, `claim_verify.py` for claims, `bearer_verify.py` for §5.3, `holdings_verify.py` for holdings records and `telemetry_aggregate_verify.py` for DAP reports. It writes the references, the compact records, their expansion and writing, the fragments and reassembly itself. It checks the floors against telemetry-aggregation's table and corpus. It requires every mutated copy of the corpus to fail, and drops each of its own checks in turn. ## How it was verified **This commit, f0dc0b8d6.** - **Locally** (git and Python checks only): `scripts/wai-conformance-lists.py selftest` and `check`, `scripts/wai-conformance-lists-selftest.sh`, `scripts/check-changelog-fragment.py` against main, `scripts/changelog-assemble.py --check` and `scripts/changelog-fragments-selftest.sh` all pass. Rebasing from the conversion's first base, #632's earlier tip 14f1abb7c, onto 946f3beda left every file's change the same. - **On a Linux x86_64 build host:** in `wai/wai-rs`, `cargo check --locked --no-default-features --tests` with `bearer_receipts,holdings`, and with `bearer_receipts,holdings,telemetry_aggregate`. Both pass. - **With the other converted branches.** This branch merges without conflict with each of the other branches converted to #632's layout (by `git merge-tree`). Merging all of them in forward, reverse and a mixed order gives one tree. On the build host, that tree passes these runs: `cargo check --tests` under the derived `WAI_FEATURES` and under `WAI_PQ_FEATURES`; `cargo check --tests` of `quickstart,app`; the catalogue's library tests; and the `wai` binary's unit tests. The module's tests, the corpus's regeneration and the Python verifier did not run on this commit here; their results come from this pull request's CI, in the new step. **On the CI hosts.** Results on the CI runners come from this pull request's own CI run. None of the runs above is a CI result. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 1 commit 2026-11-05 04:25:43 -05:00

wai/compact-return-profile, converted to the layout of the fix: no shared append point ... f0dc0b8d68

The change of wai/compact-return-profile (98192fe63..9a8353be8, 2 commits), on the fix branch: its
changelog entries as files of wai/changes/, its wai-conformance.yml edits as step files,
corpus files and test tags, and every other file as the branch has it.

- cad41c9bd bearer-receipts §13: a compact return profile for reconciliations and holdings records, with fragments
- 9a8353be8 bearer-receipts §13: a fragment delivered twice is one, holdings reader controls, references by group and by key, leaf_count before sizing

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot added 1 commit 2026-11-06 02:20:38 -05:00

wai dev's test: the stand-in room service listens without a reverse lookup ... 15698141a8

tests/cli_studio.rs failed on every run on one macOS runner and passed on
every other runner, 3 of 3 against 19 of 19 on 5 November, with "the room
service's clock on the relay: not within 30 s".

The stand-in room service starts an HTTPServer, whose server_bind calls
socket.getfqdn before the server listens. On that host, under the runner's
Python 3.15.2, getfqdn("127.0.0.1") takes 35 s, past the 30 s wai dev waits, and
a stack dump of the stalled stand-in shows it there. The stand-in relay opens a
plain socket and answers in 0.1 s. Nothing here reads the name, so the fixture
no longer asks for it.

On that host and Python the room stand-in took 35.5 s and now takes 0.12 s. A
socket.getfqdn planted to stall for 40 s fails the test without this change,
with the same message, and passes with it.

Changelog: none

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit 7dc5501005 into main 2026-11-06 13:44:54 -05:00

dcharlot referenced this pull request from a commit 2026-11-06 13:44:56 -05:00

Merge pull request 'wai: bearer-receipts §13, reconciliations and holdings records over a return path of a few bytes' (#637) from wai/compact-return-profile into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!637

No description provided.