Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions 8

receipt-ledger: version 5 holds a decode to the frames it delivered, its codestream what it read #559

Merged
dcharlot merged 3 commits from wai/decode-output into main 2026-11-01 04:38:01 -05:00
Conversation 0 Commits 3 Files changed 10 +1823 -108
10 changed files with 1823 additions and 108 deletions
Download patch file Download diff file Expand all files Collapse all files

35
wai/CHANGELOG.md
View file

@ -6,6 +6,41 @@ wire format may change and such changes are recorded here.
## Unreleased
### The receipt ledger holds a decode to the frames it delivered, as one animated PNG
- **Tariff and invoice version 5** ([extensions/receipt-ledger.md](extensions/receipt-ledger.md) §4.8): version 4's
layout, rules and prices, and one rule more. A decode (a job whose route record's operation is `media.decode`)
delivers its reconstruction, and the codestream is what it read. Version 4 gives every `int_motion` output as its
receipt and codestream and holds the codestream to what the hop delivered: true of an encode, false of a decode,
whose hop had to state an object it never delivered and commit its receipt to it. Under version 5 a decode's output
also gives `frames`, one animated PNG (PNG's third edition) of 8-bit RGB frames, each the whole picture and blended
over nothing, its default image its first frame, read whole and then frame by frame as packaging §5 reads a still's
image data, each frame's samples hashing (integer-payloads §15.1) to the receipt's frame at its index; the receipt is
still re-derived from the codestream, the decode's input; and the served hop states exactly that animation, by
content hash, and nothing else (`output_invalid` at the animation or the frame, `output_not_delivered` at its place,
or at `outputs` for an object past it). One object, not one per frame, because a route record states a hop's outputs
once each and at most 1024, and a clip runs longer and repeats frames. The frame hash binds the samples, not the
PNG's bytes, so any writer's animation of the right samples holds. An encode, and every other job, is held as in
version 4, and a decode bills its receipts' subpixels as before. A version-4 reader would refuse a decode's frames,
and a version-4 verdict must not change, so this is a version of its own; versions 1 to 4 read, verify and bill
exactly as before.
- **Animated PNG** (`png_still::encode_animation`, `png_still::animation`): a writer every byte of which is a function
of the frames and the delay, and a reader of §4.8's rules that reads the whole structure first and each frame's
samples only when asked, so a long clip is never held decoded.
- **The corpus** adds nineteen vectors: a decode held to its animation beside an encode; a decode whose record states
the codestream it read, and one stating it past the animation; a decode in version 4's form; a frame with one sample
changed, two frames swapped, a still for the animation, a frame missing, frames of another shape holding the same
samples, and a codestream not the receipt's; an animation that blends a frame over the one before, disposes of one by
no defined operation, holds a frame of part of the picture, skips a sequence number, keeps its default image apart
from its frames, counts a frame more than it holds, or has a frame with no data; a decode in version 5's form under
version 4; and a version-4 invoice under a version-5 tariff. `receipt_ledger_verify.py` reads version 5 with an
animation reader of its own, reads each frame's data with the packaging verifier's PNG reader, and drops one more
check (`decode-frames`); three more mutated copies of the corpus must fail. The ledger service (`wai/wai-ledger`)
bills version 5 as version 4.
- **Breaking:** `record:corpus:wai/live-conformance/ledger/vectors.json`: the control `invoice_version` gives an
invoice of version 6, since version 5 is now defined, and is refused `unknown_version` as before; every other vector
keeps its outcome, and the new vectors are added.
### The receipt ledger bills what the work spends: bytes out, a package's work per run and its storage, every hop's energy, the output its hop delivered
- **A job's output is the object its hop delivered** ([extensions/receipt-ledger.md](extensions/receipt-ledger.md)

92
wai/extensions/receipt-ledger.md
View file

@ -152,7 +152,7 @@ tariff the buyer did not sign bills nobody.
A reader reads exactly the bytes it is given as one tariff, in the order of §4.1, and refuses at
the first of these that holds: fewer than 4 bytes, `truncated`; a magic other than `WLT1`,
`malformed`; no fifth byte, `truncated`; a version other than 1 to 4, `unknown_version`; the
`malformed`; no fifth byte, `truncated`; a version other than 1 to 5, `unknown_version`; the
bytes end before a field the layout requires, `truncated`; bytes after `buyer_sig`, `malformed`.
It then refuses as `tariff_invalid` a tariff that breaks any of these:
@ -266,6 +266,73 @@ package prices two shorter, and a job price may be of unit 3) or an invoice of v
it states a sixth unmetered count), so version 4 is a version of its own. A tariff of versions
1 to 3, its invoices and its sources read, verify and bill exactly as before.
### 4.8 A decode's output (version 5)
A tariff of version 5 is version 4's in its layout (§4.1), its rules (§4.3), its scope and its
prices (§4.4 to §4.7), and an invoice of version 5 is version 4's (§6.1); each bills what
version 4 bills. Where this text says version 4, or a range of versions ending at 4, version 5
is included, except as this section says.
A decode (a job whose route record's `operation` is `media.decode`) delivers its
**reconstruction**, not a codestream: the codestream is what it read. Version 4 gives every
`int_motion` output as its receipt and its `codestream` and holds the codestream to what the hop
delivered (§5.1), which is true of an encode and false of a decode, whose hop would have to state
an object it never delivered and commit its receipt to it (route-record §5.2, §6). Under a tariff
of version 5 a decode's output is held to what it delivered:
1. Each output of a decode's job source is an object of exactly `receipt` (the text of a
`wai.video.int_motion` decode receipt's JSON form, integer-payloads §15.2), `codestream` (the
`WIV1` payload the decode read, hex) and `frames` (one animated PNG holding its frames, hex).
The receipt is read by integer-payloads §15.2 and re-derived from the codestream (§15.1 there):
the codestream is the decode's input, and the receipt is the evidence of what it decodes to.
2. The animation is read whole first: its chunks as packaging §5 reads a still's (every chunk's
type four letters and its CRC checked; `IHDR` first, of 8-bit RGB without interlace; an
optional `PLTE` before the image data; the `IDAT`s consecutive; ancillary chunks read past;
any other critical chunk refused; `IEND` last, empty, and nothing after it), and the chunks
of PNG's third edition that make it an animation:
- one `acTL` of 8 bytes, before any `fcTL` and the image data, counting at least one frame,
and as many `fcTL`s as it counts;
- each `fcTL` of 26 bytes, of the whole picture (its width and height `IHDR`'s, its offsets
0), its dispose operation 0 to 2 and its blend operation 0, any delay;
- the first `fcTL` before the `IDAT`s, so the default image is the first frame, and every
other after them, followed by at least one `fdAT` before the next `fcTL` or `IEND`; no
`IDAT` but the first frame's, and no `fdAT` of it;
- the sequence numbers of the `fcTL`s and `fdAT`s 0, 1, 2, … in their order, and every
`fdAT` at least 4 bytes long.
It is of the receipt's `w` × `h` and holds as many frames as the receipt has. Then each frame,
in order: its data (the `IDAT`s' for the first frame, its `fdAT`s' after their sequence
numbers for each other, concatenated) is read as packaging §5 reads a still's image data, and
the frame hash of its samples at its index (integer-payloads §15.1: BLAKE3 over
`wai:video-rframe`, 0x01, the index as a u64 and the RGB8 bytes) is the receipt's frame at
that index. Otherwise the source is refused `output_invalid`: at `outputs`, the output's
index and `frames` where the animation breaks a rule above or differs in size or count; at
`outputs`, the output's index, `frames` and the frame's index where that frame's data is not
the picture's rows or its samples are not the receipt's; and at `outputs` and the output's
index when the receipt or the codestream does not hold.
3. The objects the served attempt states it delivered (route-record §5.2) are exactly the
decode's animations, by content hash, in the order of its outputs, and nothing else: an
animation not stated at its place is refused `output_not_delivered` at `outputs`, the
output's index and `frames`, and an object stated past the last, the codestream among them,
at `outputs`.
A decode delivers its frames as one object because a clip's frames are many and repeat: a route
record states each of a hop's outputs once, and at most 1024 of them (route-record §5.2), while
a clip runs to thousands of frames and a still scene decodes to the same frame again and again.
An animated PNG is lossless, one file every browser plays and any tool splits into stills, and
its default image is its first frame, which packaging §5's reader of a still reads. The frame
hash, not the PNG's bytes, is what the receipt binds, so any writer's animation of the right
samples holds; the reference writer is `png_still::encode_animation`. A frame is the whole
picture and its samples replace the picture's, so they are its data alone: a frame blended over
the one before, or of part of the picture, would be held to samples its data does not hold.
A decode's billed subpixels are its receipts' `work_pixel_ops` (§6.3), as in version 4: the
receipt states what the frames hold. An encode, and every other job, is held as in version 4.
A version-4 reader would read a decode's `frames` as an unknown member and refuse the source, and
a version-4 verdict on a decode must not change under it, so version 5 is a version of its own.
A tariff of versions 1 to 4, its invoices and its sources read, verify and bill exactly as
before.
## 5. Sources
### 5.1 Kinds and their documents
@ -280,7 +347,7 @@ extension's verifier is given. Bytes are lowercase hexadecimal strings.
| `package` | `receipt` (the text of `package.receipt.json`), `indexes` (the index files in the receipt's component order) | packaging §9, a receipt of version 2 against its indexes |
| `ads` | `descriptor`, `grants`, `key_log`, `chains`, `policy`, `pt0_ms`, `receipts` (form-2 receipts in their JSON carriage), `pods`, `claims`, `creatives`, `issuers` | live-ads §9, with ad-presentation §9 under a policy that requires claims |
| `stitched` | `records` (pod-input records), `claims` (stitched claims), `manifests`, `responses` (VAST and VMAP documents), `receipts` (form-1 receipts in their JSON carriage), `creatives`, `issuers` | stitched-presentation §7, its signers the tariff's (§4.4) |
| `job` (versions 2 to 4) | `request`, `pricebook`, `policy`, `record`, `receipts`, `keys` (a route record's verification document, route-record §7.1, its `receipts` and `keys` not `null`) and `outputs`, a list of objects of exactly `receipt` (the text of a `wai.video.int_motion` decode receipt's JSON form, integer-payloads §15.2) and `codestream` (the `WIV1` payload it describes: the object the hop delivered, named by its content hash), or, in version 4, of exactly `package` (the text of a package receipt of version 2, `package.receipt.json`: the object the hop delivered) and `indexes` (its index files in its component order) | route-record §7, accepted with its hop receipts checked; then each output receipt read by integer-payloads §15.2 and re-derived from its `codestream` (§15.1 there), each package verified against its indexes by packaging §9, or `output_invalid` at `outputs` and its index; then each output held to what its hop delivered, or `output_not_delivered` there |
| `job` (versions 2 to 4) | `request`, `pricebook`, `policy`, `record`, `receipts`, `keys` (a route record's verification document, route-record §7.1, its `receipts` and `keys` not `null`) and `outputs`, a list of objects of exactly `receipt` (the text of a `wai.video.int_motion` decode receipt's JSON form, integer-payloads §15.2) and `codestream` (the `WIV1` payload it describes: the object the hop delivered, named by its content hash; in version 5 a decode's output also has `frames`, the animation of its frames it delivered, and its codestream is what it read, §4.8), or, in version 4, of exactly `package` (the text of a package receipt of version 2, `package.receipt.json`: the object the hop delivered) and `indexes` (its index files in its component order) | route-record §7, accepted with its hop receipts checked; then each output receipt read by integer-payloads §15.2 and re-derived from its `codestream` (§15.1 there), each package verified against its indexes by packaging §9, or `output_invalid` at `outputs` and its index; then each output held to what its hop delivered, or `output_not_delivered` there |
A source is accepted when its extension's verifier accepts it; for `ads`, when the outcome is
**accept** (live-ads §9.4), a **gap** being `source_gap`. A `stitched` source is always
@ -313,7 +380,8 @@ line bills (§6.3): they are read from the output receipts, and an output receip
the hop did not deliver bills subpixels nobody delivered for this job, as a receipt given
twice bills them twice. So the chain closes: the hop's receipt names its outputs, the record
names the hop's receipt and the outputs, and each output receipt is of one of them. Under a
tariff of version 4, a record that states no outputs (a record of version 1, or one no attempt
tariff of version 5 a decode's object is the animation of the frames it delivered, not the
codestream it read (§4.8). Under a tariff of version 4, a record that states no outputs (a record of version 1, or one no attempt
served) holds every output to none, and a job it routed is billed by the job alone; under a
tariff of version 3, a record of version 1's outputs are read as before.
@ -400,7 +468,7 @@ was asked, not the work, and a request run again is worked again.
| field | size | meaning |
|---|---|---|
| `magic` | 4 | `WLI1` (ASCII) |
| `version` | 1 | its tariff's version, 1 to 4 |
| `version` | 1 | its tariff's version, 1 to 5 |
| `seller`, `buyer` | 32 each | the tariff's |
| `tariff` | 32 | the tariff's digest |
| `prev` | 32 | the digest of the invoice before it in its chain (§6.9); zero for the first |
@ -781,7 +849,10 @@ outcome §7.3 gives it:
and per work unit, and those jobs with every hop's energy, the failed and late hops' in their
own scope, one of them unmetered; and the next invoice of that chain, billing the package it
holds for the byte-days of its own period and the same package made again by a new run, its
work billed again and its receipt cited again, beside a second package listed.
work billed again and its receipt cited again, beside a second package listed; and, under a
tariff of version 5, an `int_motion` decode whose served hop states it delivered one animated
PNG of the clip's frames, its receipt re-derived from the codestream it read, beside an encode
held to its codestream.
- **Refused**: each reason of §10 at least once, each vector a control of one defect; a source of
each kind outside the scope, a room receipt and a statement under other prices, and a source of
each timed kind outside the period; a refusal of each source kind by its own extension; and, for
@ -802,7 +873,16 @@ outcome §7.3 gives it:
(`record_duplicate`), a failed hop billed as a job and a job line at the router's list price, the unbilled
hops' energy left out or added to the billed hops' row, an unbilled hop whose figure states no
class (`job_figure_invalid`), an output of a route record of version 1, and an invoice of
version 3 under a tariff of version 4.
version 3 under a tariff of version 4; and, for version 5, a decode whose record states the
codestream it read, and one whose record states it after the animation
(`output_not_delivered`), a decode in version 4's form (`malformed`), a frame with one sample
changed, two frames swapped, a still for the animation, a frame missing, frames of another
shape holding the same samples, a codestream not the receipt's, and an animation that blends
a frame over the one before, disposes of one by no defined operation, holds a frame of part of
the picture, skips a sequence number, keeps its default image apart from its frames, counts a
frame more than it holds, or has a frame with no data (`output_invalid`), a decode in
version 5's form under a tariff of version 4, and an invoice of version 4 under a tariff of
version 5.
`wai_receipt_ledger_vectors verify` and the test `tests/receipt_ledger.rs` check that the file
regenerates byte for byte, that `wai-rs` gives every vector its outcome, that every reason of §10

9
wai/live-conformance/ledger/README.md
View file

@ -70,7 +70,14 @@ The accepted vectors:
- `version_4_storage_held_through_a_period` (version 4): the next invoice of that chain bills the
package the first holds for the byte-days of its own period, and the same package made again by
a new run (`package_run_again`), its work billed again and its receipt cited again, beside a
second package listed.
second package listed;
- `version_5_a_decode_held_to_its_frames` (version 5, §4.8): an `int_motion` decode
(`job_v5_decode`) whose served hop states it delivered one animated PNG of the clip's frames,
its receipt re-derived from the codestream it read and each frame's samples hashing to the
receipt's frame, billed at the decode's prices beside an encode held to its codestream as in
version 4. The animation is written here by `png_still::encode_animation` from the clip the
decode reads, and each refused animation is that one with one rule broken, its chunks
rewritten with good CRCs.
Every other vector is refused, a control of one defect: each reason of §10, the reused codes, and a
refusal of each source kind by its own extension.

836
wai/live-conformance/ledger/vectors.json
View file

File diff suppressed because one or more lines are too long

195
wai/live-conformance/receipt_ledger_verify.py
View file

@ -49,7 +49,9 @@ import io
import json
import os
import re
import struct
import sys
import zlib
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
@ -88,9 +90,11 @@ SCOPES = {1: "room_hops", 2: "delivery_hops", 3: "packaging", 4: "ad_decisions",
CONTENT_DOMAIN = b"moq-jwp:object\x01" # route-record §2, jwp-receipts §2.1
DAY_US = 86400 * 10 ** 6 # §4.7: a byte-day's day
JOB_UNITS = ["jobs", "subpixels", "work_units"] # §4.5; the third in version 4 (§4.7)
JOB_VERSIONS = (2, 3, 4) # §4.5, §4.6, §4.7: the versions that price jobs
WORK_VERSIONS = (3, 4) # §4.6, §6.9: priced per operation, billed by the work
JOB_VERSIONS = (2, 3, 4, 5) # §4.5 to §4.8: the versions that price jobs
WORK_VERSIONS = (3, 4, 5) # §4.6, §6.9: priced per operation, billed by the work
V4 = 4 # §4.7: bytes out, work units, storage by byte-days
V5 = 5 # §4.8: version 4, and a decode held to its frames
SPENDING = (V4, V5) # §4.7, §4.8: the versions that bill what the work spends
CAPABILITY = re.compile(r"\Awai\.[a-z0-9_.]{0,124}\Z")
OPERATION = re.compile(r"\A[a-z][a-z0-9]*\.[a-z][a-z0-9]*\Z") # §4.6, route-record §2
LABELS = {2: "ModelBased", 3: "Estimator", 4: "OnChipCounter", 5: "CalibratedInstrument"}
@ -109,12 +113,12 @@ UNITS_4[PACKAGE] = [None, None, ("byte_days", 10 ** 9)] # a package so
def units(version):
return UNITS_4 if version == V4 else UNITS
return UNITS_4 if version in SPENDING else UNITS
def scopes(version):
"""§6.1: the scopes an invoice of `version` states."""
return 6 if version == V4 else 5 if version in JOB_VERSIONS else 4
return 6 if version in SPENDING else 5 if version in JOB_VERSIONS else 4
SKIP = set()
@ -157,7 +161,7 @@ def head(b, magic):
return "malformed"
if len(b) < 5:
return "truncated"
if b[4] not in (1, 2, 3, 4):
if b[4] not in (1, 2, 3, 4, 5):
return "unknown_version"
return None
@ -184,10 +188,10 @@ def read_tariff(b):
try:
t = {"seller": r.take(32), "buyer": r.take(32), "start": r.u(8), "end": r.u(8), "currency": r.take(3)}
# §4.1: version 4 states a fourteenth room price, of bytes out; versions 1 to 3 read it as 0.
t["room"] = [r.u(4) for _ in range(14 if b[4] == V4 else 13)] + ([] if b[4] == V4 else [0])
t["room"] = [r.u(4) for _ in range(14 if b[4] in SPENDING else 13)] + ([] if b[4] in SPENDING else [0])
t["delivery"] = [r.u(8) for _ in range(4)]
# §4.1: version 4 states one package price, per 10^9 byte-days held; the others read as 0.
t["package"] = [0, 0, r.u(8)] if b[4] == V4 else [r.u(8) for _ in range(3)]
t["package"] = [0, 0, r.u(8)] if b[4] in SPENDING else [r.u(8) for _ in range(3)]
t["ads"] = [r.u(8) for _ in range(2)]
t["version"] = b[4]
t["job"] = [] # (capability, operation, unit, divisor, price)
@ -209,7 +213,7 @@ def read_tariff(b):
except UnicodeDecodeError:
return False
named = OPERATION.match(op) is not None and len(j[1]) <= 128 if t["version"] in WORK_VERSIONS else op == ""
units_priced = 3 if t["version"] == V4 else 2
units_priced = 3 if t["version"] in SPENDING else 2
return CAPABILITY.match(cap) is not None and named and 1 <= j[2] <= units_priced and j[3] >= 1
top = 5 if t["version"] in JOB_VERSIONS else 4
rules = (t["start"] < t["end"] and capitals(t["currency"]) and t["seller"] != ZERO and t["buyer"] != ZERO
@ -483,6 +487,114 @@ def content_hash(b):
return blake3(CONTENT_DOMAIN + b)
def animation(b):
"""§4.8, 2: an animated PNG of 8-bit RGB frames, each the whole picture, read whole. The
chunks under a still's rules (packaging §5: every type four letters and its CRC; IHDR first,
8-bit RGB without interlace; an optional PLTE before the image data; the IDATs consecutive;
ancillary chunks read past; any other critical chunk refused; IEND last, empty, nothing after
it), and the animation's: one acTL of 8 bytes before any fcTL and the image data, of at least
one frame; each fcTL of 26 bytes, the whole picture at offsets 0, dispose 0 to 2, blend 0, the
first before the IDATs, the others after them, each followed by at least one fdAT; the fcTL
and fdAT sequence numbers 0, 1, 2, ... in order, an fdAT at least 4 bytes; as many fcTLs as
acTL counts. Its width, height and each frame's zlib stream, the IDATs' data for the first and
its fdATs' after their sequence numbers for each other; ValueError where it is not one."""
if b[:8] != b"\x89PNG\r\n\x1a\n":
raise ValueError("not a PNG signature")
at, size, count, frames, seq = 8, None, None, [], 0
ended, plte, idat = False, False, 0 # idat: 0 none yet, 1 in the run, 2 past it
while at < len(b):
if ended:
raise ValueError("bytes after IEND")
if at + 12 > len(b):
raise ValueError("a chunk past the end")
n = struct.unpack_from(">I", b, at)[0]
kind, data = b[at + 4:at + 8], b[at + 8:at + 8 + n]
if at + 12 + n > len(b):
raise ValueError("a chunk past the end")
if not all(65 <= c <= 90 or 97 <= c <= 122 for c in kind):
raise ValueError("a chunk type that is not four letters")
if zlib.crc32(kind + data) != struct.unpack_from(">I", b, at + 8 + n)[0]:
raise ValueError("a chunk CRC")
if idat == 1 and kind != b"IDAT":
idat = 2
if kind == b"IHDR":
if size is not None or at != 8 or n != 13:
raise ValueError("IHDR is not one 13-byte chunk first")
w, h = struct.unpack(">II", data[:8])
if data[8:] != bytes([8, 2, 0, 0, 0]) or w == 0 or h == 0 or 3 * w * h > 1 << 26:
raise ValueError("not 8-bit RGB without interlace, or a size past 2^26 samples")
size = (w, h)
elif size is None:
raise ValueError("a chunk before IHDR")
elif kind == b"PLTE":
if plte or idat or n == 0 or n % 3 or n > 768:
raise ValueError("PLTE: more than one, after the image data, or not 1 to 256 entries")
plte = True
elif kind == b"acTL":
if count is not None or frames or idat or n != 8:
raise ValueError("acTL is not one 8-byte chunk before the frames")
count = struct.unpack(">I", data[:4])[0]
if count == 0:
raise ValueError("acTL counts no frame")
elif kind == b"fcTL":
if count is None or n != 26:
raise ValueError("fcTL before acTL, or not 26 bytes")
q, fw, fh, x, y = struct.unpack(">IIIII", data[:20])
if q != seq:
raise ValueError("a sequence number out of order")
seq += 1
if (fw, fh, x, y) != (size[0], size[1], 0, 0):
raise ValueError("a frame that is not the whole picture")
if data[24] > 2 or data[25] != 0:
raise ValueError("a frame's dispose or blend operation")
if len(frames) == 1 and not idat:
raise ValueError("a second frame before the image data")
if len(frames) > 1 and not frames[-1]:
raise ValueError("a frame with no fdAT")
frames.append([])
elif kind == b"IDAT":
if idat == 2:
raise ValueError("IDAT chunks that are not consecutive")
if len(frames) != 1:
raise ValueError("image data that is not the first frame's")
idat = 1
frames[0].append(data)
elif kind == b"fdAT":
if len(frames) < 2 or n < 4:
raise ValueError("fdAT before a frame after the first, or shorter than 4 bytes")
if struct.unpack(">I", data[:4])[0] != seq:
raise ValueError("a sequence number out of order")
seq += 1
frames[-1].append(data[4:])
elif kind == b"IEND":
if n:
raise ValueError("IEND with data")
ended = True
elif not kind[0] & 0x20:
raise ValueError("an unknown critical chunk")
at += 12 + n
if size is None or not ended:
raise ValueError("no IHDR or no IEND")
if count is None:
raise ValueError("no acTL: not an animation")
if not idat:
raise ValueError("no image data")
if len(frames) > 1 and not frames[-1]:
raise ValueError("a frame with no fdAT")
if len(frames) != count:
raise ValueError("acTL counts another number of frames")
return size[0], size[1], [b"".join(f) for f in frames]
def frame_samples(z, w, h):
"""§4.8, 2: a frame's zlib stream read as a still's image data is (packaging §5): the still
of that one IDAT, read by the packaging verifier. Its RGB8 bytes; ValueError otherwise."""
def chunk(kind, data):
return struct.pack(">I", len(data)) + kind + data + struct.pack(">I", zlib.crc32(kind + data))
still = b"\x89PNG\r\n\x1a\n" + chunk(b"IHDR", struct.pack(">II", w, h) + bytes([8, 2, 0, 0, 0])) + chunk(b"IDAT", z) + chunk(b"IEND", b"")
return PV.png_frame(still)[2]
def job_source(s, version):
if set(s) != {"kind", "request", "pricebook", "policy", "record", "receipts", "keys", "outputs"} \
or s["receipts"] is None or s["keys"] is None or not isinstance(s["outputs"], list):
@ -504,7 +616,7 @@ def job_source(s, version):
billed = (cap.encode("utf-8"), oer_identity(a["receipt"]))
break
unbilled = [] # §6.6, version 4: every hop's figure
if version == V4 and "unbilled-hops" not in SKIP:
if version in SPENDING and "unbilled-hops" not in SKIP:
for a in rec["attempts"]:
if a["billed"] is True:
continue
@ -517,7 +629,7 @@ def job_source(s, version):
# 2); under version 4 a record that states none holds every output to none.
if rec["version"] == 2:
delivered = {bytes.fromhex(h) for a in rec["attempts"] if a["outcome"] == "served" for h in a["outputs"]}
elif version == V4:
elif version in SPENDING:
delivered = set()
else:
delivered = None
@ -526,8 +638,44 @@ def job_source(s, version):
seen = set()
outputs, packages = [], []
hexs = re.compile(r"(?:[0-9a-f]{2})*")
# §4.8, version 5: a decode delivers one animated PNG of its frames, and its codestream is
# what it read; the served hop states exactly those animations, in order, and nothing else.
decode = version == V5 and rec["operation"] == "media.decode" and "decode-frames" not in SKIP
stated = [bytes.fromhex(h) for a in rec["attempts"] if a["outcome"] == "served" for h in a["outputs"]] if rec["version"] == 2 else []
nxt = 0
for i, x in enumerate(s["outputs"]):
if version == V4 and isinstance(x, dict) and set(x) == {"package", "indexes"}:
if decode:
if not isinstance(x, dict) or set(x) != {"receipt", "codestream", "frames"} or not isinstance(x["receipt"], str) \
or not isinstance(x["codestream"], str) or not hexs.fullmatch(x["codestream"]) \
or not isinstance(x["frames"], str) or not hexs.fullmatch(x["frames"]):
raise Refuse("malformed", [])
try:
r = IMR.read(x["receipt"])
except IMR.Bad:
raise Refuse("output_invalid", ["outputs", i])
if "output" not in SKIP and IMR.rederived(r, bytes.fromhex(x["codestream"])) is not None:
raise Refuse("output_invalid", ["outputs", i])
anim = bytes.fromhex(x["frames"])
try:
w, h, parts = animation(anim)
except ValueError:
raise Refuse("output_invalid", ["outputs", i, "frames"])
if (w, h) != (r["w"], r["h"]) or len(parts) != len(r["frames"]):
raise Refuse("output_invalid", ["outputs", i, "frames"])
for k, z in enumerate(parts):
try:
rgb = frame_samples(z, w, h)
except ValueError:
raise Refuse("output_invalid", ["outputs", i, "frames", k])
if IMR.frame_hash(k, rgb) != r["frames"][k]:
raise Refuse("output_invalid", ["outputs", i, "frames", k])
outputs.append((IMR.receipt_hash(r), r["work"]))
if "output-delivered" not in SKIP:
if nxt >= len(stated) or stated[nxt] != content_hash(anim):
raise Refuse("output_not_delivered", ["outputs", i, "frames"])
nxt += 1
continue
if version in SPENDING and isinstance(x, dict) and set(x) == {"package", "indexes"}:
# §4.7: a package a run of `package.build` made, its object its receipt's text.
if not isinstance(x["package"], str) or not isinstance(x["indexes"], list) \
or not all(isinstance(h, str) and hexs.fullmatch(h) for h in x["indexes"]):
@ -556,6 +704,8 @@ def job_source(s, version):
if h not in delivered or h in seen:
raise Refuse("output_not_delivered", ["outputs", i])
seen.add(h)
if decode and "output-delivered" not in SKIP and nxt != len(stated):
raise Refuse("output_not_delivered", ["outputs"]) # §4.8, 3: an object past the animations
return {"kind": JOB, "digest": bytes.fromhex(RRV.digest(rec)), "router": bytes.fromhex(rec["signer"]), "job": rec["job"],
"operation": rec["operation"].encode("utf-8"), "accepted": rec["accepted"], "hops": [oer_identity(a["receipt"]) for a in rec["attempts"]], "billed": billed,
"figure": fig, "outputs": outputs, "packages": packages, "unbilled": unbilled}
@ -568,7 +718,7 @@ def verify_source(s, signers, version=1):
"""§5.1: the source verified by its own extension, a stitched source with `signers`, a job
source under a tariff of version 2 or 3, or Refuse with that extension's place."""
key = json.dumps([s, sorted(k.hex() for k in signers), version if isinstance(s, dict) and s.get("kind") == "job" else version in JOB_VERSIONS,
sorted(SKIP & {"job-figure", "output", "output-delivered", "unbilled-hops", "run-work"})], sort_keys=True)
sorted(SKIP & {"job-figure", "output", "output-delivered", "unbilled-hops", "run-work", "decode-frames"})], sort_keys=True)
if key not in _SOURCES:
try:
_SOURCES[key] = read_source(s, signers, version)
@ -694,14 +844,14 @@ def billed_once(version, kind):
receipt is the evidence of what it delivered, which a re-run delivers again; its work, the route
record and the hop receipts, is billed once. In version 4 a package receipt is listed again by
each invoice that bills its storage (§4.7)."""
if kind == PACK and version == V4 and "package-again" not in SKIP:
if kind == PACK and version in SPENDING and "package-again" not in SKIP:
return False
return not (kind == VIDEO_RECEIPT and (version in WORK_VERSIONS or "output-once" in SKIP))
def hold(t, facts, previous, start, end):
"""§4.7: under version 4, a package the previous invoice lists is held through [start, end)."""
if t["version"] != V4:
if t["version"] not in SPENDING:
return
held = {i for k, i in (previous["records"] if previous else []) if k == PACK}
for k, f in enumerate(facts):
@ -756,7 +906,7 @@ def compute(t, facts):
e[1] |= set(cites)
for f in facts:
if f["kind"] == ROOM:
for unit in range(1, 15 if t["version"] == V4 and "bytes-out" not in SKIP else 14):
for unit in range(1, 15 if t["version"] in SPENDING and "bytes-out" not in SKIP else 14):
add(ROOM, f["room"], unit, f["units"][unit - 1], [(SESSION, f["digest"])])
elif f["kind"] == DELIVERY:
st = f["statement"]
@ -768,7 +918,7 @@ def compute(t, facts):
cites += [(ACK, a["digest"]) for a in f["acks"]
if a["mode"] == 1 and any(e["class"] == c and not free(c, e["grant"]) for e in a["entries"])]
add(DELIVERY, st["party"], c, n, cites)
elif f["kind"] == PACKAGE and t["version"] == V4: # §4.7: the product held, its storage alone
elif f["kind"] == PACKAGE and t["version"] in SPENDING: # §4.7: the product held, its storage alone
if f["held"] is not None:
add(PACKAGE, f["signer"], 3, f["stored"] * f["held"], [(PACK, f["digest"])]) # byte-microseconds
elif f["kind"] == PACKAGE:
@ -806,7 +956,7 @@ def compute(t, facts):
lines, total = [], 0
for (kind, source, unit) in sorted(q):
n, cited = q[(kind, source, unit)]
if t["version"] == V4 and kind == PACKAGE and unit == 3:
if t["version"] in SPENDING and kind == PACKAGE and unit == 3:
n //= DAY_US # §4.7: byte-days, rounded down once per line
if n == 0:
continue
@ -831,7 +981,7 @@ def compute(t, facts):
seen.add(d)
figs.append((2, fig))
for f in facts:
if f["kind"] == PACKAGE and t["version"] != V4: # §6.6: in version 4, its run's hop states it
if f["kind"] == PACKAGE and t["version"] not in SPENDING: # §6.6: in version 4, its run's hop states it
figs.append((3, f["figure"]))
elif f["kind"] == ADS:
figs += [(4, r["figure"]) for r in f["receipts"]]
@ -1208,6 +1358,12 @@ def mutations(doc):
m = copy.deepcopy(doc)
m["sources"]["job_v2_encode"]["outputs"] = doc["sources"]["job_v2_another_output"]["outputs"]
out.append(("job_v2_encode: its output replaced by another clip's receipt", m))
if "job_v5_decode" in doc["sources"]: # §4.8: a decode held to its frames
for other, what in (("job_v5_frames_swapped", "its first two frames swapped"), ("job_v5_a_frame_altered", "a frame with one sample changed"),
("job_v5_a_frame_missing", "its last frame left out")):
m = copy.deepcopy(doc)
m["sources"]["job_v5_decode"]["outputs"][0]["frames"] = doc["sources"][other]["outputs"][0]["frames"]
out.append((f"job_v5_decode: {what}", m))
for k, v in enumerate(doc["vectors"]):
if v["outcome"]["outcome"] == "refuse":
m = copy.deepcopy(doc)
@ -1220,7 +1376,8 @@ def mutations(doc):
CHECKS = ["signatures", "buyer-signature", "tariff-rules", "invoice-rules", "row-of-no-class", "binding", "outside-tariff",
"previous-given", "previous-overlap", "scope", "conflict", "period", "duplicate", "overlap", "rebilled", "quantity",
"summed", "citations", "mismatch", "job-figure", "output", "job-duplicate", "private-kind", "operation", "output-once",
"output-delivered", "unbilled-hops", "storage-held", "package-again", "run-work", "bytes-out", "conflict-bytes"]
"output-delivered", "unbilled-hops", "storage-held", "package-again", "run-work", "bytes-out", "conflict-bytes",
"decode-frames"]
def main(corpus_dir):

10
wai/wai-ledger/src/ledger.rs
View file

@ -68,10 +68,10 @@ impl Chain {
}
/// Whether the next invoice bills `h`: a source the chain has not billed, or, under a tariff
/// of version 4, a package the last invoice holds, whose storage each period bills (§4.7).
/// of version 4 or 5, a package the last invoice holds, whose storage each period bills (§4.7).
fn bills(&self, h: &Held) -> bool {
let held = || {
matches!(h.facts, Facts::Package(_)) && self.tariff.version == rl::VERSION_4 && self.invoices.last().is_some_and(|(i, _, _)| h.facts.heads().iter().all(|r| i.records.contains(r)))
matches!(h.facts, Facts::Package(_)) && rl::bills_spending(self.tariff.version) && self.invoices.last().is_some_and(|(i, _, _)| h.facts.heads().iter().all(|r| i.records.contains(r)))
};
!self.billed(h) || held()
}
@ -166,10 +166,10 @@ impl Ledger {
let mut all: Vec<Facts> = c.sources.iter().map(|h| h.facts.clone()).collect();
all.push(facts.clone());
rl::check_across(c.tariff.version, &all, None).map_err(|(_, r)| refuse(r.code(), json!(["source"])))?;
// A record the chain billed is billed never again; under a tariff of version 3 or 4, a
// A record the chain billed is billed never again; under a tariff of versions 3 to 5, a
// job's output receipt is cited again by a re-run, whose hop and route record are new
// (§6.9). A package source is brought once: the ledger holds it, and under version 4 bills
// its storage each period from what it holds (§4.7).
// (§6.9). A package source is brought once: the ledger holds it, and under versions 4 and
// 5 bills its storage each period from what it holds (§4.7).
let package_again = matches!(facts, Facts::Package(_)) && facts.heads().iter().all(|h| c.billed.contains(h));
if package_again || facts.records().iter().any(|r| rl::billed_once(c.tariff.version, r.0) && c.billed.contains(r)) {
return Err(refuse(Reason::RecordRebilled.code(), json!(["source"])));

7
wai/wai-rs/Cargo.toml
View file

@ -512,9 +512,10 @@ settlement = ["live_metering", "jwp_hash"]
# joules are per scope and per class, never summed across either, recomputed by a verifier; its
# chain; and the ad and delivery reports. A tariff of version 2 also prices media jobs (§4.5): a
# job source is its route record (route_record, kind 16) and its attempts' OER/2 receipts (kind
# 14), and an `int_motion` job's decode receipt (video_receipt, kind 15). SHA-256, Ed25519 and the
# sources' own readers; no new dependency.
receipt_ledger =["live_metering", "settlement", "live_ads", "video_package", "stitched_presentation", "route_record", "video_receipt"]
# 14), and an `int_motion` job's decode receipt (video_receipt, kind 15); under a tariff of
# version 5 a decode's frames, each a PNG read by packaging §5.7's reader (png_still) and held to
# its receipt's frame (§4.8). SHA-256, Ed25519 and the sources' own readers; no new dependency.
receipt_ledger =["live_metering", "settlement", "live_ads", "video_package", "stitched_presentation", "route_record", "video_receipt", "png_still"]
# Route record (extensions/route-record.md): `wai.route.record` — what a router considered for one
# media job and why it chose: the request, the price book and the routing policy as documents
# pinned by digest; the candidates, the limits that removed each, the ranking, the order and the

358
wai/wai-rs/src/png_still.rs
View file

@ -33,8 +33,13 @@
//! checked and no byte of the image data after it; its rows are exactly `1 + 3·width`
//! bytes each, and the five filters are undone.
//!
//! An animated PNG of such frames, each the whole picture, is written by
//! [`encode_animation`] and read by [`animation`]: the form a decode delivers its
//! reconstruction in ([receipt-ledger] §4.8).
//!
//! [packaging]: ../../extensions/packaging.md
//! [deriving-edge]: ../../extensions/deriving-edge.md
//! [receipt-ledger]: ../../extensions/receipt-ledger.md
use std::collections::BTreeMap;
@ -202,19 +207,69 @@ pub fn encode_grey(grey: &[u8], w: u32, h: u32) -> Vec<u8> {
encode_typed(grey, w, h, 0)
}
fn encode_typed(samples: &[u8], w: u32, h: u32, colour: u8) -> Vec<u8> {
let bpp = if colour == 0 { 1 } else { 3 };
/// The zlib stream of an 8-bit frame's rows, each filtered `Sub`, as the writer writes it.
fn compressed(samples: &[u8], w: u32, h: u32, bpp: usize) -> Vec<u8> {
let raw = filtered(samples, w, h, bpp);
let mut z = vec![0x78, 0x01];
z.extend_from_slice(&deflate_fixed(&raw));
z.extend_from_slice(&adler32(&raw).to_be_bytes());
z
}
fn ihdr(w: u32, h: u32, colour: u8) -> Vec<u8> {
let mut ihdr = Vec::with_capacity(13);
ihdr.extend_from_slice(&w.to_be_bytes());
ihdr.extend_from_slice(&h.to_be_bytes());
ihdr.extend_from_slice(&[8, colour, 0, 0, 0]);
ihdr
}
fn encode_typed(samples: &[u8], w: u32, h: u32, colour: u8) -> Vec<u8> {
let bpp = if colour == 0 { 1 } else { 3 };
let mut out = SIGNATURE.to_vec();
chunk(&mut out, b"IHDR", &ihdr);
chunk(&mut out, b"IDAT", &z);
chunk(&mut out, b"IHDR", &ihdr(w, h, colour));
chunk(&mut out, b"IDAT", &compressed(samples, w, h, bpp));
chunk(&mut out, b"IEND", &[]);
out
}
/// An animated PNG (PNG's third edition: `acTL`, `fcTL`, `fdAT`) of RGB8 frames of `w × h`,
/// each shown for `delay` seconds (a numerator and a denominator), every byte a pure function
/// of the frames and the delay: the signature; `IHDR` as a still's; `acTL` (the frame count,
/// played once); then each frame a `fcTL` of the whole picture (offsets 0, the delay, dispose
/// 0, blend 0) and its data, the first in one `IDAT`, so the default image is the first frame,
/// and each after it in one `fdAT`, the `fcTL`s and `fdAT`s numbered from 0 in their order;
/// `IEND`. Each frame's data is the zlib stream a still's is. A decode's reconstruction is
/// delivered as one ([receipt-ledger] §4.8).
///
/// [receipt-ledger]: ../../extensions/receipt-ledger.md
pub fn encode_animation<F: AsRef<[u8]>>(frames: &[F], w: u32, h: u32, delay: (u16, u16)) -> Vec<u8> {
let mut out = SIGNATURE.to_vec();
chunk(&mut out, b"IHDR", &ihdr(w, h, 2));
let mut actl = (frames.len() as u32).to_be_bytes().to_vec();
actl.extend_from_slice(&1u32.to_be_bytes());
chunk(&mut out, b"acTL", &actl);
let mut seq = 0u32;
for (k, f) in frames.iter().enumerate() {
let mut fctl = seq.to_be_bytes().to_vec();
for v in [w, h, 0, 0] {
fctl.extend_from_slice(&v.to_be_bytes());
}
fctl.extend_from_slice(&delay.0.to_be_bytes());
fctl.extend_from_slice(&delay.1.to_be_bytes());
fctl.extend_from_slice(&[0, 0]);
chunk(&mut out, b"fcTL", &fctl);
seq += 1;
let z = compressed(f.as_ref(), w, h, 3);
if k == 0 {
chunk(&mut out, b"IDAT", &z);
} else {
let mut fdat = seq.to_be_bytes().to_vec();
fdat.extend_from_slice(&z);
chunk(&mut out, b"fdAT", &fdat);
seq += 1;
}
}
chunk(&mut out, b"IEND", &[]);
out
}
@ -325,11 +380,16 @@ fn decode_typed(b: &[u8], grey_ok: bool) -> Result<(u32, u32, usize, Vec<u8>), S
if !ended {
return Err("no IEND".into());
}
Ok((w, h, bpp, rows(&z, w, h, bpp)?))
}
/// A picture's samples from its image data's zlib stream: a 32 KiB window at most (PNG's
/// CINFO ≤ 7), its Adler-32 checked, every byte of the image data in it, exactly the rows the
/// picture has, and the five filters undone.
fn rows(z: &[u8], w: u32, h: u32, bpp: usize) -> Result<Vec<u8>, String> {
let stride = bpp * w as usize;
let want = (stride + 1) * h as usize;
// The zlib stream: a 32 KiB window at most (PNG's CINFO ≤ 7), its Adler-32 checked,
// every byte of the image data in it, and exactly the rows the picture has.
let raw = inflate_zlib(&z, want)?;
let raw = inflate_zlib(z, want)?;
if raw.len() != want {
return Err("the rows are not the picture's size".into());
}
@ -352,7 +412,186 @@ fn decode_typed(b: &[u8], grey_ok: bool) -> Result<(u32, u32, usize, Vec<u8>), S
out[y * stride + i] = row[i].wrapping_add(p);
}
}
Ok((w, h, bpp, out))
Ok(out)
}
/// An animated PNG read whole by [`animation`]: its size, and each frame's image data, read
/// to its samples by [`Animation::frame`].
pub struct Animation<'a> {
pub w: u32,
pub h: u32,
/// Each frame's zlib stream in pieces: the first frame's `IDAT`s' data, each later frame's
/// `fdAT`s' data less their sequence numbers.
parts: Vec<Vec<&'a [u8]>>,
}
impl Animation<'_> {
/// How many frames it has.
pub fn frames(&self) -> usize {
self.parts.len()
}
/// Frame `k`'s RGB8 samples: its zlib stream inflated as a still's is, its rows exactly
/// the picture's and its filters undone.
pub fn frame(&self, k: usize) -> Result<Vec<u8>, String> {
let z = self.parts.get(k).ok_or("no such frame")?.concat();
rows(&z, self.w, self.h, 3)
}
}
fn be32(b: &[u8], at: usize) -> u32 {
u32::from_be_bytes(b[at..at + 4].try_into().expect("4"))
}
/// Read an animated PNG of 8-bit RGB frames, each the whole picture (receipt-ledger §4.8): the
/// chunks under a still's rules (every type four letters and its CRC checked; `IHDR` first, of
/// 8-bit RGB without interlace; an optional `PLTE` before the image data; the `IDAT`s
/// consecutive; ancillary chunks read past; any other critical chunk refused; `IEND` last,
/// empty, and nothing after it), and the animation's:
///
/// - one `acTL` of 8 bytes, before any `fcTL` and the image data, of at least one frame;
/// - each `fcTL` of 26 bytes, of the whole picture (the width and height `IHDR`'s, offsets 0),
/// dispose 0 to 2 and blend 0 (the frame's samples replace the picture's: nothing is
/// composited), any delay; the first before the `IDAT`s, so the default image is the first
/// frame, the others after them, each followed by at least one `fdAT`;
/// - the sequence numbers of the `fcTL`s and `fdAT`s 0, 1, 2, … in their order, and an `fdAT`
/// at least 4 bytes long;
/// - as many `fcTL`s as `acTL` counts.
///
/// Each frame's data is the `IDAT`s' for the first, its `fdAT`s' after their sequence numbers
/// for each other, concatenated, and is read only by [`Animation::frame`].
pub fn animation(b: &[u8]) -> Result<Animation<'_>, String> {
if b.len() < 8 || b[..8] != SIGNATURE {
return Err("not a PNG signature".into());
}
let mut at = 8usize;
let mut size: Option<(u32, u32)> = None;
let mut count: Option<u32> = None;
let mut parts: Vec<Vec<&[u8]>> = Vec::new();
let mut seq = 0u32;
let (mut ended, mut palette) = (false, false);
// 0: no IDAT yet; 1: in the IDAT run; 2: past it.
let mut idat = 0u8;
while at < b.len() {
if ended {
return Err("bytes after IEND".into());
}
let len = u32::from_be_bytes(b.get(at..at + 4).ok_or("a chunk length past the end")?.try_into().expect("4")) as usize;
let kind: [u8; 4] = b.get(at + 4..at + 8).ok_or("a chunk type past the end")?.try_into().expect("4");
let data = b.get(at + 8..at + 8 + len).ok_or("a chunk past the end")?;
let crc = u32::from_be_bytes(b.get(at + 8 + len..at + 12 + len).ok_or("a chunk CRC past the end")?.try_into().expect("4"));
if !kind.iter().all(u8::is_ascii_alphabetic) {
return Err("a chunk type that is not four letters".into());
}
if crc != crc32(&[&kind, data]) {
return Err(format!("{}: its CRC", String::from_utf8_lossy(&kind)));
}
if idat == 1 && &kind != b"IDAT" {
idat = 2;
}
match &kind {
b"IHDR" => {
if size.is_some() || at != 8 || data.len() != 13 {
return Err("IHDR is not one 13-byte chunk first".into());
}
let (w, h) = (be32(data, 0), be32(data, 4));
if data[8..] != [8, 2, 0, 0, 0] {
return Err("only 8-bit RGB, deflate, adaptive filtering, no interlace".into());
}
if w == 0 || h == 0 || (w as u64) * (h as u64) * 3 > MAX_SAMPLES {
return Err("a size outside 1 to 2^26 samples".into());
}
size = Some((w, h));
}
_ if size.is_none() => return Err("a chunk before IHDR".into()),
b"PLTE" => {
if palette || idat != 0 || data.is_empty() || data.len() % 3 != 0 || data.len() > 768 {
return Err("PLTE: more than one, after the image data, or not 1 to 256 entries".into());
}
palette = true;
}
b"acTL" => {
if count.is_some() || !parts.is_empty() || idat != 0 || data.len() != 8 {
return Err("acTL is not one 8-byte chunk before the frames".into());
}
let n = be32(data, 0);
if n == 0 {
return Err("acTL counts no frame".into());
}
count = Some(n);
}
b"fcTL" => {
let (w, h) = size.expect("IHDR first");
if count.is_none() || data.len() != 26 {
return Err("fcTL before acTL, or not 26 bytes".into());
}
if be32(data, 0) != seq {
return Err("a sequence number out of order".into());
}
seq = seq.checked_add(1).ok_or("a sequence number past 2^32")?;
if (be32(data, 4), be32(data, 8), be32(data, 12), be32(data, 16)) != (w, h, 0, 0) {
return Err("a frame that is not the whole picture".into());
}
if data[24] > 2 || data[25] != 0 {
return Err("a frame disposed of as no rule says, or blended over the one before".into());
}
// The first frame's data is the IDATs, which follow it (an IDAT before it is
// refused where it is read); every other frame's are fdATs after the IDATs, and
// the frame before it has some.
match parts.len() {
0 => {}
1 if idat == 0 => return Err("a second frame before the image data".into()),
n if n > 1 && parts[n - 1].is_empty() => return Err("a frame with no fdAT".into()),
_ => {}
}
parts.push(Vec::new());
}
b"IDAT" => {
if idat == 2 {
return Err("IDAT chunks that are not consecutive".into());
}
if parts.len() != 1 {
return Err("image data that is not the first frame's: the default image apart from the frames".into());
}
idat = 1;
parts[0].push(data);
}
b"fdAT" => {
if parts.len() < 2 || data.len() < 4 {
return Err("fdAT before a frame after the first, or shorter than 4 bytes".into());
}
if be32(data, 0) != seq {
return Err("a sequence number out of order".into());
}
seq = seq.checked_add(1).ok_or("a sequence number past 2^32")?;
parts.last_mut().expect("a frame").push(&data[4..]);
}
b"IEND" => {
if !data.is_empty() {
return Err("IEND with data".into());
}
ended = true;
}
_ if kind[0] & 0x20 == 0 => return Err(format!("an unknown critical chunk {}", String::from_utf8_lossy(&kind))),
_ => {}
}
at += 12 + len;
}
let (w, h) = size.ok_or("no IHDR")?;
if !ended {
return Err("no IEND".into());
}
let n = count.ok_or("no acTL: not an animation")?;
if idat == 0 {
return Err("no image data".into());
}
if parts.len() > 1 && parts[parts.len() - 1].is_empty() {
return Err("a frame with no fdAT".into());
}
if parts.len() as u64 != u64::from(n) {
return Err("acTL counts another number of frames".into());
}
Ok(Animation { w, h, parts })
}
// ─────────────────────────────── inflate ───────────────────────────────
@ -688,4 +927,107 @@ mod tests {
assert!(decode(b"not a png").is_err());
assert!(chunk_kinds(b"not a png").is_none());
}
/// The chunks of a PNG, and a PNG of chunks, each CRC written afresh.
fn chunks(b: &[u8]) -> Vec<([u8; 4], Vec<u8>)> {
let mut at = 8;
let mut out = Vec::new();
while at < b.len() {
let len = be32(b, at) as usize;
out.push((b[at + 4..at + 8].try_into().unwrap(), b[at + 8..at + 8 + len].to_vec()));
at += 12 + len;
}
out
}
fn png_of(cs: &[([u8; 4], Vec<u8>)]) -> Vec<u8> {
let mut out = SIGNATURE.to_vec();
for (k, d) in cs {
chunk(&mut out, k, d);
}
out
}
/// An animation reads back to its frames, a repeated frame among them, and holds the
/// chunks the writer writes; its default image is its first frame, as a still reader reads
/// it; one frame is an animation too.
#[test]
fn an_animation_round_trips() {
let (w, h) = (9, 5);
let a = frame(w, h, |x, y, c| (x * 20 + y * 7 + c) as u8);
let b = frame(w, h, |x, y, c| ((x ^ y ^ c) * 31) as u8);
let frames = [a.clone(), b.clone(), b.clone(), a.clone()];
let png = encode_animation(&frames, w, h, (1, 30));
let r = animation(&png).unwrap();
assert_eq!((r.w, r.h, r.frames()), (w, h, 4));
for (k, f) in frames.iter().enumerate() {
assert_eq!(&r.frame(k).unwrap(), f, "frame {k}");
}
assert!(r.frame(4).is_err());
assert_eq!(chunk_kinds(&png).unwrap(), [b"IHDR", b"acTL", b"fcTL", b"IDAT", b"fcTL", b"fdAT", b"fcTL", b"fdAT", b"fcTL", b"fdAT", b"IEND"].map(|k| *k).to_vec());
assert_eq!(decode(&png).unwrap(), (w, h, a.clone()), "a still reader reads the first frame");
assert_eq!(encode_animation(&frames, w, h, (1, 30)), png, "every byte a function of the frames and the delay");
let one = encode_animation(&[a.clone()], w, h, (1, 25));
assert_eq!(animation(&one).unwrap().frame(0).unwrap(), a);
assert!(animation(&encode(&a, w, h)).is_err(), "a still is not an animation");
}
/// Each of the animation's rules refused alone.
#[test]
fn the_animation_reader_refuses_what_it_does_not_read() {
let (w, h) = (6, 4);
let frames = [frame(w, h, |x, _, c| (x + c) as u8), frame(w, h, |_, y, c| (y * 9 + c) as u8), frame(w, h, |x, y, _| (x * y) as u8)];
let good = chunks(&encode_animation(&frames, w, h, (1, 10)));
assert!(animation(&png_of(&good)).is_ok());
let refused = |what: &str, f: &dyn Fn(&mut Vec<([u8; 4], Vec<u8>)>)| {
let mut cs = good.clone();
f(&mut cs);
assert!(animation(&png_of(&cs)).is_err(), "{what}");
};
// The place of frame `k`'s fcTL in the writer's chunks.
fn fctl(cs: &[([u8; 4], Vec<u8>)], k: usize) -> usize {
cs.iter().position(|c| &c.0 == b"fcTL").unwrap() + 2 * k
}
refused("no acTL", &|cs| { cs.remove(1); });
refused("acTL counting another number", &|cs| cs[1].1[3] = 4);
refused("acTL counting none", &|cs| cs[1].1[..4].copy_from_slice(&[0; 4]));
refused("acTL after the first fcTL", &|cs| cs.swap(1, 2));
refused("a frame offset", &|cs| { let i = fctl(cs, 1); cs[i].1[15] = 1; });
refused("a frame narrower than the picture", &|cs| { let i = fctl(cs, 2); cs[i].1[7] -= 1; });
refused("a frame blended over", &|cs| { let i = fctl(cs, 1); cs[i].1[25] = 1; });
refused("a frame disposed of by no rule", &|cs| { let i = fctl(cs, 1); cs[i].1[24] = 3; });
refused("a sequence number skipped", &|cs| { let i = fctl(cs, 2); cs[i].1[3] += 1; });
refused("an fdAT's sequence number repeated", &|cs| { let i = fctl(cs, 1) + 1; cs[i].1[3] -= 1; });
refused("the default image apart from the frames", &|cs| cs.swap(2, 3));
refused("the last frame with no fdAT", &|cs| { let i = fctl(cs, 2) + 1; cs.remove(i); });
refused("an fdAT of the first frame", &|cs| { let d = cs[3].1.clone(); let mut f = vec![0, 0, 0, 1]; f.extend(d); cs.insert(4, (*b"fdAT", f)); });
refused("an IDAT after the second frame", &|cs| { let i = fctl(cs, 1) + 1; cs[i].0 = *b"IDAT"; });
refused("an fcTL of 25 bytes", &|cs| { let i = fctl(cs, 1); cs[i].1.pop(); });
// A frame's data is read when the frame is: the animation reads, and that frame does not.
let mut cs = good.clone();
let i = fctl(&cs, 2) + 1;
cs[i].1.truncate(9);
let b = png_of(&cs);
let r = animation(&b).unwrap();
assert!(r.frame(1).is_ok() && r.frame(2).is_err(), "a frame's data not the picture's rows");
// An ancillary chunk between frames is read past, and the frames are unchanged.
let mut cs = good.clone();
let i = fctl(&cs, 1);
cs.insert(i, (*b"tEXt", b"Comment\0a decode".to_vec()));
let b = png_of(&cs);
let r = animation(&b).unwrap();
assert_eq!(r.frame(2).unwrap(), frames[2]);
// A frame's data may be split across fdATs, each numbered.
let mut cs = good.clone();
let i = fctl(&cs, 2) + 1;
let d = cs[i].1.clone();
let n = u32::from_be_bytes(d[..4].try_into().unwrap());
cs[i].1 = d[..10].to_vec();
let mut rest = (n + 1).to_be_bytes().to_vec();
rest.extend_from_slice(&d[10..]);
cs.insert(i + 1, (*b"fdAT", rest));
let b = png_of(&cs);
let r = animation(&b).unwrap();
assert_eq!(r.frame(2).unwrap(), frames[2]);
}
}

231
wai/wai-rs/src/receipt_ledger.rs
View file

@ -16,14 +16,15 @@
//! What this module implements, by section of the extension:
//!
//! - §4: the tariff ([`Tariff`]), its two signatures and its scope, and its job prices: per
//! serving capability in version 2, per serving capability and operation in versions 3 and
//! 4; version 4 prices a room's bytes out, the work units of a package each run that made it
//! states, and a package's storage by byte-time (§4.7);
//! serving capability in version 2, per serving capability and operation in versions 3 to
//! 5; version 4 prices a room's bytes out, the work units of a package each run that made it
//! states, and a package's storage by byte-time (§4.7); version 5 bills as version 4 does and
//! holds a decode to the frames it delivered (§4.8);
//! - §5: the sources ([`verify_source`]) and what each contributes ([`Facts`]);
//! - §6: the invoice ([`Invoice`]), its lines, citations, energy and chain ([`compute`]), and
//! what a chain bills once ([`billed_once`]): in versions 3 and 4, a job's work, never its
//! output's receipt; in version 4, a package's work once per run and its storage for each
//! period it is held, and every hop's energy, the unbilled ones in a scope of their own;
//! what a chain bills once ([`billed_once`]): in versions 3 to 5, a job's work, never its
//! output's receipt; in versions 4 and 5, a package's work once per run and its storage for
//! each period it is held, and every hop's energy, the unbilled ones in a scope of their own;
//! - §7: verification ([`verify`]); §8: the reports.
//!
//! The corpus `live-conformance/ledger/` is written by [`corpus::generate`] and checked by
@ -63,6 +64,11 @@ pub const VERSION_3: u8 = 3;
/// The version of a tariff that prices delivery by the byte, a package's work and its storage
/// by byte-time, and of its invoices, which state every hop's energy (§4.7, §6.6).
pub const VERSION_4: u8 = 4;
/// The version of a tariff that bills as version 4 does and holds a decode to what it delivered,
/// its frames, and of its invoices (§4.8).
pub const VERSION_5: u8 = 5;
/// The catalogue's operation whose hop delivers a reconstruction, not a codestream (§4.8).
pub const DECODE: &str = "media.decode";
pub const TARIFF_DOMAIN: &[u8] = b"wai:ledger-tariff\x01";
pub const ACCEPT_DOMAIN: &[u8] = b"wai:ledger-accept\x01";
pub const INVOICE_DOMAIN: &[u8] = b"wai:ledger-invoice\x01";
@ -135,11 +141,11 @@ pub mod scope {
["", "room_hops", "delivery_hops", "packaging", "ad_decisions", "jobs", "unbilled_hops"].get(s as usize).copied().unwrap_or("")
}
/// The scopes an invoice of `version` states (§6.1): 4, 5 in versions 2 and 3, 6 in 4.
/// The scopes an invoice of `version` states (§6.1): 4, 5 in versions 2 and 3, 6 in 4 and 5.
pub fn count(version: u8) -> u8 {
match version {
super::VERSION_2 | super::VERSION_3 => JOBS,
super::VERSION_4 => UNBILLED_HOPS,
super::VERSION_4 | super::VERSION_5 => UNBILLED_HOPS,
_ => AD_DECISIONS,
}
}
@ -150,14 +156,14 @@ pub fn unit_name(kind: u8, unit: u8) -> Option<&'static str> {
unit_name_in(VERSION_3, kind, unit)
}
/// The unit names of each source kind in an invoice of `version` (§6.3): version 4 adds a
/// room's `bytes_out`, and bills a package source by its `byte_days` alone (§4.7), unit 3.
/// The unit names of each source kind in an invoice of `version` (§6.3): versions 4 and 5 add a
/// room's `bytes_out`, and bill a package source by its `byte_days` alone (§4.7), unit 3.
pub fn unit_name_in(version: u8, kind: u8, unit: u8) -> Option<&'static str> {
const ROOM: [&str; 14] = [
"publisher_us", "subscriber_us_0", "subscriber_us_1", "subscriber_us_2", "subscriber_us_3", "subscriber_us_4", "subscriber_us_5", "subscriber_us_6", "subscriber_us_7",
"subscriber_us_8", "composition_us", "recorded_us", "events_sent", "bytes_out",
];
let v4 = version == VERSION_4;
let v4 = bills_spending(version);
let names: &[&str] = match kind {
source_kind::ROOM if v4 => &ROOM,
source_kind::ROOM => &ROOM[..ROOM_PRICES_1],
@ -362,25 +368,38 @@ fn read_head(b: &[u8], magic: &[u8; 4]) -> Result<u8, &'static str> {
}
match b.get(4) {
None => Err(TRUNCATED),
Some(&v) if (VERSION..=VERSION_4).contains(&v) => Ok(v),
Some(&v) if (VERSION..=VERSION_5).contains(&v) => Ok(v),
Some(_) => Err(UNKNOWN_VERSION),
}
}
/// Whether a tariff or an invoice of `version` prices jobs (§4.5, §4.6): versions 2 to 4.
/// Whether a tariff or an invoice of `version` prices jobs (§4.5, §4.6): versions 2 to 5.
pub fn prices_jobs(version: u8) -> bool {
(VERSION_2..=VERSION_4).contains(&version)
(VERSION_2..=VERSION_5).contains(&version)
}
/// Whether a tariff of `version` prices a job by its operation and bills its work (§4.6, §6.9):
/// versions 3 and 4.
/// versions 3 to 5.
pub fn bills_work(version: u8) -> bool {
version == VERSION_3 || version == VERSION_4
(VERSION_3..=VERSION_5).contains(&version)
}
/// The room prices a tariff of `version` states (§4.1): fourteen in version 4, else thirteen.
/// Whether a tariff of `version` bills what the work spends (§4.7, §6.6): a room's bytes out, a
/// package's work per run and its storage by byte-time, and every hop's energy. Versions 4 and 5.
pub fn bills_spending(version: u8) -> bool {
version == VERSION_4 || version == VERSION_5
}
/// Whether a tariff of `version` holds a decode to the frames it delivered, the codestream it
/// read being its input (§4.8): version 5.
pub fn holds_decode_to_frames(version: u8) -> bool {
version == VERSION_5
}
/// The room prices a tariff of `version` states (§4.1): fourteen in versions 4 and 5, else
/// thirteen.
pub fn room_prices(version: u8) -> usize {
if version == VERSION_4 { ROOM_PRICES } else { ROOM_PRICES_1 }
if bills_spending(version) { ROOM_PRICES } else { ROOM_PRICES_1 }
}
/// A job price (§4.5, §4.6): a capability, in version 3 an operation, a unit (1 `jobs`, 2
@ -400,9 +419,9 @@ pub struct JobPrice {
/// version 4 only (§4.7).
pub const JOB_UNITS: [&str; 3] = ["jobs", "subpixels", "work_units"];
/// The job units a tariff of `version` prices (§4.5, §4.7): two, and three in version 4.
/// The job units a tariff of `version` prices (§4.5, §4.7): two, and three in versions 4 and 5.
pub fn job_units(version: u8) -> u8 {
if version == VERSION_4 { 3 } else { 2 }
if bills_spending(version) { 3 } else { 2 }
}
fn job_capability(c: &str) -> bool {
@ -429,20 +448,21 @@ fn ascending<T: PartialOrd>(v: &[T]) -> bool {
pub struct Tariff {
/// 1; 2 for a tariff that prices jobs (§4.5); 3 for one that prices them per operation and
/// bills their work (§4.6, §6.9); 4 for one that also prices bytes out, a package's work and
/// its storage by byte-time (§4.7).
/// its storage by byte-time (§4.7); 5 for one that bills as version 4 does and holds a decode
/// to the frames it delivered (§4.8).
pub version: u8,
pub seller: [u8; 32],
pub buyer: [u8; 32],
pub start: u64,
pub end: u64,
pub currency: [u8; 3],
/// Live-metering §4.1's thirteen prices, in its order, and in version 4 a fourteenth, per
/// 10^9 bytes out; versions 1 to 3 state no fourteenth, read as 0.
/// Live-metering §4.1's thirteen prices, in its order, and in versions 4 and 5 a fourteenth,
/// per 10^9 bytes out; versions 1 to 3 state no fourteenth, read as 0.
pub room: [u32; ROOM_PRICES],
/// Settlement §7.1's four prices, in class order.
pub delivery: [u64; 4],
/// Per package receipt, per entry, per 10^9 stored bytes; in version 4 only the third, per
/// 10^9 byte-days held, the first two neither written nor read, 0 (§4.7).
/// Per package receipt, per entry, per 10^9 stored bytes; in versions 4 and 5 only the third,
/// per 10^9 byte-days held, the first two neither written nor read, 0 (§4.7).
pub package: [u64; 3],
/// Per presented impression, per billable impression.
pub ads: [u64; 2],
@ -469,7 +489,7 @@ impl Tariff {
for p in &self.room[..room_prices(self.version)] {
o.extend_from_slice(&p.to_le_bytes());
}
let package = if self.version == VERSION_4 { &self.package[2..] } else { &self.package[..] };
let package = if bills_spending(self.version) { &self.package[2..] } else { &self.package[..] };
for p in self.delivery.iter().chain(package).chain(&self.ads) {
o.extend_from_slice(&p.to_le_bytes());
}
@ -534,7 +554,7 @@ impl Tariff {
*p = r.u64()?;
}
let mut package = [0u64; 3];
for p in &mut package[if version == VERSION_4 { 2 } else { 0 }..] {
for p in &mut package[if bills_spending(version) { 2 } else { 0 }..] {
*p = r.u64()?;
}
let ads = [r.u64()?, r.u64()?];
@ -650,7 +670,7 @@ pub struct Row {
/// An invoice (§6.1).
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Invoice {
/// Its tariff's version, 1 to 4.
/// Its tariff's version, 1 to 5.
pub version: u8,
pub seller: [u8; 32],
pub buyer: [u8; 32],
@ -662,8 +682,8 @@ pub struct Invoice {
pub records: Vec<(u8, [u8; 32])>,
pub lines: Vec<Line>,
pub energy: Vec<Row>,
/// Scopes 1 to 4, in versions 2 and 3 scope 5, and in version 4 scope 6 (§6.1); an invoice
/// states only its version's.
/// Scopes 1 to 4, in versions 2 and 3 scope 5, and in versions 4 and 5 scope 6 (§6.1); an
/// invoice states only its version's.
pub unmetered: [u32; 6],
pub amount: u64,
pub sig: [u8; 64],
@ -887,7 +907,7 @@ pub struct PackageFacts {
pub total_entries: u64,
pub stored_bytes: u64,
pub figure: Figure,
/// Under a tariff of version 4, the microseconds of the invoice's period the package was
/// Under a tariff of version 4 or 5, the microseconds of the invoice's period the package was
/// held through: its receipt is in the previous invoice's records, so it was written by the
/// end of that invoice's period, at most this one's start (§4.7). `None` for a package the
/// invoice bills for the first time, and under versions 1 to 3.
@ -953,7 +973,7 @@ pub struct StitchedFacts {
pub statement: Value,
}
/// A verified job source (§5.1, versions 2 and 3).
/// A verified job source (§5.1, versions 2 to 5).
#[derive(Clone, Debug)]
pub struct JobFacts {
/// The route record's digest, and its signer: the router.
@ -972,11 +992,11 @@ pub struct JobFacts {
pub figure: Option<Figure>,
/// Each output receipt: its receipt hash and its `work_pixel_ops`.
pub outputs: Vec<([u8; 32], u64)>,
/// Under a tariff of version 4, each output package (§4.7): its receipt hash and the work
/// units it states.
/// Under a tariff of version 4 or 5, each output package (§4.7): its receipt hash and the
/// work units it states.
pub packages: Vec<([u8; 32], u64)>,
/// Under a tariff of version 4, the figure of each attempt that is not billed, a failed hop
/// or one that served past the deadline, in attempt order (§6.6); empty under versions 2
/// Under a tariff of version 4 or 5, the figure of each attempt that is not billed, a failed
/// hop or one that served past the deadline, in attempt order (§6.6); empty under versions 2
/// and 3, which state no such figure.
pub unbilled: Vec<Figure>,
}
@ -1415,6 +1435,42 @@ fn job_figure(r: &Value) -> Option<Figure> {
Some(Figure { joules, class: Some(class) })
}
/// A decode's output under a tariff of version 5 (§4.8): its receipt's hash and `work_pixel_ops`,
/// and the content hash of the animation that holds its frames.
struct DecodeOutput {
receipt: ([u8; 32], u64),
object: [u8; 32],
}
/// §4.8, 1 and 2: a decode's output of exactly `receipt`, `codestream` and `frames`; the receipt
/// read (integer-payloads §15.2) and re-derived from the codestream, which the decode read
/// (§15.1 there); `frames` one animated PNG read whole (`png_still::animation`), of the
/// receipt's size and frame count, then each frame in order, whose samples are the receipt's
/// frame at its index. A refusal is `output_invalid` at `outputs`, `i`, and for the animation
/// `frames`, for a frame `frames` and its index.
fn decode_output(x: &Map<String, Value>, i: usize) -> Result<DecodeOutput, Refusal> {
let bad = |at: Value| (Reason::OutputInvalid.code(), place(json!(["outputs", i]), at));
if !has_exactly(x, &["receipt", "codestream", "frames"]) {
return Err(bad_source());
}
let (Some(text), Some(code), Some(frames)) = (x["receipt"].as_str(), bytes_of(&x["codestream"]), bytes_of(&x["frames"])) else { return Err(bad_source()) };
let r = crate::video_receipt::VideoReceipt::from_json(text).map_err(|_| bad(json!([])))?;
if !r.verify_reconstruction_wiv1(&code) {
return Err(bad(json!([])));
}
// One frame per frame the receipt hashes, every frame's subpixels being billed (§4.5).
let a = crate::png_still::animation(&frames).map_err(|_| bad(json!(["frames"])))?;
if (a.w, a.h) != (r.w, r.h) || a.frames() != r.frames.len() {
return Err(bad(json!(["frames"])));
}
for (k, f) in r.frames.iter().enumerate() {
if !a.frame(k).is_ok_and(|rgb| crate::video_receipt::frame_hash(k as u64, &rgb).hash == f.hash) {
return Err(bad(json!(["frames", k])));
}
}
Ok(DecodeOutput { receipt: (r.receipt_hash(), r.work_pixel_ops), object: crate::route_record::content_hash(&frames) })
}
fn job_source(o: &Map<String, Value>, version: u8) -> Result<Facts, Refusal> {
use crate::route_record as rr;
if !has_exactly(o, &["kind", "request", "pricebook", "policy", "record", "receipts", "keys", "outputs"]) {
@ -1442,19 +1498,19 @@ fn job_source(o: &Map<String, Value>, version: u8) -> Result<Facts, Refusal> {
figure = Some(job_figure(&texts[j]).ok_or((Reason::JobFigureInvalid.code(), json!(["receipts", j])))?);
billed = Some((cap.to_string(), oer_identity(id).expect("an id")));
}
// Version 4 states every hop's energy: each attempt that is not billed, in a scope of its
// own (§6.6), so its figure must be one a row can state, as the billed hop's must.
// Versions 4 and 5 state every hop's energy: each attempt that is not billed, in a scope of
// its own (§6.6), so its figure must be one a row can state, as the billed hop's must.
let mut unbilled = Vec::new();
if version == VERSION_4 {
if bills_spending(version) {
for a in attempts.iter().filter(|a| a["billed"] != true) {
let j = receipt_of(a);
unbilled.push(job_figure(&texts[j]).ok_or((Reason::JobFigureInvalid.code(), json!(["receipts", j])))?);
}
}
// What the hop that served states it delivered (route-record §5.2, version 2), by content
// hash; a record of version 1 states nothing, and under a tariff of version 4 holds its
// outputs to nothing (§5.1). Under version 3 a record of version 1's outputs are read as
// before.
// hash, in its order; a record of version 1 states nothing, and under a tariff of version 4
// or 5 holds its outputs to nothing (§5.1). Under version 3 a record of version 1's outputs
// are read as before.
let delivered: Option<Vec<[u8; 32]>> = if record["version"] == 2 {
Some(
attempts
@ -1463,19 +1519,35 @@ fn job_source(o: &Map<String, Value>, version: u8) -> Result<Facts, Refusal> {
.flat_map(|a| a["outputs"].as_array().expect("a list").iter().map(|h| unhex(h.as_str().expect("a hash")).expect("hex").try_into().expect("32 bytes")))
.collect(),
)
} else if version == VERSION_4 {
} else if bills_spending(version) {
Some(Vec::new())
} else {
None
};
// Under a tariff of version 5 a decode delivers its reconstruction, one animation of its
// frames, and its codestream is what it read (§4.8).
let decode = holds_decode_to_frames(version) && record["operation"] == DECODE;
// A decode's animations are held to what its hop delivered in order: `next` is the place in
// the served attempt's outputs of the next.
let mut next = 0usize;
let mut seen: BTreeSet<[u8; 32]> = BTreeSet::new();
let (mut outs, mut packages) = (Vec::with_capacity(outputs.len()), Vec::new());
for (i, x) in outputs.iter().enumerate() {
let bad = || (Reason::OutputInvalid.code(), json!(["outputs", i]));
let x = x.as_object().ok_or_else(bad_source)?;
// Each output's object: an int_motion codestream, or in version 4 a package's receipt,
// the package a run of `package.build` made (§4.7), verified against its indexes.
let object = if version == VERSION_4 && has_exactly(x, &["package", "indexes"]) {
if decode {
let out = decode_output(x, i)?;
outs.push(out.receipt);
// The animation is the object the record states at its place (§4.8, 3).
if delivered.as_deref().unwrap_or(&[]).get(next) != Some(&out.object) {
return Err((Reason::OutputNotDelivered.code(), json!(["outputs", i, "frames"])));
}
next += 1;
continue;
}
// Each output's object: an int_motion codestream, or in versions 4 and 5 a package's
// receipt, the package a run of `package.build` made (§4.7), verified against its indexes.
let object = if bills_spending(version) && has_exactly(x, &["package", "indexes"]) {
let (Some(text), Some(indexes)) = (x["package"].as_str(), hexes(&x["indexes"])) else { return Err(bad_source()) };
let r = PackageReceiptV2::from_json(text).map_err(|_| bad())?;
let ix: Vec<&[u8]> = indexes.iter().map(Vec::as_slice).collect();
@ -1500,6 +1572,11 @@ fn job_source(o: &Map<String, Value>, version: u8) -> Result<Facts, Refusal> {
}
}
}
// A decode delivered its animations and nothing else: an object the record states past them,
// the codestream it read among them, is one its hop did not deliver (§4.8, 3).
if decode && delivered.as_ref().is_some_and(|d| d.len() != next) {
return Err((Reason::OutputNotDelivered.code(), json!(["outputs"])));
}
Ok(Facts::Job(JobFacts {
record: rr::digest(&record),
router: unhex(record["signer"].as_str().expect("a key")).expect("a key").try_into().expect("32 bytes"),
@ -1621,20 +1698,21 @@ fn overlap(version: u8, f: &Facts, g: &Facts) -> bool {
}
/// Whether a chain under a tariff of `version` bills a record of `kind` once (§6.9): every
/// record but, in versions 3 and 4, a job's output receipt, and in version 4 a package receipt.
/// record but, in versions 3 to 5, a job's output receipt, and in versions 4 and 5 a package
/// receipt.
/// A job's work, its route record and its hops' OER/2 receipts, is new on every run and billed
/// once; its output receipt is the evidence of what it delivered, which a deterministic re-run
/// delivers again. In version 4 a package is such an output: the run that made it is billed
/// once per run, its receipt cited by each, and its storage billed by every invoice whose
/// delivers again. In versions 4 and 5 a package is such an output: the run that made it is
/// billed once per run, its receipt cited by each, and its storage billed by every invoice whose
/// period it is held through (§4.7).
pub fn billed_once(version: u8, kind: u8) -> bool {
!((bills_work(version) && kind == record_kind::VIDEO_RECEIPT) || (version == VERSION_4 && kind == record_kind::PACKAGE))
!((bills_work(version) && kind == record_kind::VIDEO_RECEIPT) || (bills_spending(version) && kind == record_kind::PACKAGE))
}
/// Under a tariff of version 4, mark each package the previous invoice holds as held through
/// the period [`start`, `end`) (§4.7); under versions 1 to 3, nothing.
/// Under a tariff of version 4 or 5, mark each package the previous invoice holds as held
/// through the period [`start`, `end`) (§4.7); under versions 1 to 3, nothing.
pub fn hold(t: &Tariff, facts: &mut [Facts], previous: Option<&Invoice>, start: u64, end: u64) {
if t.version != VERSION_4 {
if !bills_spending(t.version) {
return;
}
for f in facts {
@ -1702,7 +1780,7 @@ fn is_free(class: u8, grant: &[u8; 32]) -> bool {
/// The figures of §6.2 to §6.6 over verified sources, a package's held through the period
/// marked by [`hold`].
pub fn compute(t: &Tariff, facts: &[Facts]) -> Result<Computed, Overflow> {
let v4 = t.version == VERSION_4;
let v4 = bills_spending(t.version);
let mut records: BTreeSet<(u8, [u8; 32])> = BTreeSet::new();
for f in facts {
records.extend(f.records());
@ -2360,10 +2438,10 @@ mod tests {
let at = nb.windows(2).position(|w| w == [0xc3, 0xa9]).expect("the two bytes of é");
nb[at] = 0xff;
assert_eq!(Tariff::parse(&nb).unwrap_err().0, "tariff_invalid");
// A tariff of version 5 is not defined.
let mut v5 = b.clone();
v5[4] = 5;
assert_eq!(Tariff::parse(&v5).unwrap_err().0, "unknown_version");
// A tariff of version 6 is not defined.
let mut v6 = b.clone();
v6[4] = 6;
assert_eq!(Tariff::parse(&v6).unwrap_err().0, "unknown_version");
}
fn tariff_v4() -> Tariff {
@ -2422,10 +2500,39 @@ mod tests {
assert!(!billed_once(VERSION_4, record_kind::PACKAGE) && billed_once(VERSION_3, record_kind::PACKAGE));
assert!(!billed_once(VERSION_4, record_kind::VIDEO_RECEIPT));
assert!(billed_once(VERSION_4, record_kind::OER_RECEIPT) && billed_once(VERSION_4, record_kind::ROUTE_RECORD) && billed_once(VERSION_4, record_kind::SESSION));
assert!(bills_work(VERSION_4) && prices_jobs(VERSION_4) && !prices_jobs(5));
assert!(bills_work(VERSION_4) && prices_jobs(VERSION_4) && !prices_jobs(6));
assert_eq!((scope::count(VERSION), scope::count(VERSION_2), scope::count(VERSION_3), scope::count(VERSION_4)), (4, 5, 5, 6));
}
/// A tariff of version 5 is version 4's layout under a rule of its own (§4.8): it reads back,
/// every altered byte is refused, it writes what version 4 writes but its version byte, and
/// it bills what version 4 bills; only it holds a decode to its frames.
#[test]
fn a_tariff_of_version_5_is_version_4s_layout_under_its_own_rule() {
let mut t = tariff_v4();
t.version = VERSION_5;
let t = t.sign(&key(1), &key(2));
let b = t.to_bytes();
assert_eq!(b[4], 5);
assert_eq!(Tariff::parse(&b).unwrap(), t);
for at in 0..b.len() {
let mut m = b.clone();
m[at] ^= 0x01;
assert!(Tariff::parse(&m).is_err(), "byte {at} altered and accepted");
}
let (mut b4, mut b5) = (tariff_v4().body(), t.body());
b4[4] = 0;
b5[4] = 0;
assert_eq!(b4, b5, "the layout is version 4's");
for k in [record_kind::PACKAGE, record_kind::VIDEO_RECEIPT, record_kind::OER_RECEIPT, record_kind::ROUTE_RECORD, record_kind::SESSION] {
assert_eq!(billed_once(VERSION_5, k), billed_once(VERSION_4, k), "kind {k}");
}
assert!(bills_work(VERSION_5) && prices_jobs(VERSION_5) && bills_spending(VERSION_5) && !prices_jobs(6));
assert_eq!((scope::count(VERSION_5), room_prices(VERSION_5), job_units(VERSION_5)), (6, ROOM_PRICES, 3));
assert_eq!(unit_name_in(VERSION_5, source_kind::PACKAGE, 3), Some("byte_days"));
assert!(holds_decode_to_frames(VERSION_5) && (VERSION..=VERSION_4).all(|v| !holds_decode_to_frames(v)), "only version 5 holds a decode to its frames");
}
/// What a chain bills once (§6.9): in version 3, every record but an output receipt; in
/// versions 1 and 2, every record.
#[test]
@ -2435,7 +2542,7 @@ mod tests {
}
assert!(billed_once(VERSION_2, record_kind::VIDEO_RECEIPT), "version 2 bills an output once");
assert!(!billed_once(VERSION_3, record_kind::VIDEO_RECEIPT), "version 3 bills the work, and cites the output");
assert!(prices_jobs(VERSION_2) && prices_jobs(VERSION_3) && !prices_jobs(VERSION) && !prices_jobs(5));
assert!(prices_jobs(VERSION_2) && prices_jobs(VERSION_3) && !prices_jobs(VERSION) && !prices_jobs(6));
}
/// A tariff of version 2 reads back with its job prices, and every altered byte is refused;
@ -2587,8 +2694,8 @@ mod tests {
assert_eq!(Invoice::parse(&unit.sign(&key(1)).to_bytes()).unwrap_err(), "invoice_invalid");
assert_eq!(Invoice::parse(&[b.clone(), vec![0]].concat()).unwrap_err(), "malformed");
let mut v = b.clone();
v[4] = 5;
assert_eq!(Invoice::parse(&v).unwrap_err(), "unknown_version", "version 5 is not defined (2 to 4 are, §6.1)");
v[4] = 6;
assert_eq!(Invoice::parse(&v).unwrap_err(), "unknown_version", "version 6 is not defined (2 to 5 are, §6.1)");
let mut s = b;
let n = s.len();
s[n - 1] ^= 1;

158
wai/wai-rs/src/receipt_ledger/corpus.rs
View file

@ -10,7 +10,9 @@
//! `live-conformance/routes/`, and those the version-3 vectors add (a decode, a re-run, a re-run
//! replaying a hop receipt, a job claiming another's) are routed here under that corpus's own
//! seeds, as are jobs routed under the route record's version 2 (per-operation entries, measured
//! failures, the served hop's outputs), billed under versions 3 and 4. [`generate`] reads those
//! failures, the served hop's outputs), billed under versions 3 and 4, and the decodes of
//! version 5 (§4.8), whose frames are written here as PNGs from the clip they decode.
//! [`generate`] reads those
//! files from a `wai/` directory; [`verify`] checks every vector against the outcome it states.
use std::collections::BTreeMap;
@ -547,6 +549,122 @@ pub fn generate(wai: &Path) -> Result<String, String> {
pool.insert("package_run_again", packed(accepted + 2 * DAY_US, rr::hop_receipt("package.build", "pack-0001", "pack.alpha", Some("4.625"))));
let first_sealed: Value = serde_json::from_str(pool["package_run"]["receipts"][0].as_str().ok_or("the run's hop receipt")?).map_err(|e| e.to_string())?;
pool.insert("package_run_replayed", routed(rr::package_job_document("pack-0001", accepted + 2 * DAY_US, &[rr::Hop { endpoint: "pack.alpha", error: None, start: 1_000, end: 3_001_000, receipt: first_sealed }], &base_object), vec![base_output.clone()]));
// Version 5's decodes (§4.8): a decode delivers its reconstruction as one animated PNG of its
// frames, as `png_still::encode_animation` writes one, and the codestream is what it read.
// The served hop states the animation's content hash, and its receipt commits to it. Then a
// decode whose record states the codestream it read, or that codestream past the animation;
// a frame altered; two frames swapped; a still for the animation; a frame missing; frames of
// another shape; another codestream; and an animation breaking each of its own rules alone.
const DEC: &str = "media.decode";
let decoded = crate::video_receipt::decode_wiv1(&clip).map_err(|e| format!("the corpus clip decodes: {e}"))?;
if decoded.frames.len() < 2 {
return Err("the corpus clip has fewer than two frames to swap".into());
}
let (fw, fh) = (u32::try_from(decoded.w).map_err(|_| "a width")?, u32::try_from(decoded.h).map_err(|_| "a height")?);
let animate = |frames: &[Vec<u8>]| crate::png_still::encode_animation(frames, fw, fh, (1, 25));
let anim = animate(&decoded.frames);
let output_5 = |job: &str, frames: &[u8]| -> Vec<Value> {
let r = crate::video_receipt::VideoReceipt::seal_decode_wiv1(&key("output-sink"), format!("did:example:sink/{job}"), &clip, 0, None, None).expect("a corpus payload decodes");
vec![json!({"receipt": r.to_json(), "codestream": hex(&clip), "frames": hex(frames)})]
};
let decode_5 = |job: &str, delivered: &[[u8; 32]], frames: &[u8]| routed_2(job, DEC, &[served("im.alpha", rr::hop_receipt(DEC, job, "im.alpha", Some("12.0")))], delivered, output_5(job, frames));
// A decode whose record states the animation it is given as its one object.
let held = |job: &str, frames: &[u8]| decode_5(job, &[rr_hash(frames)], frames);
pool.insert("job_v5_decode", held("job-5001", &anim));
pool.insert("job_v5_decode_names_its_input", decode_5("job-5002", &[rr_hash(&clip)], &anim));
pool.insert("job_v5_an_object_past_the_frames", decode_5("job-5003", &[rr_hash(&anim), rr_hash(&clip)], &anim));
let altered = {
let mut f = decoded.frames.clone();
f[1][0] ^= 1;
animate(&f)
};
pool.insert("job_v5_a_frame_altered", held("job-5004", &altered));
let swapped = {
let mut f = decoded.frames.clone();
f.swap(0, 1);
animate(&f)
};
pool.insert("job_v5_frames_swapped", held("job-5005", &swapped));
pool.insert("job_v5_a_still_for_the_frames", held("job-5006", &crate::png_still::encode(&decoded.frames[0], fw, fh)));
pool.insert("job_v5_a_frame_missing", held("job-5007", &animate(&decoded.frames[..decoded.frames.len() - 1])));
// Frames whose samples are the frames' bytes, each in one row: a shape not the receipt's,
// which the frame hash alone would not tell apart (it hashes the index and the bytes).
pool.insert("job_v5_frames_of_another_shape", held("job-5008", &crate::png_still::encode_animation(&decoded.frames, fw * fh, 1, (1, 25))));
let mut other_code = held("job-5009", &anim);
other_code["outputs"][0]["codestream"] = json!(hex(&other_clip));
pool.insert("job_v5_another_codestream", other_code);
// The animation's own rules (§4.8, 2), each broken alone in the animation above, rewritten
// chunk by chunk with every CRC good and, where a chunk is added or taken away, the
// sequence numbers renumbered in order.
type Chunks = Vec<([u8; 4], Vec<u8>)>;
let chunks_of = |b: &[u8]| -> Chunks {
let mut at = 8;
let mut out = Vec::new();
while at < b.len() {
let len = u32::from_be_bytes(b[at..at + 4].try_into().expect("4")) as usize;
out.push((b[at + 4..at + 8].try_into().expect("4"), b[at + 8..at + 8 + len].to_vec()));
at += 12 + len;
}
out
};
let png_of = |cs: &Chunks, renumber: bool| -> Vec<u8> {
let mut out = anim[..8].to_vec();
let mut seq = 0u32;
for (k, d) in cs {
let mut d = d.clone();
if renumber && (k == b"fcTL" || k == b"fdAT") {
d[..4].copy_from_slice(&seq.to_be_bytes());
seq += 1;
}
crate::png_still::chunk(&mut out, k, &d);
}
out
};
let base = chunks_of(&anim);
let fctl = |k: usize| base.iter().enumerate().filter(|(_, c)| &c.0 == b"fcTL").nth(k).map(|(i, _)| i).expect("the frame's fcTL");
let broken = |job: &str, renumber: bool, f: &dyn Fn(&mut Chunks)| {
let mut cs = base.clone();
f(&mut cs);
held(job, &png_of(&cs, renumber))
};
pool.insert("job_v5_a_frame_blended_over", broken("job-5010", false, &|cs| cs[fctl(1)].1[25] = 1));
pool.insert("job_v5_a_frame_disposed_of_by_no_rule", broken("job-5011", false, &|cs| cs[fctl(1)].1[24] = 3));
// The second frame as the part of the picture that is not its first column: its fcTL one
// narrower at offset 1, and its data that part's rows.
let part: Vec<u8> = decoded.frames[1].chunks_exact(3 * fw as usize).flat_map(|row| row[3..].to_vec()).collect();
let part_z = chunks_of(&crate::png_still::encode(&part, fw - 1, fh)).into_iter().find(|c| &c.0 == b"IDAT").expect("an IDAT").1;
pool.insert("job_v5_a_frame_of_part_of_the_picture", broken("job-5012", false, &|cs| {
let i = fctl(1);
cs[i].1[4..8].copy_from_slice(&(fw - 1).to_be_bytes());
cs[i].1[12..16].copy_from_slice(&1u32.to_be_bytes());
let mut d = cs[i + 1].1[..4].to_vec();
d.extend_from_slice(&part_z);
cs[i + 1].1 = d;
}));
pool.insert("job_v5_a_sequence_number_skipped", broken("job-5013", false, &|cs| {
let i = fctl(1) + 1;
let n = u32::from_be_bytes(cs[i].1[..4].try_into().expect("4"));
cs[i].1[..4].copy_from_slice(&(n + 1).to_be_bytes());
}));
// The default image apart from the animation: the first frame's data as an IDAT before the
// first fcTL, and again as an fdAT after it, the frames themselves the receipt's.
pool.insert("job_v5_the_default_image_apart", broken("job-5014", true, &|cs| {
let (i, idat) = (fctl(0), cs[fctl(0) + 1].clone());
cs.remove(i + 1);
let mut fdat = vec![0; 4];
fdat.extend_from_slice(&idat.1);
cs.insert(i + 1, (*b"fdAT", fdat));
cs.insert(i, idat);
}));
pool.insert("job_v5_an_animation_counting_another", broken("job-5015", false, &|cs| {
let n = u32::from_be_bytes(cs[1].1[..4].try_into().expect("4"));
cs[1].1[..4].copy_from_slice(&(n + 1).to_be_bytes());
}));
// The last frame's fcTL with no data after it, the count and sequence numbers its own.
pool.insert("job_v5_a_frame_with_no_data", broken("job-5016", true, &|cs| {
let last = cs.iter().rposition(|c| &c.0 == b"fdAT").expect("an fdAT");
cs.remove(last);
}));
// The rooms' prices are the lesson receipts'; the tariff's scope is every party of the
// accepted invoices.
@ -639,7 +757,7 @@ pub fn generate(wai: &Path) -> Result<String, String> {
Vector { name: "tariff_period_empty", about: "a tariff whose period is empty, signed by both", tariff: tariff_period.to_bytes(), invoice: ab.clone(), previous: None, sources: day_a.to_vec() },
// The invoice.
Vector { name: "invoice_trailing_byte", about: "an invoice with a byte after its signature", tariff: tb.clone(), invoice: [ab.clone(), vec![0]].concat(), previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_version", about: "an invoice of version 5, which this revision does not define", tariff: tb.clone(), invoice: { let mut b = ab.clone(); b[4] = 5; b }, previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_version", about: "an invoice of version 6, which this revision does not define", tariff: tb.clone(), invoice: { let mut b = ab.clone(); b[4] = 6; b }, previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_signature", about: "an invoice whose signature does not verify", tariff: tb.clone(), invoice: flip(ab.clone(), ab.len() - 1), previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_lines_unordered", about: "an invoice whose first two lines are swapped", tariff: tb.clone(), invoice: sealed(&t, [0; 32], a0, a1, &refs(&day_a), |i| i.lines.swap(0, 1)).to_bytes(), previous: None, sources: day_a.to_vec() },
Vector { name: "invoice_row_of_no_class", about: "an energy row of tag 0: joules of no one class", tariff: tb.clone(), invoice: sealed(&t, [0; 32], a0, a1, &refs(&day_a), |i| i.energy.insert(0, Row { scope: 1, tag: 0, figures: 1, joules: 1, max_ppm: 0 })).to_bytes(), previous: None, sources: day_a.to_vec() },
@ -870,6 +988,42 @@ pub fn generate(wai: &Path) -> Result<String, String> {
Vector { name: "version_3_invoice_under_a_version_4_tariff", about: "an invoice of version 3 naming a tariff of version 4", tariff: t4b.clone(), invoice: sealed_after(&t_4, None, (MONTH.0, mid), MONTH.0, mid, &refs(&day_4a), |i| { i.version = VERSION_3; i.unmetered[5] = 0; i.energy.retain(|r| r.scope != scope::UNBILLED_HOPS); i.lines.retain(|l| !(l.source_kind == source_kind::ROOM && l.unit == 14)); }).to_bytes(), previous: None, sources: day_4a.to_vec() },
]);
// Version 5 (§4.8): version 4's tariff and rules, and a decode held to the animation of its
// frames it delivered, its codestream what it read. An encode is held as in version 4.
let t_5 = {
let mut t5 = tariff_of("buyer-4", room_prices_4, vec![(source_kind::ROOM, lesson_id), package_scope, staged_scope, (source_kind::JOB, router)]);
t5.version = VERSION_5;
t5.package = [0, 0, 2_000_000];
t5.job = job_prices_4.clone();
t5.sign(&seller, &key("buyer-4"))
};
let t5b = t_5.to_bytes();
let day_5 = ["job_v2_encode", "job_v5_decode"];
let inv_5 = make_invoice(&seller, &t_5, None, MONTH.0, mid, &src(&day_5)).map_err(|e| format!("invoice 5: {e:?}"))?;
let a5 = inv_5.to_bytes();
let refused_5 = |name: &'static str, about: &'static str, source: &'static str| Vector { name, about, tariff: t5b.clone(), invoice: a5.clone(), previous: None, sources: vec![source] };
vectors.extend([
Vector { name: "version_5_a_decode_held_to_its_frames", about: "version 5: an int_motion decode whose served hop states it delivered one animated PNG of the clip's frames, its receipt re-derived from the codestream it read and every frame held to the receipt's, billed at the decode's prices beside an encode held to its codestream as in version 4", tariff: t5b.clone(), invoice: a5.clone(), previous: None, sources: day_5.to_vec() },
refused_5("version_5_a_decode_naming_the_codestream_it_read", "version 5: a decode whose route record states the codestream it read as what it delivered, its animation given", "job_v5_decode_names_its_input"),
refused_5("version_5_an_object_past_the_frames", "version 5: a decode whose record states the codestream it read after its animation", "job_v5_an_object_past_the_frames"),
refused_5("version_5_a_decode_in_version_4s_form", "version 5: a decode given as version 4 gives one, its receipt and codestream and no frames, its record stating the codestream", "job_v2_decode"),
refused_5("version_5_a_frame_altered", "version 5: a decode whose second frame has one sample changed, its record stating that animation", "job_v5_a_frame_altered"),
refused_5("version_5_frames_swapped", "version 5: a decode whose animation holds its first two frames in each other's places", "job_v5_frames_swapped"),
refused_5("version_5_a_still_for_the_frames", "version 5: a decode whose frames are given as a still of its first frame, a PNG with no animation", "job_v5_a_still_for_the_frames"),
refused_5("version_5_a_frame_missing", "version 5: a decode whose animation holds one frame fewer than its receipt hashes", "job_v5_a_frame_missing"),
refused_5("version_5_frames_of_another_shape", "version 5: a decode whose animation holds each frame's samples in one row, a shape not the receipt's", "job_v5_frames_of_another_shape"),
refused_5("version_5_a_codestream_not_the_receipts", "version 5: a decode whose codestream, what it read, is another clip than the one its receipt describes", "job_v5_another_codestream"),
refused_5("version_5_a_frame_blended_over", "version 5: a decode whose animation blends its second frame over the first", "job_v5_a_frame_blended_over"),
refused_5("version_5_a_frame_disposed_of_by_no_rule", "version 5: a decode whose animation disposes of its second frame by an operation no rule defines", "job_v5_a_frame_disposed_of_by_no_rule"),
refused_5("version_5_a_frame_of_part_of_the_picture", "version 5: a decode whose animation's second frame is the picture less its first column, at offset 1", "job_v5_a_frame_of_part_of_the_picture"),
refused_5("version_5_a_sequence_number_skipped", "version 5: a decode whose animation skips a sequence number", "job_v5_a_sequence_number_skipped"),
refused_5("version_5_the_default_image_apart", "version 5: a decode whose animation's default image is apart from its frames, the frames themselves the receipt's", "job_v5_the_default_image_apart"),
refused_5("version_5_an_animation_counting_another", "version 5: a decode whose animation counts one frame more than it holds", "job_v5_an_animation_counting_another"),
refused_5("version_5_a_frame_with_no_data", "version 5: a decode whose animation's last frame has no data", "job_v5_a_frame_with_no_data"),
Vector { name: "version_4_a_decode_given_frames", about: "version 4: a decode given in version 5's form, with frames, which version 4 does not read", tariff: t4b.clone(), invoice: a4.clone(), previous: None, sources: vec!["job_v5_decode"] },
Vector { name: "version_4_invoice_under_a_version_5_tariff", about: "an invoice of version 4 naming a tariff of version 5", tariff: t5b.clone(), invoice: sealed_after(&t_5, None, (MONTH.0, mid), MONTH.0, mid, &refs(&day_5), |i| i.version = VERSION_4).to_bytes(), previous: None, sources: day_5.to_vec() },
]);
let mut sources = Map::new();
for (n, v) in &pool {
sources.insert((*n).into(), v.clone());