receipt-ledger: tariff version 3 prices media jobs per operation and bills the work done #550
13 changed files with 1415 additions and 136 deletions
4
.github/workflows/wai-conformance.yml
vendored
4
.github/workflows/wai-conformance.yml
vendored
|
|
@ -2017,7 +2017,9 @@ jobs:
|
|||
# from the sources it takes from the metering, settlement, ads, presentation, stitched and
|
||||
# routes corpora and the packs of pack-conformance, and no other file is there; wai-rs gives every vector its
|
||||
# stated outcome; every reason of §10 and every reused code refuses a vector, and so does
|
||||
# each source kind's own extension; §10 lists exactly the reference's codes); then the
|
||||
# each source kind's own extension; §10 lists exactly the reference's codes; under a tariff
|
||||
# of version 3, jobs priced per operation and a chain that bills a job's work once, its
|
||||
# output's receipt cited again by a re-run, with version 2 billing as before); then the
|
||||
# corpus's second implementation, receipt_ledger_verify.py, written from the extension's
|
||||
# text and sharing no code with wai-rs, which verifies each source with that corpus's own
|
||||
# second implementation (live_metering_verify.py, settlement_verify.py, live_ads_verify.py
|
||||
|
|
|
|||
3
.github/workflows/wai-ledger.yml
vendored
3
.github/workflows/wai-ledger.yml
vendored
|
|
@ -2,7 +2,8 @@ name: wai-ledger service
|
|||
|
||||
# The WAI receipt ledger as a service (wai/wai-ledger, extensions/receipt-ledger.md §9): its
|
||||
# tests (the corpus's invoices issued byte for byte, the chain recovered from the directory
|
||||
# after a restart, a source billed once, one process per directory), clippy with warnings as
|
||||
# after a restart, a source billed once, under a tariff of version 3 a job's work billed once and
|
||||
# its re-run billed, one process per directory), clippy with warnings as
|
||||
# errors, and its smoke checks over HTTP against an instance started here with the corpus's
|
||||
# seller key, the invoice's verification document checked by receipt_ledger_verify.py; on
|
||||
# Linux and on the macOS runner (the host the service would be deployed to).
|
||||
|
|
|
|||
|
|
@ -6,6 +6,44 @@ wire format may change and such changes are recorded here.
|
|||
|
||||
## Unreleased
|
||||
|
||||
### The receipt ledger prices media jobs per operation, and bills the work done
|
||||
|
||||
- **Tariff version 3** ([extensions/receipt-ledger.md](extensions/receipt-ledger.md) §4.6): each job price
|
||||
also names an `operation` of the catalogue (`media.encode`, `media.decode`), and a job is priced by its serving
|
||||
capability **and** its operation, the one its route record names. Within one capability an encode (a motion search,
|
||||
then a codestream written) and a decode (a reconstruction) spend different energy for the same output units, so one
|
||||
price per capability overpriced one or underpriced the other, and a decode could not be billed at its own price. A
|
||||
job whose capability and operation the tariff does not price bills nothing. A version-2 reader cannot read a price
|
||||
that names an operation (`op_len` and `operation` follow `capability`), so this is a version of its own: a tariff and
|
||||
its invoices of version 3 (`WLT1`, `WLI1` with version byte 3, the invoice laid out as version 2's).
|
||||
- **A chain bills a job's work, not its output's identity** (§5.2, §5.3, §6.9, §12): under a tariff of version 3
|
||||
the records billed once for a job are its route record and its hops' OER/2 receipts, new on every run; its output's
|
||||
decode receipt (kind 15) is cited, and carried in its lines' citation digests, but is not billed once. The codecs
|
||||
are deterministic, so the same request run again delivers the same output receipt from a new hop with new energy:
|
||||
that re-run is billed, in the invoice that holds it or a later one, and its output receipt is cited again. A replayed
|
||||
hop receipt or route record is refused as before (`record_rebilled` across the chain, `record_duplicate` within an
|
||||
invoice), so two lines of one invoice cite one output receipt only under different hop receipts; two route records
|
||||
of one job are two runs of it, no longer `record_overlap`. A tariff of version 2 reads, verifies and bills exactly as
|
||||
before, an output once; a package keeps its own rule in every version (a delivered product, once per receipt).
|
||||
- **The corpus** adds nine vectors: encode and decode of `int_motion` at their own prices, a re-run billed in the
|
||||
next invoice of its chain, one output under two hops of one invoice billed for each, an operation the tariff does not
|
||||
price, a replayed hop receipt refused, one hop under two lines refused, prices out of order by operation, a version-2
|
||||
invoice under a version-3 tariff, and the re-run refused under version 2. Its version-3 jobs are routed by the route
|
||||
corpus's router under that corpus's seeds; `routes/vectors.json` is unchanged. `receipt_ledger_verify.py` reads
|
||||
version 3 and its rules, requires the new mutated copies to fail, and drops two more of its checks (`operation`,
|
||||
`output-once`); the ledger service (`wai/wai-ledger`) applies the chain rule of its tariff's version.
|
||||
- **Fixed:** a job price whose capability (or, in version 3, operation) is not UTF-8 text is refused
|
||||
`tariff_invalid`, as §4.3 reads it: its form is a rule, not the layout. The reference read it `malformed`, unlike the
|
||||
text and `receipt_ledger_verify.py`; no vector held such a price.
|
||||
- **Breaking:** `api:rust:wai::receipt_ledger::JobPrice::operation`: a job price states its operation, empty in
|
||||
version 2. A struct literal adds `operation: String::new()`.
|
||||
- **Breaking:** `api:rust:wai::receipt_ledger::JobFacts::operation`: a job source's operation, its route record's.
|
||||
- **Breaking:** `api:rust:wai::receipt_ledger::check_across`: takes the tariff's version first, which decides what a
|
||||
chain bills once and whether two records of one job overlap.
|
||||
- **Breaking:** `record:corpus:wai/live-conformance/ledger/vectors.json`: the control `invoice_version` gives an
|
||||
invoice of version 4, since version 3 is now defined, and is refused `unknown_version` as before; every other vector
|
||||
keeps its outcome, and the version-3 vectors are added.
|
||||
|
||||
### Media jobs in the receipt ledger, and the receipts of `wai.video.int_motion` written down
|
||||
|
||||
- **The receipt ledger prices media jobs** ([extensions/receipt-ledger.md](extensions/receipt-ledger.md) §4.5): a
|
||||
|
|
|
|||
|
|
@ -23,7 +23,10 @@
|
|||
> the delivery of each party under the one tariff with the bytes two parties were acknowledged
|
||||
> for twice. A tariff of version 2 also prices **media jobs** (§4.5): a job is billed from
|
||||
> its route record ([route-record](route-record.md)), its hops' OER/2 receipts and WAI's
|
||||
> receipt of its output, in the units those receipts state. No new transport, no payment
|
||||
> receipt of its output, in the units those receipts state. A tariff of version 3 prices a
|
||||
> job by its capability **and its operation** (§4.6), since an encode and a decode spend
|
||||
> different energy, and bills a job's **work** once, its route record and its hops' receipts,
|
||||
> its output's receipt being the evidence a re-run may cite again (§6.9). No new transport, no payment
|
||||
> and no tax: an invoice is the recomputable figures a payment rests on. The reference implementation is the
|
||||
> `receipt_ledger` module of `wai-rs` (feature `receipt_ledger`) and the service
|
||||
> `wai/wai-ledger`; the corpus is [`live-conformance/ledger/`](../live-conformance/ledger/),
|
||||
|
|
@ -112,7 +115,7 @@ records recomputes all three.
|
|||
| field | size | meaning |
|
||||
|---|---|---|
|
||||
| `magic` | 4 | `WLT1` (ASCII) |
|
||||
| `version` | 1 | 1, or 2 for a tariff that prices jobs (§4.5) |
|
||||
| `version` | 1 | 1; 2 for a tariff that prices jobs (§4.5); 3 for one that prices them per operation and bills their work (§4.6, §6.9) |
|
||||
| `seller` | 32 | the seller's key, which signs the tariff and every invoice under it |
|
||||
| `buyer` | 32 | the buyer's key, which accepts the tariff by signing it |
|
||||
| `start`, `end` | 8 each | u64: the period the tariff is good for |
|
||||
|
|
@ -121,7 +124,7 @@ records recomputes all three.
|
|||
| `delivery` | 32 | four u64: millionths of a minor unit per 10^9 acknowledged bytes of each object class, in class order (settlement §7.1) |
|
||||
| `package` | 24 | three u64: millionths of a minor unit per package receipt, per entry, and per 10^9 stored bytes |
|
||||
| `ads` | 16 | two u64: millionths of a minor unit per presented impression, and per billable impression |
|
||||
| `job` | 2 + Σ (18 + *c*) | version 2 only, absent in version 1: a u16 count *j*, then each job price (§4.5): `cap_len` (u8), `capability` (`cap_len` bytes), `unit` (u8), `divisor` (u64) and `price` (u64) |
|
||||
| `job` | 2 + Σ (18 + *c*), in version 3 2 + Σ (19 + *c* + *o*) | versions 2 and 3, absent in version 1: a u16 count *j*, then each job price (§4.5, §4.6): `cap_len` (u8), `capability` (`cap_len` bytes), in version 3 `op_len` (u8) and `operation` (`op_len` bytes), then `unit` (u8), `divisor` (u64) and `price` (u64) |
|
||||
| `scope` | 2 + 33 × *n* | a u16 count *n*, then each entry: `kind` (u8) and `id` (32), in strictly ascending order of (`kind`, `id`) |
|
||||
| `seller_sig` | 64 | §4.2 |
|
||||
| `buyer_sig` | 64 | §4.2 |
|
||||
|
|
@ -140,16 +143,17 @@ tariff the buyer did not sign bills nobody.
|
|||
|
||||
A reader reads exactly the bytes it is given as one tariff, in the order of §4.1, and refuses at
|
||||
the first of these that holds: fewer than 4 bytes, `truncated`; a magic other than `WLT1`,
|
||||
`malformed`; no fifth byte, `truncated`; a version other than 1 and 2, `unknown_version`; the
|
||||
`malformed`; no fifth byte, `truncated`; a version other than 1, 2 and 3, `unknown_version`; the
|
||||
bytes end before a field the layout requires, `truncated`; bytes after `buyer_sig`, `malformed`.
|
||||
It then refuses as `tariff_invalid` a tariff that breaks any of these:
|
||||
|
||||
- `start` is below `end`; `currency` is three capitals `A`–`Z`;
|
||||
- `seller` and `buyer` are not zero, and not the same key;
|
||||
- `scope` has at least one entry, in strictly ascending order of (`kind`, `id`), each `kind` 1 to
|
||||
4 (1 to 5 in version 2) and each `id` not zero;
|
||||
4 (1 to 5 in versions 2 and 3) and each `id` not zero;
|
||||
- in version 2, `job` has at most 255 prices, each as §4.5 requires, in strictly ascending
|
||||
order of (`capability`'s bytes, `unit`).
|
||||
order of (`capability`'s bytes, `unit`); in version 3, at most 255, each as §4.6 requires,
|
||||
in strictly ascending order of (`capability`'s bytes, `operation`'s bytes, `unit`).
|
||||
|
||||
Then the seller's signature, then the buyer's: `bad_key`, `bad_signature`.
|
||||
|
||||
|
|
@ -163,7 +167,7 @@ A scope entry names whose records the tariff bills:
|
|||
| 2 | a party's key | a settlement statement's `party` is the `id` |
|
||||
| 3 | a packager's key | a package receipt's `signer_pubkey` is the `id` |
|
||||
| 4 | an ad decider's or a stitcher's key | every receipt of an ads source has `signer_pubkey` the `id` (live-ads §3.4); a stitched source's records are accepted only when their `signer` is such an `id`: the verifier's **signers** (stitched-presentation §7.1) are the scope's entries of kind 4 |
|
||||
| 5 | a router's key (version 2) | a job source's route record has `signer` the `id` (route-record §5.1) |
|
||||
| 5 | a router's key (versions 2 and 3) | a job source's route record has `signer` the `id` (route-record §5.1) |
|
||||
|
||||
The scope is what the buyer agreed to pay for: a record signed by a party the tariff does not name
|
||||
is refused `outside_scope` (§7.2 step 5), whoever submits it.
|
||||
|
|
@ -185,6 +189,25 @@ A job the tariff has no price for, by its serving capability, is billed nothing,
|
|||
pays for a capability only at a price it signed. A package job stays a package source (§5.1),
|
||||
billed by §4.1's package prices.
|
||||
|
||||
### 4.6 Job prices per operation (version 3)
|
||||
|
||||
A tariff of version 3 prices jobs as §4.5 does, and each job price also names an `operation`:
|
||||
an operation of WAI's catalogue in route-record §2's form (two words joined by `.`, each a
|
||||
lowercase ASCII letter and then lowercase letters and digits: `media.encode`, `media.decode`),
|
||||
of at most 128 bytes. A job is priced by its serving capability **and** its operation, which is
|
||||
its route record's `operation`, the one its request named (route-record §5.1). The reason is the
|
||||
work: within one capability, an encode (a motion search, then a codestream written) and a
|
||||
decode (a reconstruction) spend different energy and compute for the same output units, so one
|
||||
price per capability overprices one of them or underprices the other, and a buyer pays each at
|
||||
the price it signed for it. A job the tariff has no price for, by its serving capability and its
|
||||
operation, is billed nothing, as a job whose capability a tariff does not price is (§4.5): a
|
||||
price for one operation never bills another.
|
||||
|
||||
Version 3 is a version of its own, not a change to version 2: a reader of version 2 cannot read
|
||||
a job price that names an operation, and the buyer of a version-2 tariff signed version 2's
|
||||
rules. Version 3 also bills a job's work rather than its output's identity (§5.2, §5.3, §6.9).
|
||||
A tariff of version 2, its invoices and its sources read, verify and bill exactly as before.
|
||||
|
||||
## 5. Sources
|
||||
|
||||
### 5.1 Kinds and their documents
|
||||
|
|
@ -199,7 +222,7 @@ extension's verifier is given. Bytes are lowercase hexadecimal strings.
|
|||
| `package` | `receipt` (the text of `package.receipt.json`), `indexes` (the index files in the receipt's component order) | packaging §9, a receipt of version 2 against its indexes |
|
||||
| `ads` | `descriptor`, `grants`, `key_log`, `chains`, `policy`, `pt0_ms`, `receipts` (form-2 receipts in their JSON carriage), `pods`, `claims`, `creatives`, `issuers` | live-ads §9, with ad-presentation §9 under a policy that requires claims |
|
||||
| `stitched` | `records` (pod-input records), `claims` (stitched claims), `manifests`, `responses` (VAST and VMAP documents), `receipts` (form-1 receipts in their JSON carriage), `creatives`, `issuers` | stitched-presentation §7, its signers the tariff's (§4.4) |
|
||||
| `job` (version 2) | `request`, `pricebook`, `policy`, `record`, `receipts`, `keys` (a route record's verification document, route-record §7.1, its `receipts` and `keys` not `null`) and `outputs`, a list of objects of exactly `receipt` (the text of a `wai.video.int_motion` decode receipt's JSON form, integer-payloads §15.2) and `codestream` (the `WIV1` payload it describes) | route-record §7, accepted with its hop receipts checked; then each output receipt read by integer-payloads §15.2 and re-derived from its `codestream` (§15.1 there), or `output_invalid` at `outputs` and its index |
|
||||
| `job` (versions 2 and 3) | `request`, `pricebook`, `policy`, `record`, `receipts`, `keys` (a route record's verification document, route-record §7.1, its `receipts` and `keys` not `null`) and `outputs`, a list of objects of exactly `receipt` (the text of a `wai.video.int_motion` decode receipt's JSON form, integer-payloads §15.2) and `codestream` (the `WIV1` payload it describes) | route-record §7, accepted with its hop receipts checked; then each output receipt read by integer-payloads §15.2 and re-derived from its `codestream` (§15.1 there), or `output_invalid` at `outputs` and its index |
|
||||
|
||||
A source is accepted when its extension's verifier accepts it; for `ads`, when the outcome is
|
||||
**accept** (live-ads §9.4), a **gap** being `source_gap`. A `stitched` source is always
|
||||
|
|
@ -211,8 +234,9 @@ refused `malformed`. Under a tariff of version 1, a `job` source is not a kind,
|
|||
`malformed`.
|
||||
|
||||
A job source's **billed hop** is the attempt of its route record that is billed (route-record
|
||||
§5.2), if any; its **serving capability** is that attempt's candidate's `capability`, and its
|
||||
**figure** is the figure of that attempt's OER/2 receipt: the unmetered marker
|
||||
§5.2), if any; its **serving capability** is that attempt's candidate's `capability`, its
|
||||
**operation** is its route record's `operation`, and its **figure** is the figure of that
|
||||
attempt's OER/2 receipt: the unmetered marker
|
||||
(oer-receipts §3.1), or `energy.joules_total` in microjoules, rounded to the nearest and a
|
||||
half to the even one, with the receipt's `energy.class`. A billed hop's receipt whose figure
|
||||
has no class, or rounds below 1 µJ or past what a u64 holds, cannot be stated in a row
|
||||
|
|
@ -241,9 +265,9 @@ the others are the ledger's own:
|
|||
| 11 | an invoice, `WLI1` | its digest (§6.7, as [revenue-share](revenue-share.md) §7.4 cites it) | no source: cited by revenue-share §7.4 |
|
||||
| 12 | a cue event | its event digest ([opportunity-report](opportunity-report.md) §7.3, live-events §3.2) | no source: cited by opportunity-report §7.3 |
|
||||
| 13 | a room policy | its digest ([opportunity-report](opportunity-report.md) §7.3, live-ads §3.2) | no source: cited by opportunity-report §7.3 |
|
||||
| 14 | an OER/2 receipt | the 32 bytes its `receipt_id` names (oer-receipts §5) | a `job` source (version 2): the receipt of each attempt of its record |
|
||||
| 15 | a `wai.video.int_motion` decode receipt | its receipt hash (integer-payloads §15.1) | a `job` source (version 2): each output receipt |
|
||||
| 16 | a route record | its digest (route-record §5.3) | a `job` source (version 2): its record |
|
||||
| 14 | an OER/2 receipt | the 32 bytes its `receipt_id` names (oer-receipts §5) | a `job` source (versions 2 and 3): the receipt of each attempt of its record |
|
||||
| 15 | a `wai.video.int_motion` decode receipt | its receipt hash (integer-payloads §15.1) | a `job` source (versions 2 and 3): each output receipt |
|
||||
| 16 | a route record | its digest (route-record §5.3) | a `job` source (versions 2 and 3): its record |
|
||||
|
||||
Kinds 17 to 127 are unassigned. Kinds **128 to 255 are reserved for private and provisional
|
||||
use**: an implementation's own records, such as a hosted service's provisional records carried
|
||||
|
|
@ -255,9 +279,13 @@ digest WAI defines covers one.
|
|||
A source's **head** is its receipt, statement or package receipt; an `ads` source's heads are its
|
||||
receipts, a `stitched` source's its accepted records, and a `job` source's its route record. A head of one source that another source of the invoice also contributes is refused
|
||||
`record_duplicate`; a per-hop record, which a room receipt and a statement can both name, is
|
||||
contributed once. An OER/2 receipt or an output receipt a `job` source contributes that an earlier
|
||||
`job` source of the invoice also contributes is refused `record_duplicate` too: one hop serves one
|
||||
job, and one output is one job's, so neither is billed for two.
|
||||
contributed once. An OER/2 receipt a `job` source contributes that an earlier `job` source of the
|
||||
invoice also contributes is refused `record_duplicate` too: one hop serves one job, so no hop is
|
||||
billed for two. Under a tariff of version 2, so is an output receipt: one output is one job's.
|
||||
Under a tariff of version 3 it is not: an output receipt is the evidence of what a job delivered,
|
||||
not the work, and the same request run again on a deterministic codec delivers the same output
|
||||
under the same receipt from a new hop that did the work again (§6.9). Two lines of one invoice
|
||||
that cite one output receipt therefore cite different hop receipts, whichever lines they are.
|
||||
|
||||
### 5.3 Placement in time
|
||||
|
||||
|
|
@ -272,9 +300,11 @@ A source is **in** a period when:
|
|||
- `job`: its route record's `accepted` is within it.
|
||||
|
||||
Two room receipts of one room whose sessions overlap, two statements of one party whose periods
|
||||
overlap, two receipts of the invoice's ads sources naming one break and one session, and two
|
||||
route records of one router naming one `job`, bill one thing twice: the later is refused
|
||||
`record_overlap`.
|
||||
overlap, two receipts of the invoice's ads sources naming one break and one session, and, under
|
||||
a tariff of version 2, two route records of one router naming one `job`, bill one thing twice:
|
||||
the later is refused `record_overlap`. Under a tariff of version 3, two route records of one job
|
||||
are two runs of it, each billed from its own hop, and no hop twice (§5.2): a job's id names what
|
||||
was asked, not the work, and a request run again is worked again.
|
||||
|
||||
## 6. The invoice
|
||||
|
||||
|
|
@ -283,7 +313,7 @@ route records of one router naming one `job`, bill one thing twice: the later is
|
|||
| field | size | meaning |
|
||||
|---|---|---|
|
||||
| `magic` | 4 | `WLI1` (ASCII) |
|
||||
| `version` | 1 | its tariff's version, 1 or 2 |
|
||||
| `version` | 1 | its tariff's version, 1, 2 or 3 |
|
||||
| `seller`, `buyer` | 32 each | the tariff's |
|
||||
| `tariff` | 32 | the tariff's digest |
|
||||
| `prev` | 32 | the digest of the invoice before it in its chain (§6.9); zero for the first |
|
||||
|
|
@ -292,7 +322,7 @@ route records of one router naming one `job`, bill one thing twice: the later is
|
|||
| `records` | 4 + 33 × *r* | a u32 count, then each record (§5.2): `kind` (u8) and `id` (32), in strictly ascending order of (`kind`, `id`) |
|
||||
| `lines` | 2 + 94 × *l* | a u16 count, then each line (§6.3), in strictly ascending order of (`source_kind`, `source`, `unit`) |
|
||||
| `energy` | 1 + 18 × *e* | a u8 count, then each row (§6.6): `scope` (u8), `tag` (u8), `figures` (u32), `joules` (u64, µJ), `max_ppm` (u32), in strictly ascending order of (`scope`, `tag`) |
|
||||
| `unmetered` | 16 or 20 | four u32: the unmetered figures of scopes 1 to 4 (§6.6); in version 2, five, scopes 1 to 5 |
|
||||
| `unmetered` | 16 or 20 | four u32: the unmetered figures of scopes 1 to 4 (§6.6); in versions 2 and 3, five, scopes 1 to 5 |
|
||||
| `amount` | 8 | u64: the sum of the lines' amounts |
|
||||
| `sig` | 64 | §6.7 |
|
||||
|
||||
|
|
@ -307,7 +337,7 @@ A line is 94 bytes:
|
|||
|
||||
| field | size | meaning |
|
||||
|---|---|---|
|
||||
| `source_kind` | 1 | 1 room, 2 delivery, 3 package, 4 ads, 5 job (version 2) |
|
||||
| `source_kind` | 1 | 1 room, 2 delivery, 3 package, 4 ads, 5 job (versions 2 and 3) |
|
||||
| `source` | 32 | the room id, the delivering party, the packager, the ad decider or the router |
|
||||
| `unit` | 1 | the unit (below) |
|
||||
| `quantity` | 8 | u64: the quantity of the unit the records state |
|
||||
|
|
@ -332,7 +362,7 @@ each:
|
|||
| 3 | 3 | `stored_bytes` | their `stored_bytes` | `package` 2 | 10^9 |
|
||||
| 4 | 1 | `presented_impressions` | under a policy that requires claims, the impressions that count (ad-presentation §9.2); of a stitched source, the presentations of its accepted records' ads (stitched-presentation §7.4), one per session | `ads` 0 | 1 |
|
||||
| 4 | 2 | `billable_impressions` | under a policy that does not, the billable impressions (quartile at least 1) | `ads` 1 | 1 |
|
||||
| 5 | *k* | the *k*-th job price's unit | over the router's job sources with a billed hop whose serving capability is that price's `capability`: one per job (`jobs`), or their output receipts' `work_pixel_ops` (`subpixels`) | the *k*-th job price's `price` | its `divisor` |
|
||||
| 5 | *k* | the *k*-th job price's unit | over the router's job sources with a billed hop whose serving capability is that price's `capability` (and, in version 3, whose operation is its `operation`): one per job (`jobs`), or each job's output receipts' `work_pixel_ops` (`subpixels`) | the *k*-th job price's `price` | its `divisor` |
|
||||
|
||||
There is one line for each source and unit whose quantity is above 0, a price of 0 included, and
|
||||
none for a quantity of 0. A delivery line's bytes are those its party's downstream parties signed
|
||||
|
|
@ -360,12 +390,17 @@ Each line cites the records it came from:
|
|||
for `presented_impressions`, each counted impression's own claim; of a stitched source, its
|
||||
stitcher's accepted records, and each presented own claim of their ads;
|
||||
- a job line: for each job it counts, its route record and its billed hop's OER/2 receipt, and,
|
||||
for `subpixels`, each of its output receipts whose `work_pixel_ops` is above 0.
|
||||
for `subpixels`, each of its output receipts whose `work_pixel_ops` is above 0. In version 3,
|
||||
an output receipt that two jobs of the line delivered is cited once and its `work_pixel_ops`
|
||||
counted for each (§6.9).
|
||||
|
||||
`cited` is their number, and `cites` is SHA-256 over the 17 bytes `wai:ledger-cites` (ASCII) and
|
||||
0x01, followed by each cited record's `kind` and identity, in ascending order of (`kind`,
|
||||
identity). Every cited record is in `records`. A buyer recomputes a line from the records it
|
||||
cites alone, and shows one line to an auditor with its records and nothing else.
|
||||
cites alone, and shows one line to an auditor with its records and nothing else. One line is
|
||||
the exception: a version-3 job line two of whose jobs delivered one output receipt names each job
|
||||
and that output once, and how many of its jobs delivered the output is stated by their sources,
|
||||
each of which pairs its job with its outputs (§5.1).
|
||||
|
||||
### 6.6 Energy
|
||||
|
||||
|
|
@ -377,7 +412,7 @@ Each record that states an energy figure states it once, in one **scope**:
|
|||
| 2 `delivery_hops` | the per-hop records named by the statements and by no room receipt of the invoice |
|
||||
| 3 `packaging` | the package receipts' `joules_micro` and class |
|
||||
| 4 `ad_decisions` | the ad receipts' `joules_micro` and class; a pod-input record states no figure |
|
||||
| 5 `jobs` (version 2) | each job source's billed hop's figure (§5.1): the hop that served the job, in the class its OER/2 receipt states; a job with no billed hop states none |
|
||||
| 5 `jobs` (versions 2 and 3) | each job source's billed hop's figure (§5.1): the hop that served the job, in the class its OER/2 receipt states; a job with no billed hop states none |
|
||||
|
||||
For each scope, `unmetered` counts its figures that are the unmetered marker, which are in no row
|
||||
and **MUST NOT** be presented as zero joules. For each scope and each tag among its metered
|
||||
|
|
@ -408,10 +443,11 @@ invoice with a row of tag 0. It refuses as `private_kind` an invoice with a reco
|
|||
|
||||
- `start` is below `end`; `currency` is three capitals `A`–`Z`;
|
||||
- each list is in the strictly ascending order §6.1 gives it; each record's `kind` is 1 to 9 (in
|
||||
version 2, also 14 to 16: kinds 10 to 13 are no ledger source's, §5.2);
|
||||
- each line's `source_kind` is 1 to 4 (1 to 5 in version 2), its `unit` a unit of its kind (§6.3;
|
||||
for kind 5, at least 1), its `quantity` and `cited` above 0;
|
||||
- each row's `scope` is 1 to 4 (1 to 5 in version 2), its `tag` 2 to 5 and its `figures` above 0.
|
||||
versions 2 and 3, also 14 to 16: kinds 10 to 13 are no ledger source's, §5.2);
|
||||
- each line's `source_kind` is 1 to 4 (1 to 5 in versions 2 and 3), its `unit` a unit of its kind
|
||||
(§6.3; for kind 5, at least 1), its `quantity` and `cited` above 0;
|
||||
- each row's `scope` is 1 to 4 (1 to 5 in versions 2 and 3), its `tag` 2 to 5 and its `figures`
|
||||
above 0.
|
||||
|
||||
Then the signature (§6.7): `bad_key`, `bad_signature`.
|
||||
|
||||
|
|
@ -419,9 +455,29 @@ Then the signature (§6.7): `bad_key`, `bad_signature`.
|
|||
|
||||
The invoices of one tariff form a **chain**: each names in `prev` the invoice before it, the first
|
||||
names zero, and their periods do not overlap. A ledger keeps one chain per tariff. No record is in
|
||||
the `records` of two invoices of a chain: a record billed once is billed never again, whatever
|
||||
period a later invoice states, and a package receipt, which states no time, is billed in the
|
||||
invoice that first holds it.
|
||||
the `records` of two invoices of a chain, but, under a tariff of version 3, an output receipt
|
||||
(kind 15): a record billed once is billed never again, whatever period a later invoice states, and
|
||||
a package receipt, which states no time, is billed in the invoice that first holds it.
|
||||
|
||||
What a chain bills once for a job is its **work**, and its output's receipt is the **evidence**.
|
||||
A run of a job is a hop that did it: new energy, and a new OER/2 receipt and route record, new
|
||||
on every run. An output receipt is not the work: the codecs are deterministic, so the same request
|
||||
run again delivers the same output under the same receipt. Under a tariff of version 3, therefore:
|
||||
|
||||
- the records a chain bills once for a job are its route record and its hops' OER/2 receipts,
|
||||
the billed hop's among them;
|
||||
- an output receipt is cited, and carried in its lines' citation digests, but is not billed once:
|
||||
a later invoice may list it again for another job's hop;
|
||||
- a re-run whose route record and hop receipt are new is billed, in the invoice that holds it;
|
||||
- a source with a hop receipt or a route record the previous invoice holds is refused
|
||||
`record_rebilled`, and one with a hop receipt an earlier source of the invoice gives is refused
|
||||
`record_duplicate` (§5.2): a replay of either is never billed;
|
||||
- two lines of one invoice may cite one output receipt only under different hop receipts.
|
||||
|
||||
Under a tariff of version 2, an output receipt is billed once as well, and a re-run of a job
|
||||
whose output an invoice of the chain holds is refused, as that tariff's buyer signed it. A
|
||||
package keeps its rule in every version: it is billed as a delivered product, once per package
|
||||
receipt, which is a rule of its own and not a job's.
|
||||
|
||||
## 7. Verification
|
||||
|
||||
|
|
@ -453,9 +509,10 @@ It runs these steps and stops at the first refusal, which it reports with the re
|
|||
that states a tariff states this one (§3, `tariff_conflict`); and it is in the invoice's period
|
||||
(§5.3, `outside_period`).
|
||||
6. **Across sources**, at `source` and the index of the later: a head contributed twice
|
||||
(`record_duplicate`); a receipt, statement or ad receipt overlapping an earlier one (§5.3,
|
||||
`record_overlap`); and a record that is in the previous invoice's `records`
|
||||
(`record_rebilled`).
|
||||
(`record_duplicate`); a receipt, statement or ad receipt overlapping an earlier one, or, in
|
||||
version 2, a route record of the router and job an earlier one names (§5.3,
|
||||
`record_overlap`); and a record that is in the previous invoice's `records`, in version 3 an
|
||||
output receipt excepted (§6.9, `record_rebilled`).
|
||||
7. **The figures**, at `invoice`, recomputed from the sources as §6.2 to §6.6 compute them and
|
||||
compared in this order:
|
||||
1. a line whose source and unit no source gives, or that states a larger quantity than the
|
||||
|
|
@ -472,9 +529,10 @@ A figure that cannot be held in its field is a difference.
|
|||
- **refuse**: the reason and where.
|
||||
- **accept**: the invoice's digest; the tariff's digest; the currency and the period; the number of
|
||||
records; each line with its source kind, source, unit name (and, for a job line, the job
|
||||
price's `capability`), quantity, price, amount and its cited records; each energy row by scope and class label with its figures, joules and `max_ppm`; each
|
||||
scope's unmetered figures and coverage (scopes 1 to 4, and in version 2 scope 5); the amount; and
|
||||
the reports of §8.
|
||||
price's `capability`, and in version 3 its `operation`), quantity, price, amount and its cited
|
||||
records; each energy row by scope and class label with its figures, joules and `max_ppm`; each
|
||||
scope's unmetered figures and coverage (scopes 1 to 4, and in versions 2 and 3 scope 5); the
|
||||
amount; and the reports of §8.
|
||||
|
||||
## 8. Reports
|
||||
|
||||
|
|
@ -586,7 +644,8 @@ from the corpora of the extensions it reads (`live-conformance/metering/`,
|
|||
`live-conformance/stitched/`, `live-conformance/routes/` and `pack-conformance/packs/`), with
|
||||
delivery statements of two parties signed for it and, for each media job, a sink's
|
||||
`wai.video.int_motion` decode receipt of its output over a payload of
|
||||
`wai-rs/tests/vectors/int_video/`. The sources are
|
||||
`wai-rs/tests/vectors/int_video/`, and, for version 3, the route records of a decode, a re-run, a
|
||||
replayed hop receipt and a claimed one, routed and sealed under the route corpus's own seeds. The sources are
|
||||
held once, by name, and each vector is a verification document (§7.1) naming its sources, with the
|
||||
outcome §7.3 gives it:
|
||||
|
||||
|
|
@ -600,7 +659,12 @@ outcome §7.3 gives it:
|
|||
version 2, a month of media jobs one router routed, beside a package: an `int_motion` job billed
|
||||
per job and per subpixel of its output's receipt, a failover billed at the fallback's price, a
|
||||
job served past its deadline and one every allowed hop failed, which bill nothing, and a job
|
||||
whose serving hop was unmetered, counted unmetered in the jobs' scope.
|
||||
whose serving hop was unmetered, counted unmetered in the jobs' scope; and, under a tariff of
|
||||
version 3, the same jobs priced per operation with an `int_motion` decode beside the encodes,
|
||||
each at its own price; the next invoice of that chain, billing the first job run again, its
|
||||
route record and hop receipt new and its output receipt one the first invoice holds; the job
|
||||
and its re-run in one invoice, their shared output receipt cited once and its subpixels billed
|
||||
for each; and a decode under a tariff that prices only the encode, which bills nothing.
|
||||
- **Refused**: each reason of §10 at least once, each vector a control of one defect; a source of
|
||||
each kind outside the scope, a room receipt and a statement under other prices, and a source of
|
||||
each timed kind outside the period; a refusal of each source kind by its own extension; and, for
|
||||
|
|
@ -608,7 +672,10 @@ outcome §7.3 gives it:
|
|||
version-1 tariff, a version-1 invoice under a version-2 tariff, a job given twice, routed twice,
|
||||
and claiming another job's output, and a job line and a jobs row that do not recompute; and an
|
||||
invoice of each version listing a record of a kind reserved for private use, and one listing
|
||||
another text's kind.
|
||||
another text's kind; and, for version 3, job prices out of order by operation, a version-2
|
||||
invoice under a version-3 tariff, a hop receipt the previous invoice billed brought again
|
||||
under a new route record, and one hop receipt claimed by two jobs billed on two lines; and,
|
||||
under a tariff of version 2, the same re-run, refused because its output receipt was billed.
|
||||
|
||||
`wai_receipt_ledger_vectors verify` and the test `tests/receipt_ledger.rs` check that the file
|
||||
regenerates byte for byte, that `wai-rs` gives every vector its outcome, that every reason of §10
|
||||
|
|
@ -633,6 +700,16 @@ copy of the corpus to fail.
|
|||
statement period and one viewer's break once (`record_overlap`), and a record once in a chain
|
||||
(`record_rebilled`). A verifier given one invoice checks it against the one before it; a buyer
|
||||
holding the chain checks every link.
|
||||
- **Work, not identity.** Under a tariff of version 3 a job is billed for the work done, its
|
||||
route record and its hops' OER/2 receipts, once per chain, and its output's receipt is the
|
||||
evidence of what it delivered (§6.9). A deterministic re-run does the work again, a new hop,
|
||||
new energy, a new receipt, and is billed; a replayed hop receipt or route record is refused,
|
||||
which is what keeps a job from being billed twice. A seller cannot bill a job again by citing
|
||||
its output again: every billing needs a hop receipt no invoice of the chain holds. What a re-run
|
||||
rests on is that hop's receipt: a router and a hop that collude can bill a re-run that never
|
||||
ran, as a party and a downstream key can bill bytes (settlement §13). Whether a request is
|
||||
worked again or answered from outputs still kept is the router's decision, not the ledger's;
|
||||
the ledger bills what was worked. A tariff of version 2 keeps billing an output once.
|
||||
- **What the figures rest on.** An invoice is as true as its records: a delivery line rests on
|
||||
acknowledgements, so a party and a downstream key that collude can bill bytes that never moved,
|
||||
as settlement §13 says; a room line rests on per-hop records and the room's events; an ad line
|
||||
|
|
|
|||
|
|
@ -43,7 +43,18 @@ The accepted vectors:
|
|||
billed hop and their output's decode receipt (`work_pixel_ops`); the failover is billed at the
|
||||
fallback's price; the job served past its deadline and the one every allowed hop failed bill
|
||||
nothing; the jobs' energy is the serving hops' figures, a counter row and a model row, and the
|
||||
job whose serving hop was unmetered is counted unmetered (coverage partial).
|
||||
job whose serving hop was unmetered is counted unmetered (coverage partial);
|
||||
- `jobs_priced_per_operation` (version 3, §4.6): the same jobs under a tariff of version 3, which
|
||||
prices each job by its serving capability and its operation, with a decode of
|
||||
`wai.video.int_motion` beside the encodes: the encodes and the decode are billed per job and per
|
||||
subpixel at their own prices, and the failover at the fallback's encode price;
|
||||
- `a_rerun_billed` (version 3, §6.9): the next invoice of that chain bills `job-0001` run again
|
||||
two days later, a new route record and a new hop receipt, its output the decode receipt the first
|
||||
invoice already lists: the work is billed once, and the output is the evidence, cited again;
|
||||
- `one_output_under_two_hops` (version 3): `job-0001` and its re-run in one invoice: both billed,
|
||||
the output receipt they share cited once and its subpixels billed for each hop;
|
||||
- `an_operation_the_tariff_does_not_price` (version 3): a tariff that prices `int_motion`'s encode
|
||||
and not its decode: the decode bills nothing, and its hop's energy is stated.
|
||||
|
||||
Every other vector is refused, a control of one defect: each reason of §10, the reused codes, and a
|
||||
refusal of each source kind by its own extension.
|
||||
|
|
@ -62,7 +73,10 @@ manifests, responses and receipts they name, by their bytes), media jobs from
|
|||
`served_after_the_deadline`, `served_by_an_unmetered_hop`, `every_allowed_survivor_failed`,
|
||||
`served_by_a_hop_whose_figure_states_no_class`, `a_failed_hop_billed`,
|
||||
`an_unknown_energy_ranks_last`), each `int_motion` job's output a decode receipt sealed here over
|
||||
`wai-rs/tests/vectors/int_video/int_video_bitstream.wiv1`, and the
|
||||
`wai-rs/tests/vectors/int_video/int_video_bitstream.wiv1`; the version-3 jobs (`job_decode`,
|
||||
`job_rerun`, `job_rerun_replays_its_hop`, `job_claims_another_hop`) routed here by that corpus's
|
||||
router under its price book and policy, their hop receipts sealed under its hop key, the replayed
|
||||
and the claimed one being `served_first_try`'s own; and the
|
||||
packages `video-base-01` and `video-staged-01` of
|
||||
[`pack-conformance/packs/`](../../pack-conformance/packs/). When one of those corpora is
|
||||
regenerated, so is this one. Nothing here was fetched from outside the repository.
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -10,12 +10,14 @@ settlement statement by settlement_verify.py (settlement §8), a package receipt
|
|||
../pack-conformance/pack_verify.py (packaging §9), an ads room by live_ads_verify.py with
|
||||
ad_presentation_verify.py (live-ads §9, ad-presentation §9), and a stream stitched by others by
|
||||
stitched_presentation_verify.py (stitched-presentation §7), whose statement the ledger reports as
|
||||
it is; and, under a tariff of version 2, a media job by route_record_verify.py (route-record §7,
|
||||
its hop receipts by the OER/2 reference verifier) and its output receipts by
|
||||
it is; and, under a tariff of version 2 or 3, a media job by route_record_verify.py (route-record
|
||||
§7, its hop receipts by the OER/2 reference verifier) and its output receipts by
|
||||
_int_motion_receipt.py (integer-payloads §15). Everything else is here:
|
||||
|
||||
- the tariff (§4): reading, rules, both signatures, its scope;
|
||||
- the invoice (§6): reading, rules, signature, chain;
|
||||
- the tariff (§4): reading, rules, both signatures, its scope, its job prices per capability
|
||||
(version 2) or per capability and operation (version 3);
|
||||
- the invoice (§6): reading, rules, signature, chain, and what a chain bills once (§6.9): in
|
||||
version 3 a job's work, its route record and hop receipts, and never its output's receipt;
|
||||
- what each source contributes (§5.2), its placement in time (§5.3), and the checks across
|
||||
sources (§7.2 step 6);
|
||||
- every line, citation, energy row and amount recomputed (§6.2 to §6.6), compared in §7.2 step
|
||||
|
|
@ -77,7 +79,9 @@ KIND_NAMES = {ROOM: "room", DELIVERY: "delivery", PACKAGE: "package", ADS: "ads"
|
|||
STITCHED = "stitched" # §5.1: a source kind billed as ads
|
||||
SCOPES = {1: "room_hops", 2: "delivery_hops", 3: "packaging", 4: "ad_decisions", 5: "jobs"} # §6.6
|
||||
JOB_UNITS = ["jobs", "subpixels"] # §4.5
|
||||
JOB_VERSIONS = (2, 3) # §4.5, §4.6: the versions that price jobs
|
||||
CAPABILITY = re.compile(r"\Awai\.[a-z0-9_.]{0,124}\Z")
|
||||
OPERATION = re.compile(r"\A[a-z][a-z0-9]*\.[a-z][a-z0-9]*\Z") # §4.6, route-record §2
|
||||
LABELS = {2: "ModelBased", 3: "Estimator", 4: "OnChipCounter", 5: "CalibratedInstrument"}
|
||||
TAGS = {v: k for k, v in LABELS.items()}
|
||||
CLASS_NAMES = ["live", "fetch", "fill", "ancillary"]
|
||||
|
|
@ -130,7 +134,7 @@ def head(b, magic):
|
|||
return "malformed"
|
||||
if len(b) < 5:
|
||||
return "truncated"
|
||||
if b[4] not in (1, 2):
|
||||
if b[4] not in (1, 2, 3):
|
||||
return "unknown_version"
|
||||
return None
|
||||
|
||||
|
|
@ -161,11 +165,12 @@ def read_tariff(b):
|
|||
t["package"] = [r.u(8) for _ in range(3)]
|
||||
t["ads"] = [r.u(8) for _ in range(2)]
|
||||
t["version"] = b[4]
|
||||
t["job"] = []
|
||||
if t["version"] == 2: # §4.5
|
||||
t["job"] = [] # (capability, operation, unit, divisor, price)
|
||||
if t["version"] in JOB_VERSIONS: # §4.5, §4.6
|
||||
for _ in range(r.u(2)):
|
||||
cap = r.take(r.u(1))
|
||||
t["job"].append((cap, r.u(1), r.u(8), r.u(8)))
|
||||
op = r.take(r.u(1)) if t["version"] == 3 else b""
|
||||
t["job"].append((cap, op, r.u(1), r.u(8), r.u(8)))
|
||||
t["scope"] = [(r.u(1), r.take(32)) for _ in range(r.u(2))]
|
||||
body_end = r.p
|
||||
seller_sig, buyer_sig = r.take(64), r.take(64)
|
||||
|
|
@ -175,15 +180,16 @@ def read_tariff(b):
|
|||
raise Refuse("malformed", ["tariff"])
|
||||
def job_price(j):
|
||||
try:
|
||||
cap = j[0].decode("utf-8")
|
||||
cap, op = j[0].decode("utf-8"), j[1].decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
return False
|
||||
return CAPABILITY.match(cap) is not None and 1 <= j[1] <= len(JOB_UNITS) and j[2] >= 1
|
||||
top = 5 if t["version"] == 2 else 4
|
||||
named = OPERATION.match(op) is not None and len(j[1]) <= 128 if t["version"] == 3 else op == ""
|
||||
return CAPABILITY.match(cap) is not None and named and 1 <= j[2] <= len(JOB_UNITS) and j[3] >= 1
|
||||
top = 5 if t["version"] in JOB_VERSIONS else 4
|
||||
rules = (t["start"] < t["end"] and capitals(t["currency"]) and t["seller"] != ZERO and t["buyer"] != ZERO
|
||||
and t["seller"] != t["buyer"] and t["scope"] and ascending(t["scope"])
|
||||
and all(1 <= k <= top and i != ZERO for k, i in t["scope"])
|
||||
and len(t["job"]) <= 255 and ascending([(j[0], j[1]) for j in t["job"]]) and all(job_price(j) for j in t["job"]))
|
||||
and len(t["job"]) <= 255 and ascending([(j[0], j[1], j[2]) for j in t["job"]]) and all(job_price(j) for j in t["job"]))
|
||||
if not rules and "tariff-rules" not in SKIP:
|
||||
raise Refuse("tariff_invalid", ["tariff"])
|
||||
body = b[:body_end]
|
||||
|
|
@ -199,12 +205,12 @@ def read_tariff(b):
|
|||
|
||||
def price(t, kind, unit):
|
||||
if kind == JOB:
|
||||
return t["job"][unit - 1][3]
|
||||
return t["job"][unit - 1][4]
|
||||
return {ROOM: t["room"], DELIVERY: t["delivery"], PACKAGE: t["package"], ADS: t["ads"]}[kind][unit - 1]
|
||||
|
||||
|
||||
def divisor(t, kind, unit):
|
||||
return t["job"][unit - 1][2] if kind == JOB else UNITS[kind][unit - 1][1]
|
||||
return t["job"][unit - 1][3] if kind == JOB else UNITS[kind][unit - 1][1]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- §6 the invoice
|
||||
|
|
@ -224,7 +230,7 @@ def read_invoice(b, at):
|
|||
"amount": r.u(8), "cited": r.u(4), "cites": r.take(32)})
|
||||
i["energy"] = [(r.u(1), r.u(1), r.u(4), r.u(8), r.u(4)) for _ in range(r.u(1))]
|
||||
i["version"] = b[4]
|
||||
v2 = i["version"] == 2
|
||||
v2 = i["version"] in JOB_VERSIONS
|
||||
i["unmetered"] = [r.u(4) for _ in range(5 if v2 else 4)] + ([] if v2 else [0])
|
||||
i["amount"] = r.u(8)
|
||||
sig = r.take(64)
|
||||
|
|
@ -472,7 +478,7 @@ def job_source(s):
|
|||
raise Refuse("output_invalid", ["outputs", i])
|
||||
outputs.append((IMR.receipt_hash(r), r["work"]))
|
||||
return {"kind": JOB, "digest": bytes.fromhex(RRV.digest(rec)), "router": bytes.fromhex(rec["signer"]), "job": rec["job"],
|
||||
"accepted": rec["accepted"], "hops": [oer_identity(a["receipt"]) for a in rec["attempts"]], "billed": billed,
|
||||
"operation": rec["operation"].encode("utf-8"), "accepted": rec["accepted"], "hops": [oer_identity(a["receipt"]) for a in rec["attempts"]], "billed": billed,
|
||||
"figure": fig, "outputs": outputs}
|
||||
|
||||
|
||||
|
|
@ -481,8 +487,8 @@ _SOURCES = {}
|
|||
|
||||
def verify_source(s, signers, version=1):
|
||||
"""§5.1: the source verified by its own extension, a stitched source with `signers`, a job
|
||||
source under a tariff of version 2, or Refuse with that extension's place."""
|
||||
key = json.dumps([s, sorted(k.hex() for k in signers), version, sorted(SKIP & {"job-figure", "output"})], sort_keys=True)
|
||||
source under a tariff of version 2 or 3, or Refuse with that extension's place."""
|
||||
key = json.dumps([s, sorted(k.hex() for k in signers), version in JOB_VERSIONS, sorted(SKIP & {"job-figure", "output"})], sort_keys=True)
|
||||
if key not in _SOURCES:
|
||||
try:
|
||||
_SOURCES[key] = read_source(s, signers, version)
|
||||
|
|
@ -499,7 +505,7 @@ def read_source(s, signers, version):
|
|||
raise Refuse("malformed", [])
|
||||
if s.get("kind") == "stitched":
|
||||
return stitched_source(s, signers)
|
||||
if s.get("kind") == "job" and version == 2:
|
||||
if s.get("kind") == "job" and version in JOB_VERSIONS:
|
||||
return job_source(s)
|
||||
readers = {"room": room_source, "delivery": delivery_source, "package": package_source, "ads": ads_source}
|
||||
if s.get("kind") not in readers:
|
||||
|
|
@ -583,8 +589,8 @@ def overlapping(a, b):
|
|||
return a[0] < b[1] and b[0] < a[1]
|
||||
|
||||
|
||||
def bills_again(f, g):
|
||||
"""§5.3: whether f bills what g, an earlier source, billed."""
|
||||
def bills_again(f, g, version):
|
||||
"""§5.3: whether f bills what g, an earlier source, billed, under a tariff of `version`."""
|
||||
if f["kind"] != g["kind"]:
|
||||
return False
|
||||
if f["kind"] == ROOM:
|
||||
|
|
@ -595,25 +601,32 @@ def bills_again(f, g):
|
|||
if f["kind"] == ADS:
|
||||
return f["room"] == g["room"] and any((x["cue"], x["session"]) == (y["cue"], y["session"])
|
||||
for x in f["receipts"] for y in g["receipts"])
|
||||
if f["kind"] == JOB:
|
||||
return f["router"] == g["router"] and f["job"] == g["job"]
|
||||
if f["kind"] == JOB: # version 2; in version 3 two records of one job are two runs of it
|
||||
return version == 2 and f["router"] == g["router"] and f["job"] == g["job"]
|
||||
return False
|
||||
|
||||
|
||||
def check_across(facts, previous):
|
||||
def billed_once(version, kind):
|
||||
"""§6.9: whether a chain bills a record of `kind` once. In version 3 a job's output receipt is
|
||||
the evidence of what it delivered, which a re-run delivers again; its work, the route record and
|
||||
the hop receipts, is billed once."""
|
||||
return not (kind == VIDEO_RECEIPT and (version == 3 or "output-once" in SKIP))
|
||||
|
||||
|
||||
def check_across(facts, previous, version=1):
|
||||
"""§7.2 step 6: (the index, the reason) of the first source refused, or None."""
|
||||
billed = set(previous["records"]) if previous else set()
|
||||
billed = {r for r in (previous["records"] if previous else []) if billed_once(version, r[0])}
|
||||
seen, jobs = set(), set()
|
||||
for k, f in enumerate(facts):
|
||||
hs = heads(f)
|
||||
if "duplicate" not in SKIP and (any(h in seen for h in hs) or len(set(hs)) != len(hs)):
|
||||
return k, "record_duplicate"
|
||||
if f["kind"] == JOB: # §5.2: one hop, one output, one job
|
||||
own = contributed(f) - set(hs)
|
||||
if f["kind"] == JOB: # §5.2: one hop, one job; in version 2, one output too
|
||||
own = {r for r in contributed(f) - set(hs) if billed_once(version, r[0])}
|
||||
if "job-duplicate" not in SKIP and own & jobs:
|
||||
return k, "record_duplicate"
|
||||
jobs |= own
|
||||
if "overlap" not in SKIP and any(bills_again(f, g) for g in facts[:k]):
|
||||
if "overlap" not in SKIP and any(bills_again(f, g, version) for g in facts[:k]):
|
||||
return k, "record_overlap"
|
||||
if "rebilled" not in SKIP and contributed(f) & billed:
|
||||
return k, "record_rebilled"
|
||||
|
|
@ -672,8 +685,8 @@ def compute(t, facts):
|
|||
if f["billed"] is None:
|
||||
continue
|
||||
cap, hop = f["billed"]
|
||||
for k, (pcap, unit, _, _) in enumerate(t["job"]):
|
||||
if pcap != cap:
|
||||
for k, (pcap, pop, unit, _, _) in enumerate(t["job"]):
|
||||
if pcap != cap or (t["version"] == 3 and pop != f["operation"] and "operation" not in SKIP):
|
||||
continue
|
||||
cites = [(ROUTE_RECORD, f["digest"]), (OER_RECEIPT, hop)]
|
||||
if unit == 1:
|
||||
|
|
@ -909,7 +922,7 @@ def verify(doc):
|
|||
if bad:
|
||||
raise Refuse(bad, ["source", k])
|
||||
facts.append(f)
|
||||
across = check_across(facts, prev) # step 6
|
||||
across = check_across(facts, prev, t["version"]) # step 6
|
||||
if across:
|
||||
raise Refuse(across[1], ["source", across[0]])
|
||||
c = compute(t, facts) # step 7
|
||||
|
|
@ -926,17 +939,19 @@ def verify(doc):
|
|||
def line_json(x, t):
|
||||
"""§7.3: a line, a job line naming its job price's unit and capability."""
|
||||
out = {"source_kind": KIND_NAMES[x["kind"]], "source": x["source"].hex(),
|
||||
"unit": JOB_UNITS[t["job"][x["unit"] - 1][1] - 1] if x["kind"] == JOB else UNITS[x["kind"]][x["unit"] - 1][0],
|
||||
"unit": JOB_UNITS[t["job"][x["unit"] - 1][2] - 1] if x["kind"] == JOB else UNITS[x["kind"]][x["unit"] - 1][0],
|
||||
"quantity": x["quantity"], "price": x["price"], "amount": x["amount"],
|
||||
"cites": [[k, i.hex()] for k, i in x["cited_records"]]}
|
||||
if x["kind"] == JOB:
|
||||
out["capability"] = t["job"][x["unit"] - 1][0].decode("utf-8")
|
||||
if t["version"] == 3:
|
||||
out["operation"] = t["job"][x["unit"] - 1][1].decode("utf-8")
|
||||
return out
|
||||
|
||||
|
||||
def accepted(inv, facts, c, t):
|
||||
unmetered, coverage = {}, {}
|
||||
for s in range(1, 6 if inv["version"] == 2 else 5):
|
||||
for s in range(1, 6 if inv["version"] in JOB_VERSIONS else 5):
|
||||
n = inv["unmetered"][s - 1]
|
||||
unmetered[SCOPES[s]] = n
|
||||
coverage[SCOPES[s]] = "partial" if n > 0 else "whole" if any(r[0] == s for r in inv["energy"]) else "none"
|
||||
|
|
@ -1030,6 +1045,31 @@ def mutations(doc):
|
|||
k = next(i for i, v in enumerate(doc["vectors"]) if v["name"] == "jobs_routed_and_served")
|
||||
m["vectors"][k]["outcome"]["coverage"]["jobs"] = "whole"
|
||||
out.append(("jobs_routed_and_served: its jobs' coverage stated whole", m))
|
||||
names = {v["name"]: i for i, v in enumerate(doc["vectors"])}
|
||||
if "jobs_priced_per_operation" in names: # version 3
|
||||
k = names["jobs_priced_per_operation"]
|
||||
lines = doc["vectors"][k]["outcome"]["lines"]
|
||||
dec = next(i for i, x in enumerate(lines) if x.get("operation") == "media.decode" and x["unit"] == "subpixels")
|
||||
enc = next(i for i, x in enumerate(lines) if x.get("operation") == "media.encode" and x["unit"] == "subpixels")
|
||||
m = copy.deepcopy(doc)
|
||||
m["vectors"][k]["outcome"]["lines"][dec]["price"] = lines[enc]["price"]
|
||||
out.append(("jobs_priced_per_operation: the decode priced as the encode", m))
|
||||
m = copy.deepcopy(doc)
|
||||
m["vectors"][k]["outcome"]["lines"][dec]["operation"] = "media.encode"
|
||||
out.append(("jobs_priced_per_operation: the decode line named an encode", m))
|
||||
k = names["one_output_under_two_hops"]
|
||||
lines = doc["vectors"][k]["outcome"]["lines"]
|
||||
sp = next(i for i, x in enumerate(lines) if x["unit"] == "subpixels")
|
||||
m = copy.deepcopy(doc)
|
||||
m["vectors"][k]["outcome"]["lines"][sp]["quantity"] //= 2
|
||||
out.append(("one_output_under_two_hops: the shared output's subpixels billed once", m))
|
||||
k = names["a_rerun_billed"]
|
||||
m = copy.deepcopy(doc)
|
||||
m["vectors"][k]["previous"] = None
|
||||
out.append(("a_rerun_billed: the invoice before it not given", m))
|
||||
m = copy.deepcopy(doc)
|
||||
m["vectors"][k]["sources"] = ["job_rerun_replays_its_hop"]
|
||||
out.append(("a_rerun_billed: its re-run replaced by one replaying the billed hop", m))
|
||||
for k, v in enumerate(doc["vectors"]):
|
||||
if v["outcome"]["outcome"] == "refuse":
|
||||
m = copy.deepcopy(doc)
|
||||
|
|
@ -1041,7 +1081,7 @@ def mutations(doc):
|
|||
|
||||
CHECKS = ["signatures", "buyer-signature", "tariff-rules", "invoice-rules", "row-of-no-class", "binding", "outside-tariff",
|
||||
"previous-given", "previous-overlap", "scope", "conflict", "period", "duplicate", "overlap", "rebilled", "quantity",
|
||||
"summed", "citations", "mismatch", "job-figure", "output", "job-duplicate", "private-kind"]
|
||||
"summed", "citations", "mismatch", "job-figure", "output", "job-duplicate", "private-kind", "operation", "output-once"]
|
||||
|
||||
|
||||
def main(corpus_dir):
|
||||
|
|
|
|||
|
|
@ -57,7 +57,8 @@ pub struct Chain {
|
|||
pub sources: Vec<Held>,
|
||||
/// The chain, in order: each invoice's bytes and its verification document.
|
||||
pub invoices: Vec<(Invoice, Vec<u8>, Value)>,
|
||||
/// The records of every invoice of the chain (§6.9).
|
||||
/// The records of every invoice of the chain (§6.9); of them, a source may not bring again
|
||||
/// those its tariff's version bills once ([`rl::billed_once`]).
|
||||
pub billed: BTreeSet<(u8, [u8; 32])>,
|
||||
}
|
||||
|
||||
|
|
@ -155,8 +156,10 @@ impl Ledger {
|
|||
rl::check_source(&c.tariff, c.tariff.start, c.tariff.end, &facts).map_err(|r| refuse(r.code(), json!(["source"])))?;
|
||||
let mut all: Vec<Facts> = c.sources.iter().map(|h| h.facts.clone()).collect();
|
||||
all.push(facts.clone());
|
||||
rl::check_across(&all, None).map_err(|(_, r)| refuse(r.code(), json!(["source"])))?;
|
||||
if facts.records().iter().any(|r| c.billed.contains(r)) {
|
||||
rl::check_across(c.tariff.version, &all, None).map_err(|(_, r)| refuse(r.code(), json!(["source"])))?;
|
||||
// A record the chain billed is billed never again; under a tariff of version 3, a job's
|
||||
// output receipt is cited again by a re-run, whose hop and route record are new (§6.9).
|
||||
if facts.records().iter().any(|r| rl::billed_once(c.tariff.version, r.0) && c.billed.contains(r)) {
|
||||
return Err(refuse(Reason::RecordRebilled.code(), json!(["source"])));
|
||||
}
|
||||
let n = c.sources.len() as u64;
|
||||
|
|
|
|||
|
|
@ -1,7 +1,8 @@
|
|||
//! The service's ledger against the committed corpus (`live-conformance/ledger/`): given the
|
||||
//! corpus's tariff and sources in the corpus's order, it issues the corpus's invoices byte for
|
||||
//! byte, with the outcomes the corpus states; it recovers its chain from its directory after a
|
||||
//! restart; it bills each source once; and one process holds its directory.
|
||||
//! restart; it bills each source once, and under a tariff of version 3 a job's work once; and
|
||||
//! one process holds its directory.
|
||||
|
||||
use std::path::PathBuf;
|
||||
|
||||
|
|
@ -107,6 +108,32 @@ fn a_source_waits_for_the_invoice_of_its_period() {
|
|||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
/// Under a tariff of version 3 the ledger bills a job's work once (receipt-ledger §6.9): it issues
|
||||
/// the corpus's chain of version 3 byte for byte, a re-run of a job it billed, whose output
|
||||
/// receipt it billed already, is held and billed in the next invoice; a hop receipt it billed,
|
||||
/// brought again under a new route record, is refused.
|
||||
#[test]
|
||||
fn a_rerun_is_billed_and_a_replayed_hop_is_not() {
|
||||
let doc = corpus();
|
||||
let (k, r) = (vector(&doc, "jobs_priced_per_operation"), vector(&doc, "a_rerun_billed"));
|
||||
let dir = scratch("rerun");
|
||||
let t = tariff_of(k);
|
||||
let mut l = Ledger::open(&dir, seller()).unwrap();
|
||||
l.add_tariff(&unhex(k["tariff"].as_str().unwrap()).unwrap()).unwrap();
|
||||
add_sources(&mut l, &doc, k);
|
||||
let (start, end) = (k["outcome"]["start"].as_u64().unwrap(), k["outcome"]["end"].as_u64().unwrap());
|
||||
let i = l.invoice(&t, start, end).unwrap();
|
||||
assert_eq!(Value::String(crate::ledger::hex(&i.invoice)), k["invoice"], "the corpus's first invoice of version 3, byte for byte");
|
||||
// The service checks a source against every one it holds, billed or not: a hop receipt held
|
||||
// already is a duplicate before it is a rebilling.
|
||||
assert_eq!(reason(l.add_source(&t, doc["sources"]["job_rerun_replays_its_hop"].clone()).unwrap_err()), "record_duplicate", "a billed hop receipt");
|
||||
add_sources(&mut l, &doc, r);
|
||||
let j = l.invoice(&t, end, r["outcome"]["end"].as_u64().unwrap()).unwrap();
|
||||
assert_eq!(Value::String(crate::ledger::hex(&j.invoice)), r["invoice"], "the re-run, billed in the next invoice");
|
||||
assert_eq!(j.outcome, r["outcome"]);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn one_process_holds_the_directory_and_tariffs_are_its_sellers() {
|
||||
let doc = corpus();
|
||||
|
|
|
|||
|
|
@ -15,9 +15,12 @@
|
|||
//!
|
||||
//! What this module implements, by section of the extension:
|
||||
//!
|
||||
//! - §4: the tariff ([`Tariff`]), its two signatures and its scope;
|
||||
//! - §4: the tariff ([`Tariff`]), its two signatures and its scope, and its job prices: per
|
||||
//! serving capability in version 2, per serving capability and operation in version 3;
|
||||
//! - §5: the sources ([`verify_source`]) and what each contributes ([`Facts`]);
|
||||
//! - §6: the invoice ([`Invoice`]), its lines, citations, energy and chain ([`compute`]);
|
||||
//! - §6: the invoice ([`Invoice`]), its lines, citations, energy and chain ([`compute`]), and
|
||||
//! what a chain bills once ([`billed_once`]): in version 3, a job's work, never its output's
|
||||
//! receipt;
|
||||
//! - §7: verification ([`verify`]); §8: the reports.
|
||||
//!
|
||||
//! The corpus `live-conformance/ledger/` is written by [`corpus::generate`] and checked by
|
||||
|
|
@ -51,6 +54,9 @@ pub const INVOICE_MAGIC: [u8; 4] = *b"WLI1";
|
|||
pub const VERSION: u8 = 1;
|
||||
/// The version of a tariff that prices jobs, and of its invoices (§4.5).
|
||||
pub const VERSION_2: u8 = 2;
|
||||
/// The version of a tariff that prices jobs per operation and bills a job's work, and of its
|
||||
/// invoices (§4.6, §6.9).
|
||||
pub const VERSION_3: u8 = 3;
|
||||
pub const TARIFF_DOMAIN: &[u8] = b"wai:ledger-tariff\x01";
|
||||
pub const ACCEPT_DOMAIN: &[u8] = b"wai:ledger-accept\x01";
|
||||
pub const INVOICE_DOMAIN: &[u8] = b"wai:ledger-invoice\x01";
|
||||
|
|
@ -310,7 +316,7 @@ impl<'a> Rd<'a> {
|
|||
}
|
||||
}
|
||||
|
||||
/// The first steps of reading (§4.3, §6.8): the magic and the version, 1 or 2.
|
||||
/// The first steps of reading (§4.3, §6.8): the magic and the version, 1, 2 or 3.
|
||||
fn read_head(b: &[u8], magic: &[u8; 4]) -> Result<u8, &'static str> {
|
||||
if b.len() < 4 {
|
||||
return Err(TRUNCATED);
|
||||
|
|
@ -320,15 +326,24 @@ fn read_head(b: &[u8], magic: &[u8; 4]) -> Result<u8, &'static str> {
|
|||
}
|
||||
match b.get(4) {
|
||||
None => Err(TRUNCATED),
|
||||
Some(&v) if v == VERSION || v == VERSION_2 => Ok(v),
|
||||
Some(&v) if v == VERSION || v == VERSION_2 || v == VERSION_3 => Ok(v),
|
||||
Some(_) => Err(UNKNOWN_VERSION),
|
||||
}
|
||||
}
|
||||
|
||||
/// A job price (§4.5): a capability, a unit (1 `jobs`, 2 `subpixels`), a divisor and a price.
|
||||
/// Whether a tariff or an invoice of `version` prices jobs (§4.5, §4.6): versions 2 and 3.
|
||||
pub fn prices_jobs(version: u8) -> bool {
|
||||
version == VERSION_2 || version == VERSION_3
|
||||
}
|
||||
|
||||
/// A job price (§4.5, §4.6): a capability, in version 3 an operation, a unit (1 `jobs`, 2
|
||||
/// `subpixels`), a divisor and a price.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord)]
|
||||
pub struct JobPrice {
|
||||
pub capability: String,
|
||||
/// Version 3: the catalogue operation it prices (§4.6). Empty in version 2, whose job
|
||||
/// prices name none.
|
||||
pub operation: String,
|
||||
pub unit: u8,
|
||||
pub divisor: u64,
|
||||
pub price: u64,
|
||||
|
|
@ -341,6 +356,13 @@ fn job_capability(c: &str) -> bool {
|
|||
(1..=128).contains(&c.len()) && c.starts_with("wai.") && c.bytes().all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'_' || b == b'.')
|
||||
}
|
||||
|
||||
/// A job price's operation (§4.6): route-record §2's form, two words joined by `.`, each a
|
||||
/// lowercase ASCII letter and then lowercase letters and digits, of at most 128 bytes.
|
||||
fn job_operation(o: &str) -> bool {
|
||||
let word = |w: &str| !w.is_empty() && w.as_bytes()[0].is_ascii_lowercase() && w.bytes().all(|c| c.is_ascii_lowercase() || c.is_ascii_digit());
|
||||
o.len() <= 128 && matches!(o.split_once('.'), Some((n, v)) if word(n) && word(v))
|
||||
}
|
||||
|
||||
fn ascending<T: PartialOrd>(v: &[T]) -> bool {
|
||||
v.windows(2).all(|w| w[0] < w[1])
|
||||
}
|
||||
|
|
@ -352,7 +374,8 @@ fn ascending<T: PartialOrd>(v: &[T]) -> bool {
|
|||
/// A tariff (§4.1).
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct Tariff {
|
||||
/// 1, or 2 for a tariff that prices jobs (§4.5).
|
||||
/// 1; 2 for a tariff that prices jobs (§4.5); 3 for one that prices them per operation and
|
||||
/// bills their work (§4.6, §6.9).
|
||||
pub version: u8,
|
||||
pub seller: [u8; 32],
|
||||
pub buyer: [u8; 32],
|
||||
|
|
@ -367,7 +390,8 @@ pub struct Tariff {
|
|||
pub package: [u64; 3],
|
||||
/// Per presented impression, per billable impression.
|
||||
pub ads: [u64; 2],
|
||||
/// Version 2: the job prices, in strictly ascending order of (capability, unit) (§4.5).
|
||||
/// Versions 2 and 3: the job prices, in strictly ascending order of (capability, unit) in
|
||||
/// version 2 (§4.5) and of (capability, operation, unit) in version 3 (§4.6).
|
||||
pub job: Vec<JobPrice>,
|
||||
/// (kind, id), strictly ascending.
|
||||
pub scope: Vec<(u8, [u8; 32])>,
|
||||
|
|
@ -392,11 +416,15 @@ impl Tariff {
|
|||
for p in self.delivery.iter().chain(&self.package).chain(&self.ads) {
|
||||
o.extend_from_slice(&p.to_le_bytes());
|
||||
}
|
||||
if self.version == VERSION_2 {
|
||||
if prices_jobs(self.version) {
|
||||
o.extend_from_slice(&u16::try_from(self.job.len()).expect("at most 65535 job prices").to_le_bytes());
|
||||
for j in &self.job {
|
||||
o.push(u8::try_from(j.capability.len()).expect("a capability of at most 255 bytes"));
|
||||
o.extend_from_slice(j.capability.as_bytes());
|
||||
if self.version == VERSION_3 {
|
||||
o.push(u8::try_from(j.operation.len()).expect("an operation of at most 255 bytes"));
|
||||
o.extend_from_slice(j.operation.as_bytes());
|
||||
}
|
||||
o.push(j.unit);
|
||||
o.extend_from_slice(&j.divisor.to_le_bytes());
|
||||
o.extend_from_slice(&j.price.to_le_bytes());
|
||||
|
|
@ -454,13 +482,21 @@ impl Tariff {
|
|||
}
|
||||
let ads = [r.u64()?, r.u64()?];
|
||||
let mut job = Vec::new();
|
||||
if version == VERSION_2 {
|
||||
if prices_jobs(version) {
|
||||
let n = r.u16()? as usize;
|
||||
// Bytes that are not text break §4.5's and §4.6's forms, a rule (§4.3): read as
|
||||
// empty, which no form admits, and refused `tariff_invalid` with the rules.
|
||||
let text = |b: &[u8]| String::from_utf8(b.to_vec()).unwrap_or_default();
|
||||
for _ in 0..n {
|
||||
let len = r.u8()? as usize;
|
||||
let cap = r.take(len)?;
|
||||
let capability = String::from_utf8(cap.to_vec()).map_err(|_| MALFORMED)?;
|
||||
job.push(JobPrice { capability, unit: r.u8()?, divisor: r.u64()?, price: r.u64()? });
|
||||
let capability = text(r.take(len)?);
|
||||
let operation = if version == VERSION_3 {
|
||||
let len = r.u8()? as usize;
|
||||
text(r.take(len)?)
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
job.push(JobPrice { capability, operation, unit: r.u8()?, divisor: r.u64()?, price: r.u64()? });
|
||||
}
|
||||
}
|
||||
let n = r.u16()? as usize;
|
||||
|
|
@ -482,10 +518,13 @@ impl Tariff {
|
|||
&& t.seller != t.buyer
|
||||
&& !t.scope.is_empty()
|
||||
&& ascending(&t.scope)
|
||||
&& t.scope.iter().all(|(k, id)| (1..=if t.version == VERSION_2 { 5 } else { 4 }).contains(k) && *id != [0; 32])
|
||||
&& t.scope.iter().all(|(k, id)| (1..=if prices_jobs(t.version) { 5 } else { 4 }).contains(k) && *id != [0; 32])
|
||||
&& t.job.len() <= 255
|
||||
&& t.job.windows(2).all(|w| (w[0].capability.as_bytes(), w[0].unit) < (w[1].capability.as_bytes(), w[1].unit))
|
||||
&& t.job.iter().all(|j| job_capability(&j.capability) && (1..=JOB_UNITS.len() as u8).contains(&j.unit) && j.divisor >= 1);
|
||||
&& t.job.windows(2).all(|w| (w[0].capability.as_bytes(), w[0].operation.as_bytes(), w[0].unit) < (w[1].capability.as_bytes(), w[1].operation.as_bytes(), w[1].unit))
|
||||
&& t.job.iter().all(|j| {
|
||||
let operation = if t.version == VERSION_3 { job_operation(&j.operation) } else { j.operation.is_empty() };
|
||||
job_capability(&j.capability) && operation && (1..=JOB_UNITS.len() as u8).contains(&j.unit) && j.divisor >= 1
|
||||
});
|
||||
if !rules {
|
||||
return Err((Reason::TariffInvalid.code(), at()));
|
||||
}
|
||||
|
|
@ -554,7 +593,7 @@ pub struct Row {
|
|||
/// An invoice (§6.1).
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct Invoice {
|
||||
/// Its tariff's version, 1 or 2.
|
||||
/// Its tariff's version, 1, 2 or 3.
|
||||
pub version: u8,
|
||||
pub seller: [u8; 32],
|
||||
pub buyer: [u8; 32],
|
||||
|
|
@ -566,7 +605,7 @@ pub struct Invoice {
|
|||
pub records: Vec<(u8, [u8; 32])>,
|
||||
pub lines: Vec<Line>,
|
||||
pub energy: Vec<Row>,
|
||||
/// Scopes 1 to 4, and in version 2 scope 5; a version-1 invoice states no fifth.
|
||||
/// Scopes 1 to 4, and in versions 2 and 3 scope 5; a version-1 invoice states no fifth.
|
||||
pub unmetered: [u32; 5],
|
||||
pub amount: u64,
|
||||
pub sig: [u8; 64],
|
||||
|
|
@ -608,7 +647,7 @@ impl Invoice {
|
|||
o.extend_from_slice(&e.joules.to_le_bytes());
|
||||
o.extend_from_slice(&e.max_ppm.to_le_bytes());
|
||||
}
|
||||
let scopes = if self.version == VERSION_2 { 5 } else { 4 };
|
||||
let scopes = if prices_jobs(self.version) { 5 } else { 4 };
|
||||
for u in &self.unmetered[..scopes] {
|
||||
o.extend_from_slice(&u.to_le_bytes());
|
||||
}
|
||||
|
|
@ -636,7 +675,7 @@ impl Invoice {
|
|||
/// Read an invoice (§6.8): its rules, then its signature.
|
||||
pub fn parse(b: &[u8]) -> Result<Invoice, &'static str> {
|
||||
let version = read_head(b, &INVOICE_MAGIC)?;
|
||||
let v2 = version == VERSION_2;
|
||||
let v2 = prices_jobs(version);
|
||||
let mut r = Rd { b, p: 5 };
|
||||
let (seller, buyer, tariff, prev) = (r.arr()?, r.arr()?, r.arr()?, r.arr()?);
|
||||
let (start, end) = (r.u64()?, r.u64()?);
|
||||
|
|
@ -847,7 +886,7 @@ pub struct StitchedFacts {
|
|||
pub statement: Value,
|
||||
}
|
||||
|
||||
/// A verified job source (§5.1, version 2).
|
||||
/// A verified job source (§5.1, versions 2 and 3).
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct JobFacts {
|
||||
/// The route record's digest, and its signer: the router.
|
||||
|
|
@ -855,6 +894,8 @@ pub struct JobFacts {
|
|||
pub router: [u8; 32],
|
||||
/// The job's id, which the router names it by.
|
||||
pub job: String,
|
||||
/// The route record's operation, which a tariff of version 3 prices by (§4.6).
|
||||
pub operation: String,
|
||||
pub accepted: u64,
|
||||
/// Each attempt's OER/2 receipt, by the 32 bytes its id names, in attempt order.
|
||||
pub hops: Vec<[u8; 32]>,
|
||||
|
|
@ -1339,6 +1380,7 @@ fn job_source(o: &Map<String, Value>) -> Result<Facts, Refusal> {
|
|||
record: rr::digest(&record),
|
||||
router: unhex(record["signer"].as_str().expect("a key")).expect("a key").try_into().expect("32 bytes"),
|
||||
job: record["job"].as_str().expect("a token").to_string(),
|
||||
operation: record["operation"].as_str().expect("an operation").to_string(),
|
||||
accepted: record["accepted"].as_u64().expect("a time"),
|
||||
hops,
|
||||
billed,
|
||||
|
|
@ -1347,11 +1389,11 @@ fn job_source(o: &Map<String, Value>) -> Result<Facts, Refusal> {
|
|||
}))
|
||||
}
|
||||
|
||||
/// §5.1: verify one source by its extension, a stitched source with `signers` (§4.4), a job
|
||||
/// source only under a tariff of version 2. A refusal's place is the extension's own, relative
|
||||
/// to the source.
|
||||
/// §5.1: verify one source by its extension, a stitched source with `signers` (§4.4); a job
|
||||
/// source is read here whatever the tariff, and only a tariff of version 2 or 3 bills it. A
|
||||
/// refusal's place is the extension's own, relative to the source.
|
||||
pub fn verify_source(v: &Value, signers: &[[u8; 32]]) -> Result<Facts, Refusal> {
|
||||
verify_source_of(v, signers, VERSION_2)
|
||||
verify_source_of(v, signers, VERSION_3)
|
||||
}
|
||||
|
||||
fn verify_source_of(v: &Value, signers: &[[u8; 32]], version: u8) -> Result<Facts, Refusal> {
|
||||
|
|
@ -1362,7 +1404,7 @@ fn verify_source_of(v: &Value, signers: &[[u8; 32]], version: u8) -> Result<Fact
|
|||
Some("package") => package_source(o),
|
||||
Some("ads") => ads_source(o),
|
||||
Some("stitched") => stitched_source(o, signers),
|
||||
Some("job") if version == VERSION_2 => job_source(o),
|
||||
Some("job") if prices_jobs(version) => job_source(o),
|
||||
_ => Err(bad_source()),
|
||||
}
|
||||
}
|
||||
|
|
@ -1436,25 +1478,36 @@ fn overlaps(a: (u64, u64), b: (u64, u64)) -> bool {
|
|||
a.0 < b.1 && b.0 < a.1
|
||||
}
|
||||
|
||||
/// Whether source `f` bills again what `g`, an earlier source, billed (§5.3).
|
||||
fn overlap(f: &Facts, g: &Facts) -> bool {
|
||||
/// Whether source `f` bills again what `g`, an earlier source, billed, under a tariff of
|
||||
/// `version` (§5.3).
|
||||
fn overlap(version: u8, f: &Facts, g: &Facts) -> bool {
|
||||
match (f, g) {
|
||||
(Facts::Room(a), Facts::Room(b)) => a.room == b.room && overlaps((a.start, a.end), (b.start, b.end)),
|
||||
(Facts::Delivery(a), Facts::Delivery(b)) => {
|
||||
a.statement.party == b.statement.party && overlaps((a.statement.start, a.statement.end), (b.statement.start, b.statement.end))
|
||||
}
|
||||
(Facts::Ads(a), Facts::Ads(b)) => a.room == b.room && a.receipts.iter().any(|x| b.receipts.iter().any(|y| x.cue == y.cue && x.session == y.session)),
|
||||
// Two records of one router and one job route one job twice (§5.3).
|
||||
(Facts::Job(a), Facts::Job(b)) => a.router == b.router && a.job == b.job,
|
||||
// In version 2, two records of one router and one job route one job twice. In version 3
|
||||
// they are two runs of it, each billed from its own hop (§5.3).
|
||||
(Facts::Job(a), Facts::Job(b)) => version == VERSION_2 && a.router == b.router && a.job == b.job,
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// §7.2 step 6: the index of the source refused and the reason, or none.
|
||||
pub fn check_across(facts: &[Facts], previous: Option<&Invoice>) -> Result<(), (usize, Reason)> {
|
||||
/// Whether a chain under a tariff of `version` bills a record of `kind` once (§6.9): every
|
||||
/// record but, in version 3, a job's output receipt. A job's work, its route record and its
|
||||
/// hops' OER/2 receipts, is new on every run and billed once; its output receipt is the
|
||||
/// evidence of what it delivered, which a deterministic re-run delivers again.
|
||||
pub fn billed_once(version: u8, kind: u8) -> bool {
|
||||
!(version == VERSION_3 && kind == record_kind::VIDEO_RECEIPT)
|
||||
}
|
||||
|
||||
/// §7.2 step 6 under a tariff of `version`: the index of the source refused and the reason, or
|
||||
/// none.
|
||||
pub fn check_across(version: u8, facts: &[Facts], previous: Option<&Invoice>) -> Result<(), (usize, Reason)> {
|
||||
let billed: BTreeSet<(u8, [u8; 32])> = previous.map(|p| p.records.iter().copied().collect()).unwrap_or_default();
|
||||
let mut heads: BTreeSet<(u8, [u8; 32])> = BTreeSet::new();
|
||||
// One hop serves one job, and one output is one job's (§5.2).
|
||||
// One hop serves one job; in version 2, one output is one job's too (§5.2).
|
||||
let mut job_records: BTreeSet<(u8, [u8; 32])> = BTreeSet::new();
|
||||
for (k, f) in facts.iter().enumerate() {
|
||||
let hs = f.heads();
|
||||
|
|
@ -1462,16 +1515,16 @@ pub fn check_across(facts: &[Facts], previous: Option<&Invoice>) -> Result<(), (
|
|||
return Err((k, Reason::RecordDuplicate));
|
||||
}
|
||||
if let Facts::Job(_) = f {
|
||||
let own: BTreeSet<(u8, [u8; 32])> = f.records().into_iter().filter(|(kind, _)| *kind != record_kind::ROUTE_RECORD).collect();
|
||||
let own: BTreeSet<(u8, [u8; 32])> = f.records().into_iter().filter(|(kind, _)| *kind != record_kind::ROUTE_RECORD && billed_once(version, *kind)).collect();
|
||||
if own.iter().any(|r| job_records.contains(r)) {
|
||||
return Err((k, Reason::RecordDuplicate));
|
||||
}
|
||||
job_records.extend(own);
|
||||
}
|
||||
if facts[..k].iter().any(|g| overlap(f, g)) {
|
||||
if facts[..k].iter().any(|g| overlap(version, f, g)) {
|
||||
return Err((k, Reason::RecordOverlap));
|
||||
}
|
||||
if f.records().iter().any(|r| billed.contains(r)) {
|
||||
if f.records().iter().any(|r| billed_once(version, r.0) && billed.contains(r)) {
|
||||
return Err((k, Reason::RecordRebilled));
|
||||
}
|
||||
heads.extend(hs);
|
||||
|
|
@ -1566,10 +1619,13 @@ pub fn compute(t: &Tariff, facts: &[Facts]) -> Result<Computed, Overflow> {
|
|||
add(source_kind::ADS, r.decider, unit, counted.len() as u128, &cites);
|
||||
}
|
||||
}
|
||||
// A job is billed at the prices of its serving capability, from its billed hop (§6.3).
|
||||
// A job is billed at the prices of its serving capability, in version 3 of its
|
||||
// operation too, from its billed hop (§6.3). An output it shares with another job is
|
||||
// cited once and counted for each (§6.5).
|
||||
Facts::Job(j) => {
|
||||
let Some((cap, hop)) = &j.billed else { continue };
|
||||
for (k, p) in t.job.iter().enumerate().filter(|(_, p)| p.capability == *cap) {
|
||||
let priced = |p: &JobPrice| p.capability == *cap && (t.version != VERSION_3 || p.operation == j.operation);
|
||||
for (k, p) in t.job.iter().enumerate().filter(|(_, p)| priced(p)) {
|
||||
let unit = u8::try_from(k + 1).expect("at most 255 job prices");
|
||||
let mut cites = vec![(record_kind::ROUTE_RECORD, j.record), (record_kind::OER_RECEIPT, *hop)];
|
||||
let n = if p.unit == 1 {
|
||||
|
|
@ -1919,7 +1975,7 @@ pub fn verify_sources(t: &Tariff, start: u64, end: u64, sources: &[Value], previ
|
|||
check_source(t, start, end, &f).map_err(|r| (r.code(), json!(["source", k])))?;
|
||||
facts.push(f);
|
||||
}
|
||||
check_across(&facts, previous).map_err(|(k, r)| (r.code(), json!(["source", k])))?;
|
||||
check_across(t.version, &facts, previous).map_err(|(k, r)| (r.code(), json!(["source", k])))?;
|
||||
Ok(facts)
|
||||
}
|
||||
|
||||
|
|
@ -1988,18 +2044,22 @@ pub fn accepted_json(a: &Accepted) -> Value {
|
|||
"unit": unit_name(l.source_kind, l.unit).unwrap_or(""), "quantity": l.quantity, "price": l.price, "amount": l.amount,
|
||||
"cites": cited.iter().map(|(k, id)| json!([k, hex(id)])).collect::<Vec<_>>(),
|
||||
});
|
||||
// A job line names its unit and capability by its job price (§7.3).
|
||||
// A job line names its unit and capability by its job price, and in version 3 its
|
||||
// operation (§7.3).
|
||||
if l.source_kind == source_kind::JOB {
|
||||
let p = &a.tariff.job[l.unit as usize - 1];
|
||||
v["unit"] = json!(JOB_UNITS[p.unit as usize - 1]);
|
||||
v["capability"] = json!(p.capability);
|
||||
if a.tariff.version == VERSION_3 {
|
||||
v["operation"] = json!(p.operation);
|
||||
}
|
||||
}
|
||||
v
|
||||
})
|
||||
.collect();
|
||||
let energy: Vec<Value> = i.energy.iter().map(|r| json!({"scope": scope::name(r.scope), "class": class_label(r.tag), "figures": r.figures, "joules": r.joules, "max_ppm": r.max_ppm})).collect();
|
||||
let (mut unmetered, mut coverage) = (Map::new(), Map::new());
|
||||
for s in 1..=if i.version == VERSION_2 { 5u8 } else { 4 } {
|
||||
for s in 1..=if prices_jobs(i.version) { 5u8 } else { 4 } {
|
||||
let n = i.unmetered[s as usize - 1];
|
||||
let has_row = i.energy.iter().any(|r| r.scope == s);
|
||||
unmetered.insert(scope::name(s).into(), n.into());
|
||||
|
|
@ -2062,14 +2122,89 @@ mod tests {
|
|||
let mut t = tariff();
|
||||
t.version = VERSION_2;
|
||||
t.job = vec![
|
||||
JobPrice { capability: "wai.video.av1".into(), unit: 1, divisor: 1, price: 7 },
|
||||
JobPrice { capability: "wai.video.int_motion".into(), unit: 2, divisor: 1_000_000, price: 9 },
|
||||
JobPrice { capability: "wai.video.av1".into(), operation: String::new(), unit: 1, divisor: 1, price: 7 },
|
||||
JobPrice { capability: "wai.video.int_motion".into(), operation: String::new(), unit: 2, divisor: 1_000_000, price: 9 },
|
||||
];
|
||||
t.scope.push((source_kind::JOB, [7; 32]));
|
||||
t.scope.sort();
|
||||
t.sign(&key(1), &key(2))
|
||||
}
|
||||
|
||||
fn tariff_v3() -> Tariff {
|
||||
let mut t = tariff_v2();
|
||||
t.version = VERSION_3;
|
||||
let p = |op: &str, unit, divisor, price| JobPrice { capability: "wai.video.int_motion".into(), operation: op.into(), unit, divisor, price };
|
||||
t.job = vec![p("media.decode", 2, 1_000_000, 10_000), p("media.encode", 1, 1, 500_000), p("media.encode", 2, 1_000_000, 20_000)];
|
||||
t.sign(&key(1), &key(2))
|
||||
}
|
||||
|
||||
/// A tariff of version 3 reads back with each job price's operation, and every altered byte
|
||||
/// is refused; its rules order the prices by (capability, operation, unit) and hold each
|
||||
/// operation to route-record §2's form (§4.3, §4.6). A version-2 price states no operation,
|
||||
/// and its bytes are as they were.
|
||||
#[test]
|
||||
fn a_tariff_of_version_3_prices_each_operation() {
|
||||
let t = tariff_v3();
|
||||
let b = t.to_bytes();
|
||||
assert_eq!(b[4], 3);
|
||||
assert_eq!(Tariff::parse(&b).unwrap(), t);
|
||||
for at in 0..b.len() {
|
||||
let mut m = b.clone();
|
||||
m[at] ^= 0x01;
|
||||
assert!(Tariff::parse(&m).is_err(), "byte {at} altered and accepted");
|
||||
}
|
||||
let refused = |f: &dyn Fn(&mut Tariff)| {
|
||||
let mut x = tariff_v3();
|
||||
f(&mut x);
|
||||
Tariff::parse(&x.sign(&key(1), &key(2)).to_bytes()).map(|_| ()).map_err(|e| e.0)
|
||||
};
|
||||
assert_eq!(refused(&|x| x.job.swap(0, 1)), Err("tariff_invalid"), "an encode price before a decode price");
|
||||
assert_eq!(refused(&|x| x.job[2].operation = "media.decode".into()), Err("tariff_invalid"), "out of order, and a decode priced twice");
|
||||
assert_eq!(refused(&|x| x.job[0].operation = String::new()), Err("tariff_invalid"), "no operation");
|
||||
for bad in ["media", "media.encode.x", "Media.encode", "media.2x", "media.", ".encode", "media-x.encode"] {
|
||||
assert_eq!(refused(&|x| x.job[0].operation = bad.into()), Err("tariff_invalid"), "{bad}");
|
||||
}
|
||||
let long = format!("media.{}", "e".repeat(122));
|
||||
assert_eq!(long.len(), 128);
|
||||
assert_eq!(refused(&|x| x.job[0].operation = long.clone()), Ok(()), "128 bytes");
|
||||
assert_eq!(refused(&|x| x.job[0].operation = format!("{long}e")), Err("tariff_invalid"), "129 bytes");
|
||||
// One price per operation: the same capability and unit under two operations.
|
||||
assert_eq!(refused(&|x| x.job.retain(|p| p.unit == 2)), Ok(()));
|
||||
// A version-2 price is the four fields it always was: one capability, then the unit.
|
||||
let v2 = tariff_v2();
|
||||
let b2 = v2.to_bytes();
|
||||
let caps: usize = v2.job.iter().map(|p| 1 + p.capability.len() + 17).sum();
|
||||
assert_eq!(b2.len(), 4 + 1 + 64 + 16 + 3 + 52 + 32 + 24 + 16 + 2 + caps + 2 + 33 * v2.scope.len() + 128);
|
||||
assert!(Tariff::parse(&b2).unwrap().job.iter().all(|p| p.operation.is_empty()));
|
||||
// A version-2 tariff writes no operation, whatever its prices hold.
|
||||
let mut named = tariff_v2();
|
||||
named.job[0].operation = "media.encode".into();
|
||||
assert_eq!(named.body(), v2.body(), "no operation written in version 2");
|
||||
// A capability or an operation that is not text breaks its form: a rule, not the layout.
|
||||
let mut not_text = tariff_v3();
|
||||
not_text.job[0].operation = "media.\u{e9}ncode".into();
|
||||
let mut nb = not_text.sign(&key(1), &key(2)).to_bytes();
|
||||
let at = nb.windows(2).position(|w| w == [0xc3, 0xa9]).expect("the two bytes of é");
|
||||
nb[at] = 0xff;
|
||||
assert_eq!(Tariff::parse(&nb).unwrap_err().0, "tariff_invalid");
|
||||
// A tariff of version 4 is not defined.
|
||||
let mut v4 = b.clone();
|
||||
v4[4] = 4;
|
||||
assert_eq!(Tariff::parse(&v4).unwrap_err().0, "unknown_version");
|
||||
}
|
||||
|
||||
/// What a chain bills once (§6.9): in version 3, every record but an output receipt; in
|
||||
/// versions 1 and 2, every record.
|
||||
#[test]
|
||||
fn a_chain_bills_a_jobs_work_once_and_cites_its_output_again() {
|
||||
for k in [record_kind::SESSION, record_kind::PACKAGE, record_kind::OER_RECEIPT, record_kind::ROUTE_RECORD] {
|
||||
assert!(billed_once(VERSION, k) && billed_once(VERSION_2, k) && billed_once(VERSION_3, k), "kind {k}");
|
||||
}
|
||||
assert!(billed_once(VERSION_2, record_kind::VIDEO_RECEIPT), "version 2 bills an output once");
|
||||
assert!(!billed_once(VERSION_3, record_kind::VIDEO_RECEIPT), "version 3 bills the work, and cites the output");
|
||||
assert!(prices_jobs(VERSION_2) && prices_jobs(VERSION_3) && !prices_jobs(VERSION) && !prices_jobs(4));
|
||||
}
|
||||
|
||||
/// A tariff of version 2 reads back with its job prices, and every altered byte is refused;
|
||||
/// its rules refuse prices out of order, a unit outside the two, a divisor of 0, a capability
|
||||
/// outside WAI's form, and a router in a tariff of version 1 (§4.3, §4.5).
|
||||
|
|
@ -2134,6 +2269,15 @@ mod tests {
|
|||
assert_eq!(with(k, &i), Err("private_kind"), "kind {k} in version 2");
|
||||
assert_eq!(with(k, &v1), Err("private_kind"), "kind {k} in version 1");
|
||||
}
|
||||
// Version 3 is laid out as version 2, and lists the same kinds.
|
||||
let t3 = tariff_v3();
|
||||
let i3 = invoice_of(&t3, [0; 32], 10, 20, &c).sign(&key(1));
|
||||
assert_eq!(i3.to_bytes().len(), i.to_bytes().len());
|
||||
assert_eq!(Invoice::parse(&i3.to_bytes()).unwrap(), i3);
|
||||
for k in [record_kind::OER_RECEIPT, record_kind::VIDEO_RECEIPT, record_kind::ROUTE_RECORD] {
|
||||
assert_eq!(with(k, &i3), Ok(()), "kind {k} in version 3");
|
||||
}
|
||||
assert_eq!(with(record_kind::RUN, &i3), Err("invoice_invalid"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -2194,8 +2338,8 @@ mod tests {
|
|||
assert_eq!(Invoice::parse(&unit.sign(&key(1)).to_bytes()).unwrap_err(), "invoice_invalid");
|
||||
assert_eq!(Invoice::parse(&[b.clone(), vec![0]].concat()).unwrap_err(), "malformed");
|
||||
let mut v = b.clone();
|
||||
v[4] = 3;
|
||||
assert_eq!(Invoice::parse(&v).unwrap_err(), "unknown_version", "version 3 is not defined (2 is, §6.1)");
|
||||
v[4] = 4;
|
||||
assert_eq!(Invoice::parse(&v).unwrap_err(), "unknown_version", "version 4 is not defined (2 and 3 are, §6.1)");
|
||||
let mut s = b;
|
||||
let n = s.len();
|
||||
s[n - 1] ^= 1;
|
||||
|
|
|
|||
|
|
@ -6,8 +6,11 @@
|
|||
//! `live-conformance/ads/` and `live-conformance/presentation/`, and packages from
|
||||
//! `pack-conformance/packs/`. The statements of two delivering parties, with the
|
||||
//! acknowledgements their viewers signed and their per-hop records, are written here from
|
||||
//! fixed seeds, as are the tariffs and the invoices. [`generate`] reads those files from a
|
||||
//! `wai/` directory; [`verify`] checks every vector against the outcome it states.
|
||||
//! fixed seeds, as are the tariffs and the invoices. Media jobs come from
|
||||
//! `live-conformance/routes/`, and those the version-3 vectors add (a decode, a re-run, a re-run
|
||||
//! replaying a hop receipt, a job claiming another's) are routed here under that corpus's own
|
||||
//! seeds. [`generate`] reads those files from a `wai/` directory; [`verify`] checks every vector
|
||||
//! against the outcome it states.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::Path;
|
||||
|
|
@ -17,6 +20,7 @@ use serde_json::{json, Map, Value};
|
|||
|
||||
use super::*;
|
||||
use crate::energy_class::{CounterFiltering, Uncertainty};
|
||||
use crate::route_record::corpus as rr;
|
||||
use crate::jwp_hash::content_hash;
|
||||
use crate::live_metering::HopRecord;
|
||||
use crate::settlement::{class, item_list, Entry, Statement};
|
||||
|
|
@ -402,6 +406,47 @@ pub fn generate(wai: &Path) -> Result<String, String> {
|
|||
// Another record of job-0001, served by another endpoint: its own hop receipt, one job.
|
||||
pool.insert("job_routed_again", job_of(&routes, "an_unknown_energy_ranks_last", vec![])?);
|
||||
pool.insert("job_unmetered_claims_another_output", job_of(&routes, "served_by_an_unmetered_hop", out("job-0001"))?);
|
||||
// Version 3: jobs routed here by the route corpus's router, its hops' receipts sealed under
|
||||
// its hop key (route-record §6). A decode of int_motion; job-0001 run again two days later,
|
||||
// a new hop, new energy and a new receipt, its output the same receipt as before; that re-run
|
||||
// replaying job-0001's own hop receipt instead; and another job whose fallback's hop claims
|
||||
// that receipt too.
|
||||
let accepted = rr::ACCEPTED_AT;
|
||||
let first_hop: Value = {
|
||||
let d = &named(&routes["vectors"], "served_first_try")?["document"];
|
||||
serde_json::from_str(d["receipts"][0].as_str().ok_or("served_first_try's hop receipt")?).map_err(|e| e.to_string())?
|
||||
};
|
||||
let served = |endpoint: &'static str, receipt: Value| rr::Hop { endpoint, error: None, start: 1_000, end: 4_000_000, receipt };
|
||||
let routed = |d: Value, outputs: Vec<Value>| -> Value {
|
||||
json!({
|
||||
"kind": "job", "request": d["request"], "pricebook": d["pricebook"], "policy": d["policy"], "record": d["record"],
|
||||
"receipts": d["receipts"], "keys": d["keys"], "outputs": outputs,
|
||||
})
|
||||
};
|
||||
pool.insert(
|
||||
"job_decode",
|
||||
routed(rr::job_document("job-0007", "media.decode", accepted, &[served("im.alpha", rr::hop_receipt("media.decode", "job-0007", "im.alpha", Some("12.5")))]), out("job-0007")),
|
||||
);
|
||||
pool.insert(
|
||||
"job_rerun",
|
||||
routed(rr::job_document("job-0001", "media.encode", accepted + 2 * DAY_US, &[served("im.alpha", rr::hop_receipt("media.encode", "job-0001", "im.alpha", Some("30.875")))]), out("job-0001")),
|
||||
);
|
||||
pool.insert("job_rerun_replays_its_hop", routed(rr::job_document("job-0001", "media.encode", accepted + 2 * DAY_US, &[served("im.alpha", first_hop.clone())]), out("job-0001")));
|
||||
pool.insert(
|
||||
"job_claims_another_hop",
|
||||
routed(
|
||||
rr::job_document(
|
||||
"job-0008",
|
||||
"media.encode",
|
||||
accepted,
|
||||
&[
|
||||
rr::Hop { endpoint: "im.alpha", error: Some("server_error"), start: 1_000, end: 2_000_000, receipt: rr::hop_receipt("media.encode", "job-0008", "im.alpha", None) },
|
||||
rr::Hop { endpoint: "av1.alpha", error: None, start: 2_000_100, end: 9_000_000, receipt: first_hop },
|
||||
],
|
||||
),
|
||||
vec![],
|
||||
),
|
||||
);
|
||||
|
||||
// The rooms' prices are the lesson receipts'; the tariff's scope is every party of the
|
||||
// accepted invoices.
|
||||
|
|
@ -494,7 +539,7 @@ pub fn generate(wai: &Path) -> Result<String, String> {
|
|||
Vector { name: "tariff_period_empty", about: "a tariff whose period is empty, signed by both", tariff: tariff_period.to_bytes(), invoice: ab.clone(), previous: None, sources: day_a.to_vec() },
|
||||
// The invoice.
|
||||
Vector { name: "invoice_trailing_byte", about: "an invoice with a byte after its signature", tariff: tb.clone(), invoice: [ab.clone(), vec![0]].concat(), previous: None, sources: day_a.to_vec() },
|
||||
Vector { name: "invoice_version", about: "an invoice of version 3, which this revision does not define", tariff: tb.clone(), invoice: { let mut b = ab.clone(); b[4] = 3; b }, previous: None, sources: day_a.to_vec() },
|
||||
Vector { name: "invoice_version", about: "an invoice of version 4, which this revision does not define", tariff: tb.clone(), invoice: { let mut b = ab.clone(); b[4] = 4; b }, previous: None, sources: day_a.to_vec() },
|
||||
Vector { name: "invoice_signature", about: "an invoice whose signature does not verify", tariff: tb.clone(), invoice: flip(ab.clone(), ab.len() - 1), previous: None, sources: day_a.to_vec() },
|
||||
Vector { name: "invoice_lines_unordered", about: "an invoice whose first two lines are swapped", tariff: tb.clone(), invoice: sealed(&t, [0; 32], a0, a1, &refs(&day_a), |i| i.lines.swap(0, 1)).to_bytes(), previous: None, sources: day_a.to_vec() },
|
||||
Vector { name: "invoice_row_of_no_class", about: "an energy row of tag 0: joules of no one class", tariff: tb.clone(), invoice: sealed(&t, [0; 32], a0, a1, &refs(&day_a), |i| i.energy.insert(0, Row { scope: 1, tag: 0, figures: 1, joules: 1, max_ppm: 0 })).to_bytes(), previous: None, sources: day_a.to_vec() },
|
||||
|
|
@ -541,9 +586,9 @@ pub fn generate(wai: &Path) -> Result<String, String> {
|
|||
// Version 2: a tariff that prices jobs (§4.5), and its invoices.
|
||||
let router: [u8; 32] = routes["keys"]["router"].as_str().and_then(unhex).and_then(|b| b.try_into().ok()).ok_or("the route corpus's router")?;
|
||||
let job_prices = vec![
|
||||
JobPrice { capability: "wai.video.av1".into(), unit: 1, divisor: 1, price: 2_000_000 },
|
||||
JobPrice { capability: "wai.video.int_motion".into(), unit: 1, divisor: 1, price: 500_000 },
|
||||
JobPrice { capability: "wai.video.int_motion".into(), unit: 2, divisor: 1_000_000, price: 30_000 },
|
||||
JobPrice { capability: "wai.video.av1".into(), operation: String::new(), unit: 1, divisor: 1, price: 2_000_000 },
|
||||
JobPrice { capability: "wai.video.int_motion".into(), operation: String::new(), unit: 1, divisor: 1, price: 500_000 },
|
||||
JobPrice { capability: "wai.video.int_motion".into(), operation: String::new(), unit: 2, divisor: 1_000_000, price: 30_000 },
|
||||
];
|
||||
let v2 = |scope: Vec<(u8, [u8; 32])>, job: Vec<JobPrice>| -> Tariff {
|
||||
let mut t2 = tariff_of("buyer-j", room_prices, scope);
|
||||
|
|
@ -588,6 +633,56 @@ pub fn generate(wai: &Path) -> Result<String, String> {
|
|||
Vector { name: "job_energy_row_altered", about: "the jobs' counter row one microjoule above its hop's figure", tariff: tjb.clone(), invoice: sealed(&t_j, [0; 32], MONTH.0, MONTH.1, &refs(&day_j), |i| { if let Some(r) = i.energy.iter_mut().find(|r| r.scope == scope::JOBS && r.tag == 4) { r.joules += 1; } }).to_bytes(), previous: None, sources: day_j.to_vec() },
|
||||
]);
|
||||
|
||||
// Version 3: job prices per operation (§4.6), and a chain that bills a job's work once and
|
||||
// cites its output's receipt again (§6.9). Encode and decode of int_motion are priced apart:
|
||||
// a decode reconstructs, an encode searches for motion and then writes the codestream.
|
||||
if routed_at != accepted {
|
||||
return Err(format!("the route corpus accepts its jobs at {routed_at}, its seeds at {accepted}"));
|
||||
}
|
||||
const IM: &str = "wai.video.int_motion";
|
||||
let jp = |capability: &str, operation: &str, unit: u8, divisor: u64, price: u64| JobPrice { capability: capability.into(), operation: operation.into(), unit, divisor, price };
|
||||
let job_prices_3 = vec![
|
||||
jp("wai.video.av1", "media.encode", 1, 1, 2_000_000),
|
||||
jp(IM, "media.decode", 1, 1, 250_000),
|
||||
jp(IM, "media.decode", 2, 1_000_000, 10_000),
|
||||
jp(IM, "media.encode", 1, 1, 500_000),
|
||||
jp(IM, "media.encode", 2, 1_000_000, 20_000),
|
||||
];
|
||||
let v3 = |job: Vec<JobPrice>| -> Tariff {
|
||||
let mut t3 = tariff_of("buyer-j", room_prices, vec![package_scope, (source_kind::JOB, router)]);
|
||||
t3.version = VERSION_3;
|
||||
t3.job = job;
|
||||
t3.sign(&seller, &key("buyer-j"))
|
||||
};
|
||||
let t_k = v3(job_prices_3.clone());
|
||||
let tkb = t_k.to_bytes();
|
||||
// The first invoice of the chain ends a day after the jobs were accepted; the re-run is
|
||||
// accepted a day later, in the next.
|
||||
let mid = accepted + DAY_US;
|
||||
let day_k = ["package_base", "job_int_motion", "job_failover", "job_late", "job_unmetered", "job_failed", "job_decode"];
|
||||
let inv_k = make_invoice(&seller, &t_k, None, MONTH.0, mid, &src(&day_k)).map_err(|e| format!("invoice k: {e:?}"))?;
|
||||
let kb = inv_k.to_bytes();
|
||||
let inv_k_rerun = make_invoice(&seller, &t_k, Some(&inv_k), mid, MONTH.1, &src(&["job_rerun"])).map_err(|e| format!("invoice k, the re-run: {e:?}"))?;
|
||||
let inv_k_both = make_invoice(&seller, &t_k, None, MONTH.0, MONTH.1, &src(&["job_int_motion", "job_rerun"])).map_err(|e| format!("invoice k, both runs: {e:?}"))?;
|
||||
let t_k_encode = v3(job_prices_3.iter().filter(|p| p.operation == "media.encode").cloned().collect());
|
||||
let inv_k_encode = make_invoice(&seller, &t_k_encode, None, MONTH.0, MONTH.1, &src(&["job_int_motion", "job_decode"])).map_err(|e| format!("invoice k, encode priced: {e:?}"))?;
|
||||
let mut decode_after_encode = job_prices_3.clone();
|
||||
decode_after_encode[1..].rotate_left(2);
|
||||
let t_k_unordered = v3(decode_after_encode);
|
||||
// Version 2 bills an output once: its chain's first half-month, then the re-run.
|
||||
let inv_j_first = make_invoice(&seller, &t_j, None, MONTH.0, mid, &src(&day_j)).map_err(|e| format!("invoice j, first: {e:?}"))?;
|
||||
vectors.extend([
|
||||
Vector { name: "jobs_priced_per_operation", about: "version 3: media jobs priced by capability and operation, beside a package: an int_motion encode and an int_motion decode at their own prices per job and per subpixel, a failover billed at the fallback's encode price, a job served past its deadline and one every allowed hop failed, billed nothing, and a job whose serving hop was unmetered", tariff: tkb.clone(), invoice: kb.clone(), previous: None, sources: day_k.to_vec() },
|
||||
Vector { name: "a_rerun_billed", about: "version 3: the next invoice of the chain bills job-0001 run again, a new route record and a new hop receipt, its output the receipt the first invoice holds", tariff: tkb.clone(), invoice: inv_k_rerun.to_bytes(), previous: Some(kb.clone()), sources: vec!["job_rerun"] },
|
||||
Vector { name: "one_output_under_two_hops", about: "version 3: job-0001 and its re-run in one invoice, two hops of one request: both billed, the output receipt they share cited once and its subpixels billed for each", tariff: tkb.clone(), invoice: inv_k_both.to_bytes(), previous: None, sources: vec!["job_int_motion", "job_rerun"] },
|
||||
Vector { name: "an_operation_the_tariff_does_not_price", about: "version 3: a tariff that prices int_motion's encode and not its decode: the decode bills nothing, its hop's energy stated", tariff: t_k_encode.to_bytes(), invoice: inv_k_encode.to_bytes(), previous: None, sources: vec!["job_int_motion", "job_decode"] },
|
||||
Vector { name: "a_hop_receipt_replayed", about: "version 3: the next invoice bills a new route record of job-0001 whose hop receipt the first invoice billed", tariff: tkb.clone(), invoice: sealed(&t_k, inv_k.digest(), mid, MONTH.1, &refs(&["job_rerun_replays_its_hop"]), |_| {}).to_bytes(), previous: Some(kb.clone()), sources: vec!["job_rerun_replays_its_hop"] },
|
||||
Vector { name: "one_hop_under_two_lines", about: "version 3: a job whose fallback's hop receipt is another job's, billed on the int_motion encode lines and the av1 line", tariff: tkb.clone(), invoice: sealed(&t_k, [0; 32], MONTH.0, MONTH.1, &refs(&["job_int_motion", "job_claims_another_hop"]), |_| {}).to_bytes(), previous: None, sources: vec!["job_int_motion", "job_claims_another_hop"] },
|
||||
Vector { name: "job_prices_unordered_by_operation", about: "a tariff of version 3 pricing int_motion's encode before its decode", tariff: t_k_unordered.to_bytes(), invoice: kb.clone(), previous: None, sources: day_k.to_vec() },
|
||||
Vector { name: "version_2_invoice_under_a_version_3_tariff", about: "an invoice of version 2 naming a tariff of version 3", tariff: tkb.clone(), invoice: sealed(&t_k, [0; 32], MONTH.0, mid, &refs(&day_k), |i| i.version = VERSION_2).to_bytes(), previous: None, sources: day_k.to_vec() },
|
||||
Vector { name: "version_2_bills_an_output_once", about: "version 2 as before: the re-run of job-0001 in the next invoice of a version-2 chain, refused because its output receipt was billed", tariff: tjb.clone(), invoice: sealed(&t_j, inv_j_first.digest(), mid, MONTH.1, &refs(&["job_rerun"]), |_| {}).to_bytes(), previous: Some(inv_j_first.to_bytes()), sources: vec!["job_rerun"] },
|
||||
]);
|
||||
|
||||
let mut sources = Map::new();
|
||||
for (n, v) in &pool {
|
||||
sources.insert((*n).into(), v.clone());
|
||||
|
|
|
|||
|
|
@ -557,6 +557,61 @@ fn vectors() -> Vec<Vector> {
|
|||
v
|
||||
}
|
||||
|
||||
// ───────────────────────────── jobs routed for the receipt ledger ─────────────────────────────
|
||||
|
||||
/// When this corpus's router accepts its jobs.
|
||||
#[cfg(feature = "receipt_ledger")]
|
||||
pub(crate) const ACCEPTED_AT: u64 = ACCEPTED;
|
||||
|
||||
/// One hop of a job routed for another corpus (receipt-ledger §11): its endpoint, its error
|
||||
/// (`None` when it served), its start and end after acceptance, and its OER/2 receipt.
|
||||
#[cfg(feature = "receipt_ledger")]
|
||||
pub(crate) struct Hop {
|
||||
pub endpoint: &'static str,
|
||||
pub error: Option<&'static str>,
|
||||
pub start: u64,
|
||||
pub end: u64,
|
||||
pub receipt: Value,
|
||||
}
|
||||
|
||||
/// The OER/2 receipt of a hop of `job` at `endpoint` for `operation`, sealed under this corpus's
|
||||
/// hop key and bound to its job as this corpus binds its own: served, its figure a counter
|
||||
/// reading of `joules`, or failed (`None`), unmetered.
|
||||
#[cfg(feature = "receipt_ledger")]
|
||||
pub(crate) fn hop_receipt(operation: &str, job: &str, endpoint: &str, joules: Option<&str>) -> Value {
|
||||
let mut r = match joules {
|
||||
Some(j) => receipt(operation, endpoint, 1, Some((j, counter(), "measured"))),
|
||||
None => receipt(operation, endpoint, -1, None),
|
||||
};
|
||||
r["subject"]["inputs"] = json!([sha(&format!("{job}/{endpoint}"))]);
|
||||
reseal_receipt(&mut r);
|
||||
r
|
||||
}
|
||||
|
||||
/// The verification document (§7.1) of this corpus's request under `job` and `operation`,
|
||||
/// routed by its router under its price book and policy, accepted at `accepted`, with
|
||||
/// `attempts` in their order and their receipts given as they are.
|
||||
#[cfg(feature = "receipt_ledger")]
|
||||
pub(crate) fn job_document(job: &str, operation: &str, accepted: u64, attempts: &[Hop]) -> Value {
|
||||
let mut request = request();
|
||||
request["job"] = json!(job);
|
||||
request["operation"] = json!(operation);
|
||||
let r = Request::read(&canonical(&request)).expect("the request reads");
|
||||
let b = PriceBook::read(&canonical(&pricebook())).expect("the price book reads");
|
||||
let p = Policy::read(&canonical(&policy(3))).expect("the policy reads");
|
||||
let atts: Vec<Attempt> =
|
||||
attempts.iter().map(|h| Attempt { endpoint: h.endpoint.to_string(), error: h.error, start: accepted + h.start, end: accepted + h.end, receipt: id(&h.receipt) }).collect();
|
||||
let record = seal(&router(), &r, &b, &p, accepted, &atts);
|
||||
json!({
|
||||
"request": canonical(&request),
|
||||
"pricebook": canonical(&pricebook()),
|
||||
"policy": canonical(&policy(3)),
|
||||
"record": canonical(&record),
|
||||
"receipts": attempts.iter().map(|h| text(&h.receipt)).collect::<Vec<_>>(),
|
||||
"keys": {HOP_SIGNER: hex(&hops().verifying_key().to_bytes())},
|
||||
})
|
||||
}
|
||||
|
||||
/// The corpus, as `vectors.json` holds it.
|
||||
pub fn generate() -> Result<String, String> {
|
||||
let mut out = Vec::new();
|
||||
|
|
|
|||
|
|
@ -115,6 +115,71 @@ fn the_corpus_covers_every_reason_and_case() {
|
|||
assert_eq!(rows, ["ModelBased", "Estimator", "OnChipCounter"], "one row per class, never one across them");
|
||||
}
|
||||
|
||||
/// Version 3 (§4.6, §6.9): one capability's encode and decode at their own prices; a re-run billed
|
||||
/// in the next invoice of its chain, the output receipt the first invoice holds cited again; one
|
||||
/// output under two hops of one invoice, cited once and billed for each; an operation the tariff
|
||||
/// does not price billing nothing; a replayed hop receipt and a hop two jobs claim refused; and
|
||||
/// version 2 billing an output once, as it did.
|
||||
#[test]
|
||||
fn version_3_prices_each_operation_and_bills_the_work() {
|
||||
use serde_json::{json, Value};
|
||||
let doc: Value = serde_json::from_str(&committed()).unwrap();
|
||||
let vectors = doc["vectors"].as_array().unwrap();
|
||||
let get = |n: &str| vectors.iter().find(|v| v["name"] == n).unwrap_or_else(|| panic!("no vector {n}"))["outcome"].clone();
|
||||
let line = |o: &Value, cap: &str, op: &str, unit: &str| -> Value {
|
||||
o["lines"]
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.find(|l| l["source_kind"] == "job" && l["capability"] == cap && l["operation"] == op && l["unit"] == unit)
|
||||
.cloned()
|
||||
.unwrap_or_else(|| panic!("no {cap} {op} {unit} line"))
|
||||
};
|
||||
let cites = |l: &Value, kind: u64| -> Vec<String> { l["cites"].as_array().unwrap().iter().filter(|c| c[0] == kind).map(|c| c[1].as_str().unwrap().to_string()).collect() };
|
||||
const IM: &str = "wai.video.int_motion";
|
||||
|
||||
let k = get("jobs_priced_per_operation");
|
||||
assert_eq!(k["outcome"], "accept");
|
||||
let (enc, dec) = (line(&k, IM, "media.encode", "subpixels"), line(&k, IM, "media.decode", "subpixels"));
|
||||
assert_eq!((enc["price"].as_u64(), dec["price"].as_u64()), (Some(20_000), Some(10_000)), "an encode and a decode at their own prices");
|
||||
assert_eq!(enc["quantity"].as_u64().unwrap(), 2 * dec["quantity"].as_u64().unwrap(), "two encodes and one decode of one clip");
|
||||
assert_eq!((line(&k, IM, "media.encode", "jobs")["quantity"].as_u64(), line(&k, IM, "media.decode", "jobs")["quantity"].as_u64()), (Some(2), Some(1)));
|
||||
assert_eq!(line(&k, "wai.video.av1", "media.encode", "jobs")["quantity"], 1, "the failover, at the fallback's encode price");
|
||||
|
||||
let r = get("a_rerun_billed");
|
||||
assert_eq!(r["outcome"], "accept");
|
||||
let rerun = line(&r, IM, "media.encode", "subpixels");
|
||||
let output = cites(&rerun, 15);
|
||||
assert_eq!(output.len(), 1);
|
||||
assert!(cites(&enc, 15).contains(&output[0]), "the re-run's output receipt is one the first invoice billed with job-0001");
|
||||
assert!(cites(&rerun, 14).iter().all(|h| !cites(&enc, 14).contains(h)), "the re-run's hop receipt is new");
|
||||
|
||||
let both = get("one_output_under_two_hops");
|
||||
assert_eq!(both["outcome"], "accept");
|
||||
let b = line(&both, IM, "media.encode", "subpixels");
|
||||
assert_eq!((cites(&b, 14).len(), cites(&b, 15), cites(&b, 16).len()), (2, output.clone(), 2), "two hops and two records, one output cited once");
|
||||
assert_eq!(b["quantity"].as_u64().unwrap(), 2 * rerun["quantity"].as_u64().unwrap(), "its subpixels billed for each hop");
|
||||
assert_eq!(line(&both, IM, "media.encode", "jobs")["quantity"], 2);
|
||||
|
||||
let n = get("an_operation_the_tariff_does_not_price");
|
||||
assert_eq!(n["outcome"], "accept");
|
||||
assert!(n["lines"].as_array().unwrap().iter().all(|l| l["operation"] == "media.encode"), "the decode bills nothing");
|
||||
assert_eq!(n["energy"].as_array().unwrap().iter().filter(|e| e["scope"] == "jobs").map(|e| e["figures"].as_u64().unwrap()).sum::<u64>(), 2, "both hops' energy stated");
|
||||
|
||||
for (name, reason, at) in [
|
||||
("a_hop_receipt_replayed", "record_rebilled", json!(["source", 0])),
|
||||
("one_hop_under_two_lines", "record_duplicate", json!(["source", 1])),
|
||||
("job_prices_unordered_by_operation", "tariff_invalid", json!(["tariff"])),
|
||||
("version_2_invoice_under_a_version_3_tariff", "tariff_mismatch", json!(["invoice"])),
|
||||
("version_2_bills_an_output_once", "record_rebilled", json!(["source", 0])),
|
||||
("output_claimed_by_two_jobs", "record_duplicate", json!(["source", 1])),
|
||||
("job_routed_twice", "record_overlap", json!(["source", 6])),
|
||||
] {
|
||||
let o = get(name);
|
||||
assert_eq!((o["reason"].as_str(), &o["at"]), (Some(reason), &at), "{name}");
|
||||
}
|
||||
}
|
||||
|
||||
/// §10 of the extension lists exactly the codes `Reason::ALL` has, each once, in order.
|
||||
#[test]
|
||||
fn the_extension_lists_the_reasons() {
|
||||
|
|
|
|||
Loading…
Reference in a new issue