open-standards/wai/wai-web/reach/README.md

# wai-web reach builds

The shipped wai-web package (`wai_web.js` and `wai_web_bg.wasm`, the wasm-bindgen web
target) runs where an engine has WebAssembly with sign-extension, non-trapping
float-to-int conversion and bulk memory, where a 64-bit integer crosses the JS boundary as
a `BigInt`, and where ES modules load. The reach builds run the same code on engines
without those features, and give the same outputs byte for byte. Nothing in the Rust
changes: every reach build is derived from the shipped package itself.

| rung | files | the engine needs | the CI proxies it runs on |
|---|---|---|---|
| shipped | `../wai_web.js`, `../wai_web_bg.wasm` (the package) | WebAssembly with sign-extension, non-trapping float-to-int and bulk memory; i64 as `BigInt`; ES modules | a current Node |
| **L1** | `wai_web_bg.l1.wasm` + `wai_web.reach.js` | the same WebAssembly features; no `BigInt` at the boundary | `node:12`, `node:14` |
| **L2** | `wai_web_bg.l2.<profile>.wasm` + `wai_web.reach.js` | MVP WebAssembly | `node:8.0.0` (receipt), `node:8`, `node:10` |
| **J** | `wai_web.j.<profile>.js` | ES5 and typed arrays; no WebAssembly | `node:0.12`, `node:8.0.0` (full), and a current Node stripped to ES5 |

- **L1** is the shipped binary with its JS interface legalised (`wasm-opt
  --legalize-js-interface`): an i64 parameter becomes a pair of i32 halves, and an i64
  result returns its low half and passes the high half through `env.setTempRet0`. Nothing
  else changes, so the code runs at the shipped build's speed.
- **L2** is the shipped binary lowered to the MVP (`--signext-lowering`,
  `--llvm-nontrapping-fptoint-lowering`, `--llvm-memory-copy-fill-lowering`) and then
  legalised. Lowered without legalisation it would still fail every export with an i64 in
  its signature, so it is never shipped alone.
- **J** is the MVP-lowered module translated to JavaScript by `wasm2js`, in one ES5
  script per profile: the runtime, the i64 table, the translation and the glue.

`wai_web.reach.js` and every J script carry the same ES5 parts:

- **The runtime** (`src/wai-reach-runtime.js`) installs what an old engine lacks and the
  glue or the translation uses (`Math.imul` / `fround` / `clz32` / `trunc`,
  `TypedArray#slice`, `TextEncoder` / `TextDecoder`, `Array#fill`, a small `Map`, a
  `Symbol.iterator` stand-in, and a few more), each only where it is missing. It also joins
  and splits i64 values at the boundary, from the table the build extracts from the binary.
  With `BigInt` a 64-bit value is a `BigInt`, as the shipped build gives it. Without
  `BigInt` it is a `Number`, exact below 2^53.
- **The glue** is the shipped `wai_web.js`, changed in a fixed, counted set of places
  (`tools/cli.mjs`, `glue`; `manifest.json` records the counts) and compiled to ES5:
  - its WebAssembly loading is replaced by two hooks the loader calls;
  - a cached memory view is rebuilt whenever `wasm.memory.buffer` is a different buffer.
    The shipped glue rebuilt one only when its buffer was detached, which a WebAssembly
    memory does when it grows and the translation's memory does not;
  - `Symbol.dispose` is read only where `Symbol` exists;
  - the 64-bit helpers take the `Number` path without `BigInt`;
  - copies out of wasm memory no longer use `TypedArray#slice`, which early engines
    implement as a view.

## Profiles

L2 and J come in profiles. `profiles.json` lists the exports each one keeps, and
`wasm-metadce` drops what they cannot reach:

| profile | what it keeps |
|---|---|
| `full` | everything |
| `verify_decode` | `verify`, plus the integer decoders (WIV1, WIH1, WIA1, WIS1, int_synth, the JSON integer forms) and the staged-delivery sink |
| `verify` | `receipt`, plus multi-rendition envelopes, pins, the capability registry, stage identities, session claims and the output tree |
| `receipt` | envelope parsing, asset provenance (content hash, Ed25519 signature, Merkle chain) and the hashes |

A page that calls a function outside its profile gets a `TypeError`. The shipped build and
L1 are always the full module.

## The loader

`wai-load.js` is ES5. It chooses a rung from feature probes alone, never from the user
agent, and its detection never throws:

1. no typed arrays: none, and the reason says to serve the classical floor renditions;
2. no WebAssembly, or a WebAssembly that refuses a minimal MVP module: **J**;
3. an early WebAssembly engine line (WebAssembly and `Error.captureStackTrace` present, no
   RegExp `dotAll` flag; the feature `noDotAll`): **J**, or the L2 receipt build when the
   page asks for the receipt profile. The full MVP module has crashed an engine of this
   line on arm64;
4. a probe module for sign-extension, non-trapping float-to-int or bulk memory (with a
   data-count section) fails `WebAssembly.validate`: **L2**;
5. a one-function module returning its i64 argument does not take and give a `BigInt`:
   **L1**;
6. otherwise: **shipped**.

```html
<script src="/wai/pkg/reach/wai-load.js"></script>
<script>
  WaiLoad.load({ profile: 'verify' }, function (err, r) {
    if (err) { /* r.reason says why; serve the classical floor */ return; }
    // r.api is wai-web's API; r.rung is 'shipped' | 'L1' | 'L2' | 'J'; r.reason says why.
    var p = new r.api.AssetProvenance(envelopeBytes, receiptJson);
  });
</script>
```

| option | meaning |
|---|---|
| `base` | the folder of the reach files, ending in `/`; by default the folder `wai-load.js` came from |
| `profile` | `full` (default), `verify_decode`, `verify` or `receipt`; used by L2 and J |
| `force` | `shipped`, `L1`, `L2` or `J`: load that rung whatever the probes say (testing). The global `WAI_REACH_FORCE` does the same |
| `shipped` | `{ glue, wasm }` URLs of the shipped package; by default `base + '../wai_web.js'` and `base + '../wai_web_bg.wasm'` |
| `host` | `{ loadScript, fetchBytes, importModule, streams }`, to load from somewhere other than the page. With `streams` (a browser's default), the shipped glue gets the wasm URL and compiles it while it downloads |

`WaiLoad.detect(opts)` returns `{ rung, profile, reason, features }` without loading
anything. After a load, `WaiLoad.state` holds what the callback received.

## Building

```
bash wai/wai-web/reach/build.sh <package-dir> [<dist-dir>]
```

`<package-dir>` holds `wai_web.js` and `wai_web_bg.wasm`. The output goes to
`<dist-dir>` (default `reach/dist/`, which git ignores) with a `manifest.json`: every
file's SHA-256 and its raw, gzip and brotli sizes, the input's SHA-256, the tool pins, and
the counts of the glue changes.

The tools are pinned:

- binaryen at release `version_123`, by the SHA-256 of each platform's archive, checked
  before use. The platforms are macOS arm64 and x86_64, and Linux x86_64 and aarch64;
- TypeScript, esbuild, acorn and binaryen's WebAssembly build of the same release by
  `package-lock.json`. `npm ci` checks their integrity hashes. J's `wasm2js` is that
  WebAssembly build, run on Node: the x86_64 Linux archive's native `wasm2js` allocates the
  translation's locals differently from the macOS archive's, while the WebAssembly build
  gives the same output on every host (and the macOS archive's output). Every other step
  gives the same bytes from every archive.

The build refuses:

- an input that uses any feature beyond the shipped build's;
- a reduced module that exports anything other than its profile;
- an L2 module whose i64 boundary differs from the shipped binary's;
- a glue whose layout it does not recognise;
- a script that does not parse as ECMAScript 5.

`manifest.json` in this folder is the build of the shipped package of record. Its `input`
says which package: the SHA-256 of both files, the commit they were built from, and how.
Given that package, `build.sh` refuses any output whose SHA-256 differs from the
manifest's; with `WAI_REACH_REQUIRE_RECORD=1` it also refuses any other package.
`WAI_REACH_RECORD=1` writes a new manifest.

## Conformance

Every rung must give the shipped build's outputs. The runners below check this:

- **`test/conformance.js`** (ES5, Node 0.12 onward) loads a rung through `wai-load.js` and
  runs `test/workloads.js` over the repository's corpora. Each workload is checked against
  an independent reference (a corpus's hashes, a sidecar's SHA-256, the receipt's signed
  fields, the registry), and its output must equal the shipped build's byte for byte. The
  workloads, by profile:
  - receipt: the hashes; envelope parsing; asset provenance over the vector in
    `test/vectors/provenance/` (its README says how it is made), including a tampered
    copy and a forged joule total;
  - verify: the capability menu against the registry; the staged corpus's content hashes;
    multi-rendition envelopes; the output tree of the staged-measurement goldens;
  - verify_decode: every committed WIV1, WIH1, WIA1 and WIS1 payload against its sidecar;
    the JSON integer forms; the committed int_synth vectors; all of `staged-conformance/`
    replayed through `StageSink`;
  - full: i64 parameters and results on both sides of 2^32.

  Under J, `WebAssembly` is deleted before anything loads. `--strip es5` also deletes
  `BigInt`, `Map`, `Symbol`, `Promise` and the other post-ES5 builtins, and makes
  `TypedArray#slice` return a view.
- **`test/run-modern.mjs`** runs on a current Node:
  - the loader must choose the shipped build, and that run is the reference;
  - every rung and profile is forced and must equal it;
  - an engine stripped to ES5 must be given J, and the same outputs;
  - the demo verify scripts (`demo/verify_*.mjs`: integer payloads, int_synth, the staged
    corpus with its guards and fuzz, the menu, the writers, the staged measurement) run
    against shipped, L1, L2 and J through a generated adapter module. The measurement
    script's written files must equal the shipped build's. They are this tree's scripts,
    or, with `--demo-commit <commit>`, that commit's scripts over that commit's corpora
    (its `wai/`, exported with `git archive`). A script this tree runs that the commit
    lacks is reported as `SKIP`.

```
node wai/wai-web/reach/test/run-modern.mjs --dist <dist> --shipped <package-dir> [--demo-commit <commit>]
node wai/wai-web/reach/test/conformance.js --dist <dist> --wai wai --rung J --profile receipt --reference <reference.json>
```

CI (`.github/workflows/wai-web-reach.yml`, on the `linux` runner) runs a leg per package:

- **The package of record.** `record/` keeps the two files the site serves under
  `/wai/pkg/`, built from the commit `manifest.json` names (`input.commit`; `input.source`
  says how). Both files must have the SHA-256 the manifest records, and `build.sh` then
  runs with `WAI_REACH_REQUIRE_RECORD=1`, so every output must have the manifest's SHA-256
  and size: these are the bytes the site serves under `/wai/pkg/reach/`. The package is
  kept rather than rebuilt in CI: the same commit and recipe give a different wasm on a
  Linux x86_64 host than on the macOS arm64 host that built it (the glue is the same), so
  the commit's build is checked on that host (the site's `build-wasm-pkg.sh recorded wai`).
  The demo verify scripts on this leg are that commit's, over that commit's corpora
  (`run-modern.mjs --demo-commit`): a script a later commit changes checks code the
  package does not hold. The reach workloads and the engine images are this tree's.
- **The tree.** wai-web is built from the pull request's own tree, so a change to wai-web
  meets the reach builds before the package of record moves to it. These outputs are not
  the recorded ones. The demo verify scripts are this tree's, so a check a pull request
  adds runs here against the code it checks.

On each leg:

1. `run-modern.mjs` runs on the outputs;
2. `conformance.js` runs in one Node image per engine line. On each line the loader must
   choose that line's rung by itself, and every output must equal the shipped build's
   from step 1.

### What each proxy stands for

The images are proxies. Each runs one JavaScript engine line on x86_64 Linux, with the
engine features below; real devices ship engines of the same lines. **No real old device
has been run.** Real devices also differ in ways no proxy reproduces: 32-bit ARM code
generation, engines with the JIT switched off, small memories, and engine families other
than the proxied one.

| proxy | engine features | rung the loader must choose | device classes it stands for |
|---|---|---|---|
| `node:0.12` | ES5 and typed arrays; no WebAssembly | J | TV sets and boxes whose web engine, or system web view, is frozen at an engine with no WebAssembly, smart TVs of model years 2014–2019 among them; phones whose system web view predates WebAssembly; broadcast-application receivers (HbbTV 2.0.x, ATSC 3.0 A/344). Console browsers that run with the JIT off, and engine families other than the proxied one, are J devices that this proxy does not cover |
| `node:8.0.0` | MVP WebAssembly; no RegExp `dotAll` flag | J, or L2 for the receipt profile | TV sets and boxes whose system web view is frozen at an engine with WebAssembly but no RegExp `dotAll` flag. The full MVP module has crashed an engine of this line on arm64; this x86_64 proxy does not crash, so it checks the choice and the outputs, not the crash |
| `node:8` | MVP WebAssembly and the `dotAll` flag; no `BigInt` | L2 | TV sets and boxes whose system web view is frozen at MVP WebAssembly with no `BigInt`, and TV platforms of model years 2019–2020 |
| `node:10` | MVP WebAssembly and `BigInt`; no sign-extension or bulk memory | L2 | TV sets and boxes whose system web view has `BigInt` but not the post-MVP features the shipped build uses |
| `node:12` | the shipped build's WebAssembly features; i64 does not cross the JS boundary as `BigInt` | L1 | TV platforms of model year 2021 |
| `node:14` | the same | L1 | TV platforms and streaming sticks whose engine has the shipped build's WebAssembly features but passes no i64 as `BigInt`, and phones' system web views of the same line |
| a current Node | every feature, with i64 as `BigInt` | shipped | current engines. Engine families other than the proxied one are not proxied here |

## Limits

- J is slower than WebAssembly, and far slower on engines without a JIT. The `receipt`
  and `verify` profiles are the ones for small or slow devices.
- Without `BigInt`, a 64-bit value is a `Number`, exact below 2^53. `IntPayloadCost.macs`
  (a 128-bit value) and the functions that return a `BigInt64Array` still need `BigInt`.
- A trap in the WebAssembly builds is an `Error('abort')` in J. The translation does not
  check memory accesses out of bounds, which trap in WebAssembly. The Rust does not make
  such accesses, and the conformance runs compare every output.
- Reduced profiles throw a `TypeError` for any function outside them.