Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions

Create on the toolchain's names: wai-app (a library), wai create --from app.oui, wai app check, wai mcp --toolset app, wai dev --show #508

Merged
dcharlot merged 14 commits from wai/create-2 into main 2026-10-27 10:15:14 -04:00
Conversation 0 Commits 14 Files changed 87 +14838 -95
87 changed files with 14838 additions and 95 deletions
Download patch file Download diff file Expand all files Collapse all files

156
.github/workflows/wai-app.yml vendored Normal file
View file

@ -0,0 +1,156 @@
name: wai-app and Create
# WAI's widgets for joule-ui and Create (wai/wai-app), and the line language they take a
# program in (joule-ui SPEC §11):
#
# unit wai-app's tests (its registry, its rules, the guide, the export's files, the
# runtime read by its listing, the bridge), clippy with warnings as errors, its
# WebAssembly surface checked for wasm32; the line language's independent verifier
# over the conformance pack, and its mutations refused; the page modules' unit tests
# (the renderer, the model streams). `wai create --from app.oui`, `wai app check` and
# `wai mcp --toolset app` are wai-rs's tests/cli_app.rs, in wai-conformance;
# e2e Create in the host's browser (wai/wai-app/e2e/run.sh), served by `wai dev --show`:
# a recorded stream renders as it arrives, the room joins a real room service, a chat
# and a poll go through it, the page's sealed log verifies, Export hands over app.oui,
# `wai create --from app.oui` makes the project and `wai app check` passes on it and
# refuses it broken, `wai dev` runs the project's own room where a host and a viewer
# answer a poll the room's clock signs, the refused lines never render, the visitor's
# own model is called with their key alone, and `wai mcp --toolset app` streams into
# the page through its bridge.
#
# The line language's own tests and its pack test run in standards-build (joule-ui-rs).
#
# Push filters match the push's delta, so every path the jobs read is listed.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
push:
branches: [main]
paths:
- 'wai/wai-app/**'
- 'joule-ui/**'
- 'wai/wai-live-web/**'
- 'wai/wai-rs/**'
- 'joule-code/joule-code-rs/crates/joule-moq-web/**'
- 'joule-code/joule-code-rs/crates/joule-moq/**'
- 'joule-code/joule-code-rs/crates/joule-moq-relay/**'
- 'joule-code/joule-code-rs/crates/joule-moq-room/**'
- 'joule-code/joule-code-rs/crates/jwp/**'
- 'joule-code/joule-code-rs/Cargo.toml'
- 'joule-code/joule-code-rs/Cargo.lock'
- 'wai/quickstart/build-web.sh'
- 'rust-toolchain.toml'
- '.github/workflows/wai-app.yml'
pull_request:
paths:
- 'wai/wai-app/**'
- 'joule-ui/**'
- 'wai/wai-live-web/**'
- 'wai/wai-rs/**'
- 'joule-code/joule-code-rs/crates/joule-moq-web/**'
- 'joule-code/joule-code-rs/crates/joule-moq/**'
- 'joule-code/joule-code-rs/crates/joule-moq-relay/**'
- 'joule-code/joule-code-rs/crates/joule-moq-room/**'
- 'joule-code/joule-code-rs/crates/jwp/**'
- 'joule-code/joule-code-rs/Cargo.toml'
- 'joule-code/joule-code-rs/Cargo.lock'
- 'wai/quickstart/build-web.sh'
- 'rust-toolchain.toml'
- '.github/workflows/wai-app.yml'
jobs:
# Forgejo matches `paths:` against the push delta, so a rebase fires this workflow
# whatever the pull request changes. This job reads the pull request's own diff against
# the paths above (scripts/pr-touches-paths.py, which fails open).
relevant:
name: wai-app · does this change touch it
runs-on: linux
outputs:
run: ${{ steps.gate.outputs.run }}
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
- id: gate
run: |
set -euo pipefail
r=$(python3 scripts/pr-touches-paths.py .github/workflows/wai-app.yml \
"${{ github.event.pull_request.base.sha }}" \
"${{ github.event.pull_request.head.sha }}")
echo "this change touches wai-app: $r"
echo "run=$r" >> "$GITHUB_OUTPUT"
# Nothing here needs macOS: it runs on the linux runner, leaving the macOS host to the
# browser job.
unit:
name: wai-app · tests, clippy, the line language's verifier (linux)
needs: relevant
if: needs.relevant.outputs.run == 'true'
runs-on: linux
timeout-minutes: 60
steps:
- uses: actions/checkout@v5
- name: Install Rust 1.98.0
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v rustup >/dev/null 2>&1 || \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
rustup toolchain install 1.98.0 --profile minimal --component clippy
rustup default 1.98.0
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- name: Tests and clippy
shell: bash
working-directory: wai/wai-app
run: |
set -euo pipefail
rustup target add wasm32-unknown-unknown
nice -n 10 cargo test --locked 2>&1 | tee "$RUNNER_TEMP/test.log"
grep -qE '^test result: ok\. [1-9][0-9]* passed; 0 failed' "$RUNNER_TEMP/test.log" || { echo "::error::no test ran"; exit 1; }
nice -n 10 cargo clippy --locked --all-targets -- -D warnings
nice -n 10 cargo clippy --locked --lib --target wasm32-unknown-unknown --no-default-features --features web -- -D warnings
- name: The line language's independent verifier, and its mutations
shell: bash
run: |
set -euo pipefail
python3 -I joule-ui/conformance/lang_verify.py
python3 -I joule-ui/conformance/lang_verify.py --mutations
- name: The page modules
shell: bash
run: node --test wai/wai-app/web/web.test.mjs
e2e:
name: wai-app · Create, browser end-to-end (macos)
needs: relevant
if: needs.relevant.outputs.run == 'true'
runs-on: macos
timeout-minutes: 90
steps:
- uses: actions/checkout@v5
# launchd's PATH omits cargo on the macOS host runner.
- name: Put cargo on PATH
shell: bash
run: echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
# Pinned to the wasm-bindgen the crates pin: the CLI and the crate must match.
- name: wasm32 target and wasm-bindgen CLI 0.2.127
shell: bash
run: |
set -euo pipefail
rustup target add wasm32-unknown-unknown
cargo install --locked wasm-bindgen-cli --version 0.2.127
- name: Create in the browser
shell: bash
env:
BROWSER_BIN: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome # allow-name: the macOS runner's browser executable, a file path
CREATE_E2E_OUT: ${{ runner.temp }}/wai-create
run: |
set -euo pipefail
test -x "$BROWSER_BIN" || { echo "::error::this runner has no browser at $BROWSER_BIN"; exit 1; }
mg=/Volumes/macos_4TB_external_1/wai-build/bin/memguard
if [ -x "$mg" ]; then export MEMGUARD="$mg"; fi
nice -n 10 bash wai/wai-app/e2e/run.sh 2>&1 | tee "$RUNNER_TEMP/create-e2e.log"
grep -qE '^create e2e: [1-9][0-9]* passed, 0 failed$' "$RUNNER_TEMP/create-e2e.log" \
|| { echo "::error::the Create end-to-end run did not pass"; exit 1; }

20
.github/workflows/wai-conformance.yml vendored
View file

@ -43,6 +43,12 @@ on:
- 'wai/wai-quantum/**'
# The integer decoders that decode through the integer operator set depend on it.
- 'wai/wai-ops/**'
# `wai create --from app.oui`, `wai app` and `wai mcp --toolset app` (the feature `app`)
# link Create's library and the joule-ui crates it builds on.
- 'wai/wai-app/**'
- 'joule-ui/joule-ui-rs/crates/joule-ui-core/**'
- 'joule-ui/joule-ui-rs/crates/joule-ui-lang/**'
- 'joule-ui/joule-ui-rs/crates/joule-ui-interaction/**'
- 'wai/*-conformance/**'
- 'wai/byte-exact-conformance/**'
# The registry export and the two library tests that read files outside
@ -637,6 +643,20 @@ jobs:
run: |
cargo test --no-default-features --features mcp --bin wai --test cli_mcp --test cli_setup_agent
cargo test --no-default-features --lib ops::
# Apps made with Create, through `wai` built with the feature `app` (wai/wai-app linked
# in). tests/cli_app.rs: `wai create --from app.oui` writes an ordinary project, its
# wai.json laid out as a template's and naming the program and the room `wai dev` runs,
# its runtime under vendor/ and LICENSE; `wai app check` passes on it and refuses it with
# its room not its app's, another audience, no room named, a runtime file changed or a
# refused line, and a watch room until its file is there; `wai create --from` refuses a
# program with errors or no room, and a directory with anything in it; `wai mcp --toolset
# app`: the tools app_guide, app_check, app_stream and app_export, a program streamed
# through the bridge to a page with the pairing code and refused to one without, an export
# inside the project directory alone, and read-only with no bridge. `--bin wai` runs the
# binary's unit tests with the app toolset in its table, and the dev front's show mode.
- name: Apps made with Create (wai create --from, wai app, wai mcp --toolset app)
working-directory: wai/wai-rs
run: cargo test --no-default-features --features app --bin wai --test cli_app
# tests/staged_moq.rs runs the MoQ trace of staged-conformance/moq/ (moq-streaming-format
# §4.1) with signed receipts: a chain for each track, a receipt held for its parent and
# admitted when it arrives, the relay's per-hop delivery timeout, dropped objects checked

31
joule-ui/NOTICE Normal file
View file

@ -0,0 +1,31 @@
joule-ui
========
The line language of SPEC.md §11 (joule-ui-rs/crates/joule-ui-lang) takes in
OpenUI Lang 1.0, the language of the open-source OpenUI project
(https://github.com/thesysdev/openui), whose specification is published under the
MIT License. joule-ui implements the language's grammar and streaming rules from
that specification's text (spec/language.md, version 1.0); no source code of the
project is included here. Its copyright and licence notice:
The MIT License (MIT)
Copyright (c) 2011-2024 Thesys Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

12
joule-ui/README.md
View file

@ -39,6 +39,18 @@ renderer is product surface, owned by the consumer.
chain, and the lifecycle are joule-ui's. See [`SPEC.md`](SPEC.md) §5–§7
and `examples/metered_surface.rs` for the full loop.
- `joule-ui-rs/crates/joule-ui-lang/` (**v0.3**) — the line language: a
model writes a surface as a short program, one widget a line
(OpenUI Lang 1.0; see [`NOTICE`](NOTICE)), and joule-ui takes it in as it
streams. After every chunk a snapshot holds the tree as far as it can be
built, each widget validated against the registry, and every error with its
line and reason; a canonical writer turns a tree back into the language.
State, queries, expressions and sequences of actions are refused
`not-declarative`: joule-ui carries no behaviour on the wire. See
[`SPEC.md`](SPEC.md) §11, the pack in [`conformance/lang/`](conformance/lang/)
with its independent verifier, and [`docs/create.md`](docs/create.md), the
design of Create, the page that builds a WAI app as a model describes it.
The surface layer makes joule-ui a peer of the declarative-UI protocols
(A2UI, MCP Apps). The interaction layer makes it the only one you can
hand to an auditor: a UI whose every interaction is energy-accounted and

379
joule-ui/SPEC.md
View file

@ -4,7 +4,7 @@
|---|---|
| **Standard** | joule-ui |
| **Status** | Draft for Review |
| **Version** | 0.2.0 |
| **Version** | 0.3.0 |
| **Licence** | Specification text CC-BY-4.0 ([`LICENSE-SPEC`](LICENSE-SPEC)); reference code Apache-2.0 ([`LICENSE`](LICENSE)) |
| **Reference** | [`joule-ui-rs/`](joule-ui-rs/) |
@ -50,6 +50,13 @@ children it accepts (`None` / `Any` / a named allow-list), and whether
unknown props are permitted. The registry is the host's contract; the renderer
is the host's product surface and is out of scope for this standard.
A schema MAY declare its **positional order** (`positional`): the order in which
a language that passes arguments by position, such as the line language of §11,
maps a call's arguments to the widget's props, the name `children` standing for
the child list. A schema that declares none has, as its order, its props in key
order followed by `children` when it accepts children. The order is absent from
the wire form when none is declared.
## §4 Validation
`validate(widget, registry)` checks a tree against the registry and returns
@ -130,6 +137,14 @@ interaction: the reference adapters seal such a span as the meter's
record of the debit MUST record it with that provenance, never an estimate
as a reading.
A host whose handler cannot run inside one call (a browser's handler awaits the
network) runs the same three phases in two steps: `begin_interaction` performs
phase 1 and samples the meter, and gives a **Pending** only when the action is
allowed; `Pending::finish` takes the handler's output and its wall-clock time on
the host's own clock, and performs phases 2 and 3. A denied action gives no
Pending, so no handler can run under it, and the denial is the Interaction phase
1 describes.
The result is an **Interaction**: the surface's content address (what the user
was looking at), the action id, the content addresses of the inputs and the
outputs (the genesis sentinel when denied), the verdict, the energy, the budget
@ -182,6 +197,9 @@ edge, not required.)
still named on the denial.
- **C4 — chain integrity.** Seal a sequence; `verify_chain` accepts it and
rejects any excision, reorder, or field mutation at the first broken link.
- **L1–L5 — the line language (§11).** Statements, ingest, streaming, the
canonical form and the refusals, from the pack in `conformance/lang/v1/`
(§11.9).
## §9 Recognized edges
@ -201,6 +219,361 @@ A renderer (the host's), a transport (the consumer's), a component library
registry, never a renderer). joule-ui pins the protocol and the receipt; the
fast-moving surfaces stay the consumer's.
## §11 The line language
A host MAY take in user interfaces written in the line-per-component language of
the open-source OpenUI project, OpenUI Lang 1.0 (MIT licence; [`NOTICE`](NOTICE)): a
program of statements `name = Component(arg, ...)`, one a line, a name usable before
the line that defines it. This section is how such a program becomes a joule-ui
widget tree (§2) against a registry (§3), checked by the validator of §4, while the
program is still arriving. The reference implementation is
[`joule-ui-rs/crates/joule-ui-lang`](joule-ui-rs/crates/joule-ui-lang/).
The language's grammar is parsed whole. What joule-ui does not carry (state,
queries and mutations, operators, conditionals, member access, built-in functions,
sequences of actions) is refused with the code `not-declarative` (§11.8): joule-ui
puts no expression and no behaviour on the wire (§2), and the smallest piece that
holds one is dropped.
A **library** is a registry and, optionally, the name of its **root widget**. Every
widget name of a library starts with an uppercase ASCII letter and holds only ASCII
letters, digits and `_`; none is `Query`, `Mutation` or `Action`; every name of a
widget's positional order (§3) is one of its props, or `children` on a widget that
accepts children, and none appears twice; the root names one of its widgets.
### §11.1 Source text
The text is prepared in this order. Each line of the prepared text keeps the number
of the source line it came from (1-based, counted in the text as received), and every
line an error names (§11.8) is that source line.
1. Every carriage return immediately followed by a line feed is removed. The text is
then a sequence of lines separated by line feeds.
2. **Host marker lines.** A line whose text starts with `]]>openui:` is a marker line.
When the text holds one, the program is only the lines that follow a marker line
whose text after `]]>openui:` starts with `content`, up to the next marker line;
marker lines themselves and every other line are removed.
3. **Fences.** A line whose first three characters are backticks (`` ``` ``) outside a
fence opens a fence. When the text holds one, the program is only the lines inside
fences, joined by line feeds; fence lines and all other lines are removed. Inside a
fence a line whose first three characters are backticks closes it, unless a
double-quoted string is open at its start: each line inside the fence is scanned
for double quotes, a `"` toggling the string open or closed, and inside an open
double-quoted string a backslash makes the next character not count (a line feed
ends that effect). Single quotes do not count here. A fence still open at the end
of the text runs to the end.
4. **Comments.** Outside a string, `//` or `#` starts a comment that runs to the end of
its line; the comment is removed and the line feed kept. A string opens at `"` or
`'` and closes at the next unescaped occurrence of the same quote character, across
line feeds; inside a string a backslash takes the next character with it.
5. **Trimming.** Space, tab, line feed and carriage return are removed from both ends
of the text. Lines removed from the front leave the line numbering too.
### §11.2 Lexing
The prepared text is read left to right into tokens. A space or a tab separates
tokens; a line feed is a **newline** token. A character no rule below matches is
skipped: lexing never fails.
- **Identifiers**: `[A-Za-z_][A-Za-z0-9_]*`; `true`, `false` and `null` are keyword
literals instead. `$` followed by an identifier is a **state name** (the `$` is part
of it); `@` followed by an identifier is an **`@` name**. A `$` or `@` not followed by
an identifier start is skipped.
- **Numbers**: a digit, or a `-` followed by a digit when the previous token is not a
value (a string, a number, usable or not, an identifier, a state name, a keyword
literal, `)` or `]`; `}` is not one), starts a number: an optional `-`, digits, then a `.` followed by
digits only when a digit follows the `.`, then, if an `e` or `E` follows, an exponent:
the marker, an optional `+` or `-`, and digits. An exponent marker with no digit
after it makes the token an **unusable number**. The number's value is the decimal
value of its text, rounded to the nearest binary64.
- **Strings** open at `"` or `'` and run to the next occurrence of the same quote; a
backslash takes the next character with it (a raw line feed is part of the string).
A double-quoted string's content (between the quotes) is decoded as a JSON string
(RFC 8259 §7); if it is not a valid one (an unknown escape, a raw control character, a
`\u` escape that is short or names a lone surrogate) the content is taken exactly as
written, backslashes included. In a single-quoted string, `\n` is a line feed, `\t` a
tab, and a backslash before any other character is dropped and the character kept.
A string still open at the end of the text is closed there: if its content then ends
in an odd number of backslashes, one more backslash is added first; the token is
marked **open**.
- **Punctuation**, longest first: `==` `!=` `>=` `<=` `&&` `||`, then `=` `>` `<` `+` `-`
`*` `/` `%` `!` `?` `:` `.` `,` `(` `)` `[` `]` `{` `}`; a single `&` is `&&` and a
single `|` is `||`.
### §11.3 Statements
A statement starts at a token that is an identifier or a state name (its **name**)
followed by `=`. Newline tokens before it are skipped; a line that does not start this
way is skipped up to its next newline token, with no error. The statement's
expression is the tokens after the `=`, newline tokens inside it left out, up to the
newline token that ends it. A newline token ends it when all of these hold:
1. the bracket depth is 0: each of `(` `[` `{` adds one, and each of `)` `]` `}` takes one
away when the depth is above 0;
2. no ternary is open: at depth 0 a `?` opens one; a `:` at depth 0 moves the innermost
open ternary to its second branch when it is still in its first (otherwise the `:`
changes nothing); a value token at depth
0 (a string, a number or an unusable number, an identifier, a state name, a keyword
literal, or a closing bracket that brings the depth to 0) gives the innermost open
ternary's current branch an operand, and closes that ternary when the branch is its
second; and
3. the next token that is not a newline is not `?`.
A statement that reaches the end of the tokens ends there.
A statement is **complete** when the host has ended the stream, or when its ending
newline has arrived and a token that is not a newline follows it. Otherwise it is
**pending**: only the last statement can be. A statement that reached the end of the
tokens (rather than a newline) while its bracket depth was above 0 or its last token was
an open string is **open**, pending or not.
A statement's **kind**: `Query(...)` or `Mutation(...)` as its whole expression makes it
a query or a mutation; otherwise a state name makes it a state declaration; otherwise it
is a value statement.
### §11.4 Expressions
An open statement is first closed: a closing bracket that does not match the innermost
open bracket is dropped, and the brackets still open are then closed, innermost first.
The statement's tokens are then parsed whole by this grammar; tokens left after the
expression are an error:
```
expression = ternary
ternary = or [ "?" ternary ":" ternary ]
or = and { "||" and }
and = equality { "&&" equality }
equality = comparison { ( "==" | "!=" ) comparison }
comparison = additive { ( ">" | "<" | ">=" | "<=" ) additive }
additive = multiplicative { ( "+" | "-" ) multiplicative }
multiplicative = unary { ( "*" | "/" | "%" ) unary }
unary = { "!" | "-" } postfix
postfix = primary { "." key | "[" expression "]" }
primary = string | number | "true" | "false" | "null"
| "[" items "]" | "{" entries "}" | "(" expression ")"
| ComponentName "(" items ")" | AtName "(" items ")"
| StateName "=" expression | StateName | identifier
items = { "," } [ expression { { "," } expression } { "," } ]
entries = { "," } [ key ":" expression { { "," } key ":" expression } { "," } ]
```
A `ComponentName` is an identifier with an uppercase first letter followed by `(`; any
other identifier is a **reference**. An unusable number is an error wherever it
stands. A **key** is the next token, whatever it is: an identifier or a string gives
its text, a number its text as written, a state name its text without `$`, a keyword
literal its word, an `@` name its text with `@`; any other token is the key `?`.
Commas between items and entries are optional, and any number of them is ignored.
A complete statement that does not parse is refused `syntax` (§11.8) at the source
line of the token where parsing stopped (the last token when the tokens ran out, the
statement's own line when it has none), and binds nothing. A pending statement that does not parse is not produced, with no error.
### §11.5 Into a joule-ui tree
**Bindings.** Statements are taken in order; each produced statement binds its name,
replacing an earlier binding, except that an open pending statement does not replace a
complete one.
**Statement refusals.** A complete state declaration, query or mutation is refused
`not-declarative` at its line. A reference to one evaluates to *refused* (below), with
no further error.
**The entry** is the binding of `root`. With no `root`, it is the first value
statement (by position in the text, among the bindings) whose expression is a call of
the library's root widget.
**Evaluation** gives each expression a value: data (a joule-ui `PropValue`), null, a
list or an object of values, a widget, an action, *unresolved*, *held*, or *refused*.
Errors go to the statement whose text holds the expression.
- A string, number or keyword gives `Text`, `Number`, `Bool`, or null.
- A reference to a name with no binding, or to a statement being evaluated already (a
cycle), is *unresolved*, and the name is listed as unresolved. Otherwise it is the
value of the bound statement's expression, evaluated where it is used.
- A list or an object gives a list or an object of its items' values.
- A state name, an operator, a conditional, member access, indexing and an assignment
are refused `not-declarative` at the line where that expression starts (a binary
operation and a conditional start at their left operand, member access and indexing at
their base, a call at its name).
- `Query(...)` or `Mutation(...)` inside an expression is refused `inline-reserved`, at its
line, its arguments not evaluated.
- `@Name(...)` with a built-in function's name (`Count`, `First`, `Last`, `Sum`, `Avg`,
`Min`, `Max`, `Filter`, `Sort`, `Round`, `Abs`, `Floor`, `Ceil`, `Each`) or a state
action's (`Set`, `Reset`, `Run`) is refused `not-declarative`, its arguments not
evaluated.
- `Action([x])` or `@Action([x])` with a list of exactly one item is that item's value
when it is an action (*held* when it is unresolved or held, *refused* when refused, and
otherwise refused `type-mismatch` at the item's line); any other argument list is
refused `not-declarative` at the call's line, unevaluated.
- Any other `@Name(args)` is the action `Action { id: Name, payload }`. Every argument is
evaluated and placed as data (below, null kept); if one was refused the action is
*refused*, else if one is not yet written the action is *held*. The payload is absent
for no argument, the argument for one, and the list of them for two or more.
- A component call is built into a widget, below.
**Placing as data** (a prop other than `children`, an item of a list or object placed as
data, an action's argument): data and actions are themselves; null is `Null` inside a
list, an object or an action's arguments; a list or an object becomes `List` or
`Object` of its items placed as data, an item that is not yet written or refused being
left out; a widget is refused `type-mismatch` (at the line where the call's or the
action's argument holding it starts); *unresolved* and *held* are **not yet written**;
*refused* stays refused.
**Building a widget.** A call of a name the registry lacks is refused
`unknown-component` at its line, its arguments not evaluated. Otherwise its arguments
map, in order, to the names of the widget's positional order (§3); arguments past its end
are refused together, `excess-args`, at the first extra one's line, and dropped
unevaluated. Every other argument is evaluated, in order; an error about an argument is
at the line where that argument starts.
- The argument mapped to `children` must give a list. Its widgets become the widget's
children in order; *unresolved*, *held*, null and *refused* items are left out; any
other item is refused `type-mismatch` and left out. A widget instead of a list, or
data, is refused `type-mismatch` and gives no children; null, *unresolved*, *held* and
*refused* give no children.
- Any other argument is placed as data. Null leaves the prop absent ("null"); a value not
yet written leaves it absent ("not yet"); a refused one, or a widget (refused
`type-mismatch`), leaves it absent ("refused").
The widget is then checked by `validate` (§4). While the stream is open, a widget that
`validate` finds missing a required prop that is absent "not yet" is *held*, with no
error. Otherwise every error at the widget itself is resolved as below and reported;
then, if one of them drops the widget, it is *refused*; else the removals are made and
the widget is checked again, until it passes:
| `validate` error | Resolution |
|---|---|
| a required prop missing | absent "not yet" (the stream has ended): `missing-required`. Absent "null": `null-required`. Absent "refused": no further error. Otherwise `missing-required`. Each drops the widget. |
| a prop of the wrong type | `type-mismatch`; a required prop drops the widget, an optional one is removed |
| a prop the widget does not declare | `type-mismatch`; the prop is removed |
| a child the widget does not accept | `disallowed-child`, at the child's own line and statement; the child is removed |
An error the table names is reported at the line of the widget's call, unless it says
otherwise. A widget that passes is the value, its children already built and checked.
**The tree** is the entry's value when that is a widget, and there is none otherwise.
Each widget of the tree has a **key**: a widget that is a statement's value takes the
statement's name, `name#2` for the second use of that statement in the evaluation and so
on; any other widget takes its parent value's key followed by `/` and its position
(`root/0`; `{key}/{slot}` for a widget in a prop's slot). The **nodes** list each
widget of the tree in pre-order with its key, the source line of its call, and its
statement.
**Orphans** are the value statements the evaluation of the entry did not reach (none
when there is no entry). The unresolved names and the orphans are reported sorted.
**At the end of the stream:** with no entry, the error is `parse-failed` (line 0, no
statement) when no statement was produced and the prepared text is not empty, and
`no-root` (line 0, no statement) otherwise; with an entry
that gives no widget, `no-root` at the entry's line; with an entry found by the root
widget rather than `root`, `no-root` as a warning at its line, and the tree renders.
**Errors** are reported sorted by line, then statement, then code (as text), then
widget, then prop (an absent widget or prop before any), then warning (not one before
one), and of the errors equal in all of these only the first is kept. While the stream
is open, errors of a statement whose binding is pending are withheld.
A snapshot MUST NOT hold a widget its registry's `validate` refuses: the tree as a whole
passes `validate`.
### §11.6 Streaming
A host passes the text a chunk at a time; after each chunk, and when the host ends the
stream, the ingest gives a **snapshot**: the tree, its nodes, the errors, the unresolved
names, the orphans, and whether the text ended inside an open string or bracket
(**incomplete**: the last statement is open). A chunk after the end starts a new stream.
1. Each snapshot is that of §11.1–§11.5 for all the text so far, statements completing
as §11.3 says. A complete statement's text no longer changes, so its value only
changes as the statements it refers to arrive.
2. The pending statement is closed for the moment (§11.2 strings, §11.4 brackets) and
evaluated; its errors are withheld until it completes.
3. An open pending statement never replaces a complete one of the same name (§11.5).
4. Text that changes what came before it (a fence line arriving after unfenced lines)
changes the preparation, and so every later snapshot.
5. **Streaming equals parsing.** The snapshot at the end of a stream is the snapshot of
the whole text taken at once, however it was cut.
### §11.7 The canonical form
A tree that passes `validate` is written one statement a widget, in pre-order, each on
its own line ending with a line feed: the root is `root`; every other widget is named
by its widget name with its first letter in lowercase, followed by a count of that
lowercase name from 1, in pre-order (`card1`, `card2`). A statement is
`name = Widget(args)`, the arguments joined by `, `: one for each name of the widget's
positional order, `children` written `[name1, name2]` when the widget has children; a
missing prop or empty `children` is `null`, and the trailing ones are left off. Values:
- text as a JSON string: `"` and `\` escaped, `\b` `\f` `\n` `\r` `\t` for those, `\u00xx`
(lowercase) for the other characters below U+0020, everything else as itself;
- a number as the shortest decimal that reads back to the same binary64 value, in
positional notation, with no exponent and no fraction for an integer (`-0` for
negative zero); a number that is not finite cannot be written;
- `true`, `false`, `null`, and lists as `[a, b]`;
- an object as `{key: value}` in key order (`, ` between entries), a key written bare
when it is an identifier other than a keyword literal, and as a JSON string otherwise;
- an action as `@Id(...)`: nothing inside for no payload, the items of a list payload of
two or more, and the payload as the single argument otherwise. The id must be an
identifier other than `Action`, a built-in function's or a state action's name.
A tree holding a widget whose name a call cannot take, a prop outside its widget's
positional order, a prop whose value is `null` (a null argument is an absent one), or an
action id the form cannot carry, is not written. The canonical form of a tree parses
back (§11.5) to the same tree, and writing that tree again gives the same text.
### §11.8 Reason codes
| Code | Raised when | What is dropped |
|---|---|---|
| `unknown-component` | a call names a widget the registry lacks | the widget |
| `missing-required` | a required prop is absent (once the stream has ended, for one not yet written) | the widget |
| `null-required` | a required prop is written `null` | the widget |
| `type-mismatch` | a value is not of its prop's type; a widget stands where data goes, or data among children; children not written as a list | an optional prop or an item; a required prop drops the widget |
| `excess-args` | more arguments than the widget's positional order | the extra arguments |
| `disallowed-child` | a child the parent's `AllowedChildren` does not accept | the child |
| `not-declarative` | state, a query, a mutation, an operator, a conditional, member access, indexing, an assignment, a built-in function, a state action, or a sequence of actions | the statement, or the value |
| `inline-reserved` | `Query` or `Mutation` inside an expression | the value |
| `syntax` | a complete statement that does not parse | the statement |
| `no-root` | at the end of the stream, no entry or an entry that is not a widget; a warning when the entry is found by the root widget | everything (not for the warning) |
| `parse-failed` | at the end of the stream, text with no statement | everything |
An error names its source line (0 for the program as a whole), its statement (empty for
the whole program), its code, the widget and the prop it concerns where there is one,
and a message for a person or a model to act on. Messages are not normative.
The widget an error names: for `unknown-component`, `missing-required`,
`null-required`, `excess-args`, and a `type-mismatch` that `validate` raised or that is
about a call's own argument (a widget in a prop's slot, the `children` argument or an
item of it), the called widget; for `disallowed-child`, the child; for a `type-mismatch`
about a widget placed as data inside a list, an object or an action's arguments, that
widget. No other error names a widget. The prop an error names: for `missing-required`,
`null-required`, and a `type-mismatch` that names the called widget, the prop
(`children` for the children argument); no other error names one.
### §11.9 Conformance
The pack in [`conformance/lang/v1/`](conformance/lang/v1/) holds the library it is
written against (`library.json`, each widget's positional order written out) and five
classes; a conforming ingest reproduces every case's expected value, errors compared
without their messages:
- **L1, statements** (`l1-statements.json`): the statements of each text at the end of the
stream, in order (name, source line, kind, complete): each produced statement with its
kind (`value`, `state`, `query`, `mutation`), and each complete statement that did not
parse with the kind `syntax`; and whether it ended open.
- **L2, ingest** (`l2-ingest.json`): each text's snapshot at the end of the stream.
- **L3, streaming** (`l3-stream.json`): each case's snapshot after every chunk and at
the end, which equals L2's snapshot of the joined chunks.
- **L4, the canonical form** (`l4-write.json`): each tree's text, which parses back to
the tree.
- **L5, refusals** (`l5-refusals.json`): negative controls, each refused with its code at
its line, and its whole snapshot.
`MANIFEST.json` names each file by its SHA-256. The independent verifier
[`conformance/lang_verify.py`](conformance/lang_verify.py) is written from this section
alone and shares no code with the reference implementation.
---
*joule-ui v0.2.0 — the surface layer (v0.1) plus the metered, sealed
interaction layer that makes a UI accountable by construction.*
*joule-ui v0.3.0 — the surface layer (v0.1), the metered, sealed interaction layer
that makes a UI accountable by construction (v0.2), and the line language a model
writes a surface in, taken in as it streams (v0.3).*

34
joule-ui/conformance/lang/README.md Normal file
View file

@ -0,0 +1,34 @@
# joule-ui line language conformance — v1
The pack of SPEC §11.9: the line language (OpenUI Lang 1.0, taken in as joule-ui
widget trees) against the pack's own library. Regenerate with:
```
cargo run -p joule-ui-lang --bin gen_lang_vectors
```
and verify, from the specification alone, with the independent verifier:
```
python3 -I conformance/lang_verify.py # every case recomputed and compared
python3 -I conformance/lang_verify.py --mutations # changed vectors are refused
```
`cargo test -p joule-ui-lang --test pack` checks that the committed files are exactly
what the reference implementation gives.
| File | Class | What a conforming ingest reproduces |
|---|---|---|
| `library.json` | — | the library: each widget's props, children rule and positional order (written out), and the root widget |
| `l1-statements.json` | **L1** | each text's statements (name, source line, kind, complete) and whether it ended open |
| `l2-ingest.json` | **L2** | each text's snapshot at the end of the stream: the tree, its nodes (key, line, statement), the errors (line, statement, code, widget, prop, warning), the unresolved names, the orphans, whether it ended open |
| `l3-stream.json` | **L3** | each case's snapshot after every chunk and at the end; the end equals the batch parse of the joined chunks |
| `l4-write.json` | **L4** | each tree's canonical text, which parses back to the tree |
| `l5-refusals.json` | **L5** | negative controls: each refused with its code at its line, and its whole snapshot |
| `MANIFEST.json` | — | each file's SHA-256 |
The tree and its values are in joule-ui's wire form (§2): a widget is
`{"name", "props", "children"}` (`children` absent when empty); a prop value is
`{"type": "text" | "number" | "bool" | "list" | "object", "value": ...}`, `{"type": "null"}`,
or `{"type": "action", "value": {"id", "payload"}}` with `payload` null or a prop value.
Errors carry no message: messages are for a person or a model, and not normative.

11
joule-ui/conformance/lang/v1/MANIFEST.json Normal file
View file

@ -0,0 +1,11 @@
{
"files": {
"l1-statements.json": "sha256:4f8ab770076dfccc30b3d83b36b7abf3100b96ed170435ebfd60c62b9370cd32",
"l2-ingest.json": "sha256:f308252782180452fc469548f62b60fe94ea0195bcbaf5512c33926a5ecc788d",
"l3-stream.json": "sha256:99ce046d1d91cefac04c0176b3d9f251b2065fbebf71e828b05115d62ad7555d",
"l4-write.json": "sha256:121d8aa3d6b3c5922d595b12ff16e2893ce4b151c7b2f4f32afbf73cfb96515f",
"l5-refusals.json": "sha256:b69fba8400e62438e057cccae1c7bbbbab9432260c226b1ae1057ebda8aca290",
"library.json": "sha256:deb301ce3984d8441a828c4e710d64d01c024e3b74f8a19c69bf2ce11ce76546"
},
"pack": "joule-ui-lang/v1"
}

257
joule-ui/conformance/lang/v1/l1-statements.json Normal file
View file

@ -0,0 +1,257 @@
{
"cases": [
{
"incomplete": false,
"input": "root = Page([a])\na = Text(\"hi\")\n",
"name": "one-a-line",
"statements": [
{
"complete": true,
"kind": "value",
"line": 1,
"name": "root"
},
{
"complete": true,
"kind": "value",
"line": 2,
"name": "a"
}
]
},
{
"incomplete": false,
"input": "\n\nhello there\nroot = Page()\n= 3\n \nx = 1\n",
"name": "blank-lines-and-prose-lines",
"statements": [
{
"complete": true,
"kind": "value",
"line": 4,
"name": "root"
},
{
"complete": true,
"kind": "value",
"line": 7,
"name": "x"
}
]
},
{
"incomplete": false,
"input": "root = Page([\n a,\n b\n])\na = Text(\"a\")\nb = Text(\"b\")",
"name": "brackets-carry-over-lines",
"statements": [
{
"complete": true,
"kind": "value",
"line": 1,
"name": "root"
},
{
"complete": true,
"kind": "value",
"line": 5,
"name": "a"
},
{
"complete": true,
"kind": "value",
"line": 6,
"name": "b"
}
]
},
{
"incomplete": false,
"input": "a = Text(\"line one\nline two\")\nroot = Page([a])",
"name": "string-carries-over-lines",
"statements": [
{
"complete": true,
"kind": "value",
"line": 1,
"name": "a"
},
{
"complete": true,
"kind": "value",
"line": 3,
"name": "root"
}
]
},
{
"incomplete": false,
"input": "x = c\n ? 1\n : 2\nroot = Page()",
"name": "ternary-continues-on-a-question-line",
"statements": [
{
"complete": true,
"kind": "value",
"line": 1,
"name": "x"
},
{
"complete": true,
"kind": "value",
"line": 4,
"name": "root"
}
]
},
{
"incomplete": false,
"input": "x = c ? 1 :\n 2\nroot = Page()",
"name": "ternary-empty-branch-continues",
"statements": [
{
"complete": true,
"kind": "value",
"line": 1,
"name": "x"
},
{
"complete": true,
"kind": "value",
"line": 3,
"name": "root"
}
]
},
{
"incomplete": false,
"input": "$s = 1\nq = Query(\"t\", {}, {})\nm = Mutation(\"u\", {})\n$w = Query(\"t\", {}, {})\nroot = Page()",
"name": "state-query-mutation-kinds",
"statements": [
{
"complete": true,
"kind": "state",
"line": 1,
"name": "$s"
},
{
"complete": true,
"kind": "query",
"line": 2,
"name": "q"
},
{
"complete": true,
"kind": "mutation",
"line": 3,
"name": "m"
},
{
"complete": true,
"kind": "query",
"line": 4,
"name": "$w"
},
{
"complete": true,
"kind": "value",
"line": 5,
"name": "root"
}
]
},
{
"incomplete": false,
"input": "root = Page([a])\na = Text(\"x\" 1 +)\nb = 1 2\n",
"name": "syntax-error-is-a-statement-of-its-own",
"statements": [
{
"complete": true,
"kind": "value",
"line": 1,
"name": "root"
},
{
"complete": true,
"kind": "syntax",
"line": 2,
"name": "a"
},
{
"complete": true,
"kind": "syntax",
"line": 3,
"name": "b"
}
]
},
{
"incomplete": false,
"input": "Here it is:\n```openui-lang\nroot = Page([a])\n```\nand a note\n```\na = Text(\"in a second fence\")\n```\n",
"name": "fenced-with-prose",
"statements": [
{
"complete": true,
"kind": "value",
"line": 3,
"name": "root"
},
{
"complete": true,
"kind": "value",
"line": 7,
"name": "a"
}
]
},
{
"incomplete": false,
"input": "root = Page([a]) // the page\n# a heading comment\na = Text(\"#not a comment // nor this\")\n",
"name": "comments",
"statements": [
{
"complete": true,
"kind": "value",
"line": 1,
"name": "root"
},
{
"complete": true,
"kind": "value",
"line": 3,
"name": "a"
}
]
},
{
"incomplete": false,
"input": "]]>openui:content?library=pack%401\nroot = Page([a])\na = Text(\"x\")\n]]>openui:context\n{\"formState\": {}}\n]]>openui:end\n",
"name": "markers",
"statements": [
{
"complete": true,
"kind": "value",
"line": 2,
"name": "root"
},
{
"complete": true,
"kind": "value",
"line": 3,
"name": "a"
}
]
},
{
"incomplete": true,
"input": "root = Page([a\na = Text(\"open",
"name": "unterminated-at-end",
"statements": [
{
"complete": true,
"kind": "syntax",
"line": 1,
"name": "root"
}
]
}
],
"class": "L1"
}

985
joule-ui/conformance/lang/v1/l2-ingest.json Normal file
View file

@ -0,0 +1,985 @@
{
"cases": [
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "card",
"line": 2,
"statement": "card"
},
{
"key": "t1",
"line": 3,
"statement": "t1"
},
{
"key": "t2",
"line": 4,
"statement": "t2"
}
],
"orphans": [],
"tree": {
"children": [
{
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "one"
}
}
},
{
"name": "Text",
"props": {
"size": {
"type": "number",
"value": 14.0
},
"text": {
"type": "text",
"value": "two"
}
}
}
],
"name": "Card",
"props": {
"subtitle": {
"type": "text",
"value": "sub"
},
"title": {
"type": "text",
"value": "Hello"
}
}
}
],
"name": "Page",
"props": {
"title": {
"type": "text",
"value": "Home"
}
}
},
"unresolved": []
},
"input": "root = Page([card], \"Home\")\ncard = Card(\"Hello\", [t1, t2], \"sub\")\nt1 = Text(\"one\")\nt2 = Text(\"two\", 14)\n",
"name": "hoisting"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "a",
"line": 2,
"statement": "a"
},
{
"key": "b",
"line": 3,
"statement": "b"
}
],
"orphans": [],
"tree": {
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "a"
}
}
},
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "b"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": [
"missing"
]
},
"input": "root = Page([a, missing, b])\na = Text(\"a\")\nb = Text(\"b\")\n",
"name": "unresolved-left-out"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "a",
"line": 2,
"statement": "a"
}
],
"orphans": [
"old"
],
"tree": {
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "a"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "root = Page([a])\na = Text(\"a\")\nold = Text(\"old\")\n",
"name": "orphans"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "root/0",
"line": 1,
"statement": "root"
},
{
"key": "root/0/0",
"line": 1,
"statement": "root"
},
{
"key": "root/0/1",
"line": 1,
"statement": "root"
},
{
"key": "root/1",
"line": 1,
"statement": "root"
}
],
"orphans": [],
"tree": {
"children": [
{
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "x"
}
}
},
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "y"
}
}
}
],
"name": "Card",
"props": {
"title": {
"type": "text",
"value": "c"
}
}
},
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "z"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "root = Page([Card(\"c\", [Text(\"x\"), Text(\"y\")]), Text(\"z\")])\n",
"name": "inline-calls-and-keys"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "t",
"line": 2,
"statement": "t"
},
{
"key": "t#2",
"line": 2,
"statement": "t"
}
],
"orphans": [],
"tree": {
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "again"
}
}
},
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "again"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "root = Page([t, t])\nt = Text(\"again\")\n",
"name": "used-twice"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "c",
"line": 2,
"statement": "c"
}
],
"orphans": [],
"tree": {
"children": [
{
"name": "Card",
"props": {
"subtitle": {
"type": "text",
"value": "sub"
},
"title": {
"type": "text",
"value": "t"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "root = Page([c])\nc = Card(\"t\", null, \"sub\")\n",
"name": "null-skips-an-optional"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "ch",
"line": 2,
"statement": "ch"
}
],
"orphans": [],
"tree": {
"children": [
{
"name": "Chart",
"props": {
"labels": {
"type": "list",
"value": [
{
"type": "text",
"value": "a"
},
{
"type": "text",
"value": "b"
}
]
},
"options": {
"type": "object",
"value": {
"colours": {
"type": "list",
"value": [
{
"type": "text",
"value": "red"
},
{
"type": "text",
"value": "blue"
}
]
},
"max value": {
"type": "null"
},
"stacked": {
"type": "bool",
"value": true
}
}
},
"values": {
"type": "list",
"value": [
{
"type": "number",
"value": 1.0
},
{
"type": "number",
"value": -2.5
},
{
"type": "number",
"value": 300.0
}
]
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "root = Page([ch])\nch = Chart([\"a\", \"b\"], [1, -2.5, 3e2], {stacked: true, \"max value\": null, colours: ['red', \"blue\"]})\n",
"name": "data-props"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "b1",
"line": 2,
"statement": "b1"
},
{
"key": "b2",
"line": 3,
"statement": "b2"
},
{
"key": "b3",
"line": 4,
"statement": "b3"
},
{
"key": "b4",
"line": 5,
"statement": "b4"
},
{
"key": "b5",
"line": 6,
"statement": "b5"
},
{
"key": "b6",
"line": 7,
"statement": "b6"
}
],
"orphans": [],
"tree": {
"children": [
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "Go",
"payload": null
}
},
"label": {
"type": "text",
"value": "none"
}
}
},
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "Open",
"payload": {
"type": "text",
"value": "x"
}
}
},
"label": {
"type": "text",
"value": "one"
}
}
},
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "Send",
"payload": {
"type": "list",
"value": [
{
"type": "text",
"value": "a"
},
{
"type": "number",
"value": 2.0
}
]
}
}
},
"label": {
"type": "text",
"value": "two"
}
}
},
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "Pick",
"payload": {
"type": "list",
"value": [
{
"type": "number",
"value": 1.0
},
{
"type": "number",
"value": 2.0
}
]
}
}
},
"label": {
"type": "text",
"value": "list"
}
}
},
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "Go",
"payload": null
}
},
"label": {
"type": "text",
"value": "wrapped"
}
}
},
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "ToAssistant",
"payload": {
"type": "list",
"value": [
{
"type": "text",
"value": "hi"
},
{
"type": "object",
"value": {
"id": {
"type": "number",
"value": 1.0
}
}
}
]
}
}
},
"label": {
"type": "text",
"value": "by name"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "root = Page([b1, b2, b3, b4, b5, b6])\nb1 = Button(\"none\", @Go())\nb2 = Button(\"one\", @Open(\"x\"))\nb3 = Button(\"two\", @Send(\"a\", 2))\nb4 = Button(\"list\", @Pick([1, 2]))\nb5 = Button(\"wrapped\", Action([@Go()]))\nb6 = Button(\"by name\", go)\ngo = @ToAssistant(\"hi\", {id: 1})\n",
"name": "actions"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "tabs",
"line": 2,
"statement": "tabs"
},
{
"key": "t1",
"line": 3,
"statement": "t1"
},
{
"key": "t1/0",
"line": 3,
"statement": "t1"
},
{
"key": "t2",
"line": 4,
"statement": "t2"
}
],
"orphans": [],
"tree": {
"children": [
{
"children": [
{
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "1"
}
}
}
],
"name": "Tab",
"props": {
"title": {
"type": "text",
"value": "One"
}
}
},
{
"name": "Tab",
"props": {
"title": {
"type": "text",
"value": "Two"
}
}
}
],
"name": "Tabs",
"props": {}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "root = Page([tabs])\ntabs = Tabs([t1, t2])\nt1 = Tab(\"One\", [Text(\"1\")])\nt2 = Tab(\"Two\")\n",
"name": "only-of-children"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "n",
"line": 2,
"statement": "n"
},
{
"key": "n/0",
"line": 2,
"statement": "n"
}
],
"orphans": [],
"tree": {
"children": [
{
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "inside"
}
}
}
],
"name": "Note",
"props": {
"author": {
"type": "text",
"value": "by"
},
"body": {
"type": "text",
"value": "the body"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "root = Page([n])\nn = Note(\"by\", \"the body\", [Text(\"inside\")])\n",
"name": "derived-order"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "a",
"line": 3,
"statement": "a"
}
],
"orphans": [],
"tree": {
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "second"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "root = Page([a])\na = Text(\"first\")\na = Text(\"second\")\n",
"name": "duplicate-later-wins"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "a",
"line": 2,
"statement": "a"
},
{
"key": "b",
"line": 3,
"statement": "b"
}
],
"orphans": [],
"tree": {
"children": [
{
"children": [
{
"name": "Card",
"props": {
"title": {
"type": "text",
"value": "b"
}
}
}
],
"name": "Card",
"props": {
"title": {
"type": "text",
"value": "a"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": [
"a"
]
},
"input": "root = Page([a])\na = Card(\"a\", [b])\nb = Card(\"b\", [a])\n",
"name": "cycle"
},
{
"expected": {
"errors": [
{
"code": "no-root",
"line": 2,
"statement": "page",
"warning": true
}
],
"incomplete": false,
"nodes": [
{
"key": "page",
"line": 2,
"statement": "page"
},
{
"key": "t",
"line": 1,
"statement": "t"
}
],
"orphans": [],
"tree": {
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "x"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "t = Text(\"x\")\npage = Page([t])\n",
"name": "fallback-entry"
},
{
"expected": {
"errors": [],
"incomplete": false,
"nodes": [
{
"key": "root",
"line": 1,
"statement": "root"
},
{
"key": "a",
"line": 2,
"statement": "a"
},
{
"key": "b",
"line": 3,
"statement": "b"
},
{
"key": "c",
"line": 4,
"statement": "c"
}
],
"orphans": [],
"tree": {
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "tab\there é"
}
}
},
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "it's"
}
}
},
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "bad \\q escape"
}
}
}
],
"name": "Page",
"props": {}
},
"unresolved": []
},
"input": "root = Page([a, b, c])\na = Text(\"tab\\there \\u00e9\")\nb = Text('it\\'s')\nc = Text(\"bad \\q escape\")\n",
"name": "escapes"
},
{
"expected": {
"errors": [
{
"code": "no-root",
"line": 0,
"statement": ""
}
],
"incomplete": false,
"nodes": [],
"orphans": [],
"tree": null,
"unresolved": []
},
"input": "",
"name": "empty"
}
],
"class": "L2"
}

1205
joule-ui/conformance/lang/v1/l3-stream.json Normal file
View file

File diff suppressed because it is too large Load diff

364
joule-ui/conformance/lang/v1/l4-write.json Normal file
View file

@ -0,0 +1,364 @@
{
"cases": [
{
"name": "page-of-cards",
"text": "root = Page([card1, card2], \"Home\")\ncard1 = Card(\"One\", [text1])\ntext1 = Text(\"a\")\ncard2 = Card(\"Two\", null, \"sub\")\n",
"tree": {
"children": [
{
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "a"
}
}
}
],
"name": "Card",
"props": {
"title": {
"type": "text",
"value": "One"
}
}
},
{
"name": "Card",
"props": {
"subtitle": {
"type": "text",
"value": "sub"
},
"title": {
"type": "text",
"value": "Two"
}
}
}
],
"name": "Page",
"props": {
"title": {
"type": "text",
"value": "Home"
}
}
}
},
{
"name": "middle-absent-is-null",
"text": "root = Page([card1])\ncard1 = Card(\"t\", null, \"s\")\n",
"tree": {
"children": [
{
"name": "Card",
"props": {
"subtitle": {
"type": "text",
"value": "s"
},
"title": {
"type": "text",
"value": "t"
}
}
}
],
"name": "Page",
"props": {}
}
},
{
"name": "numbers-and-strings",
"text": "root = Page([text1, text2, text3])\ntext1 = Text(\"quote \\\" back \\\\ line\\nfeed é \\u0001\", 0.1)\ntext2 = Text(\"neg\", -0)\ntext3 = Text(\"small\", 0.0000001)\n",
"tree": {
"children": [
{
"name": "Text",
"props": {
"size": {
"type": "number",
"value": 0.1
},
"text": {
"type": "text",
"value": "quote \" back \\ line\nfeed é \u0001"
}
}
},
{
"name": "Text",
"props": {
"size": {
"type": "number",
"value": -0.0
},
"text": {
"type": "text",
"value": "neg"
}
}
},
{
"name": "Text",
"props": {
"size": {
"type": "number",
"value": 1e-7
},
"text": {
"type": "text",
"value": "small"
}
}
}
],
"name": "Page",
"props": {}
}
},
{
"name": "data",
"text": "root = Page([chart1])\nchart1 = Chart([\"a\", null], [1, []], {\"max value\": 1000000000000000000000, \"null\": null, stacked: true})\n",
"tree": {
"children": [
{
"name": "Chart",
"props": {
"labels": {
"type": "list",
"value": [
{
"type": "text",
"value": "a"
},
{
"type": "null"
}
]
},
"options": {
"type": "object",
"value": {
"max value": {
"type": "number",
"value": 1e+21
},
"null": {
"type": "null"
},
"stacked": {
"type": "bool",
"value": true
}
}
},
"values": {
"type": "list",
"value": [
{
"type": "number",
"value": 1.0
},
{
"type": "list",
"value": []
}
]
}
}
}
],
"name": "Page",
"props": {}
}
},
{
"name": "actions",
"text": "root = Page([button1, button2, button3, button4, button5])\nbutton1 = Button(\"none\", @Go())\nbutton2 = Button(\"one\", @Open(\"x\"))\nbutton3 = Button(\"two\", @Send(\"a\", 2))\nbutton4 = Button(\"list of one\", @Pick([1]))\nbutton5 = Button(\"null\", @Clear(null), true)\n",
"tree": {
"children": [
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "Go",
"payload": null
}
},
"label": {
"type": "text",
"value": "none"
}
}
},
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "Open",
"payload": {
"type": "text",
"value": "x"
}
}
},
"label": {
"type": "text",
"value": "one"
}
}
},
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "Send",
"payload": {
"type": "list",
"value": [
{
"type": "text",
"value": "a"
},
{
"type": "number",
"value": 2.0
}
]
}
}
},
"label": {
"type": "text",
"value": "two"
}
}
},
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "Pick",
"payload": {
"type": "list",
"value": [
{
"type": "number",
"value": 1.0
}
]
}
}
},
"label": {
"type": "text",
"value": "list of one"
}
}
},
{
"name": "Button",
"props": {
"action": {
"type": "action",
"value": {
"id": "Clear",
"payload": {
"type": "null"
}
}
},
"label": {
"type": "text",
"value": "null"
},
"primary": {
"type": "bool",
"value": true
}
}
}
],
"name": "Page",
"props": {}
}
},
{
"name": "derived-order",
"text": "root = Page([note1])\nnote1 = Note(null, \"b\", [text1])\ntext1 = Text(\"in\")\n",
"tree": {
"children": [
{
"children": [
{
"name": "Text",
"props": {
"text": {
"type": "text",
"value": "in"
}
}
}
],
"name": "Note",
"props": {
"body": {
"type": "text",
"value": "b"
}
}
}
],
"name": "Page",
"props": {}
}
},
{
"name": "tabs",
"text": "root = Page([tabs1])\ntabs1 = Tabs([tab1, tab2])\ntab1 = Tab(\"x\")\ntab2 = Tab(\"y\")\n",
"tree": {
"children": [
{
"children": [
{
"name": "Tab",
"props": {
"title": {
"type": "text",
"value": "x"
}
}
},
{
"name": "Tab",
"props": {
"title": {
"type": "text",
"value": "y"
}
}
}
],
"name": "Tabs",
"props": {}
}
],
"name": "Page",
"props": {}
}
}
],
"class": "L4"
}

1161
joule-ui/conformance/lang/v1/l5-refusals.json Normal file
View file

File diff suppressed because it is too large Load diff

178
joule-ui/conformance/lang/v1/library.json Normal file
View file

@ -0,0 +1,178 @@
{
"root": "Page",
"widgets": {
"Button": {
"allow_extra_props": false,
"children": {
"kind": "none"
},
"description": "a button",
"name": "Button",
"positional": [
"label",
"action",
"primary"
],
"props": {
"action": {
"required": false,
"ty": "action"
},
"label": {
"required": true,
"ty": "text"
},
"primary": {
"required": false,
"ty": "bool"
}
}
},
"Card": {
"allow_extra_props": false,
"children": {
"kind": "any"
},
"description": "a titled container",
"name": "Card",
"positional": [
"title",
"children",
"subtitle"
],
"props": {
"subtitle": {
"required": false,
"ty": "text"
},
"title": {
"required": true,
"ty": "text"
}
}
},
"Chart": {
"allow_extra_props": false,
"children": {
"kind": "none"
},
"description": "a chart of values",
"name": "Chart",
"positional": [
"labels",
"values",
"options"
],
"props": {
"labels": {
"required": true,
"ty": "list"
},
"options": {
"required": false,
"ty": "object"
},
"values": {
"required": true,
"ty": "list"
}
}
},
"Note": {
"allow_extra_props": false,
"children": {
"kind": "any"
},
"description": "a note",
"name": "Note",
"positional": [
"author",
"body",
"children"
],
"props": {
"author": {
"required": false,
"ty": "text"
},
"body": {
"required": true,
"ty": "text"
}
}
},
"Page": {
"allow_extra_props": false,
"children": {
"kind": "any"
},
"description": "the page",
"name": "Page",
"positional": [
"children",
"title"
],
"props": {
"title": {
"required": false,
"ty": "text"
}
}
},
"Tab": {
"allow_extra_props": false,
"children": {
"kind": "any"
},
"description": "one tab",
"name": "Tab",
"positional": [
"title",
"children"
],
"props": {
"title": {
"required": true,
"ty": "text"
}
}
},
"Tabs": {
"allow_extra_props": false,
"children": {
"kind": "only_of",
"value": [
"Tab"
]
},
"description": "a strip of tabs",
"name": "Tabs",
"positional": [
"children"
],
"props": {}
},
"Text": {
"allow_extra_props": false,
"children": {
"kind": "none"
},
"description": "a run of text",
"name": "Text",
"positional": [
"text",
"size"
],
"props": {
"size": {
"required": false,
"ty": "number"
},
"text": {
"required": true,
"ty": "text"
}
}
}
}
}

1207
joule-ui/conformance/lang_verify.py Normal file
View file

File diff suppressed because it is too large Load diff

133
joule-ui/docs/create.md Normal file
View file

@ -0,0 +1,133 @@
# Create: describing an app, and the line language joule-ui takes in
A design note. The normative text is [`SPEC.md`](../SPEC.md) §11 (the line language) and
the conformance pack in [`conformance/lang/`](../conformance/lang/). The WAI side is
[`wai/wai-app/`](../../wai/wai-app/).
**Create** is a page where a person describes a WAI app ("a show for my channel, with
chat and a live poll") and watches it build itself from WAI components while a model
streams its answer. The room it builds is a real room on a room service. The person then
exports it as a starter project.
## 1. The line language onto joule-ui's tree
A model writes OpenUI Lang (the open-source project's line-per-component language, MIT
licensed; see [`NOTICE`](../NOTICE)): one statement per line, `name = Component(arg, ...)`,
names usable before they are defined. joule-ui takes it in the way it takes in A2UI
(`joule-ui-a2ui`): the language is a way to write a joule-ui **widget tree** against a
host's **registry**, and what comes out is an ordinary `Widget`, checked by joule-ui's
own `validate`.
| The language | joule-ui |
|---|---|
| the library | the `Registry`, plus the name of its root widget |
| `Component(a, b, c)` | a `Widget`; the arguments map to props in the order the widget's schema lists (`WidgetSchema::positional`, new in core) |
| the prop named `children` in that order | `Widget::children`; a list of components |
| strings, numbers, booleans, `null`, lists, objects | `Text`, `Number`, `Bool`, (absent), `List`, `Object` |
| a reference to another statement | that statement's value (hoisting) |
| `@Name(args)` in a prop | `PropValue::Action { id: "Name", payload }` |
| `$state`, `Query`, `Mutation`, operators, `? :`, member access, built-in functions, a sequence of actions | not carried: joule-ui puts no expression and no behaviour on the wire. Each is parsed (the grammar is parsed whole) and refused `not-declarative`, the smallest piece dropped |
The crate is `joule-ui-lang`: a streaming parser, an ingest into widget trees validated
against a registry, and a canonical writer back to the language. Every error names the
line and the reason (`unknown-component`, `missing-required`, `type-mismatch`,
`disallowed-child`, `not-declarative`, ...; the table is SPEC §11.8).
## 2. Streaming
The text arrives in chunks. After each chunk the parser gives a **snapshot**: the tree as
far as it can be built, the errors so far, and the names still unresolved.
- **Complete and pending.** A statement is complete once its newline has arrived and the
next non-blank character has too and is not `?` (a line starting `?` would continue a
ternary). Complete statements never change. The last statement is pending: it is
closed for the moment (an open string closed, open brackets closed in reverse order)
and built, so a title shows letter by letter as it streams.
- **What renders before the tree is whole.** The tree starts at `root`. A reference to a
statement not yet written is unresolved: in a list it is left out, in a prop it is
absent. A widget whose required prop is absent only because something has not arrived
is held back, silently, until it can render. So the first line, `root = App([room],
"Friday show")`, renders an empty app at once, and each later line fills it in.
- **How validation works on a stream.** Every widget is validated against the registry
as it is built, by joule-ui's `validate`; a widget that fails is dropped, and its error
is reported once its statement is complete (a pending statement's problems may be gone
by its next chunk). The snapshot's whole tree passes `validate` before it is handed to
a renderer: no snapshot ever holds a widget the registry refuses. At the end of the
stream the held-back cases become errors (`missing-required`, `no-root`), and the
result equals parsing the whole text at once.
## 3. The WAI registry and the renderer
`wai-app` publishes a joule-ui registry of WAI widgets, each backed by the room client
of `joule-moq-web` (the page that joins rooms, verifies every object against its receipt,
watches together and keeps co-performance):
| Widget | What it is |
|---|---|
| `App(children, title)` | the root |
| `Room(children, title)` | a live room: join with camera and microphone, or watch |
| `WatchTogether(source, children, title)` | a watch room: everyone watches one file in sync |
| `Show(children, title, audience)` | a room with co-performance: hosts on camera, an audience that votes |
| `Player(title)` | the room's video, laid out from its signed record |
| `Chat(title)` | the room's chat events, each verified; a participant sends |
| `Poll(question, options, seconds)` | a vote: the room's clock opens the window and signs the selection |
| `Participants()` | who is present, from the room's lifecycle |
| `Receipts()` | this page's sealed interaction log, verified, and the room's verified objects |
| `EnergyMeter()` | what each interaction cost, with how the figure was obtained |
| `Text(text)` | a paragraph |
Each room kind accepts only the widgets its service keeps (a watch room carries co-watch
statements alone, so it accepts no `Chat` and no `Poll`); joule-ui's `AllowedChildren`
says so, and a line that puts a `Poll` in a `WatchTogether` is refused with its line and
`disallowed-child`. An app holds one room.
The renderer (`wai/wai-app/web/`) reconciles each snapshot into the page by key (a
statement's name), so a room already joined is never torn down by the next chunk. Every
action a person fires (join, leave, send a chat, vote, play) goes through
`joule-ui-interaction` in the page's WebAssembly build: authorized, metered (a page has
no meter, so the figure is the estimator's, from the handler's wall time, and labelled
so), and sealed into the page's hash-chained log under a key the page makes. `Receipts`
shows the log and the page's own `verify_chain` of it. A browser handler is asynchronous,
so the interaction crate gains a two-step form of its lifecycle (`begin_interaction`, then
`Pending::finish`) with the same three phases; a denied action yields no `Pending`, so
its handler cannot run.
## 4. Export
The page hands the person the program, `app.oui`, in the canonical form of the writer, and
the command that makes it a project on their machine: `wai create <dir> --from app.oui`
(the WAI toolchain's operation `app.export`). The project is an ordinary WAI project, the
same kind `wai create --template` writes. Its project file, `wai.json`, is written by the
one writer every WAI project file has, and names the program (`"app": "app.oui"`) and the
room `wai dev` runs for it (`"dev": {"mode": "room" | "watch" | "show", ...}`: a watch room's
file, a show's audience inputs). Beside it are the program, the page that renders it with
the same renderer and components, their runtime under `vendor/` (each file with its SHA-256
in `vendor.json`), a README, an AGENTS.md for the next person or coding agent, and
`LICENSE`: every project `wai create` writes is Apache-2.0, and says so. No key is in it.
It runs with `wai dev` (the relay, the room service and the page on one loopback origin, the
keys made in `.wai/`, which is never committed and never served), and `wai app check`
(`app.check`) checks it: `app.oui` against the registry, the room `wai.json` names against
the app's own, and the runtime file by file. The app's files are made in Rust by one module
(`wai_app::export`), and the project file by `wai`'s, whoever asks (`wai create --from`, or
the MCP server's `app_export`), so the project format changes in one place.
## 5. How a page talks to a model
Create spends nobody's money and holds no one's key on a server. Two ways in, and the
page says which is in use:
- **Your own model.** The visitor names an endpoint that streams the common
`chat/completions` server-sent-events format (a model server they run on their own
machine, or a provider they hold a key for), a model name and a key. The page calls
it from the browser. The key stays in the page's memory; it is never stored and never
sent anywhere else.
- **Your coding agent.** `wai mcp --toolset app` is the WAI MCP server the visitor's own
agent runs on their machine, with Create's toolset. Its tools give the agent the registry
and the language (`app_guide`), check a program (`app_check`), stream a program into the
visitor's open Create page through a bridge on `127.0.0.1` the page connects to with a
pairing code (`app_stream`), and write the starter project into the agent's workspace
(`app_export`, as `wai create --from app.oui` does).
A third mode replays a recorded stream, which is how the end-to-end test drives the
page. A model hosted by the site is out of scope until the owner decides it.

10
joule-ui/joule-ui-rs/Cargo.lock generated
View file

@ -347,6 +347,16 @@ dependencies = [
"thiserror",
]
[[package]]
name = "joule-ui-lang"
version = "0.1.0"
dependencies = [
"joule-ui-core",
"serde",
"serde_json",
"sha2",
]
[[package]]
name = "joule-ui-mcp"
version = "0.1.0"

1
joule-ui/joule-ui-rs/Cargo.toml
View file

@ -8,6 +8,7 @@ members = [
"crates/joule-ui-render",
"crates/joule-ui-serve",
"crates/joule-ui-mcp",
"crates/joule-ui-lang",
]
[workspace.package]

57
joule-ui/joule-ui-rs/crates/joule-ui-core/src/lib.rs
View file

@ -227,6 +227,15 @@ pub struct WidgetSchema {
/// name. Default is strict (`false`).
#[serde(default)]
pub allow_extra_props: bool,
/// The order a positional call passes this widget's props in (SPEC §3):
/// a language that writes `Card(title, [child])` maps its arguments to
/// these names in turn, the name `children` standing for the child list.
/// Empty when the host declares none; such a widget's order is its props
/// in key order, then `children` if it accepts any (SPEC §11.5). Absent
/// from the wire form when empty, so a registry that declares none
/// serialises as it did before.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub positional: Vec<String>,
}
impl WidgetSchema {
@ -237,6 +246,7 @@ impl WidgetSchema {
props: BTreeMap::new(),
children: AllowedChildren::None,
allow_extra_props: false,
positional: Vec::new(),
}
}
pub fn prop(mut self, name: impl Into<String>, schema: PropSchema) -> Self {
@ -251,6 +261,28 @@ impl WidgetSchema {
self.allow_extra_props = true;
self
}
/// Declare the order a positional call passes the props in (SPEC §3);
/// `children` names the child list.
pub fn positional<I, S>(mut self, names: I) -> Self
where
I: IntoIterator<Item = S>,
S: Into<String>,
{
self.positional = names.into_iter().map(Into::into).collect();
self
}
/// The positional order in effect: the declared one, or the props in key
/// order followed by `children` when the widget accepts any (SPEC §11.5).
pub fn positional_order(&self) -> Vec<String> {
if !self.positional.is_empty() {
return self.positional.clone();
}
let mut order: Vec<String> = self.props.keys().cloned().collect();
if !matches!(self.children, AllowedChildren::None) {
order.push("children".into());
}
order
}
}
/// The host's published widget catalog. Validation looks every widget
@ -886,6 +918,31 @@ mod tests {
assert!(!json.contains("\"children\""), "got: {json}");
}
#[test]
fn positional_order_is_declared_or_derived() {
let declared = WidgetSchema::new("Card", "c")
.prop("title", PropSchema::required(PropType::Text))
.children(AllowedChildren::Any)
.positional(["children", "title"]);
assert_eq!(declared.positional_order(), vec!["children", "title"]);
// None declared: the props in key order, then `children`.
let derived = WidgetSchema::new("Card", "c")
.prop("title", PropSchema::required(PropType::Text))
.prop("subtitle", PropSchema::optional(PropType::Text))
.children(AllowedChildren::Any);
assert_eq!(derived.positional_order(), vec!["subtitle", "title", "children"]);
let leaf = WidgetSchema::new("Text", "t").prop("body", PropSchema::required(PropType::Text));
assert_eq!(leaf.positional_order(), vec!["body"]);
// An undeclared order is absent from the wire form, so registries that
// declare none serialise exactly as before.
let j = serde_json::to_value(&derived).unwrap();
assert!(j.get("positional").is_none(), "{j}");
let j = serde_json::to_value(&declared).unwrap();
assert_eq!(j["positional"], serde_json::json!(["children", "title"]));
let back: WidgetSchema = serde_json::from_value(j).unwrap();
assert_eq!(back, declared);
}
#[test]
fn registry_round_trips_through_json() {
let r = registry_minimal();

195
joule-ui/joule-ui-rs/crates/joule-ui-interaction/src/lib.rs
View file

@ -228,6 +228,9 @@ pub struct Interaction {
/// meter gives none, its wall-clock time is estimated at the meter's
/// [`estimator_milliwatts`](Meter::estimator_milliwatts) and graded
/// `Estimator`, so a budget binds on every host.
///
/// This is [`begin_interaction`] and [`Pending::finish`] around a handler the
/// call runs itself, timed by the host's clock.
pub fn run_interaction<O, F>(
surface: &ContentId,
action: &str,
@ -241,14 +244,54 @@ where
O: AsRef<[u8]>,
F: FnOnce() -> O,
{
let pending = match begin_interaction(surface, action, actor, inputs, meter, authorizer.as_deref_mut()) {
Ok(p) => p,
Err(denied) => return *denied,
};
let started = std::time::Instant::now();
let output = handler();
let wall = started.elapsed();
pending.finish(output.as_ref(), meter, authorizer, wall)
}
/// An interaction that was authorized and whose meter's first sample was taken,
/// its handler not yet run: phase 1 of SPEC §6 done, phases 2 and 3 waiting on
/// the handler. For a host whose handler cannot run inside one call (a
/// browser's, which awaits the network): [`begin_interaction`] gives a `Pending`
/// only when the action is allowed, so a denied action leaves nothing to run a
/// handler under, and [`Pending::finish`] meters, charges and hashes exactly as
/// [`run_interaction`] does.
#[must_use = "a Pending is finished once its handler has run"]
#[derive(Debug)]
pub struct Pending {
surface: ContentId,
action: String,
actor: String,
inputs: ContentId,
grant_id: Option<String>,
before: Option<u64>,
}
/// Phase 1 of SPEC §6: hash the inputs and authorize before anything runs. An
/// allowed action gives a [`Pending`], its meter sampled; a denied one gives its
/// sealed-ready denial (outputs the genesis sentinel, 0 µJ `Estimator`, the
/// actor named), and no `Pending` whose handler could run.
pub fn begin_interaction(
surface: &ContentId,
action: &str,
actor: &str,
inputs: &[u8],
meter: &dyn Meter,
authorizer: Option<&mut (dyn Authorizer + '_)>,
) -> Result<Pending, Box<Interaction>> {
let inputs_id = ContentId::of_bytes(inputs);
let grant_id = authorizer.as_ref().and_then(|a| a.grant_id());
// Phase 1: authorize before running.
if let Some(auth) = authorizer.as_deref_mut()
if let Some(auth) = authorizer
&& let Err(reason) = auth.authorize(action)
{
return Interaction {
return Err(Box::new(Interaction {
surface: surface.clone(),
action: action.to_string(),
inputs: inputs_id,
@ -258,56 +301,81 @@ where
budget_remaining_uj: Some(0),
grant_id,
actor: actor.to_string(),
};
}));
}
// Phase 2: meter the handler. The reading is tagged with the meter's
// DECLARED provenance — a number from a model or a stub is never promoted
// to `HwShunt`, and a meter that declares nothing is `Estimator`.
// `measured_uj` is carried only for a hardware shunt; a model/estimator
// figure rides in `estimated_uj` (SPEC §5).
let before = meter.sample_uj();
let started = std::time::Instant::now();
let output = handler();
let wall = started.elapsed();
let after = meter.sample_uj();
let energy = match (before, after) {
(Some(a), Some(b)) => {
let uj = b.saturating_sub(a);
match meter.provenance() {
Provenance::HwShunt => Energy::measured(uj),
Provenance::ModelBased => Energy {
measured_uj: None,
estimated_uj: uj,
provenance: Provenance::ModelBased,
},
Provenance::Estimator => Energy {
measured_uj: None,
estimated_uj: uj,
provenance: Provenance::Estimator,
},
}
}
// No reading for work that ran: estimate it from the handler's own
// wall-clock time rather than seal 0 µJ, which would cost it nothing
// and let a budget never bind on a host without a counter (SPEC §6).
_ => Energy::estimator(estimate_uj(wall, meter.estimator_milliwatts())),
};
// Phase 3: charge the cost, with its provenance — an estimate is debited
// as an estimate, never handed on as a reading.
let budget_remaining_uj = authorizer.map(|a| a.charge(action, &energy));
Interaction {
Ok(Pending {
surface: surface.clone(),
action: action.to_string(),
inputs: inputs_id,
outputs: ContentId::of_bytes(output.as_ref()),
verdict: Verdict::Allow,
energy,
budget_remaining_uj,
grant_id,
actor: actor.to_string(),
inputs: inputs_id,
grant_id,
before: meter.sample_uj(),
})
}
impl Pending {
/// The action this interaction fires.
pub fn action(&self) -> &str {
&self.action
}
/// Phases 2 and 3 of SPEC §6, once the handler has run: the meter's second
/// sample, the energy graded by the meter's declared provenance (or, with no
/// reading, `wall` at the meter's estimator power, never 0 µJ), the cost
/// charged with its provenance, and the outputs hashed. `wall` is the
/// handler's wall-clock time on the host's own clock.
pub fn finish(
self,
output: &[u8],
meter: &dyn Meter,
authorizer: Option<&mut (dyn Authorizer + '_)>,
wall: std::time::Duration,
) -> Interaction {
// Phase 2: meter the handler. The reading is tagged with the meter's
// DECLARED provenance — a number from a model or a stub is never promoted
// to `HwShunt`, and a meter that declares nothing is `Estimator`.
// `measured_uj` is carried only for a hardware shunt; a model/estimator
// figure rides in `estimated_uj` (SPEC §5).
let after = meter.sample_uj();
let energy = match (self.before, after) {
(Some(a), Some(b)) => {
let uj = b.saturating_sub(a);
match meter.provenance() {
Provenance::HwShunt => Energy::measured(uj),
Provenance::ModelBased => Energy {
measured_uj: None,
estimated_uj: uj,
provenance: Provenance::ModelBased,
},
Provenance::Estimator => Energy {
measured_uj: None,
estimated_uj: uj,
provenance: Provenance::Estimator,
},
}
}
// No reading for work that ran: estimate it from the handler's own
// wall-clock time rather than seal 0 µJ, which would cost it nothing
// and let a budget never bind on a host without a counter (SPEC §6).
_ => Energy::estimator(estimate_uj(wall, meter.estimator_milliwatts())),
};
// Phase 3: charge the cost, with its provenance — an estimate is debited
// as an estimate, never handed on as a reading.
let budget_remaining_uj = authorizer.map(|a| a.charge(&self.action, &energy));
Interaction {
surface: self.surface,
action: self.action,
inputs: self.inputs,
outputs: ContentId::of_bytes(output),
verdict: Verdict::Allow,
energy,
budget_remaining_uj,
grant_id: self.grant_id,
actor: self.actor,
}
}
}
@ -748,6 +816,39 @@ mod tests {
assert_eq!(verify_chain(log.entries()), Ok(log.entries().len()));
}
/// The two-step lifecycle a browser host uses gives the interaction
/// `run_interaction` gives for the same handler, meter and budget.
#[test]
fn begin_and_finish_seal_what_run_interaction_seals() {
let mut b1 = budget(10_000, "g");
let one = run_interaction(&surface(), "apply", "did:key:zA", b"in", &hw_step(1500), Some(&mut b1), || b"out".to_vec());
let mut b2 = budget(10_000, "g");
let meter = hw_step(1500);
let p = begin_interaction(&surface(), "apply", "did:key:zA", b"in", &meter, Some(&mut b2)).expect("allowed");
assert_eq!(p.action(), "apply");
let two = p.finish(b"out", &meter, Some(&mut b2), std::time::Duration::from_millis(3));
assert_eq!(one, two);
assert_eq!((b1.remaining, b1.charged), (b2.remaining, b2.charged));
// With no reading, the host's wall time is the estimate's input.
let p = begin_interaction(&surface(), "apply", "did:key:zA", b"in", &NoMeter, None).expect("allowed");
let it = p.finish(b"out", &NoMeter, None, std::time::Duration::from_millis(2));
assert_eq!(it.energy, Energy::estimator(40_000)); // 2 ms at 20 W
}
/// A denied action gives no `Pending`, so there is nothing a handler could
/// run under; the denial is the one `run_interaction` seals.
#[test]
fn a_denied_begin_gives_no_pending() {
let mut b = budget(0, "grant-xyz");
let denied = begin_interaction(&surface(), "apply", "did:key:zA", b"{}", &NoMeter, Some(&mut b)).unwrap_err();
let mut b2 = budget(0, "grant-xyz");
let same = run_interaction(&surface(), "apply", "did:key:zA", b"{}", &NoMeter, Some(&mut b2), || b"x".to_vec());
assert_eq!(*denied, same);
assert_eq!(denied.outputs, ContentId::genesis());
assert!(b.charged.is_empty());
}
/// The authorizer is handed the figure the interaction is sealed with, and
/// its provenance: a reading as a reading, an estimate as an estimate.
#[test]

21
joule-ui/joule-ui-rs/crates/joule-ui-lang/Cargo.toml Normal file
View file

@ -0,0 +1,21 @@
[package]
name = "joule-ui-lang"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
authors.workspace = true
description = "The line language ingest for joule-ui (SPEC §11): a streaming parser for the line-per-component UI language (OpenUI Lang) that builds joule-ui widget trees, each widget validated against the host's registry as it arrives, with errors that name the line and the reason; and a canonical writer from a widget tree back to the language."
[dependencies]
joule-ui-core = { path = "../joule-ui-core" }
serde = { workspace = true }
serde_json = { workspace = true }
# The pack manifest's digests (SPEC §11.9), SHA-256 so the independent verifier needs
# nothing outside the Python standard library. Already in the lockfile through ed25519-dalek.
sha2 = "0.10"
[[bin]]
name = "gen_lang_vectors"
path = "src/bin/gen_lang_vectors.rs"

17
joule-ui/joule-ui-rs/crates/joule-ui-lang/src/bin/gen_lang_vectors.rs Normal file
View file

@ -0,0 +1,17 @@
//! Write the line language's conformance pack (SPEC §11.9) to
//! `joule-ui/conformance/lang/v1/`, or to the directory given.
//!
//! cargo run -p joule-ui-lang --bin gen_lang_vectors [-- <dir>]
use std::path::PathBuf;
fn main() {
let dir = std::env::args().nth(1).map(PathBuf::from).unwrap_or_else(|| {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../conformance/lang/v1")
});
std::fs::create_dir_all(&dir).expect("the pack directory");
for (name, bytes) in joule_ui_lang::vectors::pack() {
std::fs::write(dir.join(&name), &bytes).expect("a pack file");
println!("{name} {} bytes", bytes.len());
}
}

643
joule-ui/joule-ui-rs/crates/joule-ui-lang/src/ingest.rs Normal file
View file

@ -0,0 +1,643 @@
//! From statements to a validated joule-ui tree (SPEC §11.5, §11.6).
use crate::lex::lex;
use crate::parse::{close_implicitly, parse_expr, split, Expr, Head, E};
use crate::prepare::{prepare, Prepared};
use crate::{signature_of, Code, LangError, Library, Node, Snapshot, Statement};
use joule_ui_core::{validate, AllowedChildren, PropValue, ValidationError, Widget};
use std::collections::{BTreeMap, BTreeSet, HashMap};
/// The built-in functions: they compute values, which joule-ui does not carry.
pub(crate) const BUILTINS: [&str; 14] = ["Count", "First", "Last", "Sum", "Avg", "Min", "Max", "Filter", "Sort", "Round", "Abs", "Floor", "Ceil", "Each"];
/// The built-in actions that change state.
pub(crate) const STATE_ACTIONS: [&str; 3] = ["Set", "Reset", "Run"];
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Kind {
Value,
State,
Query,
Mutation,
}
impl Kind {
fn as_str(self) -> &'static str {
match self {
Kind::Value => "value",
Kind::State => "state",
Kind::Query => "query",
Kind::Mutation => "mutation",
}
}
}
struct Bound {
line: u32,
complete: bool,
kind: Kind,
expr: E,
order: usize,
}
/// A value as a program evaluates it, before it is placed in a prop.
enum V {
Data(PropValue),
Null,
List(Vec<V>),
Object(Vec<(String, V)>),
Widget(Built),
Action(PropValue),
/// A reference to nothing written (yet), or one that closes a cycle.
Unresolved,
/// A widget held back: a required prop refers to something not yet written.
Held,
/// Refused, its error recorded (or a widget dropped).
Refused,
}
struct Built {
widget: Widget,
nodes: Vec<Node>,
}
/// Why a prop is absent.
#[derive(Clone, Copy, PartialEq, Eq)]
enum Absent {
NotYet,
Null,
Refused,
}
pub(crate) fn snapshot(lib: &Library, text: &str, finished: bool) -> Snapshot {
let prep = prepare(text);
let tokens = lex(&prep.text);
let raws = split(&tokens, finished);
let incomplete = raws.last().is_some_and(|r| r.open);
let mut errors: Vec<LangError> = Vec::new();
let mut statements: Vec<Statement> = Vec::new();
let mut binds: HashMap<String, Bound> = HashMap::new();
for (order, raw) in raws.iter().enumerate() {
let toks = if raw.open { close_implicitly(&raw.tokens) } else { raw.tokens.clone() };
let line = prep.source_line(raw.line);
let expr = match parse_expr(&toks) {
Ok(e) => e,
Err(pe) => {
// A statement that does not parse is not produced; a complete one
// is reported (SPEC §11.6 rule 2).
if raw.complete {
errors.push(LangError {
line: if toks.is_empty() { line } else { prep.source_line(pe.line) },
statement: raw.name.clone(),
code: Code::Syntax,
component: None,
prop: None,
message: format!("`{}` does not parse: {}", raw.name, pe.message),
warning: false,
});
statements.push(Statement { name: raw.name.clone(), line, kind: "syntax".into(), complete: true });
}
continue;
}
};
let kind = match &expr.expr {
Expr::Call { head: Head::Component, name, .. } if name == "Query" => Kind::Query,
Expr::Call { head: Head::Component, name, .. } if name == "Mutation" => Kind::Mutation,
_ if raw.state => Kind::State,
_ => Kind::Value,
};
statements.push(Statement { name: raw.name.clone(), line, kind: kind.as_str().into(), complete: raw.complete });
if raw.complete && kind != Kind::Value {
let what = match kind {
Kind::State => format!("`{}` is state; joule-ui carries no state (the host owns behaviour)", raw.name),
Kind::Query => format!("`{}` is a query; joule-ui carries no query (the host owns its data)", raw.name),
_ => format!("`{}` is a mutation; joule-ui carries no mutation (the host owns behaviour)", raw.name),
};
errors.push(LangError { line, statement: raw.name.clone(), code: Code::NotDeclarative, component: None, prop: None, message: what, warning: false });
}
// A later statement replaces an earlier one of the same name, except that
// a pending statement that had to be closed implicitly never replaces a
// complete one (SPEC §11.6 rule 3).
if let Some(prev) = binds.get(&raw.name)
&& prev.complete
&& !raw.complete
&& raw.open
{
continue;
}
binds.insert(raw.name.clone(), Bound { line, complete: raw.complete, kind, expr, order });
}
let mut ev = Ev {
lib,
prep: &prep,
binds: &binds,
finished,
errors: Vec::new(),
unresolved: BTreeSet::new(),
visited: BTreeSet::new(),
resolving: Vec::new(),
occurrences: HashMap::new(),
};
// The entry: the `root` statement, else the first value statement whose
// value calls the library's root widget (SPEC §11.5).
let entry: Option<(String, bool)> = if binds.contains_key("root") {
Some(("root".into(), false))
} else {
lib.root.as_ref().and_then(|r| {
let mut vals: Vec<(&String, &Bound)> = binds.iter().filter(|(_, b)| b.kind == Kind::Value).collect();
vals.sort_by_key(|(_, b)| b.order);
vals.into_iter()
.find(|(_, b)| matches!(&b.expr.expr, Expr::Call { head: Head::Component, name, .. } if name == r))
.map(|(n, _)| (n.clone(), true))
})
};
let mut tree: Option<Widget> = None;
let mut nodes: Vec<Node> = Vec::new();
if let Some((name, _)) = &entry {
let line = binds[name].line;
if let V::Widget(b) = ev.reference(name, line, "") {
tree = Some(b.widget);
nodes = b.nodes;
}
}
errors.append(&mut ev.errors);
if finished {
match &entry {
None => {
if statements.is_empty() && !prep.text.is_empty() {
errors.push(whole(Code::ParseFailed, "the text holds no statement (`name = value`, one a line)".into(), false));
} else {
let m = match &lib.root {
Some(r) => format!("no `root` statement, and no statement whose value calls `{r}`: nothing renders"),
None => "no `root` statement: nothing renders".into(),
};
errors.push(whole(Code::NoRoot, m, false));
}
}
Some((name, fallback)) => {
let line = binds[name].line;
if tree.is_none() {
errors.push(LangError {
line,
statement: name.clone(),
code: Code::NoRoot,
component: None,
prop: None,
message: format!("`{name}` is not a widget that renders: nothing renders"),
warning: false,
});
} else if *fallback {
errors.push(LangError {
line,
statement: name.clone(),
code: Code::NoRoot,
component: None,
prop: None,
message: format!("no `root` statement: `{name}`, the first to call the root widget, renders"),
warning: true,
});
}
}
}
}
// A pending statement's errors are withheld until it completes (it may be
// whole by the next chunk).
if !finished {
errors.retain(|e| binds.get(&e.statement).is_none_or(|b| b.complete));
}
// By line, statement, code, widget, prop and warning; one of each.
errors.sort_by(|a, b| sort_key(a).cmp(&sort_key(b)).then_with(|| a.message.cmp(&b.message)));
errors.dedup_by(|a, b| sort_key(a) == sort_key(b));
let orphans: Vec<String> = if entry.is_some() {
let mut v: Vec<String> = binds.iter().filter(|(n, b)| b.kind == Kind::Value && !ev.visited.contains(*n)).map(|(n, _)| n.clone()).collect();
v.sort();
v
} else {
Vec::new()
};
// Every tree a snapshot holds passes the registry's validation as a whole.
if let Some(t) = &tree {
let left = validate(t, &lib.registry);
debug_assert!(left.is_empty(), "the ingest built a tree its registry refuses: {left:?}");
if !left.is_empty() {
tree = None;
nodes.clear();
}
}
Snapshot { tree, nodes, errors, unresolved: ev.unresolved.into_iter().collect(), orphans, statements, finished, incomplete }
}
type ErrorKey<'a> = (u32, &'a str, &'static str, Option<&'a str>, Option<&'a str>, bool);
fn sort_key(e: &LangError) -> ErrorKey<'_> {
(e.line, e.statement.as_str(), e.code.as_str(), e.component.as_deref(), e.prop.as_deref(), e.warning)
}
fn whole(code: Code, message: String, warning: bool) -> LangError {
LangError { line: 0, statement: String::new(), code, component: None, prop: None, message, warning }
}
struct Ev<'a> {
lib: &'a Library,
prep: &'a Prepared,
binds: &'a HashMap<String, Bound>,
finished: bool,
errors: Vec<LangError>,
unresolved: BTreeSet<String>,
visited: BTreeSet<String>,
resolving: Vec<String>,
occurrences: HashMap<String, usize>,
}
impl Ev<'_> {
fn err(&mut self, stmt: &str, line: usize, code: Code, component: Option<&str>, prop: Option<&str>, message: String) {
self.errors.push(LangError {
line: self.prep.source_line(line),
statement: stmt.to_string(),
code,
component: component.map(String::from),
prop: prop.map(String::from),
message,
warning: false,
});
}
/// A reference to a statement: its value, evaluated where it is used.
fn reference(&mut self, name: &str, _line: u32, _from: &str) -> V {
let Some(b) = self.binds.get(name) else {
self.unresolved.insert(name.to_string());
return V::Unresolved;
};
if self.resolving.iter().any(|r| r == name) {
// A reference that closes a cycle is unresolved at that point.
self.unresolved.insert(name.to_string());
return V::Unresolved;
}
if b.kind != Kind::Value {
// State, a query, a mutation: refused at the statement already.
return V::Refused;
}
self.visited.insert(name.to_string());
let n = self.occurrences.entry(name.to_string()).or_insert(0);
*n += 1;
let key = if *n == 1 { name.to_string() } else { format!("{name}#{n}") };
self.resolving.push(name.to_string());
let v = self.eval(&b.expr, name, &key);
self.resolving.pop();
v
}
fn eval(&mut self, e: &E, stmt: &str, key: &str) -> V {
match &e.expr {
Expr::Str(s) => V::Data(PropValue::Text(s.clone())),
Expr::Num(n) => V::Data(PropValue::Number(*n)),
Expr::Bool(b) => V::Data(PropValue::Bool(*b)),
Expr::Null => V::Null,
Expr::Ref(name) => self.reference(name, 0, stmt),
Expr::State(name) => {
self.err(stmt, e.line, Code::NotDeclarative, None, None, format!("`{name}` is state; joule-ui carries no state (the host owns behaviour)"));
V::Refused
}
Expr::Array(items) => V::List(items.iter().enumerate().map(|(i, it)| self.eval(it, stmt, &format!("{key}/{i}"))).collect()),
Expr::Object(entries) => V::Object(entries.iter().map(|(k, v)| (k.clone(), self.eval(v, stmt, &format!("{key}/{k}")))).collect()),
Expr::Call { head: Head::Component, name, args } => {
if name == "Query" || name == "Mutation" {
self.err(stmt, e.line, Code::InlineReserved, None, None, format!("`{name}` is a statement of its own, not a value"));
return V::Refused;
}
if name == "Action" {
return self.action_list(args, stmt, e.line, key);
}
self.build(name, args, e.line, stmt, key)
}
Expr::Call { head: Head::At, name, args } => {
if BUILTINS.contains(&name.as_str()) {
self.err(stmt, e.line, Code::NotDeclarative, None, None, format!("`@{name}` computes a value; joule-ui carries no expression: write the value"));
return V::Refused;
}
if STATE_ACTIONS.contains(&name.as_str()) {
self.err(stmt, e.line, Code::NotDeclarative, None, None, format!("`@{name}` changes state; joule-ui carries no state (the host owns behaviour)"));
return V::Refused;
}
if name == "Action" {
return self.action_list(args, stmt, e.line, key);
}
self.action(name, args, stmt, key)
}
Expr::Unary(op, _) => self.expression(stmt, e.line, &format!("the operator `{op}`")),
Expr::Binary(op, _, _) => self.expression(stmt, e.line, &format!("the operator `{op}`")),
Expr::Ternary(..) => self.expression(stmt, e.line, "a conditional (`? :`)"),
Expr::Member(_, f) => self.expression(stmt, e.line, &format!("member access (`.{f}`)")),
Expr::Index(..) => self.expression(stmt, e.line, "indexing (`[...]` after a value)"),
Expr::Assign(name, _) => self.expression(stmt, e.line, &format!("an assignment to `{name}`")),
}
}
fn expression(&mut self, stmt: &str, line: usize, what: &str) -> V {
self.err(stmt, line, Code::NotDeclarative, None, None, format!("{what} is an expression; joule-ui carries no expression: write the value"));
V::Refused
}
/// `@Name(args)`: an action, its arguments its payload (none; the one; or a
/// list of them, SPEC §11.5).
fn action(&mut self, name: &str, args: &[E], stmt: &str, key: &str) -> V {
// Every argument is evaluated, so every reference in it is reached.
let mut items = Vec::new();
let (mut not_yet, mut refused) = (false, false);
for (i, a) in args.iter().enumerate() {
let v = self.eval(a, stmt, &format!("{key}/{i}"));
match self.data(v, stmt, a.line, true) {
Ok(Some(p)) => items.push(p),
Ok(None) => not_yet = true,
Err(()) => refused = true,
}
}
if refused {
return V::Refused;
}
if not_yet {
return V::Held;
}
let payload = match items.len() {
0 => None,
1 => Some(Box::new(items.pop().expect("one"))),
_ => Some(Box::new(PropValue::List(items))),
};
V::Action(PropValue::Action { id: name.to_string(), payload })
}
/// `Action([one])` is that one action; any other list is a sequence, which is
/// behaviour.
fn action_list(&mut self, args: &[E], stmt: &str, line: usize, key: &str) -> V {
if let [E { expr: Expr::Array(items), .. }] = args
&& let [one] = items.as_slice()
{
return match self.eval(one, stmt, &format!("{key}/0")) {
V::Action(p) => V::Action(p),
V::Unresolved | V::Held => V::Held,
V::Refused => V::Refused,
_ => {
self.err(stmt, one.line, Code::TypeMismatch, None, None, "`Action([...])` holds actions; this is not one".into());
V::Refused
}
};
}
self.err(stmt, line, Code::NotDeclarative, None, None, "a sequence of actions is behaviour the host owns; write one action".into());
V::Refused
}
/// A value placed where data goes: `Ok(None)` for what is not yet written
/// (left out), `Err` for what was refused (its error recorded). `keep_null`:
/// a `null` is a value here (in a list, an object, an action's arguments).
fn data(&mut self, v: V, stmt: &str, line: usize, keep_null: bool) -> Result<Option<PropValue>, ()> {
match v {
V::Data(p) | V::Action(p) => Ok(Some(p)),
V::Null => Ok(if keep_null { Some(PropValue::Null) } else { None }),
V::List(items) => {
let mut out = Vec::new();
for it in items {
if let Ok(Some(p)) = self.data(it, stmt, line, true) {
out.push(p);
}
}
Ok(Some(PropValue::List(out)))
}
V::Object(entries) => {
let mut out = BTreeMap::new();
for (k, it) in entries {
if let Ok(Some(p)) = self.data(it, stmt, line, true) {
out.insert(k, p);
}
}
Ok(Some(PropValue::Object(out)))
}
V::Widget(b) => {
self.err(stmt, line, Code::TypeMismatch, Some(&b.widget.name), None, format!("a widget (`{}`) stands where data goes", b.widget.name));
Err(())
}
V::Unresolved | V::Held => Ok(None),
V::Refused => Err(()),
}
}
/// A call of a registry widget: its arguments mapped to props in its
/// positional order, then validated by joule-ui, the smallest piece dropped.
fn build(&mut self, name: &str, args: &[E], line: usize, stmt: &str, key: &str) -> V {
let lib = self.lib;
let Some(schema) = lib.registry.get(name) else {
let have = lib.names().join(", ");
self.err(stmt, line, Code::UnknownComponent, Some(name), None, format!("`{name}` is not a widget of this registry; it has {have}"));
return V::Refused;
};
let order = schema.positional_order();
let sig = signature_of(schema);
if args.len() > order.len() {
self.err(
stmt,
args[order.len()].line,
Code::ExcessArgs,
Some(name),
None,
format!("`{name}` takes {} argument{}, {sig}; the extra ones are dropped", order.len(), if order.len() == 1 { "" } else { "s" }),
);
}
let mut props: BTreeMap<String, PropValue> = BTreeMap::new();
let mut absent: HashMap<String, Absent> = HashMap::new();
let mut children: Vec<(Widget, Vec<Node>)> = Vec::new();
for (slot, arg) in order.iter().zip(args.iter()) {
if slot == "children" {
match self.eval(arg, stmt, key) {
V::List(items) => {
for it in items {
match it {
V::Widget(b) => children.push((b.widget, b.nodes)),
V::Unresolved | V::Held | V::Null | V::Refused => {}
_ => self.err(stmt, arg.line, Code::TypeMismatch, Some(name), Some("children"), format!("the children of `{name}` are widgets; data stands among them")),
}
}
}
V::Null | V::Unresolved | V::Held | V::Refused => {}
V::Widget(b) => self.err(
stmt,
arg.line,
Code::TypeMismatch,
Some(name),
Some("children"),
format!("the children of `{name}` are a list: write `[{}]`", b.nodes.first().map_or("child", |n| n.key.as_str())),
),
_ => self.err(stmt, arg.line, Code::TypeMismatch, Some(name), Some("children"), format!("the children of `{name}` are a list of widgets")),
}
continue;
}
let v = self.eval(arg, stmt, &format!("{key}/{slot}"));
match v {
V::Null => {
absent.insert(slot.clone(), Absent::Null);
}
V::Unresolved | V::Held => {
absent.insert(slot.clone(), Absent::NotYet);
}
V::Refused => {
absent.insert(slot.clone(), Absent::Refused);
}
V::Widget(b) => {
self.err(stmt, arg.line, Code::TypeMismatch, Some(name), Some(slot), format!("`{slot}` of `{name}` takes data; a widget (`{}`) stands there", b.widget.name));
absent.insert(slot.clone(), Absent::Refused);
}
other => match self.data(other, stmt, arg.line, false) {
Ok(Some(p)) => {
props.insert(slot.clone(), p);
}
Ok(None) => {
absent.insert(slot.clone(), Absent::NotYet);
}
Err(()) => {
absent.insert(slot.clone(), Absent::Refused);
}
},
}
}
let mut widget = Widget { name: name.to_string(), props, children: Vec::new() };
let mut child_nodes: Vec<Vec<Node>> = Vec::new();
for (w, n) in children {
widget.children.push(w);
child_nodes.push(n);
}
// joule-ui's own validation decides; each error drops the smallest piece.
let mut props_before = widget.props.len();
loop {
let errs = validate(&widget, &lib.registry);
if errs.is_empty() {
break;
}
// Held back, silently, while a required prop is not yet written.
if !self.finished
&& errs.iter().any(|e| matches!(e, ValidationError::MissingProp { prop, path, .. } if path == "$" && absent.get(prop) == Some(&Absent::NotYet)))
{
return V::Held;
}
let mut drop_widget = false;
let mut drop_children: Vec<usize> = Vec::new();
for e in errs {
match e {
ValidationError::MissingProp { prop, path, .. } if path == "$" => match absent.get(&prop) {
Some(Absent::NotYet) => {
self.err(stmt, line, Code::MissingRequired, Some(name), Some(&prop), format!("`{name}` needs `{prop}`, and it refers to something never written: {sig}"));
drop_widget = true;
}
Some(Absent::Null) => {
self.err(stmt, line, Code::NullRequired, Some(name), Some(&prop), format!("`{prop}` of `{name}` is required and written `null`: {sig}"));
drop_widget = true;
}
// Its value was refused, and that refusal is the error
// reported: the widget goes with it.
Some(Absent::Refused) => drop_widget = true,
None => {
self.err(stmt, line, Code::MissingRequired, Some(name), Some(&prop), format!("`{name}` needs `{prop}`: {sig}"));
drop_widget = true;
}
},
ValidationError::WrongPropType { prop, expected, got, path, .. } if path == "$" => {
let required = schema.props.get(&prop).is_some_and(|p| p.required);
self.err(
stmt,
line,
Code::TypeMismatch,
Some(name),
Some(&prop),
format!("`{prop}` of `{name}` is {}, not {}{}", ty_name(expected), ty_name(got), if required { "" } else { "; it is left out" }),
);
if required {
drop_widget = true;
} else {
widget.props.remove(&prop);
}
}
ValidationError::ExtraProp { prop, path, .. } if path == "$" => {
self.err(stmt, line, Code::TypeMismatch, Some(name), Some(&prop), format!("`{name}` declares no prop `{prop}`"));
widget.props.remove(&prop);
}
ValidationError::DisallowedChild { child, path, .. } => {
let Some(i) = child_index(&path) else {
drop_widget = true;
continue;
};
let accepts = match &schema.children {
AllowedChildren::OnlyOf(list) => format!("it accepts {}", list.join(", ")),
AllowedChildren::None => "it accepts no children".into(),
AllowedChildren::Any => "it accepts any".into(),
};
let at = child_nodes.get(i).and_then(|n| n.first()).map(|n| n.line).unwrap_or(0);
self.errors.push(LangError {
line: if at > 0 { at } else { self.prep.source_line(line) },
statement: child_nodes.get(i).and_then(|n| n.first()).map_or_else(|| stmt.to_string(), |n| n.statement.clone()),
code: Code::DisallowedChild,
component: Some(child.clone()),
prop: None,
message: format!("`{name}` does not accept `{child}`: {accepts}"),
warning: false,
});
drop_children.push(i);
}
// A child is valid before its parent holds it, so nothing below
// the parent can fail here; if it did, the parent goes.
_ => drop_widget = true,
}
}
if drop_widget {
return V::Refused;
}
drop_children.sort_unstable();
drop_children.dedup();
let changed = !drop_children.is_empty() || widget.props.len() != props_before;
for i in drop_children.into_iter().rev() {
widget.children.remove(i);
child_nodes.remove(i);
}
if !changed {
// Nothing left to drop but the widget itself.
return V::Refused;
}
props_before = widget.props.len();
}
let mut nodes = vec![Node { key: key.to_string(), line: self.prep.source_line(line), statement: stmt.to_string() }];
for n in child_nodes {
nodes.extend(n);
}
V::Widget(Built { widget, nodes })
}
}
fn child_index(path: &str) -> Option<usize> {
let rest = path.strip_prefix("$.children[")?;
let end = rest.find(']')?;
if end + 1 != rest.len() {
return None;
}
rest[..end].parse().ok()
}
fn ty_name(t: joule_ui_core::PropType) -> &'static str {
use joule_ui_core::PropType as T;
match t {
T::Null => "null",
T::Bool => "a boolean",
T::Number => "a number",
T::Text => "text",
T::List => "a list",
T::Object => "an object",
T::Action => "an action",
T::Any => "any value",
}
}

351
joule-ui/joule-ui-rs/crates/joule-ui-lang/src/lex.rs Normal file
View file

@ -0,0 +1,351 @@
//! Lexing (SPEC §11.2). Lexing never fails: a character that matches no rule
//! is skipped. A string still open at the end of the text is closed there, and
//! its token says so.
/// A token's kind.
#[derive(Debug, Clone, PartialEq)]
pub enum Tok {
/// An identifier: a component name (uppercase first letter) or a
/// reference (lowercase or `_`).
Ident(String),
/// A state name, `$` included.
State(String),
/// An `@` name, the `@` left off.
At(String),
Str(String),
/// A number and the text it was written as.
Num(f64, String),
/// A number whose exponent marker has no digits after it: not usable.
BadNum(String),
True,
False,
Null,
/// An operator or a punctuation mark: `=` `==` `!=` `>` `<` `>=` `<=` `+`
/// `-` `*` `/` `%` `&&` `||` `!` `?` `:` `.` `,` `(` `)` `[` `]` `{` `}`.
P(&'static str),
Newline,
}
impl Tok {
/// Whether this token ends a value: a literal, a name, or a closing bracket.
/// A `-` after one is subtraction, not a sign (SPEC §11.2).
pub fn ends_value(&self) -> bool {
matches!(
self,
Tok::Ident(_) | Tok::State(_) | Tok::Str(_) | Tok::Num(..) | Tok::BadNum(_) | Tok::True | Tok::False | Tok::Null | Tok::P(")") | Tok::P("]")
)
}
}
/// A token, the prepared line it starts on (0-based), and, for a string, whether
/// the text ended inside it.
#[derive(Debug, Clone, PartialEq)]
pub struct Token {
pub tok: Tok,
pub line: usize,
pub unterminated: bool,
}
fn ident_start(c: char) -> bool {
c.is_ascii_alphabetic() || c == '_'
}
fn ident_char(c: char) -> bool {
c.is_ascii_alphanumeric() || c == '_'
}
/// Lex the prepared text.
pub fn lex(text: &str) -> Vec<Token> {
let cs: Vec<char> = text.chars().collect();
let mut out: Vec<Token> = Vec::new();
let mut i = 0;
let mut line = 0usize;
let n = cs.len();
while i < n {
let c = cs[i];
let start_line = line;
let push = |tok: Tok, out: &mut Vec<Token>| out.push(Token { tok, line: start_line, unterminated: false });
match c {
' ' | '\t' => {
i += 1;
}
'\n' => {
push(Tok::Newline, &mut out);
line += 1;
i += 1;
}
'"' | '\'' => {
let (tok, used, lines, open) = string(&cs[i..], c);
out.push(Token { tok, line: start_line, unterminated: open });
line += lines;
i += used;
}
'$' if cs.get(i + 1).is_some_and(|&d| ident_start(d)) => {
let mut j = i + 1;
while j < n && ident_char(cs[j]) {
j += 1;
}
push(Tok::State(cs[i..j].iter().collect()), &mut out);
i = j;
}
'@' if cs.get(i + 1).is_some_and(|&d| ident_start(d)) => {
let mut j = i + 1;
while j < n && ident_char(cs[j]) {
j += 1;
}
push(Tok::At(cs[i + 1..j].iter().collect()), &mut out);
i = j;
}
c if ident_start(c) => {
let mut j = i;
while j < n && ident_char(cs[j]) {
j += 1;
}
let word: String = cs[i..j].iter().collect();
let tok = match word.as_str() {
"true" => Tok::True,
"false" => Tok::False,
"null" => Tok::Null,
_ => Tok::Ident(word),
};
push(tok, &mut out);
i = j;
}
c if c.is_ascii_digit()
|| (c == '-'
&& cs.get(i + 1).is_some_and(|d| d.is_ascii_digit())
&& !out.last().is_some_and(|t| t.tok.ends_value())) =>
{
let (tok, used) = number(&cs[i..]);
push(tok, &mut out);
i += used;
}
_ => {
let two: String = cs[i..(i + 2).min(n)].iter().collect();
let p2 = match two.as_str() {
"==" => Some("=="),
"!=" => Some("!="),
">=" => Some(">="),
"<=" => Some("<="),
"&&" => Some("&&"),
"||" => Some("||"),
_ => None,
};
if let Some(p) = p2 {
push(Tok::P(p), &mut out);
i += 2;
continue;
}
let p1 = match c {
'=' => Some("="),
'>' => Some(">"),
'<' => Some("<"),
'+' => Some("+"),
'-' => Some("-"),
'*' => Some("*"),
'/' => Some("/"),
'%' => Some("%"),
'&' => Some("&&"),
'|' => Some("||"),
'!' => Some("!"),
'?' => Some("?"),
':' => Some(":"),
'.' => Some("."),
',' => Some(","),
'(' => Some("("),
')' => Some(")"),
'[' => Some("["),
']' => Some("]"),
'{' => Some("{"),
'}' => Some("}"),
_ => None,
};
if let Some(p) = p1 {
push(Tok::P(p), &mut out);
}
// Anything else is skipped: lexing never fails.
i += 1;
}
}
}
out
}
/// A number from `-?[0-9]+(\.[0-9]+)?([eE][+-]?[0-9]+)?`, with an `e` or `E`
/// after the digits always taken as the exponent marker (SPEC §11.2).
fn number(cs: &[char]) -> (Tok, usize) {
let mut j = 0;
if cs[0] == '-' {
j = 1;
}
while j < cs.len() && cs[j].is_ascii_digit() {
j += 1;
}
if j + 1 < cs.len() && cs[j] == '.' && cs[j + 1].is_ascii_digit() {
j += 1;
while j < cs.len() && cs[j].is_ascii_digit() {
j += 1;
}
}
let mut bad = false;
if j < cs.len() && (cs[j] == 'e' || cs[j] == 'E') {
j += 1;
if j < cs.len() && (cs[j] == '+' || cs[j] == '-') {
j += 1;
}
let digits = j;
while j < cs.len() && cs[j].is_ascii_digit() {
j += 1;
}
bad = j == digits;
}
let text: String = cs[..j].iter().collect();
if bad {
return (Tok::BadNum(text), j);
}
let v: f64 = text.parse().unwrap_or(0.0);
(Tok::Num(v, text), j)
}
/// A string from its opening quote: the token, the characters used, the line
/// feeds inside it, and whether the text ended inside it (it is then closed
/// there, SPEC §11.2).
fn string(cs: &[char], q: char) -> (Tok, usize, usize, bool) {
let mut j = 1;
let mut lines = 0;
let mut closed = false;
while j < cs.len() {
let c = cs[j];
if c == '\\' {
if j + 1 < cs.len() && cs[j + 1] == '\n' {
lines += 1;
}
j += 2;
continue;
}
if c == '\n' {
lines += 1;
}
if c == q {
closed = true;
break;
}
j += 1;
}
let end = j.min(cs.len());
let mut raw: String = cs[1..end].iter().collect();
let used = if closed { end + 1 } else { cs.len() };
if !closed {
// Ended mid-escape: a backslash closes the escape first.
let trailing = raw.chars().rev().take_while(|&c| c == '\\').count();
if trailing % 2 == 1 {
raw.push('\\');
}
}
let value = if q == '"' { decode_dq(&raw) } else { decode_sq(&raw) };
(Tok::Str(value), used, lines, !closed)
}
/// A double-quoted string's content decoded as a JSON string (RFC 8259 §7); a
/// content that is not one (an unknown escape, a raw control character, a
/// short or lone-surrogate `\u` escape) is taken as written (SPEC §11.2).
pub fn decode_dq(raw: &str) -> String {
let quoted = format!("\"{raw}\"");
match serde_json::from_str::<String>(&quoted) {
Ok(s) => s,
Err(_) => raw.to_string(),
}
}
/// A single-quoted string's content: `\'`, `\\`, `\n` and `\t` are escapes;
/// a backslash before any other character is dropped and the character kept.
pub fn decode_sq(raw: &str) -> String {
let mut out = String::with_capacity(raw.len());
let mut it = raw.chars();
while let Some(c) = it.next() {
if c != '\\' {
out.push(c);
continue;
}
match it.next() {
Some('n') => out.push('\n'),
Some('t') => out.push('\t'),
Some(o) => out.push(o),
None => {}
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
fn toks(s: &str) -> Vec<Tok> {
lex(s).into_iter().map(|t| t.tok).collect()
}
#[test]
fn names_numbers_strings() {
assert_eq!(
toks("x = Card(\"hi\", 12.5, $s, @Go)"),
vec![
Tok::Ident("x".into()),
Tok::P("="),
Tok::Ident("Card".into()),
Tok::P("("),
Tok::Str("hi".into()),
Tok::P(","),
Tok::Num(12.5, "12.5".into()),
Tok::P(","),
Tok::State("$s".into()),
Tok::P(","),
Tok::At("Go".into()),
Tok::P(")"),
]
);
}
#[test]
fn minus_is_a_sign_only_after_no_value() {
assert_eq!(toks("[1 -2]"), vec![Tok::P("["), Tok::Num(1.0, "1".into()), Tok::P("-"), Tok::Num(2.0, "2".into()), Tok::P("]")]);
assert_eq!(toks("[-2]"), vec![Tok::P("["), Tok::Num(-2.0, "-2".into()), Tok::P("]")]);
assert_eq!(toks("1."), vec![Tok::Num(1.0, "1".into()), Tok::P(".")]);
assert_eq!(toks("1e"), vec![Tok::BadNum("1e".into())]);
assert_eq!(toks("2E-3"), vec![Tok::Num(0.002, "2E-3".into())]);
}
#[test]
fn single_ampersand_and_bar_are_the_doubles_and_strays_skip() {
assert_eq!(toks("a & b | c ; d"), vec![
Tok::Ident("a".into()), Tok::P("&&"), Tok::Ident("b".into()), Tok::P("||"), Tok::Ident("c".into()), Tok::Ident("d".into())
]);
}
#[test]
fn string_escapes() {
assert_eq!(toks(r#""a\né""#), vec![Tok::Str("a\né".into())]);
// Not a JSON string (an unknown escape): taken as written.
assert_eq!(toks(r#""a\qb""#), vec![Tok::Str("a\\qb".into())]);
assert_eq!(toks(r"'it\'s\x'"), vec![Tok::Str("it'sx".into())]);
// A raw line feed is not JSON either: taken as written, and lines counted.
let t = lex("x = \"a\nb\"\ny");
assert_eq!(t[2].tok, Tok::Str("a\nb".into()));
assert_eq!(t[4].line, 2);
}
#[test]
fn an_open_string_is_closed_at_the_end() {
let t = lex("\"abc");
assert_eq!((t[0].tok.clone(), t[0].unterminated), (Tok::Str("abc".into()), true));
// Mid-escape: a backslash closes the escape first.
assert_eq!(lex("\"abc\\")[0].tok, Tok::Str("abc\\".into()));
// A partial \u escape is not JSON: taken as written.
assert_eq!(lex("\"a\\u00")[0].tok, Tok::Str("a\\u00".into()));
}
#[test]
fn lone_surrogates_are_taken_as_written() {
assert_eq!(decode_dq(r"\ud800"), r"\ud800");
assert_eq!(decode_dq(r"😀"), "😀");
}
}

329
joule-ui/joule-ui-rs/crates/joule-ui-lang/src/lib.rs Normal file
View file

@ -0,0 +1,329 @@
//! The line language ingest for joule-ui (SPEC §11).
//!
//! A model writes a user interface as a short program, one statement a line:
//!
//! ```text
//! root = App([room], "Friday show")
//! room = Show([chat, poll])
//! chat = Chat()
//! poll = Poll("Which song next?", ["The first", "The second"])
//! ```
//!
//! This is OpenUI Lang, the open-source project's language (MIT licence; see
//! `joule-ui/NOTICE`). joule-ui takes it in the way it takes in A2UI: the
//! program is a way to write a joule-ui [`Widget`] tree against a host's
//! [`Registry`], and what comes out is an ordinary widget tree that joule-ui's
//! own [`validate`](joule_ui_core::validate) has checked, widget by widget.
//!
//! - [`StreamParser`] takes the text a chunk at a time and gives a
//! [`Snapshot`] after each: the tree as far as it can be built, every error
//! so far with its line and reason, the names still unresolved. Every tree a
//! snapshot holds passes `validate`.
//! - [`parse`] is the same over the whole text at once; at the end of a stream
//! the two agree.
//! - [`write`] writes a tree back to the language in its canonical form, which
//! parses back to the same tree.
//!
//! What joule-ui does not carry — state (`$name`), queries and mutations,
//! operators, conditionals, member access, built-in functions, sequences of
//! actions — is parsed and refused with the code `not-declarative`: joule-ui
//! puts no expression and no behaviour on the wire, and the smallest piece that
//! holds one is dropped.
#![forbid(unsafe_code)]
mod ingest;
pub mod lex;
pub mod parse;
pub mod prepare;
pub mod vectors;
mod write;
pub use write::{number_text, write, WriteError};
/// Names the language reserves for its own call forms: no widget takes them.
pub const RESERVED: [&str; 3] = ["Query", "Mutation", "Action"];
use joule_ui_core::{AllowedChildren, Registry, Widget, WidgetSchema};
use serde::{Deserialize, Serialize};
/// The library a program is written against: the host's registry (each
/// widget's props, children and positional order) and the name of its root
/// widget, the one a program without a `root` statement may start from
/// (SPEC §11.5).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Library {
pub registry: Registry,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub root: Option<String>,
}
/// Why a registry cannot be written in the language.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum LibraryError {
/// A widget name that does not start with an uppercase letter cannot be a
/// call's head.
NotCallable(String),
/// A positional name that is not a declared prop, or `children` on a widget
/// that takes none.
UnknownPositional { widget: String, name: String },
/// A positional name listed twice.
DuplicatePositional { widget: String, name: String },
/// The root names no widget of the registry.
UnknownRoot(String),
/// A widget named like one of the language's own call forms.
Reserved(String),
}
impl std::fmt::Display for LibraryError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
LibraryError::NotCallable(w) => write!(f, "widget `{w}` does not start with an uppercase letter, so no call can name it"),
LibraryError::UnknownPositional { widget, name } => write!(f, "widget `{widget}` lists `{name}` in its positional order, and it is not one of its props"),
LibraryError::DuplicatePositional { widget, name } => write!(f, "widget `{widget}` lists `{name}` twice in its positional order"),
LibraryError::UnknownRoot(r) => write!(f, "the root `{r}` is not a widget of the registry"),
LibraryError::Reserved(w) => write!(f, "`{w}` is one of the language's own call forms, so no widget takes the name"),
}
}
}
impl std::error::Error for LibraryError {}
impl Library {
/// A library over `registry`, checked: every widget callable, every
/// positional name a prop it declares (or `children`, when it takes
/// children), none twice, and the root one of its widgets.
pub fn new(registry: Registry, root: Option<String>) -> Result<Self, LibraryError> {
let mut names: Vec<&str> = registry.names().collect();
names.sort_unstable();
for name in names {
let s = registry.get(name).expect("listed");
if !name.starts_with(|c: char| c.is_ascii_uppercase()) || !name.chars().all(|c| c.is_ascii_alphanumeric() || c == '_') {
return Err(LibraryError::NotCallable(name.into()));
}
if RESERVED.contains(&name) {
return Err(LibraryError::Reserved(name.into()));
}
let order = s.positional_order();
for (i, p) in order.iter().enumerate() {
let known = if p == "children" { !matches!(s.children, AllowedChildren::None) } else { s.props.contains_key(p) };
if !known {
return Err(LibraryError::UnknownPositional { widget: name.into(), name: p.clone() });
}
if order[..i].contains(p) {
return Err(LibraryError::DuplicatePositional { widget: name.into(), name: p.clone() });
}
}
}
if let Some(r) = &root
&& registry.get(r).is_none()
{
return Err(LibraryError::UnknownRoot(r.clone()));
}
Ok(Library { registry, root })
}
/// A widget's signature as an error hint and a guide print it:
/// `Poll(question*, options*, seconds)`, `*` marking a required prop.
pub fn signature(&self, name: &str) -> Option<String> {
let s = self.registry.get(name)?;
Some(signature_of(s))
}
/// The widget names, sorted.
pub fn names(&self) -> Vec<String> {
let mut v: Vec<String> = self.registry.names().map(String::from).collect();
v.sort();
v
}
}
pub(crate) fn signature_of(s: &WidgetSchema) -> String {
let args: Vec<String> = s
.positional_order()
.iter()
.map(|p| {
let req = s.props.get(p).is_some_and(|ps| ps.required);
if req { format!("{p}*") } else { p.clone() }
})
.collect();
format!("{}({})", s.name, args.join(", "))
}
/// The reason codes of the ingest (SPEC §11.8).
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum Code {
/// A call names a widget the registry does not have. The widget is dropped.
UnknownComponent,
/// A required prop is absent. The widget is dropped.
MissingRequired,
/// A required prop is written `null`. The widget is dropped.
NullRequired,
/// A value is not of the type its prop declares, or a widget stands where
/// data must, or data where a widget must. An optional prop or a list
/// element is left out; a required prop drops the widget.
TypeMismatch,
/// More arguments than the widget has positions. The extra ones are dropped.
ExcessArgs,
/// A widget a parent's `AllowedChildren` does not accept. The child is left out.
DisallowedChild,
/// State, a query or a mutation, an operator, a conditional, member access,
/// a built-in function, or a sequence of actions: joule-ui carries no
/// expression and no behaviour. The smallest piece that holds one is dropped.
NotDeclarative,
/// `Query` or `Mutation` written inside a value. It evaluates to nothing.
InlineReserved,
/// A complete statement whose expression does not parse. It is dropped.
Syntax,
/// At the end of the stream, no usable entry: nothing renders. A warning
/// when the entry is the first statement that calls the root widget.
NoRoot,
/// At the end of the stream, text that holds no statement at all.
ParseFailed,
}
impl Code {
pub fn as_str(self) -> &'static str {
match self {
Code::UnknownComponent => "unknown-component",
Code::MissingRequired => "missing-required",
Code::NullRequired => "null-required",
Code::TypeMismatch => "type-mismatch",
Code::ExcessArgs => "excess-args",
Code::DisallowedChild => "disallowed-child",
Code::NotDeclarative => "not-declarative",
Code::InlineReserved => "inline-reserved",
Code::Syntax => "syntax",
Code::NoRoot => "no-root",
Code::ParseFailed => "parse-failed",
}
}
}
/// One problem: the source line, the statement it is in, its reason code,
/// and what it is about, in words a model can act on.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
pub struct LangError {
/// The 1-based line of the source text (as it arrived, fences and prose
/// counted); 0 for the whole program.
pub line: u32,
/// The statement's name; empty for the whole program.
pub statement: String,
pub code: Code,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub component: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub prop: Option<String>,
pub message: String,
/// A warning: the page renders through it.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub warning: bool,
}
/// A widget of the tree, in pre-order beside it: the key a renderer keeps its
/// instance under across snapshots, and where it was written.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Node {
/// The statement's name for a widget that is a statement's value (`room`;
/// `room#2` where one statement is used twice), else its parent's key and
/// its position among the children (`root/0`).
pub key: String,
/// The source line of its call.
pub line: u32,
/// The statement whose text holds its call.
pub statement: String,
}
/// A statement, as the program holds it now.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Statement {
pub name: String,
pub line: u32,
/// `value`, `state`, `query`, `mutation`, or `syntax` for one that does not
/// parse.
pub kind: String,
pub complete: bool,
}
/// What the program renders after a chunk (SPEC §11.6).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct Snapshot {
/// The tree from the entry, every widget of it accepted by the registry;
/// none while there is no usable entry.
pub tree: Option<Widget>,
/// The tree's widgets in pre-order, with their keys and lines.
pub nodes: Vec<Node>,
/// Every error so far, by line. A pending statement's are withheld until it
/// completes; a widget held back for something not yet written is not an
/// error until the stream ends.
pub errors: Vec<LangError>,
/// Names referred to and not (yet) defined, sorted.
pub unresolved: Vec<String>,
/// Value statements the entry does not reach, sorted.
pub orphans: Vec<String>,
/// The statements in the order written, each binding its name (a later one
/// replaces an earlier, SPEC §11.5).
pub statements: Vec<Statement>,
/// The host has ended the stream.
pub finished: bool,
/// The text ended inside an open string or bracket.
pub incomplete: bool,
}
impl Snapshot {
/// The errors with a given code.
pub fn errors_with(&self, code: Code) -> Vec<&LangError> {
self.errors.iter().filter(|e| e.code == code).collect()
}
}
/// The streaming parser (SPEC §11.6): push the text a chunk at a time, read a
/// snapshot after each; `finish` when the host ends the stream.
#[derive(Debug, Clone)]
pub struct StreamParser<'l> {
lib: &'l Library,
text: String,
finished: bool,
}
impl<'l> StreamParser<'l> {
pub fn new(lib: &'l Library) -> Self {
StreamParser { lib, text: String::new(), finished: false }
}
/// Take the next chunk. A chunk after `finish` starts a new stream.
pub fn push(&mut self, chunk: &str) -> Snapshot {
if self.finished {
self.text.clear();
self.finished = false;
}
self.text.push_str(chunk);
self.snapshot()
}
/// The host ends the stream: the last statement completes.
pub fn finish(&mut self) -> Snapshot {
self.finished = true;
self.snapshot()
}
/// The snapshot of the text so far.
pub fn snapshot(&self) -> Snapshot {
ingest::snapshot(self.lib, &self.text, self.finished)
}
/// The text taken so far.
pub fn text(&self) -> &str {
&self.text
}
}
/// Parse a whole program at once: the snapshot a stream of the same text gives
/// at its end.
pub fn parse(lib: &Library, text: &str) -> Snapshot {
ingest::snapshot(lib, text, true)
}
#[cfg(test)]
mod tests;

487
joule-ui/joule-ui-rs/crates/joule-ui-lang/src/parse.rs Normal file
View file

@ -0,0 +1,487 @@
//! Statements (SPEC §11.3) and expressions (SPEC §11.4).
//!
//! The grammar is parsed whole, the parts joule-ui does not carry included, so
//! a statement is refused for what it says (`not-declarative`), not because the
//! parser could not read it.
use crate::lex::{Tok, Token};
/// One statement as the splitter found it.
#[derive(Debug, Clone, PartialEq)]
pub struct RawStmt {
/// The bound name (`$` included for a state name).
pub name: String,
/// Whether the name is a state name.
pub state: bool,
/// The prepared line the statement starts on (0-based).
pub line: usize,
/// The expression's tokens, line feeds inside it left out.
pub tokens: Vec<Token>,
/// Complete: its line feed arrived and the next character after it that is
/// not white space has too, and is not `?` (SPEC §11.6 rule 1); or the
/// stream has ended.
pub complete: bool,
/// The text ended inside it, in an open string or an open bracket, so it
/// is closed implicitly (SPEC §11.6 rule 2).
pub open: bool,
}
/// Split the token stream into statements (SPEC §11.3). `finished`: the host
/// has ended the stream, so the last statement is complete too.
pub fn split(tokens: &[Token], finished: bool) -> Vec<RawStmt> {
let mut out = Vec::new();
let mut pos = 0;
let n = tokens.len();
let skip_line = |pos: &mut usize| {
while *pos < n && tokens[*pos].tok != Tok::Newline {
*pos += 1;
}
};
while pos < n {
while pos < n && tokens[pos].tok == Tok::Newline {
pos += 1;
}
if pos >= n {
break;
}
let (name, state) = match &tokens[pos].tok {
Tok::Ident(s) => (s.clone(), false),
Tok::State(s) => (s.clone(), true),
_ => {
skip_line(&mut pos);
continue;
}
};
if pos + 1 >= n || tokens[pos + 1].tok != Tok::P("=") {
skip_line(&mut pos);
continue;
}
let line = tokens[pos].line;
pos += 2;
let mut expr = Vec::new();
let mut depth = 0usize;
// Open ternaries at depth 0: (in its `:` branch, that branch has an operand).
let mut ternaries: Vec<(bool, bool)> = Vec::new();
let mut complete = finished;
let mut ended_at_boundary = false;
while pos < n {
let t = &tokens[pos];
if t.tok == Tok::Newline {
if depth == 0 && ternaries.is_empty() {
let mut peek = pos + 1;
while peek < n && tokens[peek].tok == Tok::Newline {
peek += 1;
}
if peek < n && tokens[peek].tok == Tok::P("?") {
pos += 1;
continue;
}
ended_at_boundary = true;
if peek < n {
complete = true;
}
break;
}
pos += 1;
continue;
}
let mut operand = false;
match &t.tok {
Tok::P("(") | Tok::P("[") | Tok::P("{") => depth += 1,
Tok::P(")") | Tok::P("]") | Tok::P("}") => {
if depth > 0 {
depth -= 1;
operand = depth == 0;
}
}
Tok::P("?") if depth == 0 => ternaries.push((false, false)),
Tok::P(":") if depth == 0 => {
if let Some(top) = ternaries.last_mut()
&& !top.0
{
*top = (true, false);
}
}
Tok::Ident(_) | Tok::State(_) | Tok::Str(_) | Tok::Num(..) | Tok::BadNum(_) | Tok::True | Tok::False | Tok::Null => {
operand = depth == 0;
}
_ => {}
}
if operand && let Some(top) = ternaries.last_mut() {
top.1 = true;
if top.0 {
ternaries.pop();
}
}
expr.push(t.clone());
pos += 1;
}
let open = !ended_at_boundary && (depth > 0 || expr.last().is_some_and(|t| t.unterminated));
out.push(RawStmt { name, state, line, tokens: expr, complete, open });
}
out
}
/// An expression and the prepared line it starts on.
#[derive(Debug, Clone, PartialEq)]
pub struct E {
pub expr: Expr,
pub line: usize,
}
#[derive(Debug, Clone, PartialEq)]
pub enum Head {
/// A component name: an identifier with an uppercase first letter.
Component,
/// An `@` name.
At,
}
#[derive(Debug, Clone, PartialEq)]
pub enum Expr {
Str(String),
Num(f64),
Bool(bool),
Null,
Ref(String),
State(String),
Array(Vec<E>),
Object(Vec<(String, E)>),
Call { head: Head, name: String, args: Vec<E> },
Unary(&'static str, Box<E>),
Binary(&'static str, Box<E>, Box<E>),
Ternary(Box<E>, Box<E>, Box<E>),
Member(Box<E>, String),
Index(Box<E>, Box<E>),
/// `$name = expression` inside an expression.
Assign(String, Box<E>),
}
/// Why an expression did not parse, and the prepared line where it stopped.
#[derive(Debug, Clone, PartialEq)]
pub struct ParseError {
pub line: usize,
pub message: String,
}
/// Close a pending statement implicitly (SPEC §11.6 rule 2): a closing bracket
/// that matches no open one is dropped, and the brackets still open are closed
/// in reverse order. (An open string was closed by the lexer.)
pub fn close_implicitly(tokens: &[Token]) -> Vec<Token> {
let mut out = Vec::with_capacity(tokens.len() + 4);
let mut stack: Vec<&'static str> = Vec::new();
for t in tokens {
match t.tok {
Tok::P("(") => stack.push(")"),
Tok::P("[") => stack.push("]"),
Tok::P("{") => stack.push("}"),
Tok::P(c @ (")" | "]" | "}")) => {
if stack.last() == Some(&c) {
stack.pop();
} else {
continue;
}
}
_ => {}
}
out.push(t.clone());
}
let line = tokens.last().map_or(0, |t| t.line);
while let Some(c) = stack.pop() {
out.push(Token { tok: Tok::P(c), line, unterminated: false });
}
out
}
/// Parse a statement's expression: the whole of its tokens, nothing left over.
pub fn parse_expr(tokens: &[Token]) -> Result<E, ParseError> {
let mut p = Parser { t: tokens, i: 0 };
if tokens.is_empty() {
return Err(ParseError { line: 0, message: "a statement with no expression".into() });
}
let e = p.ternary()?;
if p.i < tokens.len() {
return Err(p.err("a token after the end of the expression"));
}
Ok(e)
}
struct Parser<'a> {
t: &'a [Token],
i: usize,
}
impl Parser<'_> {
fn peek(&self) -> Option<&Tok> {
self.t.get(self.i).map(|t| &t.tok)
}
fn at(&self, p: &str) -> bool {
matches!(self.peek(), Some(Tok::P(q)) if *q == p)
}
fn line(&self) -> usize {
self.t.get(self.i).or_else(|| self.t.last()).map_or(0, |t| t.line)
}
fn err(&self, m: &str) -> ParseError {
ParseError { line: self.line(), message: m.to_string() }
}
fn expect(&mut self, p: &str) -> Result<(), ParseError> {
if self.at(p) {
self.i += 1;
Ok(())
} else {
Err(self.err(&format!("expected `{p}`")))
}
}
fn ternary(&mut self) -> Result<E, ParseError> {
let c = self.or()?;
if self.at("?") {
self.i += 1;
let a = self.ternary()?;
self.expect(":")?;
let b = self.ternary()?;
let line = c.line;
return Ok(E { expr: Expr::Ternary(Box::new(c), Box::new(a), Box::new(b)), line });
}
Ok(c)
}
fn binary_level(&mut self, ops: &[&'static str], next: fn(&mut Self) -> Result<E, ParseError>) -> Result<E, ParseError> {
let mut l = next(self)?;
loop {
let op = match self.peek() {
Some(Tok::P(p)) => ops.iter().find(|o| **o == *p).copied(),
_ => None,
};
let Some(op) = op else { return Ok(l) };
self.i += 1;
let r = next(self)?;
let line = l.line;
l = E { expr: Expr::Binary(op, Box::new(l), Box::new(r)), line };
}
}
fn or(&mut self) -> Result<E, ParseError> {
self.binary_level(&["||"], Self::and)
}
fn and(&mut self) -> Result<E, ParseError> {
self.binary_level(&["&&"], Self::equality)
}
fn equality(&mut self) -> Result<E, ParseError> {
self.binary_level(&["==", "!="], Self::comparison)
}
fn comparison(&mut self) -> Result<E, ParseError> {
self.binary_level(&[">", "<", ">=", "<="], Self::additive)
}
fn additive(&mut self) -> Result<E, ParseError> {
self.binary_level(&["+", "-"], Self::multiplicative)
}
fn multiplicative(&mut self) -> Result<E, ParseError> {
self.binary_level(&["*", "/", "%"], Self::unary)
}
fn unary(&mut self) -> Result<E, ParseError> {
if self.at("!") || self.at("-") {
let op = if self.at("!") { "!" } else { "-" };
let line = self.line();
self.i += 1;
let e = self.unary()?;
return Ok(E { expr: Expr::Unary(op, Box::new(e)), line });
}
self.postfix()
}
fn postfix(&mut self) -> Result<E, ParseError> {
let mut e = self.primary()?;
loop {
if self.at(".") {
self.i += 1;
let f = self.key()?;
let line = e.line;
e = E { expr: Expr::Member(Box::new(e), f), line };
} else if self.at("[") {
self.i += 1;
let ix = self.ternary()?;
self.expect("]")?;
let line = e.line;
e = E { expr: Expr::Index(Box::new(e), Box::new(ix)), line };
} else {
return Ok(e);
}
}
}
/// An object key or a field name: a name, a string, a number as written, a
/// component name, a keyword, or a state name without its `$`; any other
/// token is the key `?` (SPEC §11.4).
fn key(&mut self) -> Result<String, ParseError> {
let Some(t) = self.peek().cloned() else { return Err(self.err("expected a key")) };
self.i += 1;
Ok(match t {
Tok::Ident(s) | Tok::Str(s) => s,
Tok::State(s) => s[1..].to_string(),
Tok::Num(_, text) | Tok::BadNum(text) => text,
Tok::True => "true".into(),
Tok::False => "false".into(),
Tok::Null => "null".into(),
Tok::At(s) => format!("@{s}"),
_ => "?".into(),
})
}
/// Elements up to `close`, commas between them optional and ignored.
fn list(&mut self, close: &str) -> Result<Vec<E>, ParseError> {
let mut v = Vec::new();
loop {
while self.at(",") {
self.i += 1;
}
if self.at(close) {
self.i += 1;
return Ok(v);
}
if self.peek().is_none() {
return Err(self.err(&format!("expected `{close}`")));
}
v.push(self.ternary()?);
}
}
fn primary(&mut self) -> Result<E, ParseError> {
let line = self.line();
let Some(t) = self.peek().cloned() else { return Err(self.err("expected a value")) };
self.i += 1;
let expr = match t {
Tok::Str(s) => Expr::Str(s),
Tok::Num(v, _) => Expr::Num(v),
Tok::BadNum(text) => {
self.i -= 1;
return Err(self.err(&format!("`{text}` is not a number: its exponent has no digits")));
}
Tok::True => Expr::Bool(true),
Tok::False => Expr::Bool(false),
Tok::Null => Expr::Null,
Tok::P("[") => Expr::Array(self.list("]")?),
Tok::P("{") => {
let mut entries = Vec::new();
loop {
while self.at(",") {
self.i += 1;
}
if self.at("}") {
self.i += 1;
break;
}
if self.peek().is_none() {
return Err(self.err("expected `}`"));
}
let k = self.key()?;
self.expect(":")?;
entries.push((k, self.ternary()?));
}
Expr::Object(entries)
}
Tok::P("(") => {
let e = self.ternary()?;
self.expect(")")?;
return Ok(e);
}
Tok::Ident(name) => {
if self.at("(") && name.starts_with(|c: char| c.is_ascii_uppercase()) {
self.i += 1;
Expr::Call { head: Head::Component, name, args: self.list(")")? }
} else {
Expr::Ref(name)
}
}
Tok::At(name) => {
self.expect("(")?;
Expr::Call { head: Head::At, name, args: self.list(")")? }
}
Tok::State(name) => {
if self.at("=") {
self.i += 1;
Expr::Assign(name, Box::new(self.ternary()?))
} else {
Expr::State(name)
}
}
_ => {
self.i -= 1;
return Err(self.err("expected a value"));
}
};
Ok(E { expr, line })
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::lex::lex;
fn names(src: &str, finished: bool) -> Vec<(String, bool, bool)> {
split(&lex(src), finished).into_iter().map(|s| (s.name, s.complete, s.open)).collect()
}
#[test]
fn statements_complete_when_the_next_character_arrives() {
assert_eq!(names("a = 1\nb = 2", false), vec![("a".into(), true, false), ("b".into(), false, false)]);
assert_eq!(names("a = 1\n", false), vec![("a".into(), false, false)]);
assert_eq!(names("a = 1\n", true), vec![("a".into(), true, false)]);
// A next line starting `?` continues the statement.
assert_eq!(names("a = c\n? 1\n: 2\nb = 3", false), vec![("a".into(), true, false), ("b".into(), false, false)]);
assert_eq!(names("a = c\n", false), vec![("a".into(), false, false)]);
}
#[test]
fn brackets_strings_and_ternaries_carry_a_statement_over_line_feeds() {
assert_eq!(names("a = A([\n b,\n c\n])\nd = 1", false).len(), 2);
assert_eq!(names("a = \"x\ny\"\nd = 1", false).len(), 2);
// A `:` branch still empty continues; an operand closes it.
assert_eq!(names("a = c ? 1 :\n 2\nd = 1", false).len(), 2);
assert_eq!(names("a = c ? 1\n : 2\nd = 1", false).len(), 2);
let s = split(&lex("a = A([b"), false);
assert!(s[0].open && !s[0].complete);
}
#[test]
fn lines_without_the_shape_are_skipped() {
assert_eq!(names("hello there\n= 3\nx = 1\n", true), vec![("x".into(), true, false)]);
}
fn p(src: &str) -> Result<E, ParseError> {
let s = split(&lex(src), true);
parse_expr(&s[0].tokens)
}
#[test]
fn calls_arrays_objects() {
let e = p("x = Card(\"t\", [a, b,], {k: 1, \"q r\": [2] $s: 3})").unwrap();
let Expr::Call { name, args, .. } = e.expr else { panic!() };
assert_eq!(name, "Card");
assert_eq!(args.len(), 3);
let Expr::Object(entries) = &args[2].expr else { panic!() };
let keys: Vec<&str> = entries.iter().map(|(k, _)| k.as_str()).collect();
assert_eq!(keys, vec!["k", "q r", "s"]);
}
#[test]
fn precedence_and_postfix() {
let e = p("x = a + b * c > 1 && !d || e ? f.g[0] : -h").unwrap();
assert!(matches!(e.expr, Expr::Ternary(..)));
assert!(matches!(p("x = m.0.1").unwrap().expr, Expr::Member(_, ref f) if f == "0.1"));
}
#[test]
fn syntax_errors() {
assert!(p("x = Card(\"a\"").is_err());
assert!(p("x = count(1)").is_err(), "a call head is uppercase or @");
assert!(p("x = 1e").is_err());
assert!(p("x = @Go").is_err());
assert!(p("x = 1 2").is_err());
}
#[test]
fn implicit_closing_drops_strays_and_closes_in_reverse() {
let s = split(&lex("x = A([b, {k: \"v"), false);
let closed = close_implicitly(&s[0].tokens);
assert!(parse_expr(&closed).is_ok());
let s = split(&lex("x = A(]"), false);
assert!(parse_expr(&close_implicitly(&s[0].tokens)).is_ok());
}
}

208
joule-ui/joule-ui-rs/crates/joule-ui-lang/src/prepare.rs Normal file
View file

@ -0,0 +1,208 @@
//! Source preparation (SPEC §11.1): the host's marker lines, fences, comments
//! and trimming, keeping for each line of the prepared text the line of the
//! source it came from, so that every error can name the line a person sees.
/// The prepared program text and, for each of its lines, the 1-based line of
/// the source text it came from.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Prepared {
pub text: String,
pub lines: Vec<u32>,
}
impl Prepared {
/// The source line of the prepared line `i` (0-based); 0 past the end.
pub fn source_line(&self, i: usize) -> u32 {
self.lines.get(i).copied().unwrap_or(0)
}
}
/// The prefix of a host marker line (SPEC §11.1 step 2).
pub const MARKER: &str = "]]>openui:";
const FENCE: &str = "```";
/// Prepare `source` (SPEC §11.1): line feeds normalised, marker sections,
/// fences, comments, then trimming.
pub fn prepare(source: &str) -> Prepared {
// 1. A carriage return before a line feed goes with it.
let normalised = source.replace("\r\n", "\n");
let mut lines: Vec<(u32, &str)> = normalised.split('\n').enumerate().map(|(i, l)| (i as u32 + 1, l)).collect();
// 2. Marker sections: with any marker line, the program is the lines of
// `content` sections alone.
if lines.iter().any(|(_, l)| l.starts_with(MARKER)) {
let mut kept = Vec::new();
let mut in_content = false;
for (n, l) in lines {
if let Some(rest) = l.strip_prefix(MARKER) {
in_content = rest.starts_with("content");
continue;
}
if in_content {
kept.push((n, l));
}
}
lines = kept;
}
// 3. Fences: with any fence, the program is the fences' contents, joined.
if lines.iter().any(|(_, l)| l.starts_with(FENCE)) {
let mut kept = Vec::new();
let mut in_fence = false;
let mut in_dq = false;
let mut escaped = false;
for (n, l) in lines {
if !in_fence {
if l.starts_with(FENCE) {
in_fence = true;
in_dq = false;
escaped = false;
}
continue;
}
if !in_dq && l.starts_with(FENCE) {
in_fence = false;
continue;
}
kept.push((n, l));
// Double-quoted strings shield a fence line; a string may run on
// over a line feed, so the state carries to the next line.
for c in l.chars() {
if escaped {
escaped = false;
} else if in_dq && c == '\\' {
escaped = true;
} else if c == '"' {
in_dq = !in_dq;
}
}
// The line feed between this line and the next is a character of
// the string too: it ends an escape.
escaped = false;
}
lines = kept;
}
let mut map: Vec<u32> = lines.iter().map(|(n, _)| *n).collect();
let joined: String = lines.iter().map(|(_, l)| *l).collect::<Vec<_>>().join("\n");
// 4. Comments: `//` or `#` outside a string, to the end of its line.
let stripped = strip_comments(&joined);
// 5. Trimming: leading and trailing whitespace (space, tab, line feed,
// carriage return). Lines dropped at the front leave the map too.
let is_ws = |c: char| matches!(c, ' ' | '\t' | '\n' | '\r');
let lead: &str = &stripped[..stripped.len() - stripped.trim_start_matches(is_ws).len()];
let dropped = lead.matches('\n').count();
map.drain(..dropped.min(map.len()));
let text = stripped.trim_matches(is_ws).to_string();
let n_lines = text.split('\n').count();
map.truncate(n_lines);
Prepared { text, lines: map }
}
/// Remove comments (SPEC §11.1 step 4): a `//` or a `#` outside a string starts
/// one, and it runs to the end of its line; the line feed stays. Strings, in
/// double or single quotes, may hold line feeds; a backslash in one escapes
/// the next character.
pub fn strip_comments(text: &str) -> String {
let chars: Vec<char> = text.chars().collect();
let mut out = String::with_capacity(text.len());
let mut quote: Option<char> = None;
let mut i = 0;
while i < chars.len() {
let c = chars[i];
if let Some(q) = quote {
out.push(c);
if c == '\\' {
if let Some(&n) = chars.get(i + 1) {
out.push(n);
i += 2;
continue;
}
} else if c == q {
quote = None;
}
i += 1;
continue;
}
if c == '"' || c == '\'' {
quote = Some(c);
out.push(c);
i += 1;
continue;
}
if c == '#' || (c == '/' && chars.get(i + 1) == Some(&'/')) {
while i < chars.len() && chars[i] != '\n' {
i += 1;
}
continue;
}
out.push(c);
i += 1;
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn plain_text_keeps_its_lines() {
let p = prepare("a = 1\nb = 2\n");
assert_eq!(p.text, "a = 1\nb = 2");
assert_eq!(p.lines, vec![1, 2]);
}
#[test]
fn leading_blank_lines_shift_the_map() {
let p = prepare("\n\n a = 1\n\nb = 2 \n\n");
assert_eq!(p.text, "a = 1\n\nb = 2");
assert_eq!(p.lines, vec![3, 4, 5]);
}
#[test]
fn fences_keep_their_contents_and_prose_goes() {
let src = "Here is your app:\n```openui-lang\nroot = A([x])\nx = B(\"a\")\n```\nEnjoy.\n```\ny = C()\n```";
let p = prepare(src);
assert_eq!(p.text, "root = A([x])\nx = B(\"a\")\ny = C()");
assert_eq!(p.lines, vec![3, 4, 8]);
}
#[test]
fn a_fence_line_inside_a_double_quoted_string_does_not_close() {
let src = "```\nx = T(\"a\n```\nb\")\n```";
let p = prepare(src);
assert_eq!(p.text, "x = T(\"a\n```\nb\")");
}
#[test]
fn an_unterminated_fence_runs_to_the_end() {
let p = prepare("prose\n```\nroot = A()\nx = B(");
assert_eq!(p.text, "root = A()\nx = B(");
assert_eq!(p.lines, vec![3, 4]);
}
#[test]
fn comments_go_but_not_inside_strings() {
assert_eq!(strip_comments("a = 1 // one\nb = \"#fff\" # col"), "a = 1 \nb = \"#fff\" ");
assert_eq!(strip_comments("s = 'it''s // no'"), "s = 'it''s // no'");
assert_eq!(strip_comments("s = \"a\\\" // still\" // gone"), "s = \"a\\\" // still\" ");
}
#[test]
fn marker_sections_keep_content_alone() {
let src = "]]>openui:content?library=x%401\nroot = A()\n]]>openui:context\n{\"formState\": {}}\n]]>openui:end";
let p = prepare(src);
assert_eq!(p.text, "root = A()");
assert_eq!(p.lines, vec![2]);
}
#[test]
fn crlf_is_one_line_feed() {
let p = prepare("a = 1\r\nb = 2\r\n");
assert_eq!(p.text, "a = 1\nb = 2");
}
}

363
joule-ui/joule-ui-rs/crates/joule-ui-lang/src/tests.rs Normal file
View file

@ -0,0 +1,363 @@
//! The ingest's behaviour, rule by rule (SPEC §11).
use crate::vectors::{ingest_cases, refusal_cases, stream_cases, test_library, write_cases};
use crate::*;
use joule_ui_core::{validate, PropValue, Widget};
fn lib() -> Library {
test_library()
}
fn text_of(w: &Widget) -> Option<&str> {
match w.props.get("text") {
Some(PropValue::Text(s)) => Some(s),
_ => None,
}
}
fn names(w: &Widget) -> Vec<String> {
let mut v = vec![w.name.clone()];
for c in &w.children {
v.extend(names(c));
}
v
}
#[test]
fn a_program_becomes_a_validated_tree() {
let l = lib();
let s = parse(&l, "root = Page([card], \"Home\")\ncard = Card(\"Hello\", [t1, t2], \"sub\")\nt1 = Text(\"one\")\nt2 = Text(\"two\", 14)\n");
assert!(s.errors.is_empty(), "{:?}", s.errors);
let t = s.tree.expect("a tree");
assert_eq!(names(&t), vec!["Page", "Card", "Text", "Text"]);
assert_eq!(t.props["title"], PropValue::Text("Home".into()));
assert_eq!(t.children[0].props["subtitle"], PropValue::Text("sub".into()));
assert_eq!(t.children[0].children[1].props["size"], PropValue::Number(14.0));
assert!(validate(&t, &l.registry).is_empty());
let keys: Vec<&str> = s.nodes.iter().map(|n| n.key.as_str()).collect();
assert_eq!(keys, vec!["root", "card", "t1", "t2"]);
let lines: Vec<u32> = s.nodes.iter().map(|n| n.line).collect();
assert_eq!(lines, vec![1, 2, 3, 4]);
}
#[test]
fn unresolved_references_are_left_out_and_listed() {
let s = parse(&lib(), "root = Page([a, missing, b])\na = Text(\"a\")\nb = Text(\"b\")\n");
assert_eq!(s.unresolved, vec!["missing"]);
assert!(s.errors.is_empty());
assert_eq!(s.tree.unwrap().children.len(), 2);
}
#[test]
fn inline_widgets_take_their_parents_key() {
let s = parse(&lib(), "root = Page([Card(\"c\", [Text(\"x\"), Text(\"y\")]), Text(\"z\")])\n");
let keys: Vec<&str> = s.nodes.iter().map(|n| n.key.as_str()).collect();
assert_eq!(keys, vec!["root", "root/0", "root/0/0", "root/0/1", "root/1"]);
}
#[test]
fn every_refusal_names_its_code_and_line_and_the_rest_renders() {
let l = lib();
for c in refusal_cases() {
let s = parse(&l, c.input);
let hit = s.errors.iter().find(|e| e.code.as_str() == c.code && e.line == c.line);
assert!(hit.is_some(), "{}: expected {} at line {}, got {:?}", c.name, c.code, c.line, s.errors);
assert!(!hit.unwrap().message.is_empty());
// Everything else renders: the widget written to be kept is in the tree.
if c.input.contains("\"kept\"") {
let t = s.tree.as_ref().unwrap_or_else(|| panic!("{}: nothing rendered", c.name));
let mut found = false;
fn any_kept(w: &Widget, found: &mut bool) {
if w.props.values().any(|v| *v == PropValue::Text("kept".into())) {
*found = true;
}
for c in &w.children {
any_kept(c, found);
}
}
any_kept(t, &mut found);
assert!(found, "{}: the kept widget is gone", c.name);
}
if let Some(t) = &s.tree {
assert!(validate(t, &l.registry).is_empty(), "{}: the tree must validate", c.name);
}
}
}
#[test]
fn a_widget_the_registry_does_not_have_never_renders() {
let l = lib();
let s = parse(&l, "root = Page([a, b])\na = Text(\"kept\")\nb = Chartt([1], [2])\n");
assert_eq!(names(s.tree.as_ref().unwrap()), vec!["Page", "Text"]);
let e = &s.errors_with(Code::UnknownComponent)[0];
assert_eq!((e.line, e.statement.as_str(), e.component.as_deref()), (3, "b", Some("Chartt")));
assert!(e.message.contains("Card"), "the hint lists what the registry has: {}", e.message);
}
#[test]
fn a_line_that_fails_validation_never_renders() {
let l = lib();
// A required prop of the wrong type: the widget is dropped.
let s = parse(&l, "root = Page([a, b])\na = Text(\"kept\")\nb = Text(42)\n");
assert_eq!(names(s.tree.as_ref().unwrap()), vec!["Page", "Text"]);
assert_eq!(s.errors_with(Code::TypeMismatch)[0].line, 3);
// A child its parent does not accept: the child is left out.
let s = parse(&l, "root = Page([tabs])\ntabs = Tabs([t1, x])\nt1 = Tab(\"kept\")\nx = Text(\"not a tab\")\n");
assert_eq!(names(s.tree.as_ref().unwrap()), vec!["Page", "Tabs", "Tab"]);
let e = &s.errors_with(Code::DisallowedChild)[0];
assert_eq!((e.line, e.component.as_deref()), (4, Some("Text")));
// An optional prop of the wrong type: the prop is left out, the widget stays.
let s = parse(&l, "root = Page([a])\na = Text(\"kept\", \"big\")\n");
let t = s.tree.unwrap();
assert!(!t.children[0].props.contains_key("size"));
assert_eq!(text_of(&t.children[0]), Some("kept"));
}
#[test]
fn nothing_the_host_owns_crosses_into_the_tree() {
let l = lib();
for (src, line) in [
("root = Page([a])\na = Text($name)\n", 2),
("root = Page([a])\na = Text(\"a\" + \"b\")\n", 2),
("root = Page([a])\na = Text(x ? \"y\" : \"z\")\n", 2),
("root = Page([a])\na = Text(d.title)\n", 2),
("root = Page([a])\na = Text(\"t\", @Count([1]))\n", 2),
("root = Page([a])\na = Button(\"t\", @Set($s, 1))\n", 2),
("root = Page([a])\na = Button(\"t\", Action([@A(), @B()]))\n", 2),
("root = Page()\n$s = 1\n", 2),
("root = Page()\nq = Query(\"t\", {}, {})\n", 2),
] {
let s = parse(&l, src);
let e = s.errors_with(Code::NotDeclarative);
assert_eq!(e.len(), 1, "{src}: {:?}", s.errors);
assert_eq!(e[0].line, line, "{src}");
}
}
#[test]
fn actions_carry_their_payload() {
let l = lib();
let s = parse(&l, ingest_cases().iter().find(|c| c.name == "actions").unwrap().input);
assert!(s.errors.is_empty(), "{:?}", s.errors);
let t = s.tree.unwrap();
let a = |i: usize| t.children[i].props["action"].clone();
let act = |id: &str, p: Option<PropValue>| PropValue::Action { id: id.into(), payload: p.map(Box::new) };
assert_eq!(a(0), act("Go", None));
assert_eq!(a(1), act("Open", Some("x".into())));
assert_eq!(a(2), act("Send", Some(PropValue::List(vec!["a".into(), PropValue::Number(2.0)]))));
assert_eq!(a(3), act("Pick", Some(PropValue::List(vec![PropValue::Number(1.0), PropValue::Number(2.0)]))));
assert_eq!(a(4), act("Go", None));
assert!(matches!(a(5), PropValue::Action { ref id, .. } if id == "ToAssistant"));
}
#[test]
fn the_entry_falls_back_to_the_root_widget_with_a_warning() {
let s = parse(&lib(), "t = Text(\"x\")\npage = Page([t])\n");
assert!(s.tree.is_some());
let e = &s.errors_with(Code::NoRoot)[0];
assert!(e.warning);
let s = parse(&lib(), "a = Text(\"x\")\n");
assert!(s.tree.is_none());
assert!(!s.errors_with(Code::NoRoot)[0].warning);
}
#[test]
fn a_cycle_is_unresolved_where_it_closes() {
let s = parse(&lib(), "root = Page([a])\na = Card(\"a\", [b])\nb = Card(\"b\", [a])\n");
assert_eq!(s.unresolved, vec!["a"]);
assert_eq!(names(s.tree.as_ref().unwrap()), vec!["Page", "Card", "Card"]);
}
// ── streaming ───────────────────────────────────────────────────────────────
/// Split `text` at the given byte offsets (each on a character boundary).
fn chunks_at(text: &str, cuts: &[usize]) -> Vec<String> {
let mut v = Vec::new();
let mut last = 0;
for &c in cuts {
if c > last && c < text.len() && text.is_char_boundary(c) {
v.push(text[last..c].to_string());
last = c;
}
}
v.push(text[last..].to_string());
v
}
fn programs() -> Vec<String> {
let mut v: Vec<String> = ingest_cases().iter().map(|c| c.input.to_string()).collect();
v.extend(refusal_cases().iter().map(|c| c.input.to_string()));
v.extend(stream_cases().iter().map(|c| c.chunks.concat()));
v
}
/// Streaming equals parsing (SPEC §11.6): whatever the chunking, the snapshot at
/// the end of the stream is the batch parse of the whole text; and every
/// snapshot on the way holds only a tree the registry accepts.
#[test]
fn streaming_equals_parsing_for_every_chunking() {
let l = lib();
let mut seed: u64 = 0x5eed_1234_abcd_0001;
for text in programs() {
let batch = parse(&l, &text);
for round in 0..12 {
let cuts: Vec<usize> = if round == 0 {
(1..text.len()).collect() // a byte at a time
} else {
let mut c: Vec<usize> = (0..round)
.map(|_| {
seed ^= seed << 13;
seed ^= seed >> 7;
seed ^= seed << 17;
(seed as usize) % text.len().max(1)
})
.collect();
c.sort_unstable();
c
};
let mut p = StreamParser::new(&l);
for ch in chunks_at(&text, &cuts) {
let s = p.push(&ch);
if let Some(t) = &s.tree {
assert!(validate(t, &l.registry).is_empty(), "{text:?}: a snapshot's tree must validate");
}
assert_eq!(s.nodes.len(), s.tree.as_ref().map_or(0, |t| names(t).len()));
}
let end = p.finish();
assert_eq!(end, batch, "{text:?} cut at {cuts:?}");
}
}
}
#[test]
fn the_app_renders_progressively_as_lines_arrive() {
let l = lib();
let text = "root = Page([card], \"Home\")\ncard = Card(\"Hello\", [t1, t2])\nt1 = Text(\"one\")\nt2 = Text(\"two\")\n";
let mut p = StreamParser::new(&l);
let mut counts = Vec::new();
for ch in chunks_at(text, &(1..text.len()).collect::<Vec<_>>()) {
let s = p.push(&ch);
counts.push(s.tree.as_ref().map_or(0, |t| names(t).len()));
assert!(s.errors.is_empty(), "no error while a valid program streams: {:?}", s.errors);
}
// The page shows after its first line, before anything else has arrived ...
let first_line = text.find('\n').unwrap();
assert!(counts[first_line] >= 1, "{counts:?}");
// ... and the widgets only ever grow in number.
assert!(counts.windows(2).all(|w| w[1] >= w[0]), "{counts:?}");
assert_eq!(*counts.last().unwrap(), 4);
}
#[test]
fn a_pending_title_shows_as_it_streams() {
let l = lib();
let mut p = StreamParser::new(&l);
p.push("root = Page([c])\nc = Card(\"A");
let s = p.push(" long");
assert_eq!(s.tree.unwrap().children[0].props["title"], PropValue::Text("A long".into()));
let s = p.push(" title\")");
assert_eq!(s.tree.unwrap().children[0].props["title"], PropValue::Text("A long title".into()));
}
#[test]
fn a_widget_waits_silently_for_its_required_props() {
let l = lib();
let mut p = StreamParser::new(&l);
let s = p.push("root = Page([c])\nc = Card(");
assert_eq!(names(s.tree.as_ref().unwrap()), vec!["Page"]);
assert!(s.errors.is_empty());
// A reference not yet written holds it back silently too ...
let s = p.push("title)\nd = 1\n");
assert!(s.errors.is_empty(), "{:?}", s.errors);
assert_eq!(s.unresolved, vec!["title"]);
// ... until the stream ends without it.
let s = p.finish();
assert_eq!(s.errors_with(Code::MissingRequired)[0].line, 2);
}
#[test]
fn a_pending_statement_never_replaces_a_complete_one() {
let l = lib();
let mut p = StreamParser::new(&l);
let s = p.push("root = Page([a])\na = Text(\"done\")\nx = 1\na = Text(\"partial");
assert_eq!(text_of(&s.tree.unwrap().children[0]), Some("done"));
let s = p.push("\")\n");
// Closed now, needing no implicit closing: the later statement binds.
assert_eq!(text_of(&s.tree.unwrap().children[0]), Some("partial"));
}
#[test]
fn a_fence_arriving_late_restarts_the_parse() {
let l = lib();
let mut p = StreamParser::new(&l);
let s = p.push("root = Page([a])\na = Text(\"x\")\n");
assert_eq!(names(s.tree.as_ref().unwrap()), vec!["Page", "Text"]);
let s = p.push("```\nroot = Page()\n```");
assert_eq!(names(s.tree.as_ref().unwrap()), vec!["Page"]);
}
#[test]
fn the_reported_line_is_the_line_as_it_arrived() {
let s = parse(&lib(), "Here is the app.\n\n```openui-lang\nroot = Page([a, b])\na = Text(\"kept\")\nb = Text(42)\n```\n");
assert_eq!(s.errors_with(Code::TypeMismatch)[0].line, 6);
}
// ── the writer ──────────────────────────────────────────────────────────────
#[test]
fn the_canonical_form_parses_back_to_the_same_tree() {
let l = lib();
for (name, tree) in write_cases() {
let text = write(&tree, &l).unwrap_or_else(|e| panic!("{name}: {e}"));
let s = parse(&l, &text);
assert!(s.errors.is_empty(), "{name}: {:?}\n{text}", s.errors);
assert_eq!(s.tree.as_ref(), Some(&tree), "{name}:\n{text}");
// And it is a fixed point.
assert_eq!(write(s.tree.as_ref().unwrap(), &l).unwrap(), text);
}
// Every tree the ingest cases build writes and reads back.
for c in ingest_cases() {
let s = parse(&l, c.input);
if let Some(t) = s.tree {
let text = write(&t, &l).unwrap();
assert_eq!(parse(&l, &text).tree, Some(t), "{}", c.name);
}
}
}
#[test]
fn the_canonical_form_is_one_statement_a_widget_in_pre_order() {
let l = lib();
let tree = write_cases().into_iter().find(|(n, _)| *n == "page-of-cards").unwrap().1;
assert_eq!(
write(&tree, &l).unwrap(),
"root = Page([card1, card2], \"Home\")\ncard1 = Card(\"One\", [text1])\ntext1 = Text(\"a\")\ncard2 = Card(\"Two\", null, \"sub\")\n"
);
}
#[test]
fn what_cannot_be_written_is_refused() {
let l = lib();
let null = Widget::leaf("Page").child(Widget::leaf("Text").prop("text", "t").prop("size", PropValue::Null));
assert!(matches!(write(&null, &l), Err(WriteError::Invalid(_))), "a null number is not a number");
let bad_id = Widget::leaf("Page").child(Widget::leaf("Button").prop("label", "x").prop("action", PropValue::Action { id: "not a name".into(), payload: None }));
assert_eq!(write(&bad_id, &l), Err(WriteError::ActionId("not a name".into())));
let inf = Widget::leaf("Page").child(Widget::leaf("Text").prop("text", "t").prop("size", f64::INFINITY));
assert!(matches!(write(&inf, &l), Err(WriteError::NotFinite { .. })));
assert!(matches!(write(&Widget::leaf("Ghost"), &l), Err(WriteError::Invalid(_))));
}
#[test]
fn a_library_is_checked() {
use joule_ui_core::{AllowedChildren, PropSchema, PropType, Registry, WidgetSchema};
let r = Registry::new().with(WidgetSchema::new("card", "lower"));
assert_eq!(Library::new(r, None).unwrap_err(), LibraryError::NotCallable("card".into()));
let r = Registry::new().with(WidgetSchema::new("Card", "c").prop("t", PropSchema::required(PropType::Text)).positional(["t", "children"]));
assert!(matches!(Library::new(r, None), Err(LibraryError::UnknownPositional { .. })));
let r = Registry::new().with(WidgetSchema::new("Card", "c").children(AllowedChildren::Any).positional(["children", "children"]));
assert!(matches!(Library::new(r, None), Err(LibraryError::DuplicatePositional { .. })));
let r = Registry::new().with(WidgetSchema::new("Action", "a"));
assert_eq!(Library::new(r, None).unwrap_err(), LibraryError::Reserved("Action".into()));
assert!(matches!(Library::new(Registry::new(), Some("Page".into())), Err(LibraryError::UnknownRoot(_))));
assert_eq!(lib().signature("Card").unwrap(), "Card(title*, children, subtitle)");
}

345
joule-ui/joule-ui-rs/crates/joule-ui-lang/src/vectors.rs Normal file
View file

@ -0,0 +1,345 @@
//! The conformance pack of the line language (SPEC §11.9): its library, its
//! cases, and the vectors the reference implementation gives for them. The
//! generator (`gen_lang_vectors`) writes them to `joule-ui/conformance/lang/v1/`;
//! the pack test regenerates them and compares byte for byte; the independent
//! verifier (`joule-ui/conformance/lang_verify.py`) recomputes them from the
//! specification text.
use crate::{parse, write, Library, Snapshot, StreamParser};
use joule_ui_core::{AllowedChildren, PropSchema, PropType, PropValue, Registry, Widget, WidgetSchema};
use serde_json::{json, Value};
use sha2::{Digest, Sha256};
use std::collections::BTreeMap;
/// The pack's library: a page of cards, text, buttons, tabs and a chart, its
/// root `Page`.
pub fn test_library() -> Library {
let r = Registry::new()
.with(
WidgetSchema::new("Page", "the page")
.prop("title", PropSchema::optional(PropType::Text))
.children(AllowedChildren::Any)
.positional(["children", "title"]),
)
.with(
WidgetSchema::new("Card", "a titled container")
.prop("title", PropSchema::required(PropType::Text))
.prop("subtitle", PropSchema::optional(PropType::Text))
.children(AllowedChildren::Any)
.positional(["title", "children", "subtitle"]),
)
.with(
WidgetSchema::new("Text", "a run of text")
.prop("text", PropSchema::required(PropType::Text))
.prop("size", PropSchema::optional(PropType::Number))
.positional(["text", "size"]),
)
.with(
WidgetSchema::new("Button", "a button")
.prop("label", PropSchema::required(PropType::Text))
.prop("action", PropSchema::optional(PropType::Action))
.prop("primary", PropSchema::optional(PropType::Bool))
.positional(["label", "action", "primary"]),
)
.with(
WidgetSchema::new("Tabs", "a strip of tabs")
.children(AllowedChildren::OnlyOf(vec!["Tab".into()]))
.positional(["children"]),
)
.with(
WidgetSchema::new("Tab", "one tab")
.prop("title", PropSchema::required(PropType::Text))
.children(AllowedChildren::Any)
.positional(["title", "children"]),
)
.with(
WidgetSchema::new("Chart", "a chart of values")
.prop("labels", PropSchema::required(PropType::List))
.prop("values", PropSchema::required(PropType::List))
.prop("options", PropSchema::optional(PropType::Object))
.positional(["labels", "values", "options"]),
)
// No declared order: its props in key order, then `children`.
.with(
WidgetSchema::new("Note", "a note")
.prop("body", PropSchema::required(PropType::Text))
.prop("author", PropSchema::optional(PropType::Text))
.children(AllowedChildren::Any),
);
Library::new(r, Some("Page".into())).expect("the pack's library is well-formed")
}
/// A case of the batch classes: a name and the program text.
pub struct Case {
pub name: &'static str,
pub input: &'static str,
}
/// The statement cases (L1): how text splits into statements.
pub fn statement_cases() -> Vec<Case> {
vec![
Case { name: "one-a-line", input: "root = Page([a])\na = Text(\"hi\")\n" },
Case { name: "blank-lines-and-prose-lines", input: "\n\nhello there\nroot = Page()\n= 3\n \nx = 1\n" },
Case { name: "brackets-carry-over-lines", input: "root = Page([\n a,\n b\n])\na = Text(\"a\")\nb = Text(\"b\")" },
Case { name: "string-carries-over-lines", input: "a = Text(\"line one\nline two\")\nroot = Page([a])" },
Case { name: "ternary-continues-on-a-question-line", input: "x = c\n ? 1\n : 2\nroot = Page()" },
Case { name: "ternary-empty-branch-continues", input: "x = c ? 1 :\n 2\nroot = Page()" },
Case { name: "state-query-mutation-kinds", input: "$s = 1\nq = Query(\"t\", {}, {})\nm = Mutation(\"u\", {})\n$w = Query(\"t\", {}, {})\nroot = Page()" },
Case { name: "syntax-error-is-a-statement-of-its-own", input: "root = Page([a])\na = Text(\"x\" 1 +)\nb = 1 2\n" },
Case { name: "fenced-with-prose", input: "Here it is:\n```openui-lang\nroot = Page([a])\n```\nand a note\n```\na = Text(\"in a second fence\")\n```\n" },
Case { name: "comments", input: "root = Page([a]) // the page\n# a heading comment\na = Text(\"#not a comment // nor this\")\n" },
Case { name: "markers", input: "]]>openui:content?library=pack%401\nroot = Page([a])\na = Text(\"x\")\n]]>openui:context\n{\"formState\": {}}\n]]>openui:end\n" },
Case { name: "unterminated-at-end", input: "root = Page([a\na = Text(\"open" },
]
}
/// The ingest cases (L2): the tree, its nodes, its errors.
pub fn ingest_cases() -> Vec<Case> {
vec![
Case { name: "hoisting", input: "root = Page([card], \"Home\")\ncard = Card(\"Hello\", [t1, t2], \"sub\")\nt1 = Text(\"one\")\nt2 = Text(\"two\", 14)\n" },
Case { name: "unresolved-left-out", input: "root = Page([a, missing, b])\na = Text(\"a\")\nb = Text(\"b\")\n" },
Case { name: "orphans", input: "root = Page([a])\na = Text(\"a\")\nold = Text(\"old\")\n" },
Case { name: "inline-calls-and-keys", input: "root = Page([Card(\"c\", [Text(\"x\"), Text(\"y\")]), Text(\"z\")])\n" },
Case { name: "used-twice", input: "root = Page([t, t])\nt = Text(\"again\")\n" },
Case { name: "null-skips-an-optional", input: "root = Page([c])\nc = Card(\"t\", null, \"sub\")\n" },
Case { name: "data-props", input: "root = Page([ch])\nch = Chart([\"a\", \"b\"], [1, -2.5, 3e2], {stacked: true, \"max value\": null, colours: ['red', \"blue\"]})\n" },
Case { name: "actions", input: "root = Page([b1, b2, b3, b4, b5, b6])\nb1 = Button(\"none\", @Go())\nb2 = Button(\"one\", @Open(\"x\"))\nb3 = Button(\"two\", @Send(\"a\", 2))\nb4 = Button(\"list\", @Pick([1, 2]))\nb5 = Button(\"wrapped\", Action([@Go()]))\nb6 = Button(\"by name\", go)\ngo = @ToAssistant(\"hi\", {id: 1})\n" },
Case { name: "only-of-children", input: "root = Page([tabs])\ntabs = Tabs([t1, t2])\nt1 = Tab(\"One\", [Text(\"1\")])\nt2 = Tab(\"Two\")\n" },
Case { name: "derived-order", input: "root = Page([n])\nn = Note(\"by\", \"the body\", [Text(\"inside\")])\n" },
Case { name: "duplicate-later-wins", input: "root = Page([a])\na = Text(\"first\")\na = Text(\"second\")\n" },
Case { name: "cycle", input: "root = Page([a])\na = Card(\"a\", [b])\nb = Card(\"b\", [a])\n" },
Case { name: "fallback-entry", input: "t = Text(\"x\")\npage = Page([t])\n" },
Case { name: "escapes", input: "root = Page([a, b, c])\na = Text(\"tab\\there \\u00e9\")\nb = Text('it\\'s')\nc = Text(\"bad \\q escape\")\n" },
Case { name: "empty", input: "" },
]
}
/// The refusals (L5): negative controls, each refused with its code at its line.
pub struct Refusal {
pub name: &'static str,
pub input: &'static str,
pub code: &'static str,
pub line: u32,
}
pub fn refusal_cases() -> Vec<Refusal> {
vec![
Refusal { name: "unknown-component", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Chartt([1], [2])\n", code: "unknown-component", line: 3 },
Refusal { name: "unknown-component-inline", input: "root = Page([\n Text(\"kept\"),\n Graph(1)\n])\n", code: "unknown-component", line: 3 },
Refusal { name: "missing-required", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Card()\n", code: "missing-required", line: 3 },
Refusal { name: "missing-required-never-written", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Text(words)\n", code: "missing-required", line: 3 },
Refusal { name: "null-required", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Text(null)\n", code: "null-required", line: 3 },
Refusal { name: "type-mismatch-required", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Text(42)\n", code: "type-mismatch", line: 3 },
Refusal { name: "type-mismatch-optional-left-out", input: "root = Page([a])\na = Text(\"kept\", \"big\")\n", code: "type-mismatch", line: 2 },
Refusal { name: "type-mismatch-widget-as-data", input: "root = Page([a])\na = Card(Text(\"no\"))\n", code: "type-mismatch", line: 2 },
Refusal { name: "type-mismatch-bare-child", input: "root = Page(a)\na = Text(\"no list\")\n", code: "type-mismatch", line: 1 },
Refusal { name: "type-mismatch-data-child", input: "root = Page([a, \"text\"])\na = Text(\"kept\")\n", code: "type-mismatch", line: 1 },
Refusal { name: "excess-args", input: "root = Page([a])\na = Text(\"kept\", 12, \"extra\")\n", code: "excess-args", line: 2 },
Refusal { name: "disallowed-child", input: "root = Page([tabs])\ntabs = Tabs([t1, x])\nt1 = Tab(\"kept\")\nx = Text(\"not a tab\")\n", code: "disallowed-child", line: 4 },
Refusal { name: "state-statement", input: "root = Page([a])\na = Text(\"kept\")\n$count = 0\n", code: "not-declarative", line: 3 },
Refusal { name: "state-in-a-prop", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Text($name)\n", code: "not-declarative", line: 3 },
Refusal { name: "query-statement", input: "root = Page([a])\na = Text(\"kept\")\nrows = Query(\"list\", {}, {rows: []})\n", code: "not-declarative", line: 3 },
Refusal { name: "operator", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Text(\"a\" + \"b\")\n", code: "not-declarative", line: 3 },
Refusal { name: "conditional", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Text(x\n ? \"y\"\n : \"z\")\n", code: "not-declarative", line: 3 },
Refusal { name: "member-access", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Text(data.title)\n", code: "not-declarative", line: 3 },
Refusal { name: "builtin-function", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Text(\"n\", @Count([1, 2]))\n", code: "not-declarative", line: 3 },
Refusal { name: "state-action", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Button(\"x\", @Set($s, 1))\n", code: "not-declarative", line: 3 },
Refusal { name: "action-sequence", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Button(\"x\", Action([@A(), @B()]))\n", code: "not-declarative", line: 3 },
Refusal { name: "inline-reserved", input: "root = Page([a, b])\na = Text(\"kept\")\nb = Text(\"x\", Query(\"t\", {}, 1))\n", code: "inline-reserved", line: 3 },
Refusal { name: "syntax", input: "root = Page([a])\na = Text(\"kept\")\nb = Text(\"x\" \"y\" +)\n", code: "syntax", line: 3 },
Refusal { name: "no-root", input: "a = Text(\"nothing calls the root\")\n", code: "no-root", line: 0 },
Refusal { name: "root-not-a-widget", input: "root = 42\n", code: "no-root", line: 1 },
Refusal { name: "parse-failed", input: "just some prose a model wrote\n", code: "parse-failed", line: 0 },
]
}
/// A streaming case (L3): the chunks in order.
pub struct StreamCase {
pub name: &'static str,
pub chunks: Vec<&'static str>,
}
pub fn stream_cases() -> Vec<StreamCase> {
vec![
StreamCase {
name: "worked-example",
chunks: vec!["root = Page([card], \"Ho", "me\")\ncard = Card(\"Hel", "lo\", [t1, t2])\nt1 = Te", "xt(\"one\")\nt2 = Text(\"two\")\n"],
},
StreamCase { name: "pending-title-grows", chunks: vec!["root = Page([c])\nc = Card(\"A", " long", " title\")"] },
StreamCase { name: "held-until-required-arrives", chunks: vec!["root = Page([c])\nc = Card(", "\"t\")\n"] },
StreamCase { name: "errors-wait-for-completion", chunks: vec!["root = Page([a, b])\na = Text(\"ok\")\nb = Text(4", "2)\nc = 1"] },
StreamCase { name: "pending-does-not-overwrite", chunks: vec!["root = Page([a])\na = Text(\"done\")\nx = 1\na = Text(\"partial", "\")\n"] },
StreamCase { name: "ternary-across-chunks", chunks: vec!["root = Page([a])\na = Text(\"t\")\nx = c\n", "? 1 : 2\ny = 3"] },
StreamCase { name: "fence-arrives-late", chunks: vec!["root = Page([a])\na = Text(\"x\")\n", "```\nroot = Page()\n```"] },
StreamCase { name: "open-string-mid-escape", chunks: vec!["root = Page([a])\na = Text(\"back\\", "slash\")"] },
StreamCase { name: "unknown-then-known", chunks: vec!["root = Page([a, b])\nb = Text(\"b\")\na = Te", "x", "t(\"a\")\n"] },
]
}
/// The writer cases (L4): trees and the canonical text of each.
pub fn write_cases() -> Vec<(&'static str, Widget)> {
let action = |id: &str, p: Option<PropValue>| PropValue::Action { id: id.into(), payload: p.map(Box::new) };
let mut obj = BTreeMap::new();
obj.insert("stacked".into(), PropValue::Bool(true));
obj.insert("max value".into(), PropValue::Number(1e21));
obj.insert("null".into(), PropValue::Null);
vec![
(
"page-of-cards",
Widget::leaf("Page")
.prop("title", "Home")
.child(Widget::leaf("Card").prop("title", "One").child(Widget::leaf("Text").prop("text", "a")))
.child(Widget::leaf("Card").prop("title", "Two").prop("subtitle", "sub")),
),
("middle-absent-is-null", Widget::leaf("Page").child(Widget::leaf("Card").prop("title", "t").prop("subtitle", "s"))),
(
"numbers-and-strings",
Widget::leaf("Page").child(Widget::leaf("Text").prop("text", "quote \" back \\ line\nfeed é \u{1}").prop("size", 0.1)).child(Widget::leaf("Text").prop("text", "neg").prop("size", -0.0)).child(
Widget::leaf("Text").prop("text", "small").prop("size", 1e-7),
),
),
(
"data",
Widget::leaf("Page").child(
Widget::leaf("Chart")
.prop("labels", PropValue::List(vec!["a".into(), PropValue::Null]))
.prop("values", PropValue::List(vec![PropValue::Number(1.0), PropValue::List(vec![])]))
.prop("options", PropValue::Object(obj)),
),
),
(
"actions",
Widget::leaf("Page")
.child(Widget::leaf("Button").prop("label", "none").prop("action", action("Go", None)))
.child(Widget::leaf("Button").prop("label", "one").prop("action", action("Open", Some("x".into()))))
.child(Widget::leaf("Button").prop("label", "two").prop("action", action("Send", Some(PropValue::List(vec!["a".into(), PropValue::Number(2.0)])))))
.child(Widget::leaf("Button").prop("label", "list of one").prop("action", action("Pick", Some(PropValue::List(vec![PropValue::Number(1.0)])))))
.child(Widget::leaf("Button").prop("label", "null").prop("action", action("Clear", Some(PropValue::Null))).prop("primary", true)),
),
("derived-order", Widget::leaf("Page").child(Widget::leaf("Note").prop("body", "b").child(Widget::leaf("Text").prop("text", "in")))),
("tabs", Widget::leaf("Page").child(Widget::leaf("Tabs").child(Widget::leaf("Tab").prop("title", "x")).child(Widget::leaf("Tab").prop("title", "y")))),
]
}
/// A snapshot as the pack states it: the tree, its nodes, the errors without
/// their messages (prose for a model, not normative), the unresolved names,
/// the orphans, and whether the text ended open.
pub fn snapshot_json(s: &Snapshot) -> Value {
let errors: Vec<Value> = s
.errors
.iter()
.map(|e| {
let mut m = serde_json::Map::new();
m.insert("line".into(), json!(e.line));
m.insert("statement".into(), json!(e.statement));
m.insert("code".into(), json!(e.code.as_str()));
if let Some(c) = &e.component {
m.insert("component".into(), json!(c));
}
if let Some(p) = &e.prop {
m.insert("prop".into(), json!(p));
}
if e.warning {
m.insert("warning".into(), json!(true));
}
Value::Object(m)
})
.collect();
json!({
"tree": s.tree,
"nodes": s.nodes,
"errors": errors,
"unresolved": s.unresolved,
"orphans": s.orphans,
"incomplete": s.incomplete,
})
}
/// Canonical JSON: keys sorted, two-space indent, a final line feed.
pub fn canonical(v: &Value) -> Vec<u8> {
fn sort(v: &Value) -> Value {
match v {
Value::Object(m) => {
let b: BTreeMap<&String, Value> = m.iter().map(|(k, x)| (k, sort(x))).collect();
Value::Object(b.into_iter().map(|(k, x)| (k.clone(), x)).collect())
}
Value::Array(a) => Value::Array(a.iter().map(sort).collect()),
o => o.clone(),
}
}
let mut out = serde_json::to_vec_pretty(&sort(v)).expect("json");
out.push(b'\n');
out
}
/// The library as the pack publishes it: each widget's schema, its positional
/// order in effect written out, by name.
pub fn library_json(lib: &Library) -> Value {
let mut widgets = serde_json::Map::new();
for name in lib.names() {
let s = lib.registry.get(&name).expect("listed");
let mut v = serde_json::to_value(s).expect("schema");
v["positional"] = json!(s.positional_order());
widgets.insert(name, v);
}
json!({ "root": lib.root, "widgets": widgets })
}
/// Every file of the pack, by name, with its bytes; `MANIFEST.json` last.
pub fn pack() -> Vec<(String, Vec<u8>)> {
let lib = test_library();
let mut files: Vec<(String, Vec<u8>)> = Vec::new();
files.push(("library.json".into(), canonical(&library_json(&lib))));
let l1: Vec<Value> = statement_cases()
.iter()
.map(|c| {
let s = parse(&lib, c.input);
json!({ "name": c.name, "input": c.input, "statements": s.statements, "incomplete": s.incomplete })
})
.collect();
files.push(("l1-statements.json".into(), canonical(&json!({ "class": "L1", "cases": l1 }))));
let l2: Vec<Value> = ingest_cases()
.iter()
.map(|c| json!({ "name": c.name, "input": c.input, "expected": snapshot_json(&parse(&lib, c.input)) }))
.collect();
files.push(("l2-ingest.json".into(), canonical(&json!({ "class": "L2", "cases": l2 }))));
let l3: Vec<Value> = stream_cases()
.iter()
.map(|c| {
let mut p = StreamParser::new(&lib);
let after: Vec<Value> = c.chunks.iter().map(|ch| snapshot_json(&p.push(ch))).collect();
let end = snapshot_json(&p.finish());
json!({ "name": c.name, "chunks": c.chunks, "after_each": after, "at_end": end })
})
.collect();
files.push(("l3-stream.json".into(), canonical(&json!({ "class": "L3", "cases": l3 }))));
let l4: Vec<Value> = write_cases()
.iter()
.map(|(name, tree)| {
let text = write(tree, &lib).expect("the writer cases are writable");
json!({ "name": name, "tree": tree, "text": text })
})
.collect();
files.push(("l4-write.json".into(), canonical(&json!({ "class": "L4", "cases": l4 }))));
let l5: Vec<Value> = refusal_cases()
.iter()
.map(|c| json!({ "name": c.name, "input": c.input, "code": c.code, "line": c.line, "expected": snapshot_json(&parse(&lib, c.input)) }))
.collect();
files.push(("l5-refusals.json".into(), canonical(&json!({ "class": "L5", "cases": l5 }))));
let mut digests = serde_json::Map::new();
for (name, bytes) in &files {
digests.insert(name.clone(), json!(format!("sha256:{}", hex(&Sha256::digest(bytes)))));
}
files.push(("MANIFEST.json".into(), canonical(&json!({ "pack": "joule-ui-lang/v1", "files": digests }))));
files
}
fn hex(b: &[u8]) -> String {
b.iter().map(|x| format!("{x:02x}")).collect()
}

179
joule-ui/joule-ui-rs/crates/joule-ui-lang/src/write.rs Normal file
View file

@ -0,0 +1,179 @@
//! The canonical writer (SPEC §11.7): a widget tree back to the language, one
//! statement a widget, in pre-order, so that streamed it renders from the top
//! down, and parsed it gives the same tree.
use crate::ingest::{BUILTINS, STATE_ACTIONS};
use crate::{Library, RESERVED};
use joule_ui_core::{validate, PropValue, ValidationError, Widget};
use std::collections::HashMap;
/// Why a tree cannot be written.
#[derive(Debug, Clone, PartialEq)]
pub enum WriteError {
/// The tree does not pass the registry's validation.
Invalid(Vec<ValidationError>),
/// A widget whose name does not start with an uppercase letter, or is
/// reserved (`Query`, `Mutation`, `Action`): no call can name it.
NotCallable(String),
/// A prop the widget's positional order does not list.
NotPositional { widget: String, prop: String },
/// A prop whose value is `null`: a `null` argument is an absent one.
NullProp { widget: String, prop: String },
/// A number that is not finite.
NotFinite { widget: String, prop: String },
/// An action id that is not a name a call can carry (`@Id(...)`), or one the
/// language reserves.
ActionId(String),
}
impl std::fmt::Display for WriteError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
WriteError::Invalid(e) => write!(f, "the tree does not validate against the registry ({} error(s))", e.len()),
WriteError::NotCallable(w) => write!(f, "no call can name the widget `{w}`"),
WriteError::NotPositional { widget, prop } => write!(f, "`{prop}` is not in the positional order of `{widget}`"),
WriteError::NullProp { widget, prop } => write!(f, "`{prop}` of `{widget}` is null, and a null argument is an absent one"),
WriteError::NotFinite { widget, prop } => write!(f, "`{prop}` of `{widget}` holds a number that is not finite"),
WriteError::ActionId(id) => write!(f, "the action id `{id}` cannot be written as `@{id}(...)`"),
}
}
}
impl std::error::Error for WriteError {}
fn is_name(s: &str) -> bool {
let mut c = s.chars();
c.next().is_some_and(|f| f.is_ascii_alphabetic() || f == '_') && c.all(|x| x.is_ascii_alphanumeric() || x == '_')
}
/// Write `tree` in the canonical form (SPEC §11.7).
pub fn write(tree: &Widget, lib: &Library) -> Result<String, WriteError> {
let errs = validate(tree, &lib.registry);
if !errs.is_empty() {
return Err(WriteError::Invalid(errs));
}
// Names, in pre-order: `root`, then each widget's name with a lowercase
// first letter and a count of its own, from 1.
let mut flat: Vec<&Widget> = Vec::new();
fn walk<'a>(w: &'a Widget, out: &mut Vec<&'a Widget>) {
out.push(w);
for c in &w.children {
walk(c, out);
}
}
walk(tree, &mut flat);
let mut counts: HashMap<String, usize> = HashMap::new();
let mut names: Vec<String> = Vec::with_capacity(flat.len());
for (i, w) in flat.iter().enumerate() {
if i == 0 {
names.push("root".into());
continue;
}
let mut base = String::new();
let mut cs = w.name.chars();
if let Some(f) = cs.next() {
base.push(f.to_ascii_lowercase());
}
base.extend(cs);
let n = counts.entry(base.clone()).or_insert(0);
*n += 1;
names.push(format!("{base}{n}"));
}
// The index of each widget's first child in `flat`.
let mut out = String::new();
let mut index = 0usize;
fn emit(w: &Widget, lib: &Library, names: &[String], index: &mut usize, out: &mut String) -> Result<(), WriteError> {
let me = *index;
*index += 1;
if !w.name.starts_with(|c: char| c.is_ascii_uppercase()) || RESERVED.contains(&w.name.as_str()) || !is_name(&w.name) {
return Err(WriteError::NotCallable(w.name.clone()));
}
let schema = lib.registry.get(&w.name).expect("validated");
let order = schema.positional_order();
for p in w.props.keys() {
if !order.contains(p) {
return Err(WriteError::NotPositional { widget: w.name.clone(), prop: p.clone() });
}
}
// The children's names: each child's index in pre-order.
let mut child_names = Vec::new();
let mut at = me + 1;
for c in &w.children {
child_names.push(names[at].clone());
at += count(c);
}
let mut args: Vec<Option<String>> = Vec::new();
for slot in &order {
if slot == "children" {
args.push(if child_names.is_empty() { None } else { Some(format!("[{}]", child_names.join(", "))) });
continue;
}
match w.props.get(slot) {
None => args.push(None),
Some(PropValue::Null) => return Err(WriteError::NullProp { widget: w.name.clone(), prop: slot.clone() }),
Some(v) => args.push(Some(value(v, &w.name, slot)?)),
}
}
while args.last().is_some_and(Option::is_none) {
args.pop();
}
let args: Vec<String> = args.into_iter().map(|a| a.unwrap_or_else(|| "null".into())).collect();
out.push_str(&format!("{} = {}({})\n", names[me], w.name, args.join(", ")));
for c in &w.children {
emit(c, lib, names, index, out)?;
}
Ok(())
}
fn count(w: &Widget) -> usize {
1 + w.children.iter().map(count).sum::<usize>()
}
emit(tree, lib, &names, &mut index, &mut out)?;
Ok(out)
}
/// A number as the canonical form writes it: the shortest decimal that reads
/// back to the same value, in positional notation, with no exponent and no
/// trailing `.0` (`-0` for negative zero).
pub fn number_text(n: f64) -> String {
format!("{n}")
}
fn value(v: &PropValue, widget: &str, prop: &str) -> Result<String, WriteError> {
Ok(match v {
PropValue::Null => "null".into(),
PropValue::Bool(b) => b.to_string(),
PropValue::Number(n) => {
if !n.is_finite() {
return Err(WriteError::NotFinite { widget: widget.into(), prop: prop.into() });
}
number_text(*n)
}
PropValue::Text(s) => serde_json::to_string(s).expect("a string serialises"),
PropValue::List(items) => {
let v: Result<Vec<String>, WriteError> = items.iter().map(|i| value(i, widget, prop)).collect();
format!("[{}]", v?.join(", "))
}
PropValue::Object(m) => {
let mut parts = Vec::new();
for (k, x) in m {
let key = if is_name(k) && !matches!(k.as_str(), "true" | "false" | "null") { k.clone() } else { serde_json::to_string(k).expect("a string serialises") };
parts.push(format!("{key}: {}", value(x, widget, prop)?));
}
format!("{{{}}}", parts.join(", "))
}
PropValue::Action { id, payload } => {
if !is_name(id) || id == "Action" || BUILTINS.contains(&id.as_str()) || STATE_ACTIONS.contains(&id.as_str()) {
return Err(WriteError::ActionId(id.clone()));
}
let args = match payload.as_deref() {
None => String::new(),
Some(PropValue::List(items)) if items.len() >= 2 => {
let v: Result<Vec<String>, WriteError> = items.iter().map(|i| value(i, widget, prop)).collect();
v?.join(", ")
}
Some(p) => value(p, widget, prop)?,
};
format!("@{id}({args})")
}
})
}

20
joule-ui/joule-ui-rs/crates/joule-ui-lang/tests/pack.rs Normal file
View file

@ -0,0 +1,20 @@
//! The published pack (`joule-ui/conformance/lang/v1/`) is exactly what the
//! reference implementation gives: every file regenerated and compared byte for
//! byte, and no file in the directory that the pack does not hold.
use std::path::PathBuf;
#[test]
fn the_published_pack_is_what_the_implementation_gives() {
let dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../conformance/lang/v1");
let pack = joule_ui_lang::vectors::pack();
for (name, bytes) in &pack {
let on_disk = std::fs::read(dir.join(name)).unwrap_or_else(|e| panic!("{name}: {e} (regenerate with `cargo run -p joule-ui-lang --bin gen_lang_vectors`)"));
assert!(on_disk == *bytes, "{name} differs from what the implementation gives (regenerate with `cargo run -p joule-ui-lang --bin gen_lang_vectors`)");
}
let mut listed: Vec<String> = std::fs::read_dir(&dir).unwrap().map(|e| e.unwrap().file_name().to_string_lossy().into_owned()).collect();
listed.sort();
let mut expected: Vec<String> = pack.into_iter().map(|(n, _)| n).collect();
expected.sort();
assert_eq!(listed, expected, "the pack directory holds exactly the pack");
}

35
wai/CHANGELOG.md
View file

@ -6,6 +6,41 @@ wire format may change and such changes are recorded here.
## v0.4.0 — current
### Create: a WAI app described, built as a model writes it, and made a project by `wai create --from app.oui`
- **The widgets** ([`wai-app`](wai-app/), a library with no binary): a joule-ui registry of WAI
widgets, `App`, `Room`, `WatchTogether`, `Show`, `Player`, `Chat`, `Poll`, `Participants`,
`Receipts`, `EnergyMeter` and `Text`, each backed by the room client of `joule-moq-web`
running in a frame of the page. Each kind of room accepts only the widgets its room service
keeps: a watch room holds no `Chat` and no `Poll`, and a line that asks for one is refused
`disallowed-child` at its line. An app holds one room; a poll has 2 to 32 options; a show's
audience is 1 to 255; a watch room's source is a file within the app.
- **Create**: a page where a person describes an app and a model writes it in joule-ui's line
language ([joule-ui SPEC](../joule-ui/SPEC.md) §11), which the page parses as it streams and
renders widget by widget, every widget validated against the registry. The page calls no
model of its own and holds no key: the visitor's own model (its address and their key,
called from the page alone), their coding agent (`wai mcp --toolset app`, whose bridge on
127.0.0.1 the page reads with a pairing code), or a recorded stream. Every action on the page
is sealed into a hash-chained log the page signs with its own key and verifies
(joule-ui-interaction; the figures are estimates, labelled `Estimator`: a page has no meter).
Export hands over the program, `app.oui`, and the command that makes it a project.
- **`wai create <dir> --from app.oui`** (`app.export`; the feature `app`): an ordinary WAI
project, the same kind `wai create --template` writes: its `wai.json`, written by the one
writer every project file has, names the program (`app`) and the room `wai dev` runs for it
(`dev.mode` `room`, `watch` or `show`, a watch room's file, a show's `audience`); beside it
the program in its canonical form, the page and its runtime under `vendor/` (each file
checked against the runtime's listing), a README and an AGENTS.md. No key is in it.
**`wai app guide`** and **`wai app check`** (`app.guide`, `app.check`): the guide a model
writes against, and the app, the room its project file names and its runtime checked.
**`wai mcp --toolset app`**: `app_guide`, `app_check`, `app_stream` (into the person's open
Create page through the bridge) and `app_export`.
- **Every project `wai create` writes carries its licence**, Apache-2.0, as `LICENSE`, and every
example's README says so.
- **`wai dev --show [--audience N]`** (or `"mode": "show"` in `wai.json`): a room with
co-performance, its relay holding the floor to the room's clock. `wai dev`'s front never
serves a path with a segment that starts with a dot, so a project's `.wai/`, where its room's
keys are, stays off the origin.
### Coding agents: `wai mcp`, `wai setup-agent`, and the room service's API described
- **`wai mcp`** (`agent.mcp`; the feature `mcp`, which the quickstart's `wai` holds): WAI's

11
wai/README.md
View file

@ -111,6 +111,7 @@ operation of the toolchain, named by it with `_` for the dot, and the tools come
|---|---|
| `core` (the default) | `envelope_inspect`, `registry_list`, `registry_show`, `receipt_verify`, `package_build`, `conformance_list`, `conformance_run`, `docs_search` |
| `dev` | `dev_start`, `dev_stop`, `room_grant`, `room_join`, `room_leave`, `room_session` |
| `app` (a build with the feature `app`) | `app_guide`, `app_check`, `app_stream`, `app_export`: Create's, below |
`receipt_verify` verifies a delivery receipt, a claim, a live-room session or a stream audit
bundle, and answers with the verifier's verdict and reason code. The `dev` tools, marked
@ -126,8 +127,14 @@ cargo install --path . --no-default-features --features mcp # `wai` with `wai
cd /path/to/your/project && wai setup-agent --toolset core,dev
```
The quickstart's `wai` holds `wai mcp` too. The room service's HTTP API is described, OpenAPI
3.1, in [`joule-moq-room/openapi.json`](../joule-code/joule-code-rs/crates/joule-moq-room/openapi.json),
The quickstart's `wai` holds `wai mcp` too.
**Create** ([`wai-app`](wai-app/)) builds a WAI app from a description: a model writes it in
joule-ui's line language and the page renders it as it streams. Its program, `app.oui`, becomes
a project with `wai create my-app --from app.oui` (a build with the feature `app`): an ordinary
WAI project that `wai app check` checks and `wai dev` runs.
The room service's HTTP API is described, OpenAPI 3.1, in [`joule-moq-room/openapi.json`](../joule-code/joule-code-rs/crates/joule-moq-room/openapi.json),
each operation's `operationId` its name in the toolchain's catalogue.
## SDK consumption

2
wai/examples/android-app/README.md
View file

@ -27,3 +27,5 @@ README covers it.
**With a coding assistant.** Run `wai setup-agent` in this directory: it writes the
instructions an assistant reads and connects it to WAI's tools.
**Licence.** Apache-2.0: `wai create` writes the licence beside the project, as `LICENSE`.

2
wai/examples/apple-app/README.md
View file

@ -29,3 +29,5 @@ that the bundle verifies. The player opens a window: give it an HLS address (the
**With a coding assistant.** Run `wai setup-agent` in this directory: it writes the
instructions an assistant reads and connects it to WAI's tools.
**Licence.** Apache-2.0: `wai create` writes the licence beside the project, as `LICENSE`.

2
wai/examples/encoder-ingest/README.md
View file

@ -28,3 +28,5 @@ first build fetches the repository and compiles the relay and the gateway.
**With a coding assistant.** Run `wai setup-agent` in this directory: it writes the
instructions an assistant reads and connects it to WAI's tools.
**Licence.** Apache-2.0: `wai create` writes the licence beside the project, as `LICENSE`.

2
wai/examples/messaging/README.md
View file

@ -29,3 +29,5 @@ first tab's inbox rings, and answering opens the call, camera and microphone on
**With a coding assistant.** Run `wai setup-agent` in this directory: it writes the
instructions an assistant reads and connects it to WAI's tools.
**Licence.** Apache-2.0: `wai create` writes the licence beside the project, as `LICENSE`.

2
wai/examples/package-and-deliver/README.md
View file

@ -27,3 +27,5 @@ prints its commands, live packaging among them.
**With a coding assistant.** Run `wai setup-agent` in this directory: it writes the
instructions an assistant reads and connects it to WAI's tools.
**Licence.** Apache-2.0: `wai create` writes the licence beside the project, as `LICENSE`.

2
wai/examples/remote-desktop/README.md
View file

@ -29,3 +29,5 @@ session's records; the independent verifier in the WAI repository checks them:
**With a coding assistant.** Run `wai setup-agent` in this directory: it writes the
instructions an assistant reads and connects it to WAI's tools.
**Licence.** Apache-2.0: `wai create` writes the licence beside the project, as `LICENSE`.

2
wai/examples/verify-receipts/README.md
View file

@ -27,3 +27,5 @@ command line.
**With a coding assistant.** Run `wai setup-agent` in this directory: it writes the
instructions an assistant reads and connects it to WAI's tools.
**Licence.** Apache-2.0: `wai create` writes the licence beside the project, as `LICENSE`.

2
wai/examples/video-room/README.md
View file

@ -23,3 +23,5 @@ origin; the API is described at https://wai.transaction.science/docs/api/.
**With a coding assistant.** Run `wai setup-agent` in this directory: it writes the
instructions an assistant reads and connects it to WAI's tools.
**Licence.** Apache-2.0: `wai create` writes the licence beside the project, as `LICENSE`.

2
wai/examples/watch-together/README.md
View file

@ -26,3 +26,5 @@ but a WAV file. Or run `wai dev --watch film.webm --watch-ms 90000`.
**With a coding assistant.** Run `wai setup-agent` in this directory: it writes the
instructions an assistant reads and connects it to WAI's tools.
**Licence.** Apache-2.0: `wai create` writes the licence beside the project, as `LICENSE`.

1
wai/wai-app/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
/e2e/node_modules/

553
wai/wai-app/Cargo.lock generated Normal file
View file

@ -0,0 +1,553 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "arrayvec"
version = "0.7.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3fb67a6e08acf24fdeccbac2cb6ac4305825bd1f117462e0e6f2f193345ad56"
[[package]]
name = "base64ct"
version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
[[package]]
name = "blake3"
version = "1.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d9e454fc11f76977dc803893aff6304ed33d6a26efae8696573bea74baa27ae"
dependencies = [
"arrayvec",
"cc",
"cfg-if",
"constant_time_eq",
"cpufeatures 0.3.1",
]
[[package]]
name = "block-buffer"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
dependencies = [
"generic-array",
]
[[package]]
name = "bs58"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf88ba1141d185c399bee5288d850d63b8369520c1eafc32a0430b5b6c287bf4"
dependencies = [
"tinyvec",
]
[[package]]
name = "bumpalo"
version = "3.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
[[package]]
name = "cc"
version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "86e055fa6e3e40ad6187ba25b6855eeb635d5271f8e0f0e1e9a5826b8d75bea0"
dependencies = [
"find-msvc-tools",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600"
[[package]]
name = "const-oid"
version = "0.9.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8"
[[package]]
name = "constant_time_eq"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b"
[[package]]
name = "cpufeatures"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
dependencies = [
"libc",
]
[[package]]
name = "cpufeatures"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566"
dependencies = [
"libc",
]
[[package]]
name = "crypto-common"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
"typenum",
]
[[package]]
name = "curve25519-dalek"
version = "4.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be"
dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"curve25519-dalek-derive",
"digest",
"fiat-crypto",
"rustc_version",
"subtle",
"zeroize",
]
[[package]]
name = "curve25519-dalek-derive"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "der"
version = "0.7.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb"
dependencies = [
"const-oid",
"zeroize",
]
[[package]]
name = "digest"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [
"block-buffer",
"crypto-common",
]
[[package]]
name = "ed25519"
version = "2.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53"
dependencies = [
"pkcs8",
"signature",
]
[[package]]
name = "ed25519-dalek"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9"
dependencies = [
"curve25519-dalek",
"ed25519",
"serde",
"sha2",
"subtle",
"zeroize",
]
[[package]]
name = "fiat-crypto"
version = "0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d"
[[package]]
name = "find-msvc-tools"
version = "0.1.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1c66915cc5391097de7a2b0b972e8505aa33c0df401fd3a6811dbdb32881ba48"
[[package]]
name = "generic-array"
version = "0.14.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
dependencies = [
"typenum",
"version_check",
]
[[package]]
name = "getrandom"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0"
dependencies = [
"cfg-if",
"libc",
"wasi",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "joule-ui-core"
version = "0.1.0"
dependencies = [
"serde",
"serde_json",
"thiserror",
]
[[package]]
name = "joule-ui-interaction"
version = "0.1.0"
dependencies = [
"blake3",
"ed25519-dalek",
"serde",
"serde_json",
"thiserror",
]
[[package]]
name = "joule-ui-lang"
version = "0.1.0"
dependencies = [
"joule-ui-core",
"serde",
"serde_json",
"sha2",
]
[[package]]
name = "libc"
version = "0.2.191"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65cd87c9a6d60536580b316ad985726104730a5151c99845f2490428ef91240a"
[[package]]
name = "memchr"
version = "2.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "pkcs8"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7"
dependencies = [
"der",
"spki",
]
[[package]]
name = "proc-macro2"
version = "1.0.107"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rand_core"
version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c"
dependencies = [
"getrandom",
]
[[package]]
name = "rustc_version"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
dependencies = [
"semver",
]
[[package]]
name = "rustversion"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
[[package]]
name = "semver"
version = "1.0.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd"
[[package]]
name = "serde"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.9",
]
[[package]]
name = "serde_json"
version = "1.0.154"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7e9cc8b1b85264074fbcc02a88680c4096b1e47df8f739dceb03bf482f04bd6"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "sha2"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283"
dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"digest",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "signature"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de"
dependencies = [
"rand_core",
]
[[package]]
name = "spki"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d"
dependencies = [
"base64ct",
"der",
]
[[package]]
name = "subtle"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "syn"
version = "2.0.119"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "syn"
version = "3.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d78c8dee4c7bf0e14673097256fed6142ce9d3b85a408189d07482442145823b"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "thiserror"
version = "2.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09e52cb86a36cede5cb101bf8908837b3e4c6e5e59fe7fd85c23fb56200d189e"
dependencies = [
"thiserror-impl",
]
[[package]]
name = "thiserror-impl"
version = "2.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe5197923287db20a58125f0bc85c062f7f2c892de97b18c356f9efb14b28524"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.9",
]
[[package]]
name = "tinyvec"
version = "1.13.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fd3ca314f692efd6c868f8408f53fe444634a845f96c028b97d35f6a1f79f0ee"
[[package]]
name = "typenum"
version = "1.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
[[package]]
name = "unicode-ident"
version = "1.0.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a2c754d6c33795a1c324727428e5a7dedb5b06195f9890bdbcba760d3e246563"
[[package]]
name = "version_check"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "wai-app"
version = "0.1.0"
dependencies = [
"bs58",
"ed25519-dalek",
"joule-ui-core",
"joule-ui-interaction",
"joule-ui-lang",
"serde",
"serde_json",
"sha2",
"wasm-bindgen",
]
[[package]]
name = "wasi"
version = "0.11.1+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
[[package]]
name = "wasm-bindgen"
version = "0.2.127"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b70935747edd64d89de3efa29d73789b806c15798f8e7dca4d8ac356b50ce70"
dependencies = [
"cfg-if",
"once_cell",
"rustversion",
"wasm-bindgen-macro",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.127"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77775f8f3f7217702089053b94958f8f54061a3f663417df76e19cbdcca29bc1"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
]
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.127"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e11d33f857dc2fb11b8bc75aee111aa9cbeb12cd9f25efd3d4c2a3dd4e235284"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn 2.0.119",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.127"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ef64dbcc55df09c7e5a46182d181c2cfa3e925f3da937ea764728b4bbb9dcbf"
dependencies = [
"unicode-ident",
]
[[package]]
name = "zeroize"
version = "1.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e13084392c5e4bc371903e2935a5eaeed24905a7511356b883835e18a78f6879"
[[package]]
name = "zmij"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"

40
wai/wai-app/Cargo.toml Normal file
View file

@ -0,0 +1,40 @@
[package]
name = "wai-app"
version = "0.1.0"
edition = "2024"
rust-version = "1.97"
license = "Apache-2.0"
description = "WAI's widgets for joule-ui, and Create, as a library with no binary: a registry of WAI widgets (rooms, watching together, a show, chat, a poll, the room's video, receipts, energy) for agent-emitted surfaces, the guide a model writes against, the plan of the room an app needs, the files of a starter project (which `wai create --from app.oui` writes), a WebAssembly build for the page that renders an app while a model streams it, and the bridge on loopback a coding agent streams a program into that page through (`wai mcp --toolset app`)."
repository = "https://git.transaction.science/Transaction-Science/open-standards"
publish = false
# Its own workspace and Cargo.lock, like the other crates beside wai-rs.
[workspace]
# A library: `wai` (wai-rs, the feature `app`) is its command line and its MCP server, and
# the page's WebAssembly build is the cdylib.
[lib]
crate-type = ["cdylib", "rlib"]
[dependencies]
joule-ui-core = { path = "../../joule-ui/joule-ui-rs/crates/joule-ui-core" }
joule-ui-lang = { path = "../../joule-ui/joule-ui-rs/crates/joule-ui-lang" }
joule-ui-interaction = { path = "../../joule-ui/joule-ui-rs/crates/joule-ui-interaction" }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
ed25519-dalek = "2"
sha2 = "0.10"
bs58 = "0.5"
# Pinned to the wasm-bindgen CLI the build runs: the two must match exactly.
wasm-bindgen = { version = "=0.2.127", optional = true }
[features]
default = []
# The page's WebAssembly surface (src/web.rs), built for wasm32-unknown-unknown with
# `--no-default-features --features web`.
web = ["dep:wasm-bindgen"]
[profile.release]
opt-level = "s"
lto = true
codegen-units = 1

68
wai/wai-app/README.md Normal file
View file

@ -0,0 +1,68 @@
# wai-app
WAI's widgets for joule-ui, and **Create**: describe a WAI app, watch it build itself
from WAI widgets while a model writes it, try its room, and take it away as a project. A
library with no binary of its own: `wai`, built with its feature `app`, is its command line
and its MCP server. The design is in [`joule-ui/docs/create.md`](../../joule-ui/docs/create.md);
the line language a model writes the app in is joule-ui's (`joule-ui/SPEC.md` §11).
## The widgets
A joule-ui registry (`src/registry.rs`), each widget backed by the room client of
`joule-moq-web` (`web/components.js`):
| Widget | What it is | Holds |
|---|---|---|
| `App(children, title)` | the app | one room; `Text`, `Receipts`, `EnergyMeter` |
| `Room(children, title)` | a live room: join with camera and microphone, or watch | `Player`, `Chat`, and any room's widgets |
| `WatchTogether(source, children, title)` | a watch room: one file, watched in sync | any room's widgets (a watch room keeps the shared controls alone) |
| `Show(children, title, audience)` | a show: hosts on camera, an audience that votes | `Player`, `Chat`, `Poll`, and any room's widgets |
| `Player(title)` | the room's video, laid out from its signed record | |
| `Chat(title)` | the room's chat, each message a signed, verified event | |
| `Poll(question, options, seconds)` | a vote the show's lead opens and its clock closes and signs | |
| `Participants(title)` | who is present, from the room's lifecycle | |
| `Receipts(title)` | the page's sealed log of every action, verified | |
| `EnergyMeter(title)` | each action's energy, and how it was obtained | |
| `Text(text)` | a paragraph | |
Any room's widgets: `Participants`, `Receipts`, `EnergyMeter`, `Text`. An app holds one
room; a poll has 2 to 32 options; a show's audience is 1 to 255; a watch room's source is
a file within the app (`src/plan.rs`). A line that breaks a rule is refused with its line
and a reason code, and does not render.
## The pieces
- `src/guide.rs`: what a model is told before it writes an app, made from the registry
(`wai app guide`, the tool `app_guide`).
- `src/lib.rs`: `check` and `Create`, a program as a snapshot the page renders, whole or
as it streams, with the room the app needs (`wai app check`, the tool `app_check`).
- `src/export.rs`: the files of a starter project for a program, and the facts its project
file holds: `app.oui`, `index.html`, `README.md`, `AGENTS.md`, `.gitignore`,
`vendor.json`, `vendor/` (the runtime), and for a watch room `media/README.md`.
`wai create <dir> --from app.oui` writes them, with `wai.json` and `LICENSE` from the
writer every `wai create` project has. No key is ever in it: `wai dev` makes development
keys in `.wai/` at run time.
- `src/runtime.rs`: the runtime an export carries, read by its listing
(`create/wai-app-runtime.json`) and checked file by file.
- `src/bridge.rs`: the bridge on 127.0.0.1 that carries a program from a coding agent into
the person's open Create page (`wai mcp --toolset app`, the tool `app_stream`).
- `src/web.rs` (feature `web`): the page's WebAssembly surface, with the interaction log
every action on the page is sealed into (joule-ui-interaction).
- `web/`: the streaming renderer, the components, the room frame, the Create page and the
exported app's page; `web/runtime.mjs` lays the runtime out as the site serves it.
## Run it
```
cargo test # registry, plan, guide, export, runtime, bridge
node --test web/web.test.mjs # the renderer and the model streams
e2e/run.sh # Create in a browser, with wai dev, wai create --from and wai app check
wai app guide # what a model is told (wai built with --features app)
wai app check app.oui # a program's tree, errors and room
wai create my-app --from app.oui --runtime <dir> # a starter project; then wai app check and wai dev in it
wai mcp --toolset app # Create's tools, for a coding agent
```
## Licence
Apache-2.0, and every project `wai create` writes from an app is too.

438
wai/wai-app/e2e/create.e2e.mjs Normal file
View file

@ -0,0 +1,438 @@
// Create, end to end, in the host's browser: a recorded model stream fed to the page, the
// app rendering as it arrives, its room joined on a real room service that `wai dev` runs, a
// poll opened and answered, the app taken away as app.oui and made a project by
// `wai create --from app.oui`, and that project checked and run with `wai dev`.
//
// node create.e2e.mjs --rt <runtime dir> --relay <joule-moq-relay> --room <joule-moq-room>
// --wai <wai, built with app> --web <wai dev's room client> --out <dir>
// [--browser <executable>] [--site <dist>]
//
// With --site, the WAI site's build is served instead of the runtime's directory, and its
// /create/ page is driven: the page as the site serves it, with its runtime under /wai-app/.
//
// Scenarios, in order:
// create the `show` recording replayed into Create, served by `wai dev --show` (its relay,
// its room service): widgets appear while the stream is still arriving and only grow
// in number; every snapshot's tree validates; the room widget's frame loads the room
// client while later lines still stream; the host joins (the room's clock records
// it), sends a chat, opens the poll; the page's sealed log holds each action and
// verifies; Export hands over app.oui, the program in its canonical form, and the
// command that makes it a project; `wai create --from app.oui` writes the project
// built the exported project: `wai app check` passes, and refuses it with its room not
// its app's, another audience, no room named, a runtime file changed, or a refused
// line; `wai dev` in the project runs its own relay and room service from wai.json,
// its keys in .wai/ (which .gitignore keeps out and the front never serves); there a
// host joins, chats and opens the poll, a viewer watches, reads the chat and votes,
// and both pages show the result the room's clock signed
// refused the `watch-refused` recording: the Chat and Poll a watch room cannot keep never
// render, each refused `disallowed-child` at its line, and export stays off
// own the visitor's own model: a stand-in chat/completions server on loopback streams
// the room recording; the page renders it, and the key it was given goes to that
// endpoint alone, in its Authorization header, and is stored nowhere
// agent the visitor's coding agent: `wai mcp --toolset app` driven over its stdio streams
// the show into the page through its bridge, the page connected with the pairing
// code; `app_export` writes the project `wai create --from` writes
//
// Every wait is bounded; nothing listens beyond loopback.
import fs from "node:fs";
import path from "node:path";
import net from "node:net";
import { createRequire } from "node:module";
import { spawn, execFileSync } from "node:child_process";
import { fileURLToPath } from "node:url";
const HERE = path.dirname(fileURLToPath(import.meta.url));
const LIB = path.resolve(HERE, "../../../joule-code/joule-code-rs/crates/joule-moq-web/e2e/lib.mjs");
const { serveSameOrigin, browserArgs, memguardedBrowser, killStrays, until, sleep } = await import(LIB);
const args = Object.fromEntries(process.argv.slice(2).reduce((a, x, i, all) => (x.startsWith("--") ? [...a, [x.slice(2), all[i + 1] && !all[i + 1].startsWith("--") ? all[i + 1] : "1"]] : a), []));
const need = (k) => { if (!args[k]) { console.error(`create e2e: --${k} is required`); process.exit(2); } return path.resolve(args[k]); };
const RELAY = need("relay"), ROOM = need("room"), WAI = need("wai"), WEB = need("web");
// The runtime's directory (e2e/run.sh lays it out), served as the site with Create's
// development page at /create/index.html; or, with --site, a WAI site build, whose /create/
// page loads the runtime from /wai-app/.
const SITE = args.site ? path.resolve(args.site) : null;
const RT = SITE ? path.join(SITE, "wai-app") : need("rt");
const ROOT = SITE || RT;
const CREATE = SITE ? "/create/" : "/create/index.html";
const OUT = need("out");
const ONLY = (args.only || "create,built,refused,own,agent").split(",");
// Each run starts from no results: the room service's keygen refuses a key file a previous
// run left.
fs.rmSync(OUT, { recursive: true, force: true });
fs.mkdirSync(OUT, { recursive: true });
const require = createRequire(path.join(process.env.PLAYWRIGHT_HOST || path.join(HERE, "node_modules"), "x"));
const { chromium } = require("playwright");
const STEP = 60000;
let failed = 0;
const results = [];
function check(name, ok, detail = "") {
results.push({ name, ok, detail });
if (!ok) failed++;
console.log(`${ok ? "ok " : "FAIL"} ${name}${detail ? ` — ${detail}` : ""}`);
}
process.on("unhandledRejection", (e) => { console.error("unhandled:", (e && e.stack) || e); killStrays(); process.exit(1); });
const freePort = () => new Promise((ok) => { const s = net.createServer(); s.listen(0, "127.0.0.1", () => { const p = s.address().port; s.close(() => ok(p)); }); });
const procs = [];
function run(bin, argv, name, dir, env = {}) {
const log = fs.openSync(path.join(dir, `${name}.log`), "w");
const p = spawn(bin, argv, { stdio: ["ignore", log, log], env: { ...process.env, NO_COLOR: "1", ...env }, cwd: env.CWD || dir });
p.logFile = path.join(dir, `${name}.log`);
procs.push(p);
return p;
}
const tail = (f) => { try { return fs.readFileSync(f, "utf8").split("\n").slice(-8).join(" | "); } catch { return ""; } };
// `wai` in `cwd`: its exit status and its output.
const wai = (argv, cwd) => {
try {
return { code: 0, out: execFileSync(WAI, argv, { cwd, stdio: ["ignore", "pipe", "pipe"] }).toString() };
} catch (e) {
return { code: e.status, out: `${e.stdout || ""}${e.stderr || ""}` };
}
};
const envelope = (out) => JSON.parse(out.split("\n").find((l) => l.startsWith("{")) || "{}");
// `wai dev` in `cwd` (a project, or with --site a directory of pages), on a free port: its
// relay and room service, with this host's binaries and room client. Resolves once it is ready.
async function waiDev(cwd, name, extra = []) {
const port = await freePort();
const dir = path.join(OUT, name);
fs.mkdirSync(dir, { recursive: true });
const p = run(WAI, ["dev", "--port", String(port), "--web", WEB, "--relay", RELAY, "--room", ROOM, ...extra], name, dir, { CWD: cwd });
await until(`wai dev (${name}) is ready`, 90000, () => {
if (p.exitCode != null) throw new Error(`wai dev exited: ${tail(p.logFile)}`);
return /wai dev: ready on/.test(fs.readFileSync(p.logFile, "utf8"));
}, 300);
return { port, url: `http://127.0.0.1:${port}`, proc: p, dir };
}
let browser;
async function launch() {
const bin = args.browser ? path.resolve(args.browser) : null;
const opts = { headless: true, args: browserArgs() };
if (bin) opts.executablePath = memguardedBrowser(bin, OUT).executable;
browser = await chromium.launch(opts);
console.log(`browser: ${browser.version()}`);
}
async function page(url, name) {
const ctx = await browser.newContext({ permissions: ["camera", "microphone"], acceptDownloads: true });
const p = await ctx.newPage();
const logs = [];
p.on("console", (m) => logs.push(`[${m.type()}] ${m.text()}`));
p.on("pageerror", (e) => logs.push(`[pageerror] ${e.stack || e}`));
p.on("response", (r) => { if (r.status() >= 400) logs.push(`[http ${r.status()}] ${r.url()}`); });
p.dump = () => fs.writeFileSync(path.join(OUT, `${name}.console.log`), logs.join("\n"));
p.logs = logs;
await p.goto(url, { waitUntil: "domcontentloaded", timeout: STEP });
return p;
}
const roomOf = (p) => p.evaluate(() => { const r = (window.createTest || window.appTest).ctx.room.current; return r ? { phase: r.phase(), me: r.me(), co: r.co(), chats: r.chats(), checked: r.checked(), present: r.present() } : null; });
const recording = (n) => JSON.parse(fs.readFileSync(path.join(RT, "create/recorded", `wai-app-${n}.json`), "utf8"));
const SHOW = recording("show");
const SHOW_TEXT = SHOW.chunks.join("");
let shared = null; // the Create page's `wai dev`, the page, the exported project
async function scenarioCreate() {
const dir = path.join(OUT, "create");
fs.mkdirSync(dir, { recursive: true });
// The Create page, served by `wai dev` with a show's room: the site is the runtime.
const dev = await waiDev(ROOT, "create-dev", ["--site", ROOT, "--show", "--audience", "1", "--dir", path.join(OUT, "create-dev", "state")]);
check("create: wai dev runs a show's relay and room service behind the Create page", true, dev.url);
const srv = { url: dev.url, close: async () => {} };
const p = await page(`${srv.url}${CREATE}`, "create-host");
await until("Create loads its session", STEP, () => p.evaluate(() => !!window.createTest));
// Watch the page while the stream arrives: when each widget appears, and how much of the
// program had arrived then.
await p.evaluate(() => {
window.__seen = [];
new MutationObserver(() => {
for (const el of document.querySelectorAll("#app [data-widget]")) {
if (el.__seen) continue;
el.__seen = true;
window.__seen.push({ widget: el.dataset.widget, key: el.dataset.key, chars: window.createTest.session.program().length, state: document.getElementById("create").dataset.state });
}
}).observe(document.getElementById("app"), { childList: true, subtree: true });
});
await p.selectOption("#recording", "show");
await p.click("#build");
await until("the stream ends", STEP, () => p.evaluate(() => window.createTest.done >= 1));
const snaps = await p.evaluate(() => window.createTest.snapshots);
const seen = await p.evaluate(() => window.__seen);
const total = SHOW_TEXT.length;
const counts = snaps.map((s) => s.widgets);
check("create: the app renders while the stream arrives, the first widget after a fraction of it", seen.length >= 8 && seen[0].state === "streaming" && seen[0].chars < total / 4, `${seen.length} widgets; first ${seen[0] && seen[0].widget} at ${seen[0] && seen[0].chars} of ${total} characters`);
check("create: widgets appear in steps over the stream, never fewer than before", counts.every((c, i) => i === 0 || c >= counts[i - 1]) && new Set(counts).size >= 6 && seen.filter((x) => x.state === "streaming").length >= 6, `${snaps.length} snapshots; counts ${[...new Set(counts)].join(" → ")}`);
const room = seen.find((x) => x.widget === "Show");
check("create: the room widget (and its room client) appears while later lines still stream", !!room && room.chars < total / 2, room ? `Show at ${room.chars} of ${total}` : "no Show");
const fin = await p.evaluate(() => window.createTest.last);
check("create: the final snapshot has no error, one show, eight widgets", fin.finished && fin.errors.length === 0 && fin.room && fin.room.kind === "show" && fin.nodes.length === 8, JSON.stringify({ errors: fin.errors, room: fin.room && fin.room.kind, n: fin.nodes.length }));
// The host joins the show: the room client in its frame enters the room service's room.
await until("the room client is ready", STEP, () => p.evaluate(() => { const r = window.createTest.ctx.room.current; return r && r.phase() === "idle"; }), 300);
await p.click('[data-widget="Show"] [data-act="join"]');
const host = await until("the host is in the room, publishing", STEP, async () => { const r = await roomOf(p); return r && r.phase === "publishing" && r; }, 300);
const joined = await until("the room's clock records the host joining", STEP, () => p.evaluate((me) => {
const s = window.createTest.ctx.room.current.state();
return s.timeline.some((e) => e.kind === "join" && e.who === "room") && (s.room && s.room.present || []).some((x) => x.principal === me);
}, host.me), 300);
check("create: the host joins a real room: the room service granted its key, the relay took its proof, the room's clock recorded it", !!joined, `principal ${host.me.slice(0, 16)}…`);
// A chat, sent from the Chat widget, signed and verified back into the room's record.
await p.fill('[data-widget="Chat"] input', "Welcome to the show");
await p.click('[data-widget="Chat"] button[type="submit"]');
await until("the host's chat is in the room's verified record", STEP, async () => (await roomOf(p)).chats.some((c) => c.text === "Welcome to the show"), 300);
check("create: a chat sent from the widget comes back as a verified event of the room", true);
// The lead opens the poll.
await until("the host leads the show", STEP, async () => { const r = await roomOf(p); return r.co && r.co.lead; }, 300);
await p.click('[data-widget="Poll"] button.wai-primary');
const w = await until("the room's clock opens the poll's window", STEP, async () => { const r = await roomOf(p); return r.co && r.co.windows.find((x) => x.question === "Which song next?"); }, 300);
check("create: the poll opened from the widget is a window the room's clock signed", !!w, `window ${w.digest.slice(0, 16)}…, closes at ${w.close}`);
// The page's sealed log: every action, each verified as a link of the chain.
const log = await p.evaluate(() => ({ log: window.createTest.session.log(), v: window.createTest.session.verify(), actor: window.createTest.session.actor }));
const acts = log.log.map((e) => e.action);
check("create: the page sealed each action into its hash-chained log, which verifies", ["room.join", "chat.send", "poll.open"].every((a) => acts.includes(a)) && log.v.verified === log.log.length && log.log.every((e) => e.actor === log.actor && e.energy.provenance === "Estimator"), `${acts.join(", ")}; verified ${log.v.verified}`);
const shown = await p.evaluate(() => document.querySelector('[data-widget="Receipts"] .wai-verdict').dataset.ok);
check("create: the Receipts widget shows the page's own verification", shown === "true");
const cut = await p.evaluate(async () => {
const raw = JSON.parse(window.createTest.session.logRaw());
raw.splice(0, 1);
const at = new URL("./wai-app-session.js", new URL(document.getElementById("create").dataset.runtime, location.href));
return (await import(at.href)).verifyLog(JSON.stringify(raw));
});
check("create: the log with its first link cut out does not verify", JSON.parse(cut).broken_at === 0, cut);
// Export: the program, as app.oui, and the command that makes it a project.
const dl = p.waitForEvent("download", { timeout: STEP });
await p.click("#export");
const d = await dl;
const oui = path.join(dir, "app.oui");
await d.saveAs(oui);
const canonical = await p.evaluate(() => window.createTest.session.canonical());
const command = await p.evaluate(() => document.getElementById("export-command").textContent);
check("create: Export hands over app.oui, the program in its canonical form, and the command that makes it a project", d.suggestedFilename() === "app.oui" && fs.readFileSync(oui, "utf8") === canonical && /^wai create [a-z0-9-]+ --from app\.oui$/.test(command), command);
const project = path.join(dir, "exported");
const made = wai(["create", "exported", "--from", "app.oui", "--runtime", RT], dir);
const files = made.code === 0 ? fs.readdirSync(project).sort() : [];
const wj = made.code === 0 ? JSON.parse(fs.readFileSync(path.join(project, "wai.json"), "utf8")) : { dev: {} };
check("create: `wai create --from app.oui` writes the project: the program, wai.json naming its show, the page, its runtime, README, AGENTS.md and LICENSE",
made.code === 0 && ["app.oui", "wai.json", "index.html", "README.md", "AGENTS.md", "LICENSE", "vendor.json", "vendor", ".gitignore"].every((f) => files.includes(f)) && wj.app === "app.oui" && wj.dev.mode === "show" && wj.dev.audience === 1 && fs.readFileSync(path.join(project, "app.oui"), "utf8") === canonical,
made.code === 0 ? `${files.join(" ")}; dev ${JSON.stringify(wj.dev)}` : made.out);
const vendor = made.code === 0 ? JSON.parse(fs.readFileSync(path.join(project, "vendor.json"), "utf8")) : { files: [] };
check("create: the project carries the runtime under vendor/, and no key", vendor.files.length > 10 && !fs.readdirSync(project).some((f) => /key/i.test(f)), `${vendor.files.length} runtime files`);
shared = { dev, srv, p, project, window: w, dir };
}
async function scenarioBuilt() {
if (!shared) return check("built: needs the create scenario", false);
const { project, dir } = shared;
const ok = wai(["app", "check", "--json"], project);
const okv = envelope(ok.out);
check("built: `wai app check` passes on the exported project", ok.code === 0 && okv.ok === true, ((okv.result && okv.result.checked) || []).join("; "));
// Negative controls: the check refuses a project that is not whole.
const broken = path.join(dir, "exported-broken");
const edit = (f, from, to) => { const t = fs.readFileSync(f, "utf8"); if (!from.test(t)) throw new Error(`${f}: ${from}`); fs.writeFileSync(f, t.replace(from, to)); };
for (const [what, mutate, expect] of [
["whose room is not its app's", (d) => edit(path.join(d, "wai.json"), /"mode": "show", "audience": 1/, '"mode": "room"'), /dev\.mode is "room"/],
["whose show has another audience", (d) => edit(path.join(d, "wai.json"), /"audience": 1/, '"audience": 7'), /dev\.audience is 7/],
["that names no room", (d) => edit(path.join(d, "wai.json"), /, "mode": "show", "audience": 1/, ""), /names no room/],
["with a runtime file changed", (d) => fs.appendFileSync(path.join(d, "vendor/create/wai-app-components.js"), "\n// changed\n"), /wai-app-components\.js: its SHA-256/],
["with a line the registry refuses", (d) => edit(path.join(d, "app.oui"), /^(\w+) = Chat\(.*\)$/m, "$1 = Hologram()"), /line \d+: unknown-component/],
]) {
fs.rmSync(broken, { recursive: true, force: true });
fs.cpSync(project, broken, { recursive: true });
mutate(broken);
const r = wai(["app", "check", "--json"], broken);
const v = envelope(r.out);
const failures = ((v.result && v.result.failures) || []).join(" | ");
check(`built: \`wai app check\` refuses a project ${what}`, r.code === 1 && expect.test(failures), failures.slice(0, 200));
}
// The project's own development setup: `wai dev` in it, from its wai.json.
const dev = await waiDev(project, "exported-dev");
const keys = path.join(project, ".wai/dev/room-keys.toml");
const leaked = await fetch(`${dev.url}/.wai/dev/room-keys.toml`).then((r) => r.status).catch((e) => String(e));
check("built: wai dev makes the room's keys in .wai/, which .gitignore keeps out and the front never serves", fs.existsSync(keys) && fs.readFileSync(path.join(project, ".gitignore"), "utf8").includes(".wai/") && leaked !== 200, `GET /.wai/dev/room-keys.toml: ${leaked}`);
const h = await page(`${dev.url}/`, "exported-host");
await until("the exported app renders", STEP, () => h.evaluate(() => window.appTest && document.querySelectorAll("#wai-app [data-widget]").length === 8));
check("built: the exported app renders its eight widgets from app.oui", true);
await until("its room client is ready", STEP, () => h.evaluate(() => { const r = window.appTest.ctx.room.current; return r && r.phase() === "idle"; }), 300);
await h.click('[data-widget="Show"] [data-act="join"]');
await until("the host is in the project's room, publishing", STEP, async () => { const r = await roomOf(h); return r && r.phase === "publishing" && r; }, 300);
await h.fill('[data-widget="Chat"] input', "Welcome to the show");
await h.click('[data-widget="Chat"] button[type="submit"]');
await until("the host's chat is in the room's verified record", STEP, async () => (await roomOf(h)).chats.some((c) => c.text === "Welcome to the show"), 300);
await until("the host leads the show", STEP, async () => { const r = await roomOf(h); return r.co && r.co.lead; }, 300);
const v = await page(`${dev.url}/`, "exported-viewer");
await until("the viewer's app renders", STEP, () => v.evaluate(() => window.appTest && document.querySelectorAll("#wai-app [data-widget]").length === 8));
await until("its room client is ready", STEP, () => v.evaluate(() => { const r = window.appTest.ctx.room.current; return r && r.phase() === "idle"; }), 300);
await v.click('[data-widget="Show"] [data-act="watch"]');
const viewer = await until("the viewer watches", STEP, async () => { const r = await roomOf(v); return r && r.phase === "watching" && r; }, 300);
check("built: in the project's own room, a host joins and a viewer enters", !!viewer.me, viewer.me.slice(0, 16));
await until("the viewer reads the host's chat", STEP, async () => (await roomOf(v)).chats.some((c) => c.text === "Welcome to the show"), 300);
check("built: the viewer reads the host's chat, verified", true);
await h.click('[data-widget="Poll"] button.wai-primary');
await until("a window open now", STEP, async () => { const x = await roomOf(v); return x.co && x.co.windows.find((y) => y.question === "Which song next?" && y.close > x.co.now); }, 300);
await until("the viewer's poll offers its options", STEP, () => v.evaluate(() => [...document.querySelectorAll('[data-widget="Poll"] button.wai-option')].some((b) => !b.disabled)), 300);
await v.click('[data-widget="Poll"] button.wai-option[data-option="1"]');
const chosen = await until("the room's clock selects and signs the result", STEP * 2, async () => {
const t = await v.evaluate(() => document.querySelector('[data-widget="Poll"] .wai-poll-result').textContent);
return /chose “The loud one”, with 1 vote/.test(t) && t;
}, 500);
const hostSees = await until("the host's page shows the same result", STEP, async () => { const t = await h.evaluate(() => document.querySelector('[data-widget="Poll"] .wai-poll-result').textContent); return /The loud one/.test(t) && t; }, 500);
check("built: the viewer's vote is selected by the room's clock, and both pages show it", !!chosen && !!hostSees, chosen);
const vlog = await v.evaluate(() => ({ acts: window.appTest.session.log().map((e) => e.action), v: window.appTest.session.verify() }));
check("built: the exported app sealed the viewer's actions too", vlog.acts.includes("room.watch") && vlog.acts.includes("poll.vote") && vlog.v.verified === vlog.acts.length, vlog.acts.join(", "));
h.dump();
v.dump();
dev.proc.kill("SIGTERM");
}
async function scenarioRefused() {
const srv = shared ? shared.srv : await serveSameOrigin(ROOT, { service: "http://127.0.0.1:9", relayPort: 9 });
const p = await page(`${srv.url}${CREATE}`, "create-refused");
await until("Create loads", STEP, () => p.evaluate(() => !!window.createTest));
await p.evaluate(() => {
window.__ever = new Set();
new MutationObserver(() => { for (const el of document.querySelectorAll("#app [data-widget]")) window.__ever.add(el.dataset.widget); }).observe(document.getElementById("app"), { childList: true, subtree: true });
});
await p.selectOption("#recording", "watch-refused");
await p.click("#build");
await until("the stream ends", STEP, () => p.evaluate(() => window.createTest.done >= 1));
const ever = await p.evaluate(() => [...window.__ever]);
const fin = await p.evaluate(() => window.createTest.last);
const errs = fin.errors.map((e) => `${e.line}:${e.code}:${e.component}`);
check("refused: the Chat and the Poll a watch room cannot keep never render", !ever.includes("Chat") && !ever.includes("Poll") && ever.includes("WatchTogether") && ever.includes("Participants"), ever.join(", "));
check("refused: each is refused disallowed-child at its line", errs.includes("3:disallowed-child:Chat") && errs.includes("4:disallowed-child:Poll"), errs.join(" "));
const listed = await p.evaluate(() => [...document.querySelectorAll("#problems li")].map((li) => li.dataset.code));
const off = await p.evaluate(() => document.getElementById("export").disabled);
check("refused: the page lists the refusals, and export stays off", listed.filter((c) => c === "disallowed-child").length === 2 && off, listed.join(" "));
if (!shared) await srv.close();
}
// The visitor's own model: a stand-in `chat/completions` server on loopback (another origin
// than the page's) that streams the room recording as a model would, token by token. The key
// typed in the page goes to that endpoint alone, in its Authorization header, and is stored
// nowhere.
async function scenarioOwn() {
const dir = path.join(OUT, "own");
fs.mkdirSync(dir, { recursive: true });
const rec = recording("room");
const KEY = "test-key-0f3a9c";
const seen = [];
const http = await import("node:http");
const model = http.createServer((req, res) => {
const cors = { "access-control-allow-origin": req.headers.origin || "*", "access-control-allow-headers": "authorization, content-type", "access-control-allow-methods": "POST" };
if (req.method === "OPTIONS") { res.writeHead(204, cors); return res.end(); }
let body = "";
req.on("data", (d) => { body += d; });
req.on("end", async () => {
seen.push({ path: req.url, auth: req.headers.authorization, body: JSON.parse(body || "{}") });
res.writeHead(200, { ...cors, "content-type": "text/event-stream" });
res.write('data: {"choices":[{"delta":{"role":"assistant"}}]}\n\n');
for (const c of rec.chunks) {
res.write(`data: ${JSON.stringify({ choices: [{ delta: { content: c } }] })}\n\n`);
await sleep(5);
}
res.end("data: [DONE]\n\n");
});
});
await new Promise((ok) => model.listen(0, "127.0.0.1", ok));
const endpoint = `http://127.0.0.1:${model.address().port}/v1/chat/completions`;
const srv = shared ? shared.srv : await serveSameOrigin(ROOT, { service: "http://127.0.0.1:9", relayPort: 9 });
const p = await page(`${srv.url}${CREATE}`, "create-own");
const requests = [];
p.on("request", (r) => requests.push({ url: r.url(), auth: r.headers().authorization || null }));
await until("Create loads", STEP, () => p.evaluate(() => !!window.createTest));
await p.check('input[name="source"][value="own"]');
await p.fill("#own-endpoint", endpoint);
await p.fill("#own-model", "a-local-model");
await p.fill("#own-key", KEY);
await p.fill("#prompt", rec.prompt);
await p.click("#build");
await until("the stream from the visitor's model ends", STEP, () => p.evaluate(() => window.createTest.done >= 1));
const fin = await p.evaluate(() => window.createTest.last);
const says = await p.evaluate(() => document.getElementById("model-says").textContent);
check("own: the page streams the app from the visitor's own model and renders it", fin.errors.length === 0 && fin.room && fin.room.kind === "live" && fin.nodes.length === 6, `${fin.nodes.length} widgets; "${says.slice(0, 90)}…"`);
const call = seen[0] || {};
check("own: the model is asked with the registry's guide as its instructions and the person's description", call.body && call.body.stream === true && call.body.model === "a-local-model" && /You build a WAI app/.test(call.body.messages[0].content) && call.body.messages[1].content === rec.prompt);
const withKey = requests.filter((r) => r.auth && r.auth.includes(KEY)).map((r) => r.url);
const stored = await p.evaluate((k) => JSON.stringify({ ...localStorage }).includes(k) || JSON.stringify({ ...sessionStorage }).includes(k) || document.cookie.includes(k), KEY);
check("own: the key goes to the visitor's endpoint alone, in its Authorization header, and is stored nowhere", call.auth === `Bearer ${KEY}` && withKey.length === 1 && withKey[0] === endpoint && !stored, `${requests.length} requests; the key in ${withKey.length}`);
p.dump();
model.close();
if (!shared) await srv.close();
}
// The visitor's coding agent: `wai mcp --toolset app` on this machine, driven over its stdio as
// an agent drives it; the page connects to its bridge with the pairing code, and the agent
// streams the show into it, then exports it as `wai create --from` does.
async function scenarioAgent() {
const dir = path.join(OUT, "agent");
fs.mkdirSync(dir, { recursive: true });
const mcp = spawn(WAI, ["mcp", "--toolset", "app", "--project", dir], { stdio: ["pipe", "pipe", "pipe"] });
procs.push(mcp);
let errText = "";
mcp.stderr.on("data", (d) => { errText += d; });
const first = await until("the MCP server names its bridge", 10000, () => errText.match(/http:\/\/127\.0\.0\.1:(\d+), pairing code (\w+)/));
const [, port, code] = first;
let buf = "";
const waiting = [];
mcp.stdout.on("data", (d) => {
buf += d;
let nl;
while ((nl = buf.indexOf("\n")) >= 0) {
const line = buf.slice(0, nl);
buf = buf.slice(nl + 1);
const w = waiting.shift();
if (w) w(JSON.parse(line));
}
});
let id = 0;
const rpc = (method, params) => new Promise((ok) => { waiting.push(ok); mcp.stdin.write(JSON.stringify({ jsonrpc: "2.0", id: ++id, method, params }) + "\n"); });
await rpc("initialize", { protocolVersion: "2025-06-18", capabilities: {}, clientInfo: { name: "e2e", version: "0" } });
const srv = shared ? shared.srv : await serveSameOrigin(ROOT, { service: "http://127.0.0.1:9", relayPort: 9 });
const p = await page(`${srv.url}${CREATE}`, "create-agent");
await until("Create loads", STEP, () => p.evaluate(() => !!window.createTest));
await p.check('input[name="source"][value="agent"]');
await p.fill("#agent-bridge", `http://127.0.0.1:${port}`);
await p.fill("#agent-code", code);
await p.click("#build");
await until("the page is connected to the bridge", STEP, () => p.evaluate(() => /Connected/.test(document.getElementById("agent-state").textContent)), 200);
const r = await rpc("tools/call", { name: "app_stream", arguments: { program: SHOW_TEXT, chunk: 12, pace_ms: 10 } });
check("agent: the agent's stream reaches the connected page", r.result && !r.result.isError && r.result.structuredContent.delivered === 1, JSON.stringify(r.result && r.result.structuredContent).slice(0, 120));
await until("the page renders the agent's app", STEP, () => p.evaluate(() => window.createTest.done >= 1));
const fin = await p.evaluate(() => window.createTest.last);
const counts = await p.evaluate(() => window.createTest.snapshots.map((x) => x.widgets));
check("agent: the page renders the app as the agent streams it", fin.errors.length === 0 && fin.nodes.length === 8 && new Set(counts).size >= 5 && counts.every((c, i) => !i || c >= counts[i - 1]), `${counts.length} snapshots, ${fin.nodes.length} widgets`);
const wrong = await p.evaluate(async (u) => (await fetch(u)).status, `http://127.0.0.1:${port}/create/stream?code=wrong`).catch((e) => String(e));
check("agent: the bridge refuses a page without the pairing code", wrong === 403 || /Failed to fetch/.test(String(wrong)), String(wrong));
const e = await rpc("tools/call", { name: "app_export", arguments: { program: SHOW_TEXT, dir: "agent-export", runtime_dir: RT } });
const ex = path.join(dir, "agent-export");
const checked = wai(["app", "check"], ex);
check("agent: app_export writes the project `wai create --from` writes, and its check passes", e.result && !e.result.isError && fs.existsSync(path.join(ex, "wai.json")) && checked.code === 0, JSON.stringify(e.result && e.result.structuredContent).slice(0, 160));
p.dump();
mcp.stdin.end();
if (!shared) await srv.close();
}
try {
await launch();
for (const [name, f] of [["create", scenarioCreate], ["built", scenarioBuilt], ["refused", scenarioRefused], ["own", scenarioOwn], ["agent", scenarioAgent]]) {
if (!ONLY.includes(name)) continue;
console.log(`== ${name}`);
try { await f(); } catch (e) { check(`${name}: ran to the end`, false, (e && e.stack) || String(e)); }
}
if (shared) shared.p.dump();
} finally {
if (browser) await browser.close().catch(() => {});
for (const p of procs) if (p.exitCode == null) p.kill("SIGTERM");
for (const p of procs) if (p.exitCode == null) p.kill("SIGTERM");
killStrays();
}
fs.writeFileSync(path.join(OUT, "results.json"), JSON.stringify({ results, failed }, null, 1));
console.log(`create e2e: ${results.length - failed} passed, ${failed} failed`);
process.exit(failed ? 1 : 0);

58
wai/wai-app/e2e/package-lock.json generated Normal file
View file

@ -0,0 +1,58 @@
{
"name": "wai-app-e2e",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "wai-app-e2e",
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.60.0"
}
},
"node_modules/fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/playwright": {
"version": "1.60.0",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.60.0.tgz",
"integrity": "sha512-hheHdokM8cdqCb0lcE3s+zT4t4W+vvjpGxsZlDnikarzx8tSzMebh3UiFtgqwFwnTnjYQcsyMF8ei2mCO/tpeA==",
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.60.0"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"fsevents": "2.3.2"
}
},
"node_modules/playwright-core": {
"version": "1.60.0",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.60.0.tgz",
"integrity": "sha512-9bW6zvX/m0lEbgTKJ6YppOKx8H3VOPBMOCFh2irXFOT4BbHgrx5hPjwJYLT40Lu+4qtD36qKc/Hn56StUW57IA==",
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=18"
}
}
}
}

10
wai/wai-app/e2e/package.json Normal file
View file

@ -0,0 +1,10 @@
{
"name": "wai-app-e2e",
"private": true,
"description": "Create, end to end: a browser automation package drives the host browser against a room service and a relay on loopback.",
"license": "Apache-2.0",
"type": "module",
"dependencies": {
"playwright": "1.60.0"
}
}

84
wai/wai-app/e2e/run.sh Executable file
View file

@ -0,0 +1,84 @@
#!/bin/bash
# Build what Create's end-to-end test needs, and run it.
#
# wai/wai-app/e2e/run.sh [--only create,built,refused,own,agent]
#
# 1. joule-moq-relay and joule-moq-room (native, release), and `wai` with its feature `app`
# (wai/wai-rs, in a target directory of its own: two workspaces build wai-rs differently);
# 2. the WebAssembly packages, through wasm-bindgen's web target, each with its own --out-name,
# into the runtime directory: joule-moq-web (room/moq/pkg/wai_room_moq), wai-live-web
# (room/live/pkg/wai_room_live) and wai-app with its feature `web` (create/pkg/wai_app);
# 3. the runtime laid out as the site serves it (web/runtime.mjs): create/ and room/; and the
# room client `wai dev` serves at /room/ (wai/quickstart/build-web.sh);
# 4. the page modules' unit tests (node --test web/web.test.mjs);
# 5. the browser-automation package from e2e/package-lock.json (`npm ci --ignore-scripts`),
# outside the tree unless PLAYWRIGHT_HOST names one installed;
# 6. e2e/create.e2e.mjs in the browser BROWSER_BIN names (the host's installed one), else the
# package's own.
#
# Environment: CARGO_TARGET_DIR (one target directory for every workspace), BROWSER_BIN,
# PLAYWRIGHT_HOST, CREATE_E2E_OUT (where the runtime and the results go), CARGO_LOCKED
# (default --locked). Nothing listens beyond loopback.
set -euo pipefail
HERE=$(cd "$(dirname "$0")" && pwd)
UI=$(cd "$HERE/.." && pwd)
ROOT=$(cd "$UI/../.." && pwd)
JC="$ROOT/joule-code/joule-code-rs"
WLW="$ROOT/wai/wai-live-web"
LOCKED=${CARGO_LOCKED---locked}
TARGET=${CARGO_TARGET_DIR:-$UI/target}
export CARGO_TARGET_DIR="$TARGET"
OUT=${CREATE_E2E_OUT:-${TMPDIR:-/tmp}/wai-create-e2e}
RT="$OUT/runtime"
rm -rf "$OUT/results" "$RT" "$OUT/web"
mkdir -p "$OUT/results" "$RT"
step() { echo "== $*"; }
step "native: joule-moq-relay, joule-moq-room, wai (feature app)"
(cd "$JC" && cargo build $LOCKED --release -p joule-moq-relay --bin joule-moq-relay)
(cd "$JC/crates/joule-moq-room" && cargo build $LOCKED --release --bin joule-moq-room)
WAI_TARGET="$TARGET/wai-app-cli"
(cd "$ROOT/wai/wai-rs" && CARGO_TARGET_DIR="$WAI_TARGET" cargo build $LOCKED --release --no-default-features --features app --bin wai)
step "wasm: joule-moq-web, wai-live-web, wai-app"
command -v wasm-bindgen >/dev/null || { echo "the wasm-bindgen CLI 0.2.127 is needed"; exit 2; }
wasm-bindgen --version | grep -q ' 0\.2\.127$' || { echo "the wasm-bindgen CLI must be 0.2.127, the version the crates pin"; exit 2; }
(cd "$JC" && cargo build $LOCKED --release -p joule-moq-web --lib --target wasm32-unknown-unknown)
wasm-bindgen --target web --no-typescript --out-dir "$RT/room/moq/pkg" --out-name wai_app_room_moq "$TARGET/wasm32-unknown-unknown/release/joule_moq_web.wasm"
(cd "$WLW" && cargo build $LOCKED --release --lib --target wasm32-unknown-unknown)
wasm-bindgen --target web --no-typescript --out-dir "$RT/room/live/pkg" --out-name wai_app_room_live "$TARGET/wasm32-unknown-unknown/release/wai_live_web.wasm"
(cd "$UI" && cargo build $LOCKED --release --lib --target wasm32-unknown-unknown --no-default-features --features web)
wasm-bindgen --target web --no-typescript --out-dir "$RT/create/pkg" --out-name wai_app "$TARGET/wasm32-unknown-unknown/release/wai_app.wasm"
ls -l "$RT"/create/pkg/*.wasm "$RT"/room/*/pkg/*.wasm
step "the runtime, as the site serves it"
node "$UI/web/runtime.mjs" "$RT"
step "the room client wai dev serves"
bash "$ROOT/wai/quickstart/build-web.sh" "$OUT/web"
step "unit tests of the page's modules"
node --test "$UI/web/web.test.mjs"
step "browser automation"
if [ -z "${PLAYWRIGHT_HOST:-}" ]; then
PW="$OUT/pw"
mkdir -p "$PW"
cp "$HERE/package.json" "$HERE/package-lock.json" "$PW/"
(cd "$PW" && npm ci --ignore-scripts --no-audit --no-fund)
export PLAYWRIGHT_HOST="$PW/node_modules"
fi
browser=()
[ -n "${BROWSER_BIN:-}" ] && browser=(--browser "$BROWSER_BIN")
step "end-to-end"
node "$HERE/create.e2e.mjs" \
--rt "$RT" \
--relay "$TARGET/release/joule-moq-relay" \
--room "$TARGET/release/joule-moq-room" \
--wai "$WAI_TARGET/release/wai" \
--web "$OUT/web" \
--out "$OUT/results" \
${browser[@]+"${browser[@]}"} \
"$@"

182
wai/wai-app/src/bridge.rs Normal file
View file

@ -0,0 +1,182 @@
//! The bridge a coding agent streams a program into a person's open Create page through:
//! an HTTP server on 127.0.0.1 that a page reads server-sent events from, with the pairing
//! code the bridge makes at start. `wai mcp --toolset app` runs it, and its tool
//! `app_stream` sends through it.
//!
//! Nothing is sent off the machine: the bridge listens on 127.0.0.1 alone, and a page reads
//! from it only with the pairing code.
use std::collections::hash_map::RandomState;
use std::hash::{BuildHasher, Hasher};
use std::io::{BufRead, BufReader, Write};
use std::net::{TcpListener, TcpStream};
use std::sync::{Arc, Mutex};
use std::time::Duration;
use serde_json::Value;
/// A pairing code: 10 letters and digits a page presents to read the bridge.
fn pairing_code() -> String {
const A: &[u8] = b"abcdefghjkmnpqrstuvwxyz23456789";
let mut out = String::new();
for i in 0..10u64 {
let mut h = RandomState::new().build_hasher();
h.write_u64(i);
h.write_u128(std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_nanos()).unwrap_or(0));
out.push(A[(h.finish() % A.len() as u64) as usize] as char);
}
out
}
/// The bridge: Create pages connected by server-sent events, each with the code.
pub struct Bridge {
pub port: u16,
pub code: String,
listeners: Arc<Mutex<Vec<TcpStream>>>,
}
impl Bridge {
/// Listen on 127.0.0.1:`port` (0 for any free port).
pub fn start(port: u16) -> std::io::Result<Bridge> {
let l = TcpListener::bind(("127.0.0.1", port))?;
let port = l.local_addr()?.port();
let code = pairing_code();
let listeners: Arc<Mutex<Vec<TcpStream>>> = Arc::new(Mutex::new(Vec::new()));
let (ls, c) = (listeners.clone(), code.clone());
std::thread::spawn(move || {
for s in l.incoming().flatten() {
let (ls, c) = (ls.clone(), c.clone());
std::thread::spawn(move || {
let _ = serve(s, &c, &ls);
});
}
});
Ok(Bridge { port, code, listeners })
}
/// The pages connected now.
pub fn listening(&self) -> usize {
self.listeners.lock().map(|l| l.len()).unwrap_or(0)
}
/// Stream `program` to every connected page, `chunk` characters at a time with `pace_ms`
/// between chunks, between a `start` and an `end` event: the chunks sent, and the pages
/// the last one reached.
pub fn stream(&self, program: &str, chunk: usize, pace_ms: u64) -> (usize, usize) {
self.send("start", &Value::Object(Default::default()));
let chars: Vec<char> = program.chars().collect();
let (mut n, mut reached) = (0, 0);
for piece in chars.chunks(chunk.max(1)) {
let text: String = piece.iter().collect();
reached = self.send("chunk", &Value::Object([("text".to_string(), Value::String(text))].into_iter().collect()));
n += 1;
if pace_ms > 0 {
std::thread::sleep(Duration::from_millis(pace_ms));
}
}
self.send("end", &Value::Object(Default::default()));
(n, reached)
}
/// Send one event to every connected page; the pages it reached.
pub fn send(&self, event: &str, data: &Value) -> usize {
let msg = format!("event: {event}\ndata: {data}\n\n");
let Ok(mut ls) = self.listeners.lock() else { return 0 };
ls.retain_mut(|s| s.write_all(msg.as_bytes()).and_then(|_| s.flush()).is_ok());
ls.len()
}
}
fn serve(mut s: TcpStream, code: &str, listeners: &Arc<Mutex<Vec<TcpStream>>>) -> std::io::Result<()> {
s.set_read_timeout(Some(Duration::from_secs(10)))?;
let mut r = BufReader::new(s.try_clone()?);
let mut first = String::new();
r.read_line(&mut first)?;
let mut origin = None;
loop {
let mut h = String::new();
if r.read_line(&mut h)? == 0 || h == "\r\n" || h == "\n" {
break;
}
if let Some((k, v)) = h.split_once(':')
&& k.eq_ignore_ascii_case("origin")
{
origin = Some(v.trim().to_string());
}
}
let mut parts = first.split_whitespace();
let (method, target) = (parts.next().unwrap_or(""), parts.next().unwrap_or(""));
let allow = |o: &Option<String>| match o {
Some(o) if !o.contains(['\r', '\n']) => format!("Access-Control-Allow-Origin: {o}\r\nVary: Origin\r\n"),
_ => String::new(),
};
if method == "OPTIONS" {
// A page on a public origin asks before it reads a loopback address.
let resp = format!(
"HTTP/1.1 204 No Content\r\n{}Access-Control-Allow-Methods: GET\r\nAccess-Control-Allow-Headers: *\r\nAccess-Control-Allow-Private-Network: true\r\nContent-Length: 0\r\nConnection: close\r\n\r\n",
allow(&origin)
);
return s.write_all(resp.as_bytes());
}
let (path, query) = target.split_once('?').unwrap_or((target, ""));
let given = query.split('&').find_map(|kv| kv.strip_prefix("code=")).unwrap_or("");
if method == "GET" && path == "/create/stream" {
if given != code {
let resp = format!("HTTP/1.1 403 Forbidden\r\n{}Content-Length: 0\r\nConnection: close\r\n\r\n", allow(&origin));
return s.write_all(resp.as_bytes());
}
s.set_read_timeout(None)?;
let head = format!(
"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nCache-Control: no-store\r\n{}Access-Control-Allow-Private-Network: true\r\nConnection: keep-alive\r\n\r\nevent: hello\ndata: {{}}\n\n",
allow(&origin)
);
s.write_all(head.as_bytes())?;
s.flush()?;
if let Ok(mut l) = listeners.lock() {
l.push(s);
}
return Ok(());
}
let resp = format!("HTTP/1.1 404 Not Found\r\n{}Content-Length: 0\r\nConnection: close\r\n\r\n", allow(&origin));
s.write_all(resp.as_bytes())
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
use std::io::Read;
#[test]
fn the_bridge_refuses_a_page_without_the_code_and_streams_to_one_with_it() {
let b = Bridge::start(0).unwrap();
let get = |path: &str| {
let mut s = TcpStream::connect(("127.0.0.1", b.port)).unwrap();
write!(s, "GET {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nOrigin: https://site.example\r\n\r\n").unwrap();
s
};
let mut bad = get("/create/stream?code=wrong");
let mut resp = String::new();
bad.read_to_string(&mut resp).unwrap();
assert!(resp.starts_with("HTTP/1.1 403"), "{resp}");
let good = get(&format!("/create/stream?code={}", b.code));
let mut r = BufReader::new(good);
let mut head = String::new();
r.read_line(&mut head).unwrap();
assert!(head.starts_with("HTTP/1.1 200"), "{head}");
for _ in 0..50 {
if b.listening() == 1 {
break;
}
std::thread::sleep(Duration::from_millis(20));
}
assert_eq!(b.send("chunk", &json!({ "text": "root = App(" })), 1);
let mut seen = String::new();
while !seen.contains("root = App(") {
let mut l = String::new();
r.read_line(&mut l).unwrap();
seen.push_str(&l);
}
assert!(seen.contains("event: chunk"));
}
}

255
wai/wai-app/src/export.rs Normal file
View file

@ -0,0 +1,255 @@
//! The files of a starter project for a program, and the facts its project file holds.
//! `wai create --from app.oui` (catalogue `app.export`) writes them, with the project file
//! (`wai.json`) written by the one writer `wai create --template` writes its own with, so a
//! project exported from Create and one made from a template have one shape. Every file an
//! export holds, and its layout, is decided here, whoever asks for it (the command line, the
//! MCP server), so that the project format changes in one place.
use serde::{Deserialize, Serialize};
use crate::{guide, Checked};
/// A file of the runtime the project carries under `vendor/` (the WAI widgets,
/// the renderer, the room client and their WebAssembly builds), by its path in
/// the runtime and its SHA-256. The host supplies the bytes.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct RuntimeFile {
pub path: String,
pub sha256: String,
}
/// What a project file holds: its text, or a runtime file the host copies in.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case", tag = "kind", content = "value")]
pub enum Contents {
Text(String),
Runtime(String),
}
/// One file of the project, by its path in the project.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ProjectFile {
pub path: String,
pub contents: Contents,
}
/// Why a program is not exported.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case", tag = "kind")]
pub enum ExportError {
/// The program has errors; the export is the app as it renders with none.
Errors { count: usize },
/// The program holds no room, so there is nothing for a room service to keep.
NoRoom,
/// The tree cannot be written in the canonical form.
Unwritable { reason: String },
/// A runtime file's path leaves `vendor/`.
BadRuntimePath { path: String },
}
impl std::fmt::Display for ExportError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
ExportError::Errors { count } => write!(f, "the app has {count} error(s); fix them before exporting"),
ExportError::NoRoom => write!(f, "the app holds no room (a Room, a WatchTogether or a Show)"),
ExportError::Unwritable { reason } => write!(f, "the app cannot be written: {reason}"),
ExportError::BadRuntimePath { path } => write!(f, "the runtime path `{path}` leaves vendor/"),
}
}
}
impl std::error::Error for ExportError {}
/// The program's file in the project.
pub const APP: &str = "app.oui";
/// The files every export holds, in the order they are written, beside `vendor/` (and
/// `media/README.md` for a watch room). The project file and the licence are `wai create`'s.
pub const PROJECT_FILES: [&str; 6] = [APP, "index.html", "README.md", "AGENTS.md", ".gitignore", "vendor.json"];
/// What the project file (`wai.json`) of an export holds: its title and summary, and the room
/// `wai dev` runs for the app.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ProjectFacts {
pub title: String,
pub summary: String,
/// `wai dev`'s mode for the app's room: `room` (a `Room`), `watch` (a `WatchTogether`) or
/// `show` (a `Show`).
pub mode: String,
/// A watch room's file, as the project holds it (`media/<source>`).
pub watch: Option<String>,
/// A show's inputs per viewer per poll.
pub audience: Option<u8>,
}
/// A starter project: its files, and its project file's facts.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Export {
pub files: Vec<ProjectFile>,
pub project: ProjectFacts,
}
/// `wai dev`'s mode for a room of `kind` (a plan's `live`, `watch` or `show`).
pub fn dev_mode(kind: &str) -> &'static str {
match kind {
"watch" => "watch",
"show" => "show",
_ => "room",
}
}
fn safe_runtime_path(p: &str) -> bool {
!p.is_empty()
&& !p.starts_with('/')
&& p.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-' | '/'))
&& p.split('/').all(|x| !x.is_empty() && x != "." && x != "..")
}
fn fill(template: &str, vars: &[(&str, &str)]) -> String {
let mut s = template.to_string();
for (k, v) in vars {
s = s.replace(&format!("{{{{{k}}}}}"), v);
}
s
}
fn html_text(s: &str) -> String {
s.replace('&', "&amp;").replace('<', "&lt;").replace('>', "&gt;")
}
/// The starter project for `program`: every file of the project but `wai.json` and `LICENSE`
/// (which `wai create` writes), the runtime files named by their runtime paths for the host to
/// copy in, and the facts its project file holds.
pub fn export(program: &str, runtime: &[RuntimeFile]) -> Result<Export, ExportError> {
let checked: Checked = crate::check(program);
let errors = checked.errors.iter().filter(|e| !e.warning).count();
if errors > 0 {
return Err(ExportError::Errors { count: errors });
}
let (Some(tree), Some(plan)) = (checked.tree.as_ref(), checked.room.as_ref()) else {
return Err(ExportError::NoRoom);
};
for r in runtime {
if !safe_runtime_path(&r.path) {
return Err(ExportError::BadRuntimePath { path: r.path.clone() });
}
}
let lib = crate::registry::library();
let oui = joule_ui_lang::write(tree, &lib).map_err(|e| ExportError::Unwritable { reason: e.to_string() })?;
// The app's title, else its room's.
let title = match tree.props.get("title") {
Some(joule_ui_core::PropValue::Text(s)) => Some(s.clone()),
_ => None,
}
.or_else(|| plan.title.clone());
let title = title.unwrap_or_else(|| "A WAI app".into());
let (summary, what) = match plan.kind.as_str() {
"watch" => (
"a watch room, where everyone watches one file together, in sync.",
"A watch room: press Watch, and the file plays at the room's position; play, pause, seek or change the speed, and everyone's player follows.",
),
"show" => (
"a show, with hosts on camera and an audience that watches and votes.",
"A show: the first host to join leads and opens the polls; viewers press Watch and vote, and the room's clock signs each result.",
),
_ => (
"a live room, where people join with their camera and microphone, or watch.",
"A live room: press Join to take part with your camera and microphone, or Watch; every frame is checked against its signed receipt before it is shown.",
),
};
let watch = (plan.kind == "watch").then(|| format!("media/{}", plan.source.as_deref().unwrap_or_default()));
let watch_note = match &watch {
Some(w) => format!("\nA watch room plays one file: put it at `{w}`, and if it is not a WAV file, set `dev.watch_ms` in\n`wai.json` to its length in milliseconds.\n"),
None => String::new(),
};
let guide_text = guide::guide();
let vars: [(&str, &str); 5] = [("TITLE", &html_text(&title)), ("SUMMARY", summary), ("WHAT", what), ("GUIDE", &guide_text), ("WATCH", &watch_note)];
let vendor = serde_json::json!({
"note": "The runtime under vendor/, each file with its SHA-256; `wai app check` verifies them.",
"files": runtime,
});
let mut files = vec![
ProjectFile { path: APP.into(), contents: Contents::Text(oui) },
ProjectFile { path: "index.html".into(), contents: Contents::Text(fill(include_str!("../templates/index.html"), &vars)) },
ProjectFile { path: "README.md".into(), contents: Contents::Text(fill(include_str!("../templates/README.md"), &vars)) },
ProjectFile { path: "AGENTS.md".into(), contents: Contents::Text(fill(include_str!("../templates/AGENTS.md"), &vars)) },
ProjectFile { path: ".gitignore".into(), contents: Contents::Text(include_str!("../templates/gitignore").into()) },
ProjectFile { path: "vendor.json".into(), contents: Contents::Text(serde_json::to_string_pretty(&vendor).expect("json") + "\n") },
];
if let (Some(w), Some(source)) = (&watch, plan.source.as_deref()) {
files.push(ProjectFile {
path: "media/README.md".into(),
contents: Contents::Text(format!("Put the file the room watches here, as `{source}` (`wai.json` names it `{w}`), and if it is not a WAV file set `dev.watch_ms` in `../wai.json` to its length in milliseconds.\n")),
});
}
for r in runtime {
files.push(ProjectFile { path: format!("vendor/{}", r.path), contents: Contents::Runtime(r.path.clone()) });
}
let project = ProjectFacts {
title,
summary: format!("A WAI app made with Create: {summary}"),
mode: dev_mode(&plan.kind).into(),
watch,
audience: (plan.kind == "show").then_some(plan.audience),
};
Ok(Export { files, project })
}
#[cfg(test)]
mod tests {
use super::*;
const SHOW: &str = "root = App([show], \"Friday night\")\nshow = Show([player, chat, poll], \"Live\", 2)\nplayer = Player()\nchat = Chat()\npoll = Poll(\"Next?\", [\"A\", \"B\"])\n";
fn rt() -> Vec<RuntimeFile> {
vec![RuntimeFile { path: "create/app.js".into(), sha256: "00".repeat(32) }]
}
fn text<'a>(files: &'a [ProjectFile], p: &str) -> &'a str {
match &files.iter().find(|f| f.path == p).unwrap_or_else(|| panic!("{p} is not in the export")).contents {
Contents::Text(t) => t,
Contents::Runtime(_) => panic!("{p} is a runtime file"),
}
}
#[test]
fn an_export_holds_every_project_file_and_the_room_its_project_file_names() {
let e = export(SHOW, &rt()).unwrap();
for p in PROJECT_FILES {
assert!(e.files.iter().any(|f| f.path == p), "{p} is missing");
}
assert!(!e.files.iter().any(|f| f.path == "wai.json" || f.path == "LICENSE"), "wai create writes those");
assert_eq!((e.project.mode.as_str(), e.project.audience, e.project.watch.as_deref()), ("show", Some(2), None));
assert_eq!(e.project.title, "Friday night");
assert!(e.files.iter().any(|f| f.path == "vendor/create/app.js" && f.contents == Contents::Runtime("create/app.js".into())));
// The program is the canonical form of the tree.
let oui = text(&e.files, APP);
let back = crate::check(oui);
assert!(back.errors.is_empty());
assert_eq!(joule_ui_lang::write(back.tree.as_ref().unwrap(), &crate::registry::library()).unwrap(), oui);
assert!(text(&e.files, ".gitignore").contains(".wai/"));
assert!(text(&e.files, "AGENTS.md").contains("- Poll(question*, options*, seconds)"));
assert!(text(&e.files, "index.html").contains("<title>Friday night</title>"));
let readme = text(&e.files, "README.md");
assert!(readme.contains("wai dev") && readme.contains("wai app check") && readme.contains("Apache-2.0"), "{readme}");
}
#[test]
fn each_room_kind_has_its_mode() {
let live = export("root = App([r], \"x\")\nr = Room([Chat()])\n", &[]).unwrap();
assert_eq!((live.project.mode.as_str(), live.project.audience, live.project.watch), ("room", None, None));
let watch = export("root = App([w])\nw = WatchTogether(\"film.mp4\")\n", &[]).unwrap();
assert_eq!((watch.project.mode.as_str(), watch.project.watch.as_deref()), ("watch", Some("media/film.mp4")));
assert!(watch.files.iter().any(|f| f.path == "media/README.md"));
assert!(text(&watch.files, "README.md").contains("media/film.mp4"));
}
#[test]
fn what_is_not_exported() {
assert_eq!(export("root = App([t])\nt = Text(\"no room\")\n", &[]), Err(ExportError::NoRoom));
assert_eq!(export("root = App([r])\nr = Room([Poll(\"q\", [\"a\", \"b\"])])\n", &[]), Err(ExportError::Errors { count: 1 }));
let bad = [RuntimeFile { path: "../etc/x".into(), sha256: String::new() }];
assert!(matches!(export(SHOW, &bad), Err(ExportError::BadRuntimePath { .. })));
}
}

62
wai/wai-app/src/guide.rs Normal file
View file

@ -0,0 +1,62 @@
//! What a model is told before it writes an app: the language as joule-ui takes
//! it in, and every widget of the registry, generated from the registry so the
//! two never disagree.
use crate::registry::{library, ROOT};
/// The example a model is shown.
pub const EXAMPLE: &str = "root = App([show], \"Friday night\")\nshow = Show([player, chat, poll, people], \"Live from the studio\")\nplayer = Player()\nchat = Chat(\"Talk\")\npoll = Poll(\"Which song next?\", [\"The first one\", \"The quiet one\", \"The loud one\"])\npeople = Participants()\n";
/// The model's instructions: the system prompt of a Create session.
pub fn guide() -> String {
let lib = library();
let mut g = String::new();
g.push_str(
"You build a WAI app from a person's description. Write it as a program in a line language, one widget a line, and nothing else: no prose, no code fence.\n\n\
Rules:\n\
1. Each line is `name = Widget(arguments)`. The first line is `root = App([...], \"Title\")`.\n\
2. Arguments go in the order the widget's signature lists them; `*` marks a required one. Write `null` to skip one in the middle; leave out the ones at the end you do not need.\n\
3. A widget's children are a list, even when there is one: `[chat]`. Give each child a name and define it on its own line below; a name may be used before its line.\n\
4. Values are plain: text in double quotes, whole or decimal numbers, true or false, lists `[a, b]`, objects `{key: value}`. There are no variables (`$name`), no expressions or conditions, no queries and no `@` functions: the app's behaviour is the widgets' own.\n\
5. An app holds one room. Choose the room for what the person asks: a Room when people join on camera and talk; a WatchTogether when everyone watches one file in sync; a Show when hosts perform and an audience watches and votes. Put in the room only the widgets its signature lists below.\n\
6. Write the layout first (root, then the room), then each widget, so the app appears as you write.\n\n\
Widgets (`*` required):\n",
);
let mut names = lib.names();
// The root first, then the rooms, then the rest, as a model reads them.
names.sort_by_key(|n| match n.as_str() {
ROOT => (0, n.clone()),
"Room" | "WatchTogether" | "Show" => (1, n.clone()),
_ => (2, n.clone()),
});
for n in names {
let s = lib.registry.get(&n).expect("listed");
let sig = lib.signature(&n).expect("listed");
let holds = match &s.children {
joule_ui_core::AllowedChildren::OnlyOf(list) => format!(" Holds: {}.", list.join(", ")),
_ => String::new(),
};
g.push_str(&format!("- {sig}: {}{holds}\n", s.description));
}
g.push_str("\nExample, for \"a show for my channel with chat and a live poll\":\n");
g.push_str(EXAMPLE);
g
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_guide_lists_every_widget_and_its_example_is_valid() {
let g = guide();
for n in library().names() {
assert!(g.contains(&format!("- {n}(")), "{n} is missing from the guide");
}
let s = joule_ui_lang::parse(&library(), EXAMPLE);
assert!(s.errors.is_empty(), "{:?}", s.errors);
let (t, _, p, e) = crate::plan::apply(s.tree.as_ref().unwrap(), &s.nodes);
assert!(t.is_some() && e.is_empty());
assert_eq!(p.unwrap().kind, "show");
}
}

229
wai/wai-app/src/lib.rs Normal file
View file

@ -0,0 +1,229 @@
//! WAI's widgets for joule-ui, and Create, as a library (catalogue operations `app.guide`,
//! `app.check`, `app.stream`, `app.export`). `wai`, built with its feature `app`, is its
//! command line (`wai app guide|check`, `wai create --from app.oui`) and its MCP server
//! (`wai mcp --toolset app`).
//!
//! - [`registry`]: the WAI registry, a joule-ui `Registry` of typed widgets
//! (rooms of three kinds, the room's video, chat, a poll, who is present, the
//! page's receipts, energy) whose children rules are what each room's service
//! keeps.
//! - [`guide`]: what a model is told before it writes an app, generated from the
//! registry.
//! - [`check`] and [`Create`]: a program (joule-ui's line language, joule-ui/SPEC.md §11) as
//! a snapshot the page renders, whole or as it streams, with WAI's own rules
//! ([`plan`]) applied: one room, a poll's options, a show's audience, a watch
//! room's source; and the room the app needs.
//! - [`export`]: the files of a starter project for a program, and the facts its project
//! file (`wai.json`, written by `wai create`) holds.
//! - `bridge` (not on wasm32): the bridge on 127.0.0.1 a coding agent streams a program into
//! a person's open Create page through.
//! - `runtime` (not on wasm32): the runtime an export carries under `vendor/`, read by its
//! listing and checked file by file.
//! - `web` (feature): the page's WebAssembly surface, with the interaction log
//! every action on the page is sealed into.
#![forbid(unsafe_code)]
pub mod export;
pub mod guide;
pub mod plan;
pub mod registry;
#[cfg(not(target_arch = "wasm32"))]
pub mod bridge;
#[cfg(not(target_arch = "wasm32"))]
pub mod runtime;
#[cfg(feature = "web")]
pub mod web;
use joule_ui_core::Widget;
use joule_ui_lang::{Node, Snapshot, Statement, StreamParser};
use serde::{Deserialize, Serialize};
pub use plan::{AppError, PollPlan, RoomPlan};
/// One error, from the line language or from WAI's own rules, in one shape.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ErrorOut {
pub line: u32,
pub statement: String,
pub code: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub component: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub prop: Option<String>,
pub message: String,
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub warning: bool,
}
/// What the page renders: the snapshot of the line language, WAI's rules
/// applied, and the room the app needs.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct Checked {
pub tree: Option<Widget>,
pub nodes: Vec<Node>,
pub errors: Vec<ErrorOut>,
pub unresolved: Vec<String>,
pub orphans: Vec<String>,
pub statements: Vec<Statement>,
pub finished: bool,
pub incomplete: bool,
pub room: Option<RoomPlan>,
}
/// A snapshot with WAI's rules applied (joule-ui/SPEC.md §11.6 for the snapshot itself): a
/// widget that breaks one is dropped from the tree, its error withheld while its
/// statement is pending, like the line language's own.
pub fn checked(s: Snapshot) -> Checked {
let mut errors: Vec<ErrorOut> = s
.errors
.iter()
.map(|e| ErrorOut {
line: e.line,
statement: e.statement.clone(),
code: e.code.as_str().into(),
component: e.component.clone(),
prop: e.prop.clone(),
message: e.message.clone(),
warning: e.warning,
})
.collect();
let (tree, nodes, room) = match &s.tree {
Some(t) => {
// A statement is pending while its binding is the last, incomplete one.
let pending = |name: &str| !s.finished && s.statements.iter().rev().find(|x| x.name == name).is_some_and(|x| !x.complete);
let (t2, n2, room, app) = plan::apply_streaming(t, &s.nodes, &pending);
for a in app {
if !pending(&a.statement) {
errors.push(ErrorOut { line: a.line, statement: a.statement, code: a.code, component: a.component, prop: a.prop, message: a.message, warning: false });
}
}
(t2, n2, room)
}
None => (None, Vec::new(), None),
};
if s.finished && tree.is_some() && room.is_none() {
errors.push(ErrorOut {
line: 0,
statement: String::new(),
code: "no-room".into(),
component: None,
prop: None,
message: "the app holds no room: add a Room, a WatchTogether or a Show to App's children".into(),
warning: true,
});
}
errors.sort_by(|a, b| (a.line, &a.statement, &a.code).cmp(&(b.line, &b.statement, &b.code)));
Checked { tree, nodes, errors, unresolved: s.unresolved, orphans: s.orphans, statements: s.statements, finished: s.finished, incomplete: s.incomplete, room }
}
/// Check a whole program: the snapshot at the end of its stream, WAI's rules
/// applied.
pub fn check(program: &str) -> Checked {
checked(joule_ui_lang::parse(&registry::library(), program))
}
/// A Create session's program as it streams.
pub struct Create {
lib: joule_ui_lang::Library,
text: String,
finished: bool,
}
impl Default for Create {
fn default() -> Self {
Self::new()
}
}
impl Create {
pub fn new() -> Self {
Create { lib: registry::library(), text: String::new(), finished: false }
}
/// The next chunk; after `finish`, the first chunk of a new program.
pub fn push(&mut self, chunk: &str) -> Checked {
if self.finished {
self.text.clear();
self.finished = false;
}
self.text.push_str(chunk);
self.snapshot()
}
/// The stream has ended.
pub fn finish(&mut self) -> Checked {
self.finished = true;
self.snapshot()
}
/// Start again with no program.
pub fn reset(&mut self) {
self.text.clear();
self.finished = false;
}
pub fn text(&self) -> &str {
&self.text
}
pub fn snapshot(&self) -> Checked {
let mut p = StreamParser::new(&self.lib);
let s = if self.finished {
p.push(&self.text);
p.finish()
} else {
p.push(&self.text)
};
checked(s)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_stream_and_a_check_agree_at_the_end() {
let text = guide::EXAMPLE;
let mut c = Create::new();
let mut widgets = Vec::new();
for ch in text.as_bytes().chunks(7) {
let s = c.push(std::str::from_utf8(ch).unwrap());
widgets.push(s.nodes.len());
}
assert!(widgets.windows(2).all(|w| w[1] >= w[0]), "{widgets:?}");
assert_eq!(c.finish(), check(text));
}
/// The recorded streams, cut one character at a time: no widget that has
/// appeared ever goes, and every key appears once (the renderer keeps its
/// instance), until the stream ends as a check of the whole text has it.
#[test]
fn widgets_only_ever_arrive_while_a_recording_streams() {
for rec in [include_str!("../web/recorded/wai-app-show.json"), include_str!("../web/recorded/wai-app-room.json"), include_str!("../web/recorded/wai-app-watch-refused.json")] {
let v: serde_json::Value = serde_json::from_str(rec).unwrap();
let text: String = v["chunks"].as_array().unwrap().iter().map(|c| c.as_str().unwrap()).collect();
let mut c = Create::new();
let mut seen: Vec<String> = Vec::new();
for ch in text.chars() {
let s = c.push(&ch.to_string());
let keys: Vec<String> = s.nodes.iter().map(|n| n.key.clone()).collect();
for k in &seen {
assert!(keys.contains(k), "{k} went away after {:?}", c.text());
}
for k in keys {
if !seen.contains(&k) {
seen.push(k);
}
}
}
assert_eq!(c.finish(), check(&text));
}
}
#[test]
fn a_line_against_the_rooms_rules_is_refused_with_its_line() {
let c = check("root = App([w])\nw = WatchTogether(\"film.mp4\", [chat])\nchat = Chat()\n");
let e = c.errors.iter().find(|e| e.code == "disallowed-child").expect("refused");
assert_eq!((e.line, e.component.as_deref()), (3, Some("Chat")));
assert!(e.message.contains("WatchTogether"));
assert_eq!(c.nodes.len(), 2, "the chat is not rendered");
}
}

279
wai/wai-app/src/plan.rs Normal file
View file

@ -0,0 +1,279 @@
//! What an app asks of the room service, and the rules of an app that joule-ui's
//! structural validation cannot state (one room, a poll's options, a show's
//! audience, a watch room's source). A widget that breaks one is dropped from
//! the tree the page renders, and the error names its line, as a line the
//! registry refuses is.
use joule_ui_core::{PropValue, Widget};
use joule_ui_lang::Node;
use serde::{Deserialize, Serialize};
use crate::registry::ROOMS;
/// An app's own error: the shape of a line-language error (joule-ui/SPEC.md §11.8), its
/// code one of WAI's (`one-room`, `poll-options`, `poll-seconds`,
/// `audience-range`, `watch-source`, `no-room`).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct AppError {
pub line: u32,
pub statement: String,
pub code: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub component: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub prop: Option<String>,
pub message: String,
}
/// A poll as the room service is asked for it.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct PollPlan {
pub question: String,
pub options: Vec<String>,
pub seconds: u32,
}
/// The room an app needs: its kind, which decides the room service's
/// configuration, and what the app shows in it.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct RoomPlan {
/// `live` (a Room), `watch` (a WatchTogether) or `show` (a Show).
pub kind: String,
/// The widget's key in the snapshot, so the page finds it.
pub key: String,
pub title: Option<String>,
/// A watch room's file, as the app names it.
pub source: Option<String>,
/// A show's inputs per viewer per poll.
pub audience: u8,
/// The widgets inside the room, by name, in order.
pub features: Vec<String>,
pub polls: Vec<PollPlan>,
}
/// The default seconds a poll stays open, and its bounds.
pub const POLL_SECONDS: (u32, u32, u32) = (30, 5, 300);
fn text(w: &Widget, p: &str) -> Option<String> {
match w.props.get(p) {
Some(PropValue::Text(s)) => Some(s.clone()),
_ => None,
}
}
fn whole(n: f64, lo: f64, hi: f64) -> Option<u32> {
(n.fract() == 0.0 && n >= lo && n <= hi).then_some(n as u32)
}
/// A watch room's source: a file within the app, named by a relative path of
/// letters, digits, `.`, `_`, `-` and `/`, no part of it `.` or `..`.
pub fn source_ok(s: &str) -> bool {
!s.is_empty()
&& s.len() <= 200
&& !s.starts_with('/')
&& s.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-' | '/'))
&& s.split('/').all(|part| !part.is_empty() && part != "." && part != "..")
}
/// Check the app's own rules on a validated tree and its nodes (pre-order):
/// the tree with every widget that breaks one dropped, its nodes, the room the
/// app needs (none when it holds none), and the errors.
///
/// `pending(statement)`: whether a statement is still arriving. A pending
/// statement's values are checked once it completes (a poll's `seconds` reads
/// `2` before it reads `20`, its options arrive one by one), so its widget is
/// kept meanwhile; a second room is never rendered, pending or not.
pub fn apply_streaming(tree: &Widget, nodes: &[Node], pending: &dyn Fn(&str) -> bool) -> (Option<Widget>, Vec<Node>, Option<RoomPlan>, Vec<AppError>) {
let mut errors = Vec::new();
let mut index = 0usize;
let mut rooms = 0usize;
let mut out_nodes = Vec::new();
let Some(out) = prune(tree, nodes, &mut index, &mut out_nodes, &mut errors, &mut rooms, pending) else {
// The root itself broke a rule: nothing renders.
return (None, Vec::new(), None, errors);
};
// The plan, from the room kept.
let mut plan = None;
if ROOMS.contains(&out.name.as_str()) {
plan = Some(plan_of(&out, out_nodes[0].key.clone()));
} else {
let mut k = 1usize;
for c in &out.children {
if ROOMS.contains(&c.name.as_str()) {
let key = out_nodes.get(k).map(|n| n.key.clone()).unwrap_or_default();
plan = Some(plan_of(c, key));
break;
}
k += count(c);
}
}
(Some(out), out_nodes, plan, errors)
}
/// [`apply_streaming`] on a whole program: every statement checked.
pub fn apply(tree: &Widget, nodes: &[Node]) -> (Option<Widget>, Vec<Node>, Option<RoomPlan>, Vec<AppError>) {
apply_streaming(tree, nodes, &|_| false)
}
fn count(w: &Widget) -> usize {
1 + w.children.iter().map(count).sum::<usize>()
}
fn err(errors: &mut Vec<AppError>, n: &Node, code: &str, component: &str, prop: Option<&str>, message: String) {
errors.push(AppError { line: n.line, statement: n.statement.clone(), code: code.into(), component: Some(component.into()), prop: prop.map(String::from), message });
}
#[allow(clippy::too_many_arguments)]
fn prune(w: &Widget, nodes: &[Node], index: &mut usize, out_nodes: &mut Vec<Node>, errors: &mut Vec<AppError>, rooms: &mut usize, pending: &dyn Fn(&str) -> bool) -> Option<Widget> {
let me = *index;
let node = nodes[me].clone();
*index += count(w);
let mut keep = true;
// A statement still arriving: its values are checked once it is whole.
let later = pending(&node.statement);
match w.name.as_str() {
n if ROOMS.contains(&n) => {
*rooms += 1;
if *rooms > 1 {
err(errors, &node, "one-room", n, None, format!("an app holds one room; this `{n}` is a second, and is left out"));
keep = false;
}
if n == "WatchTogether" && !later && !text(w, "source").is_some_and(|s| source_ok(&s)) {
err(errors, &node, "watch-source", n, Some("source"), "a watch room's `source` is a file within the app: a relative path such as \"film.mp4\" or \"media/film.mp4\", of letters, digits, `.`, `_`, `-` and `/`".into());
keep = false;
}
if n == "Show"
&& !later
&& let Some(PropValue::Number(a)) = w.props.get("audience")
&& whole(*a, 1.0, 255.0).is_none()
{
err(errors, &node, "audience-range", n, Some("audience"), "a show's `audience` is a whole number from 1 to 255".into());
keep = false;
}
}
"Poll" if !later => {
let ok = match w.props.get("options") {
Some(PropValue::List(items)) => {
(2..=32).contains(&items.len()) && items.iter().all(|i| matches!(i, PropValue::Text(s) if !s.is_empty() && s.chars().count() <= 100))
}
_ => false,
};
if !ok {
err(errors, &node, "poll-options", "Poll", Some("options"), "a poll's `options` are a list of 2 to 32 texts, each of 1 to 100 characters".into());
keep = false;
}
if let Some(PropValue::Number(s)) = w.props.get("seconds")
&& whole(*s, POLL_SECONDS.1 as f64, POLL_SECONDS.2 as f64).is_none()
{
err(errors, &node, "poll-seconds", "Poll", Some("seconds"), format!("a poll's `seconds` is a whole number from {} to {}", POLL_SECONDS.1, POLL_SECONDS.2));
keep = false;
}
}
_ => {}
}
if !keep {
return None;
}
out_nodes.push(node);
let mut kept = Widget { name: w.name.clone(), props: w.props.clone(), children: Vec::new() };
let mut i = me + 1;
for c in &w.children {
let mut ci = i;
if let Some(k) = prune(c, nodes, &mut ci, out_nodes, errors, rooms, pending) {
kept.children.push(k);
}
i += count(c);
}
Some(kept)
}
fn plan_of(room: &Widget, key: String) -> RoomPlan {
let kind = match room.name.as_str() {
"WatchTogether" => "watch",
"Show" => "show",
_ => "live",
};
let audience = match room.props.get("audience") {
Some(PropValue::Number(a)) => *a as u8,
_ => 1,
};
let polls = room
.children
.iter()
.filter(|c| c.name == "Poll")
.map(|p| PollPlan {
question: text(p, "question").unwrap_or_default(),
options: match p.props.get("options") {
Some(PropValue::List(items)) => items.iter().filter_map(|i| if let PropValue::Text(s) = i { Some(s.clone()) } else { None }).collect(),
_ => Vec::new(),
},
seconds: match p.props.get("seconds") {
Some(PropValue::Number(s)) => *s as u32,
_ => POLL_SECONDS.0,
},
})
.collect();
RoomPlan {
kind: kind.into(),
key,
title: text(room, "title"),
source: if kind == "watch" { text(room, "source") } else { None },
audience: if kind == "show" { audience } else { 0 },
features: room.children.iter().map(|c| c.name.clone()).collect(),
polls,
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::registry::library;
fn run(src: &str) -> (Widget, Option<RoomPlan>, Vec<AppError>) {
let s = joule_ui_lang::parse(&library(), src);
assert!(s.errors.iter().all(|e| e.warning), "{:?}", s.errors);
let (t, _n, p, e) = apply(s.tree.as_ref().unwrap(), &s.nodes);
(t.unwrap(), p, e)
}
#[test]
fn a_show_with_chat_and_a_poll() {
let (_, p, e) = run("root = App([show], \"Friday\")\nshow = Show([player, chat, poll], \"Live\", 2)\nplayer = Player()\nchat = Chat()\npoll = Poll(\"Next?\", [\"A\", \"B\"], 20)\n");
assert!(e.is_empty());
let p = p.unwrap();
assert_eq!((p.kind.as_str(), p.key.as_str(), p.audience), ("show", "show", 2));
assert_eq!(p.features, vec!["Player", "Chat", "Poll"]);
assert_eq!(p.polls, vec![PollPlan { question: "Next?".into(), options: vec!["A".into(), "B".into()], seconds: 20 }]);
}
#[test]
fn a_second_room_is_left_out_with_its_line() {
let (t, p, e) = run("root = App([a, b])\na = Room([])\nb = Show([])\n");
assert_eq!(e.len(), 1);
assert_eq!((e[0].code.as_str(), e[0].line, e[0].statement.as_str()), ("one-room", 3, "b"));
assert_eq!(t.children.len(), 1);
assert_eq!(p.unwrap().kind, "live");
}
#[test]
fn a_poll_needs_two_to_thirty_two_options() {
let (t, _, e) = run("root = App([s])\ns = Show([p, q])\np = Poll(\"one?\", [\"only\"])\nq = Poll(\"ok?\", [\"y\", \"n\"], 3)\n");
let codes: Vec<(&str, u32)> = e.iter().map(|x| (x.code.as_str(), x.line)).collect();
assert_eq!(codes, vec![("poll-options", 3), ("poll-seconds", 4)]);
assert!(t.children[0].children.is_empty());
}
#[test]
fn a_watch_source_stays_within_the_app() {
for bad in ["/etc/passwd", "../x.mp4", "https://example.com/x.mp4", "a//b", ""] {
assert!(!source_ok(bad), "{bad}");
}
assert!(source_ok("media/film-1.mp4"));
let (_, p, e) = run("root = App([w])\nw = WatchTogether(\"film.mp4\")\n");
assert!(e.is_empty());
assert_eq!(p.unwrap().source.as_deref(), Some("film.mp4"));
let (_, p, e) = run("root = App([w])\nw = WatchTogether(\"../film.mp4\")\n");
assert_eq!(e[0].code, "watch-source");
assert!(p.is_none());
}
}

147
wai/wai-app/src/registry.rs Normal file
View file

@ -0,0 +1,147 @@
//! The WAI widgets for joule-ui: each a typed schema, its positional order the
//! one a model writes it in, its children the ones its room's service keeps.
use joule_ui_core::{AllowedChildren, PropSchema, PropType, Registry, WidgetSchema};
use joule_ui_lang::Library;
/// The root widget.
pub const ROOT: &str = "App";
/// The three kinds of room, by widget.
pub const ROOMS: [&str; 3] = ["Room", "WatchTogether", "Show"];
/// What any room may also hold.
const IN_ANY_ROOM: [&str; 4] = ["Participants", "Receipts", "EnergyMeter", "Text"];
fn only(extra: &[&str]) -> AllowedChildren {
AllowedChildren::OnlyOf(extra.iter().chain(IN_ANY_ROOM.iter()).map(|s| s.to_string()).collect())
}
/// The WAI registry.
pub fn registry() -> Registry {
use PropSchema as P;
use PropType as T;
Registry::new()
.with(
WidgetSchema::new(
"App",
"The app: its title and what it holds. An app holds one room (a Room, a WatchTogether or a Show), and text, receipts and an energy meter beside it.",
)
.prop("title", P::optional(T::Text))
.children(AllowedChildren::OnlyOf(
["Room", "WatchTogether", "Show", "Text", "Receipts", "EnergyMeter"].iter().map(|s| s.to_string()).collect(),
))
.positional(["children", "title"]),
)
.with(
WidgetSchema::new("Text", "A paragraph of text.")
.prop("text", P::required(T::Text))
.positional(["text"]),
)
.with(
WidgetSchema::new(
"Room",
"A live room: people join with their camera and microphone, or watch. Every audio and video object is checked against its signed receipt before it is shown. Holds a Player for the room's video and a Chat; those who join can chat, those who watch read it.",
)
.prop("title", P::optional(T::Text))
.children(only(&["Player", "Chat"]))
.positional(["children", "title"]),
)
.with(
WidgetSchema::new(
"WatchTogether",
"A watch room: everyone watches one file together, in sync; anyone may play, pause, seek or change the speed for everyone. `source` is the file's name within the app (for example \"film.mp4\"). A watch room carries the shared controls alone: it holds no Chat and no Poll.",
)
.prop("source", P::required(T::Text))
.prop("title", P::optional(T::Text))
.children(only(&[]))
.positional(["source", "children", "title"]),
)
.with(
WidgetSchema::new(
"Show",
"A show: hosts on camera, the first of them leading; an audience that watches and votes. Holds a Player for the hosts' video, a Chat for the hosts, and Polls the lead opens and the audience answers. `audience` is how many inputs each viewer may send to one poll (1 to 255; 1 when left out).",
)
.prop("title", P::optional(T::Text))
.prop("audience", P::optional(T::Number))
.children(only(&["Player", "Chat", "Poll"]))
.positional(["children", "title", "audience"]),
)
.with(
WidgetSchema::new("Player", "The room's video: each participant's camera laid out from the room's signed record, every frame checked before it is shown.")
.prop("title", P::optional(T::Text))
.positional(["title"]),
)
.with(
WidgetSchema::new("Chat", "The room's chat: each message a signed event, verified before it is shown. Those who joined write; those who watch read.")
.prop("title", P::optional(T::Text))
.positional(["title"]),
)
.with(
WidgetSchema::new(
"Poll",
"A vote in a Show: the lead opens it, each viewer chooses one option, and the room's clock closes it and signs the result. `options` is a list of 2 to 32 texts; `seconds` how long it stays open (5 to 300; 30 when left out).",
)
.prop("question", P::required(T::Text))
.prop("options", P::required(T::List))
.prop("seconds", P::optional(T::Number))
.positional(["question", "options", "seconds"]),
)
.with(
WidgetSchema::new("Participants", "Who is in the room now, from the room's signed lifecycle.")
.prop("title", P::optional(T::Text))
.positional(["title"]),
)
.with(
WidgetSchema::new(
"Receipts",
"This page's receipts: every action a person takes here is sealed into a hash-chained log this page signs and verifies, and the room's objects checked against their receipts are counted.",
)
.prop("title", P::optional(T::Text))
.positional(["title"]),
)
.with(
WidgetSchema::new("EnergyMeter", "What each action cost in energy, and how the figure was obtained (a page has no meter, so its figures are estimates, labelled so).")
.prop("title", P::optional(T::Text))
.positional(["title"]),
)
}
/// The WAI library: the registry and its root.
pub fn library() -> Library {
Library::new(registry(), Some(ROOT.into())).expect("the WAI registry is a well-formed library")
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn every_widget_is_reachable_from_the_root() {
let r = registry();
let mut seen = std::collections::BTreeSet::new();
let mut todo = vec![ROOT.to_string()];
while let Some(n) = todo.pop() {
if !seen.insert(n.clone()) {
continue;
}
if let AllowedChildren::OnlyOf(list) = &r.get(&n).unwrap().children {
todo.extend(list.iter().cloned());
}
}
let mut all: Vec<String> = r.names().map(String::from).collect();
all.sort();
assert_eq!(seen.into_iter().collect::<Vec<_>>(), all);
}
#[test]
fn a_watch_room_holds_no_chat_and_no_poll() {
let r = registry();
let AllowedChildren::OnlyOf(w) = &r.get("WatchTogether").unwrap().children else { panic!() };
assert!(!w.iter().any(|c| c == "Chat" || c == "Poll"));
let AllowedChildren::OnlyOf(s) = &r.get("Show").unwrap().children else { panic!() };
assert!(s.iter().any(|c| c == "Poll"));
let AllowedChildren::OnlyOf(l) = &r.get("Room").unwrap().children else { panic!() };
assert!(!l.iter().any(|c| c == "Poll"), "only a show's clock opens and closes a vote");
}
}

68
wai/wai-app/src/runtime.rs Normal file
View file

@ -0,0 +1,68 @@
//! The Create runtime an export carries under `vendor/`: the files `web/runtime.mjs` lays out
//! (the WAI widgets, the renderer, the room frame, the room client and their WebAssembly
//! builds), listed with their SHA-256 in `create/wai-app-runtime.json`.
use std::path::Path;
use sha2::{Digest, Sha256};
use crate::export::RuntimeFile;
/// The runtime's listing, by its path in a runtime directory.
pub const LISTING: &str = "create/wai-app-runtime.json";
/// Lowercase hexadecimal SHA-256.
pub fn sha256_hex(b: &[u8]) -> String {
Sha256::digest(b).iter().map(|x| format!("{x:02x}")).collect()
}
/// The runtime in `dir`: each file its listing names, with its bytes, each checked against
/// the SHA-256 the listing gives it.
pub fn read(dir: &Path) -> Result<Vec<(RuntimeFile, Vec<u8>)>, String> {
let listing = dir.join(LISTING);
let text = std::fs::read_to_string(&listing).map_err(|e| format!("{}: {e} (lay the runtime out with wai/wai-app/web/runtime.mjs)", listing.display()))?;
#[derive(serde::Deserialize)]
struct Listing {
files: Vec<RuntimeFile>,
}
let l: Listing = serde_json::from_str(&text).map_err(|e| format!("{}: {e}", listing.display()))?;
if l.files.is_empty() {
return Err(format!("{} lists no file", listing.display()));
}
let mut out = Vec::new();
for f in l.files {
if f.path.split('/').any(|s| s.is_empty() || s == "." || s == "..") || f.path.starts_with('/') {
return Err(format!("{}: the runtime path `{}` leaves the runtime", listing.display(), f.path));
}
let p = dir.join(&f.path);
let bytes = std::fs::read(&p).map_err(|e| format!("{}: {e}", p.display()))?;
let got = sha256_hex(&bytes);
if got != f.sha256 {
return Err(format!("{}: its SHA-256 is {got}, the runtime's listing says {}", f.path, f.sha256));
}
out.push((f, bytes));
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_runtime_is_read_by_its_listing_and_a_changed_file_is_refused() {
let d = std::env::temp_dir().join(format!("wai-app-runtime-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(d.join("create")).unwrap();
std::fs::write(d.join("create/a.js"), b"export {};\n").unwrap();
let listing = serde_json::json!({"files": [{"path": "create/a.js", "sha256": sha256_hex(b"export {};\n")}]});
std::fs::write(d.join(LISTING), listing.to_string()).unwrap();
let rt = read(&d).unwrap();
assert_eq!((rt.len(), rt[0].0.path.as_str(), rt[0].1.len()), (1, "create/a.js", 11));
std::fs::write(d.join("create/a.js"), b"export {}; // changed\n").unwrap();
assert!(read(&d).unwrap_err().contains("SHA-256"));
std::fs::write(d.join(LISTING), serde_json::json!({"files": [{"path": "../x", "sha256": ""}]}).to_string()).unwrap();
assert!(read(&d).unwrap_err().contains("leaves"));
let _ = std::fs::remove_dir_all(&d);
}
}

194
wai/wai-app/src/web.rs Normal file
View file

@ -0,0 +1,194 @@
//! The page's WebAssembly surface (feature `web`): a Create session, its
//! program streamed into snapshots, and the interaction log every action a
//! person takes on the page is sealed into (joule-ui/SPEC.md §6, §7).
//!
//! A page has no energy meter, so each interaction's figure is the estimator's,
//! the handler's wall time on the page's clock at the default estimator power,
//! graded `Estimator` (joule-ui/SPEC.md §5). No capability gates the page's own actions: the
//! room's grants gate what the room does.
use ed25519_dalek::SigningKey;
use joule_ui_interaction::{begin_interaction, verify_chain, ContentId, InteractionLog, NoMeter, Pending, SealedInteraction, Verdict};
use serde_json::{json, Value};
use wasm_bindgen::prelude::*;
use crate::{registry, Checked, Create};
fn hex(b: &[u8]) -> String {
b.iter().map(|x| format!("{x:02x}")).collect()
}
/// A `did:key` for an Ed25519 key: multibase base58btc of the multicodec
/// prefix `0xed 0x01` and the key.
pub fn did_key(key: &[u8; 32]) -> String {
let mut mc = vec![0xed, 0x01];
mc.extend_from_slice(key);
format!("did:key:z{}", bs58::encode(mc).into_string())
}
fn entry_view(e: &SealedInteraction) -> Value {
let it = &e.interaction;
json!({
"seq": e.seq,
"action": it.action,
"actor": it.actor,
"verdict": it.verdict.tag(),
"reason": match &it.verdict { Verdict::Deny(r) => Some(r.clone()), Verdict::Allow => None },
"energy": { "estimated_uj": it.energy.estimated_uj, "measured_uj": it.energy.measured_uj, "provenance": it.energy.provenance.label() },
"surface": it.surface.0,
"inputs": it.inputs.0,
"outputs": it.outputs.0,
"prev": e.prev.0,
"id": e.content_id().0,
"signature": hex(&e.signature),
"signer": hex(&e.signer),
})
}
/// A Create session in the page.
#[wasm_bindgen]
pub struct Session {
create: Create,
log: InteractionLog,
actor: String,
pending: Vec<Option<Pending>>,
surface: ContentId,
}
#[wasm_bindgen]
impl Session {
/// A session whose log is signed by the Ed25519 key from `seed` (32 bytes the
/// page draws from its random generator and keeps in memory alone).
#[wasm_bindgen(constructor)]
pub fn new(seed: &[u8]) -> Result<Session, JsError> {
let seed: [u8; 32] = seed.try_into().map_err(|_| JsError::new("a session key is 32 bytes"))?;
let sk = SigningKey::from_bytes(&seed);
let actor = did_key(&sk.verifying_key().to_bytes());
Ok(Session { create: Create::new(), log: InteractionLog::new(sk), actor, pending: Vec::new(), surface: ContentId::genesis() })
}
/// The principal every action on this page is attributed to: the page's own key.
pub fn actor(&self) -> String {
self.actor.clone()
}
fn view(&mut self, c: Checked) -> String {
if let Some(t) = &c.tree {
self.surface = ContentId::of_bytes(&serde_json::to_vec(t).expect("a tree serialises"));
}
let mut v = serde_json::to_value(&c).expect("a snapshot serialises");
v["surface"] = json!(self.surface.0);
v.to_string()
}
/// The next chunk of the program: the snapshot, as JSON.
pub fn push(&mut self, chunk: &str) -> String {
let c = self.create.push(chunk);
self.view(c)
}
/// The stream has ended: the final snapshot.
pub fn finish(&mut self) -> String {
let c = self.create.finish();
self.view(c)
}
/// Start a new program (the log goes on).
pub fn reset(&mut self) {
self.create.reset();
}
/// The program so far.
pub fn program(&self) -> String {
self.create.text().to_string()
}
/// Phase 1 of an interaction (joule-ui/SPEC.md §6), before its handler runs: `{"pending": id}`
/// for an allowed action, or `{"sealed": entry}` for a denied one, whose handler
/// must not run.
#[wasm_bindgen(js_name = beginAction)]
pub fn begin_action(&mut self, action: &str, inputs: &str) -> String {
match begin_interaction(&self.surface, action, &self.actor, inputs.as_bytes(), &NoMeter, None) {
Ok(p) => {
self.pending.push(Some(p));
json!({ "pending": self.pending.len() - 1 }).to_string()
}
Err(denied) => {
let e = self.log.record(*denied);
json!({ "sealed": entry_view(e) }).to_string()
}
}
}
/// Phases 2 and 3, once the handler ran for `wall_ms` on the page's clock and
/// produced `output`: the interaction sealed as the next link of the log.
#[wasm_bindgen(js_name = finishAction)]
pub fn finish_action(&mut self, id: usize, output: &str, wall_ms: f64) -> Result<String, JsError> {
let p = self.pending.get_mut(id).and_then(Option::take).ok_or_else(|| JsError::new("no such pending action"))?;
let wall = if wall_ms.is_finite() && wall_ms > 0.0 { std::time::Duration::from_secs_f64(wall_ms / 1000.0) } else { std::time::Duration::ZERO };
let it = p.finish(output.as_bytes(), &NoMeter, None, wall);
Ok(entry_view(self.log.record(it)).to_string())
}
/// The log, every link as the page shows it.
pub fn log(&self) -> String {
Value::Array(self.log.entries().iter().map(entry_view).collect()).to_string()
}
/// The log as sealed, for anyone to verify (`verifyLog`).
#[wasm_bindgen(js_name = logRaw)]
pub fn log_raw(&self) -> String {
serde_json::to_string(self.log.entries()).expect("entries serialise")
}
/// The page's own check of its log (joule-ui/SPEC.md §7): `{"verified": n}` or `{"broken_at": i}`.
pub fn verify(&self) -> String {
verify_result(self.log.entries())
}
/// The energy of every allowed interaction, in microjoules.
#[wasm_bindgen(js_name = totalUj)]
pub fn total_uj(&self) -> f64 {
self.log.total_uj() as f64
}
/// The program in its canonical form (joule-ui/SPEC.md §11.7): the `app.oui` the page
/// hands the person, which `wai create --from app.oui` makes a project of.
pub fn canonical(&self) -> Result<String, JsError> {
let c = self.create.snapshot();
let t = c.tree.ok_or_else(|| JsError::new("nothing renders yet"))?;
joule_ui_lang::write(&t, &registry::library()).map_err(|e| JsError::new(&e.to_string()))
}
}
fn verify_result(entries: &[SealedInteraction]) -> String {
match verify_chain(entries) {
Ok(n) => json!({ "verified": n }).to_string(),
Err(i) => json!({ "broken_at": i }).to_string(),
}
}
/// Verify a log as `logRaw` gives it.
#[wasm_bindgen(js_name = verifyLog)]
pub fn verify_log(raw: &str) -> Result<String, JsError> {
let entries: Vec<SealedInteraction> = serde_json::from_str(raw).map_err(|e| JsError::new(&e.to_string()))?;
Ok(verify_result(&entries))
}
/// The WAI registry as a library: each widget's schema and positional order, and the root.
#[wasm_bindgen]
pub fn registry() -> String {
joule_ui_lang::vectors::library_json(&registry::library()).to_string()
}
/// What a model is told before it writes an app.
#[wasm_bindgen]
pub fn guide() -> String {
crate::guide::guide()
}
/// Check a whole program: its final snapshot, as JSON.
#[wasm_bindgen]
pub fn check(program: &str) -> String {
serde_json::to_string(&crate::check(program)).expect("a snapshot serialises")
}

21
wai/wai-app/templates/AGENTS.md Normal file
View file

@ -0,0 +1,21 @@
# Working on this project
This is a WAI app. `app.oui` is the app, written in joule-ui's line language (SPEC §11 of
joule-ui): one widget a line, `name = Widget(arguments)`, `root` first. The page renders
it with the WAI widgets in `vendor/create/`, and `wai dev` runs the room that `wai.json` names.
- To change the app, edit `app.oui`, then run `wai app check`. A line the registry does not
accept is refused with its line and a reason code; fix that line.
- Keep `wai.json`'s `dev` block matching the app's room: a `Room` is `"mode": "room"`; a
`WatchTogether` is `"mode": "watch"` with `"watch": "media/<source>"` (and `"watch_ms"` for a
file that is not WAV); a `Show` is `"mode": "show"` with `"audience"` equal to its audience.
- Run it with `wai dev`; `wai dev --check` checks every piece and stops.
- Do not edit `vendor/`: it is the runtime, checked file by file against `vendor.json`.
- Never put a key, a token or any other secret in this repository. Development keys are
made in `.wai/` by `wai dev` and stay there.
- WAI's tools for a coding agent: `wai mcp --toolset core,dev,app` (`wai setup-agent`
registers it). `app_check` checks a program, as `wai app check` does.
## The language and the widgets
{{GUIDE}}

38
wai/wai-app/templates/README.md Normal file
View file

@ -0,0 +1,38 @@
# {{TITLE}}
A WAI app made with Create: {{SUMMARY}}
It is an ordinary WAI project, made by `wai create --from app.oui`. `app.oui` is the app, one
widget a line. The WAI widgets that render it, and the room client under them, are in `vendor/`.
`wai.json` names the room that `wai dev` runs for it.
## Run it
```sh
wai app check # the app, the room wai.json names for it, and the runtime under vendor/
wai dev # the relay, the room service and this page on http://127.0.0.1:8080/
```
{{WATCH}}
## What you see
{{WHAT}} The page's receipts show every action you take there, sealed into a log the
page signs and checks.
## Change it
Edit `app.oui` (the widgets and their signatures are in `AGENTS.md`), then run
`wai app check`. It refuses a line the registry does not accept, naming the line and the
reason. A change to the app's room (its kind, a show's audience, a watch room's file) goes into
`wai.json` too, and `wai app check` says when the two disagree.
## Development and production
Everything `wai dev` starts listens on 127.0.0.1 alone, and its keys are made in `.wai/`,
which is never committed. In production, the room service runs with keys of its own, kept
outside the repository, behind a front that serves this page with TLS and passes `/room-api/`
to the room service and `/moq` to the relay on the same origin. No model key or other secret is
part of this project.
## Licence
Apache-2.0: see `LICENSE`.

3
wai/wai-app/templates/gitignore Normal file
View file

@ -0,0 +1,3 @@
# Development keys, logs and configuration (`wai dev`), and build output: never committed.
.wai/
dist/

15
wai/wai-app/templates/index.html Normal file
View file

@ -0,0 +1,15 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{{TITLE}}</title>
<link rel="stylesheet" href="./vendor/create/wai-app.css">
</head>
<body>
<!-- The app: app.oui rendered by the WAI widgets, its room on the room service behind this
page's own origin (/room-api, and the relay at /moq). -->
<main id="wai-app" class="wai-app" data-program="./app.oui" data-service="/room-api" data-frame="./vendor/create/wai-app-room-frame.html"></main>
<script type="module" src="./vendor/create/wai-app-app.js"></script>
</body>
</html>

23
wai/wai-app/web/app.js Normal file
View file

@ -0,0 +1,23 @@
// An app made with Create, as its exported page runs it: the program (app.oui) parsed by
// wai-app's WebAssembly build, rendered by the same widgets Create renders, its room on the
// room service behind this page's origin, every action sealed into the page's log.
import { openSession } from "./session.js";
import { Renderer } from "./renderer.js";
import { components, context } from "./components.js";
const main = document.getElementById("wai-app");
const session = await openSession();
const program = await (await fetch(main.dataset.program, { cache: "no-store" })).text();
session.push(program);
const snap = session.finish();
const ctx = context({ session, service: main.dataset.service, frame: main.dataset.frame });
const renderer = new Renderer(main, components, ctx);
renderer.render(snap);
const bad = snap.errors.filter((e) => !e.warning);
if (bad.length) {
const p = document.createElement("pre");
p.className = "wai-problems";
p.textContent = bad.map((e) => `app.oui line ${e.line}: ${e.message} (${e.code})`).join("\n");
main.after(p);
}
window.appTest = { session, ctx, renderer, snapshot: snap };

461
wai/wai-app/web/components.js Normal file
View file

@ -0,0 +1,461 @@
// The WAI widgets of wai-app's registry, as components of the streaming renderer.
//
// A room widget (Room, WatchTogether, Show) runs the room client of joule-moq-web in a frame
// of this page's own origin (room-frame.html): the page that makes its own key, asks the
// room service for a grant, proves the key to the relay, checks every audio and video
// object against its signed receipt before it is shown, keeps a watch room in sync and
// follows a show's polls. The widgets inside the room drive that client through its own
// interface (`roomTest`), from this page: every action a person takes (join, watch, leave,
// send a chat, open a poll, vote, play, pause) goes through the page's session and is
// sealed into its log (session.js).
const h = (tag, attrs = {}, ...kids) => {
const el = document.createElement(tag);
for (const [k, v] of Object.entries(attrs)) {
if (v == null || v === false) continue;
if (k === "class") el.className = v;
else if (k === "text") el.textContent = v;
else if (k.startsWith("on")) el.addEventListener(k.slice(2), v);
else el.setAttribute(k, v === true ? "" : String(v));
}
for (const c of kids) if (c != null) el.append(c);
return el;
};
const short = (p) => (p ? `${p.slice(0, 8)}…` : "");
const text = (props, k, d = "") => (typeof props[k] === "string" ? props[k] : d);
// ---------------------------------------------------------------------------- the room
// The page's one room: the widget that holds it sets it, the widgets inside it read it.
export class RoomSlot {
constructor() { this.current = null; this.subs = new Set(); }
set(handle) { this.current = handle; for (const f of this.subs) f(handle); }
clear(handle) { if (this.current === handle) this.set(null); }
on(f) { this.subs.add(f); if (this.current) f(this.current); return () => this.subs.delete(f); }
}
// The room client in its frame, and what the widgets ask of it.
export class RoomHandle {
constructor(kind, frame) {
this.kind = kind;
this.frame = frame;
this.subs = new Set();
this.last = null;
this.timer = setInterval(() => this.tick(), 600);
}
get t() {
try { return this.frame.contentWindow && this.frame.contentWindow.roomTest; } catch { return null; }
}
// Ready to enter: the client has read its room service and waits for a role.
async ready(ms = 30000) {
const end = Date.now() + ms;
for (;;) {
const t = this.t;
if (t && t.enter && t.state && t.state.phase === "idle") return t;
if (t && t.state && t.state.phase === "refused") throw new Error("the room client refused its endpoints");
if (Date.now() > end) throw new Error("the room client did not load");
await new Promise((r) => setTimeout(r, 150));
}
}
async enter(role) {
const t = await this.ready();
const phase = await t.enter(role);
this.tick();
return { phase, principal: this.me() };
}
async leave() { const t = this.t; if (t && t.leave) await t.leave(); this.tick(); return "left"; }
state() { const t = this.t; return (t && t.state) || null; }
phase() { const s = this.state(); return s ? s.phase : "loading"; }
me() { return (this.last && this.last.me && this.last.me.principal) || null; }
role() { const t = this.t; return t && t.role; }
snapshot() { try { return this.t && this.t.snapshot ? this.t.snapshot() : null; } catch { return null; } }
chats() {
const s = this.state();
return ((s && s.timeline) || []).filter((e) => e.kind === "chat").map((e) => ({ t: e.t, who: e.who, text: e.payload && e.payload.text, digest: e.digest }));
}
async sendChat(text) { const v = await this.t.sendChat(text); this.tick(); return { t: v && v.t, digest: v && v.digest }; }
co() { try { return this.t && this.t.coState ? this.t.coState() : null; } catch { return null; } }
async openPoll(question, options, seconds) {
const r = await this.t.coRequest("/co/window", { rule: "plurality", options, k: 1, seconds, question });
if (r.status !== 200) throw new Error(`the room's clock answered ${r.status}: ${JSON.stringify(r.body)}`);
return { status: r.status };
}
async vote(window, option) { const v = await this.t.coInput({ window, type: "vote", option }); return { window, option, t: v && v.t }; }
async watch(name, arg) { const at = await this.t.watchControl(name, arg); return { name, arg, at }; }
// The watch room's player position, in microseconds.
position() {
try { const m = this.frame.contentDocument.getElementById("cowatch-media"); return Math.round((m.currentTime || 0) * 1e6); } catch { return 0; }
}
// Objects received and checked against their receipts, over every participant and track.
checked() {
const s = this.state();
const out = { received: 0, verified: 0, refused: 0 };
for (const sub of Object.values((s && s.subs) || {})) {
for (const m of ["audio", "video", "screen"]) {
const x = sub[m];
if (!x) continue;
out.received += x.received || 0;
out.verified += x.verified || 0;
out.refused += x.refused || 0;
}
}
return out;
}
present() {
const s = this.state();
return ((s && s.room && s.room.present) || []).map((p) => p.principal);
}
setView(view) {
try { this.frame.contentDocument.body.dataset.view = view; } catch { /* not loaded yet: the query has it */ }
}
tick() {
this.last = this.snapshot();
for (const f of this.subs) { try { f(this); } catch { /* a view's own fault */ } }
}
on(f) { this.subs.add(f); f(this); return () => this.subs.delete(f); }
destroy() {
clearInterval(this.timer);
this.subs.clear();
try { if (this.t && this.t.leave && ["publishing", "watching"].includes(this.phase())) this.t.leave(); } catch { /* gone */ }
}
}
const KIND = {
Room: { kind: "live", label: "Live room", join: "Join with camera and microphone", watch: "Watch", view: "none" },
WatchTogether: { kind: "watch", label: "Watching together", join: null, watch: "Watch together", view: "watch" },
Show: { kind: "show", label: "Show", join: "Join as a host", watch: "Watch", view: "none" },
};
const PHASE = {
loading: "The room client is loading…",
idle: "Not in the room yet.",
entering: "Entering…",
publishing: "In the room, on camera and microphone.",
watching: "Watching.",
full: "The room is full: you can watch.",
left: "You left the room.",
refused: "This page cannot reach the room service.",
};
function roomWidget(name) {
const k = KIND[name];
return (ctx, props) => {
const title = h("h2", { class: "wai-room-title", text: text(props, "title", k.label) });
const status = h("span", { class: "wai-room-status", role: "status", "aria-live": "polite", text: PHASE.loading });
const join = k.join ? h("button", { type: "button", class: "wai-btn wai-primary", "data-act": "join", text: k.join }) : null;
const watch = h("button", { type: "button", class: "wai-btn", "data-act": "watch", text: k.watch });
const leave = h("button", { type: "button", class: "wai-btn", "data-act": "leave", hidden: true, text: "Leave" });
const frame = h("iframe", { class: "wai-frame", title: `${k.label}: the room`, allow: "camera; microphone; autoplay", loading: "eager" });
const stage = h("div", { class: "wai-room-stage", "data-view": k.view }, frame);
const slot = h("div", { class: "wai-room-children" });
const el = h("section", { class: "wai-room", "data-kind": k.kind },
h("header", { class: "wai-room-head" }, title, h("span", { class: "wai-kind", text: k.label })),
h("div", { class: "wai-room-controls" }, join, watch, leave, status),
stage, slot);
const u = new URL(ctx.config.frame, location.href);
u.searchParams.set("service", (ctx.config.services && ctx.config.services[k.kind]) || ctx.config.service);
u.searchParams.set("view", k.view);
frame.src = u.toString();
const handle = new RoomHandle(k.kind, frame);
handle.stage = stage;
handle.views = new Set();
handle.showVideo = (on) => {
if (on) handle.views.add("video"); else handle.views.delete("video");
const view = k.view === "watch" ? "watch" : handle.views.size ? "stage" : "none";
stage.dataset.view = view;
handle.setView(view);
};
// The frame's height follows its content.
const fit = () => {
try {
const d = frame.contentDocument;
if (d && d.body) frame.style.height = `${Math.max(0, d.body.scrollHeight)}px`;
} catch { /* not ours yet */ }
};
frame.addEventListener("load", () => {
handle.setView(stage.dataset.view);
try { new ResizeObserver(fit).observe(frame.contentDocument.body); } catch { /* no observer */ }
fit();
});
const busy = (b) => { for (const x of [join, watch, leave]) if (x) x.disabled = b; };
const act = (action, role) => async () => {
busy(true);
try {
if (role) await ctx.session.fire(action, { room: k.kind, role }, () => handle.enter(role));
else await ctx.session.fire(action, { room: k.kind }, () => handle.leave());
} catch (e) {
status.textContent = `${PHASE[handle.phase()] || ""} ${e.message || e}`;
}
busy(false);
render();
};
if (join) join.addEventListener("click", act("room.join", "publisher"));
watch.addEventListener("click", act("room.watch", "viewer"));
leave.addEventListener("click", act("room.leave", null));
const render = () => {
const p = handle.phase();
const inRoom = p === "publishing" || p === "watching";
status.textContent = PHASE[p] || p;
if (join) join.hidden = inRoom || p === "left";
watch.hidden = inRoom || p === "left";
leave.hidden = !inRoom;
el.dataset.phase = p;
};
handle.on(render);
ctx.room.set(handle);
return {
el,
slot,
handle,
update(next) { title.textContent = text(next, "title", k.label); },
destroy() { ctx.room.clear(handle); handle.destroy(); },
};
};
}
// The widgets inside a room find it through the page's room slot.
function inRoom(ctx, draw) {
let handle = null;
let off = null;
const offSlot = ctx.room.on((hd) => {
if (off) off();
handle = hd;
off = hd ? hd.on(() => draw(hd)) : null;
if (!hd) draw(null);
});
return { get handle() { return handle; }, stop() { if (off) off(); offSlot(); } };
}
// ---------------------------------------------------------------------------- the widgets
function App(ctx, props) {
const title = h("h1", { class: "wai-app-title", text: text(props, "title") });
const slot = h("div", { class: "wai-app-children" });
const el = h("div", { class: "wai-app-shell" }, title, slot);
return { el, slot, update(p) { title.textContent = text(p, "title"); } };
}
function Text(ctx, props) {
const el = h("p", { class: "wai-text", text: text(props, "text") });
return { el, update(p) { el.textContent = text(p, "text"); } };
}
function Player(ctx, props) {
const el = h("div", { class: "wai-player", text: text(props, "title", "The room's video") });
const r = inRoom(ctx, () => {});
const on = ctx.room.on((hd) => { if (hd && hd.showVideo) hd.showVideo(true); });
return {
el,
update(p) { el.textContent = text(p, "title", "The room's video"); },
destroy() { on(); const hd = r.handle; if (hd && hd.showVideo) hd.showVideo(false); r.stop(); },
};
}
function Chat(ctx, props) {
const head = h("h3", { class: "wai-h", text: text(props, "title", "Chat") });
const list = h("ul", { class: "wai-chat-list", "aria-live": "polite" });
const input = h("input", { class: "wai-input", maxlength: "500", autocomplete: "off", "aria-label": "Your message" });
const send = h("button", { type: "submit", class: "wai-btn", text: "Send" });
const note = h("p", { class: "wai-note" });
const form = h("form", { class: "wai-chat-form" }, input, send);
const el = h("section", { class: "wai-chat" }, head, list, form, note);
let shown = 0;
const r = inRoom(ctx, (hd) => {
const chats = hd ? hd.chats() : [];
if (chats.length !== shown) {
list.replaceChildren(...chats.map((c) => h("li", { "data-digest": c.digest || "" }, h("b", { text: `${c.who}: ` }), c.text || "")));
shown = chats.length;
list.scrollTop = list.scrollHeight;
}
const p = hd ? hd.phase() : "loading";
const can = p === "publishing";
input.disabled = send.disabled = !can;
note.textContent = can ? "Each message is signed by your key and checked by everyone before it shows." : p === "watching" ? "Viewers read the chat; those who join write it." : "Join the room to write.";
});
form.addEventListener("submit", async (e) => {
e.preventDefault();
const t = input.value.trim();
const hd = r.handle;
if (!t || !hd) return;
input.value = "";
try { await ctx.session.fire("chat.send", { text: t }, () => hd.sendChat(t)); } catch (err) { note.textContent = String(err.message || err); }
});
return { el, update(p) { head.textContent = text(p, "title", "Chat"); }, destroy() { r.stop(); } };
}
function Poll(ctx, props) {
let p = props;
const q = h("h3", { class: "wai-h wai-poll-q" });
const opts = h("div", { class: "wai-poll-options" });
const open = h("button", { type: "button", class: "wai-btn wai-primary", hidden: true, text: "Open the poll" });
const result = h("p", { class: "wai-poll-result", role: "status", "aria-live": "polite" });
const note = h("p", { class: "wai-note" });
const el = h("section", { class: "wai-poll" }, q, opts, open, result, note);
const options = () => (Array.isArray(p.options) ? p.options : []).map(String);
const question = () => text(p, "question");
const seconds = () => (typeof p.seconds === "number" ? p.seconds : 30);
let voted = null;
const draw = (hd) => {
q.textContent = question();
const co = hd ? hd.co() : null;
const labels = options();
const windows = co ? co.windows.filter((w) => w.question === question()) : [];
const w = windows[windows.length - 1] || null;
const now = co ? co.now : 0;
const isOpen = !!w && w.close > now;
const audience = co && co.audience && co.audience.limit > 0;
open.hidden = !(co && co.lead) || isOpen;
open.disabled = false;
if (opts.dataset.labels !== JSON.stringify(labels) || opts.dataset.open !== String(isOpen && audience)) {
opts.dataset.labels = JSON.stringify(labels);
opts.dataset.open = String(isOpen && audience);
opts.replaceChildren(...labels.map((label, i) => {
const b = h("button", { type: "button", class: "wai-btn wai-option", "data-option": String(i), text: label, disabled: !(isOpen && audience) });
b.addEventListener("click", async () => {
const ww = w;
if (!ww || !hd) return;
for (const x of opts.querySelectorAll("button")) x.disabled = true;
try { await ctx.session.fire("poll.vote", { question: question(), option: i, label }, () => hd.vote(ww.digest, i)); voted = i; } catch (e) { note.textContent = String(e.message || e); }
});
return b;
}));
}
if (w && w.selection) {
const pick = w.selection.picks[0];
const credit = w.selection.credits[0];
result.textContent = pick && pick.option != null ? `The room chose “${labels[pick.option] || `option ${pick.option + 1}`}”, with ${credit ? credit.count : 0} vote${credit && credit.count === 1 ? "" : "s"}; the room's clock signed the result.` : "The poll closed with no vote.";
el.dataset.state = "closed";
} else if (isOpen) {
result.textContent = `Open: closes in ${Math.max(0, Math.round((w.close - now) / 1e6))} s · ${w.inputs} vote${w.inputs === 1 ? "" : "s"} so far${voted != null ? ` · you chose “${labels[voted]}”` : ""}`;
el.dataset.state = "open";
} else if (w) {
result.textContent = "Closed: the room's clock is counting.";
el.dataset.state = "counting";
} else {
result.textContent = co ? (co.lead ? "Open it when you are ready." : "The host opens the poll.") : "Join or watch the show to take part.";
el.dataset.state = "waiting";
}
note.textContent = audience ? "Your vote is signed by your key; the room's clock selects and signs the result." : co && co.lead ? "You lead this show: you open its polls." : "";
};
const r = inRoom(ctx, draw);
open.addEventListener("click", async () => {
const hd = r.handle;
if (!hd) return;
open.disabled = true;
try { await ctx.session.fire("poll.open", { question: question(), options: options() }, () => hd.openPoll(question(), options().length, seconds())); } catch (e) { note.textContent = String(e.message || e); }
});
draw(null);
return { el, update(next) { p = next; draw(r.handle); }, destroy() { r.stop(); } };
}
function Participants(ctx, props) {
const head = h("h3", { class: "wai-h", text: text(props, "title", "In the room") });
const list = h("ul", { class: "wai-people" });
const el = h("section", { class: "wai-participants" }, head, list);
const r = inRoom(ctx, (hd) => {
const ps = hd ? hd.present() : [];
const me = hd ? hd.me() : null;
const json = JSON.stringify([ps, me]);
if (list.dataset.v === json) return;
list.dataset.v = json;
list.replaceChildren(...(ps.length ? ps.map((p) => h("li", { "data-principal": p, text: p === me ? `you (${short(p)})` : short(p) })) : [h("li", { class: "wai-note", text: "No one yet." })]));
});
return { el, update(p) { head.textContent = text(p, "title", "In the room"); }, destroy() { r.stop(); } };
}
function Receipts(ctx, props) {
const head = h("h3", { class: "wai-h", text: text(props, "title", "Receipts") });
const verdict = h("p", { class: "wai-verdict", role: "status" });
const table = h("table", { class: "wai-log" }, h("thead", {}, h("tr", {}, ...["#", "Action", "Verdict", "Energy", "Link"].map((t) => h("th", { text: t })))));
const body = h("tbody");
table.append(body);
const room = h("p", { class: "wai-note" });
const el = h("section", { class: "wai-receipts" }, head, verdict, h("div", { class: "wai-scroll" }, table), room);
const draw = () => {
const log = ctx.session.log();
const v = ctx.session.verify();
verdict.textContent = v.verified != null
? `${v.verified} action${v.verified === 1 ? "" : "s"} sealed by this page's key (${short(ctx.session.actor.replace("did:key:", ""))}), the chain verified link by link.`
: `The chain breaks at link ${v.broken_at}.`;
verdict.dataset.ok = String(v.verified != null);
body.replaceChildren(...log.slice(-12).map((e) => h("tr", { "data-seq": String(e.seq) },
h("td", { text: String(e.seq) }), h("td", { text: e.action }), h("td", { text: e.verdict }),
h("td", { text: `${e.energy.estimated_uj} µJ (${e.energy.provenance})` }), h("td", { class: "wai-mono", text: e.id.slice(3, 15) }))));
};
const off = ctx.session.on(draw);
const r = inRoom(ctx, (hd) => {
if (!hd) { room.textContent = ""; return; }
const c = hd.checked();
room.textContent = `The room: ${c.verified} of ${c.received} objects received checked against their receipts${c.refused ? `, ${c.refused} refused` : ""}.`;
});
draw();
return { el, update(p) { head.textContent = text(p, "title", "Receipts"); }, destroy() { off(); r.stop(); } };
}
function EnergyMeter(ctx, props) {
const head = h("h3", { class: "wai-h", text: text(props, "title", "Energy") });
const total = h("p", { class: "wai-energy-total" });
const list = h("ul", { class: "wai-energy" });
const note = h("p", { class: "wai-note", text: "A page has no energy meter: each figure is an estimate (the action's time on this page at 20 W), labelled Estimator, never a reading." });
const el = h("section", { class: "wai-energymeter" }, head, total, list, note);
const draw = () => {
const log = ctx.session.log();
const by = new Map();
for (const e of log) if (e.verdict === "allow") by.set(e.action, (by.get(e.action) || 0) + e.energy.estimated_uj);
const sum = ctx.session.totalUj();
total.textContent = `${(sum / 1e6).toFixed(3)} J over ${log.length} action${log.length === 1 ? "" : "s"} (Estimator)`;
list.replaceChildren(...[...by.entries()].map(([a, uj]) => h("li", { text: `${a}: ${(uj / 1e3).toFixed(1)} mJ` })));
};
const off = ctx.session.on(draw);
draw();
return { el, update(p) { head.textContent = text(p, "title", "Energy"); }, destroy() { off(); } };
}
function WatchTogetherControls(ctx) {
// The shared player's controls, each a sealed action: they apply to everyone in the room.
const mk = (label, act) => h("button", { type: "button", class: "wai-btn", "data-act": act, text: label });
const play = mk("Play", "play"), pause = mk("Pause", "pause"), back = mk("Back 10 s", "back"), fwd = mk("Ahead 10 s", "ahead");
const el = h("div", { class: "wai-watch-controls", hidden: true }, play, pause, back, fwd);
const r = inRoom(ctx, (hd) => { el.hidden = !hd || hd.phase() !== "watching"; });
const go = (name, arg) => async () => {
const hd = r.handle;
if (!hd) return;
const a = typeof arg === "function" ? arg(hd) : arg;
try { await ctx.session.fire(`watch.${name}`, { arg: a }, () => hd.watch(name, a)); } catch { /* shown by the room */ }
};
play.addEventListener("click", go("play", 0));
pause.addEventListener("click", go("pause", 0));
back.addEventListener("click", go("seek", (hd) => Math.max(0, hd.position() - 10e6)));
fwd.addEventListener("click", go("seek", (hd) => hd.position() + 10e6));
return { el, stop: () => r.stop() };
}
function WatchTogether(ctx, props, node) {
const c = roomWidget("WatchTogether")(ctx, props, node);
const controls = WatchTogetherControls(ctx);
c.el.insertBefore(controls.el, c.slot);
const destroy = c.destroy;
c.destroy = () => { controls.stop(); destroy(); };
return c;
}
export const components = {
App,
Text,
Room: roomWidget("Room"),
WatchTogether,
Show: roomWidget("Show"),
Player,
Chat,
Poll,
Participants,
Receipts,
EnergyMeter,
};
// The context the components share: the page's session, its one room, and where the room
// service (`service`, or one for each kind of room, `services`) and the room client's frame
// are, each on this page's origin.
export function context({ session, service, services = {}, frame }) {
return { session, room: new RoomSlot(), config: { service, services, frame } };
}

188
wai/wai-app/web/create.js Normal file
View file

@ -0,0 +1,188 @@
// Create: describe a WAI app, watch it build itself from WAI widgets as a model writes it,
// try its room, and take its program (app.oui) away: `wai create <dir> --from app.oui` makes
// it a starter project on the person's machine (catalogue `app.export`).
//
// The page calls no model of its own and holds no key on any server. A program comes from
// the visitor's own model (an endpoint and a key they type, called from this page alone),
// from their coding agent (`wai mcp --toolset app`, through its bridge on 127.0.0.1), or from a
// recorded stream; the page says which. Each chunk is parsed by wai-app's WebAssembly build
// (joule-ui's line language), and the snapshot, every widget of it accepted by the WAI
// registry, is rendered at once.
import { openSession, guide } from "./session.js";
import { Renderer } from "./renderer.js";
import { components, context } from "./components.js";
import { ownModel, bridge, replay } from "./model.js";
const root = document.getElementById("create");
const $ = (id) => document.getElementById(id);
const session = await openSession();
// The room service each kind of room is tried on (`data-service-live`, `-watch`, `-show`),
// else `data-service`; every one on this page's own origin.
const services = { live: root.dataset.serviceLive, watch: root.dataset.serviceWatch, show: root.dataset.serviceShow };
const ctx = context({ session, service: root.dataset.service, services, frame: root.dataset.frame });
const renderer = new Renderer($("app"), components, ctx);
const test = (window.createTest = { session, ctx, renderer, snapshots: [], last: null, source: null, done: 0, errors: [] });
const SAYS = {
none: "No model is called until you choose one. This page has no model of its own and holds no key.",
own: (u) => `Model: your own, at ${u}, called from this page with the key you typed. The key stays in this tab's memory and goes to that address alone; it is never stored.`,
agent: (u) => `Model: your coding agent, which streams the app to this page through the bridge on your machine (${u}).`,
replay: (n) => `Model: none. This replays a recorded stream (${n}).`,
};
const says = (t) => { $("model-says").textContent = t; };
function sourceNow() {
const r = document.querySelector('input[name="source"]:checked');
return r ? r.value : "replay";
}
function showSource() {
const s = sourceNow();
for (const id of ["own", "agent", "replay"]) $(id).hidden = id !== s;
says(SAYS.none);
}
for (const r of document.querySelectorAll('input[name="source"]')) r.addEventListener("change", showSource);
showSource();
// ---------------------------------------------------------------------------- the stream
let running = 0;
async function build(chunks, label) {
const run = ++running;
session.reset();
test.snapshots = [];
test.source = label;
test.errors = [];
$("export").disabled = true;
$("download").hidden = true;
$("export-status").textContent = "";
$("export-command").textContent = "";
root.dataset.state = "streaming";
try {
for await (const c of chunks) {
if (run !== running) return;
show(session.push(c));
}
if (run !== running) return;
show(session.finish());
root.dataset.state = "done";
test.done++;
} catch (e) {
root.dataset.state = "failed";
test.errors.push(String(e.message || e));
says(`The stream stopped: ${e.message || e}`);
}
}
function show(s) {
renderer.render(s);
test.last = s;
test.snapshots.push({ widgets: s.nodes.length, keys: s.nodes.map((n) => n.key), errors: s.errors.map((e) => `${e.line}:${e.code}`), chars: session.program().length, finished: s.finished });
drawProgram(s);
drawProblems(s);
const ok = s.finished && s.room && !s.errors.some((e) => !e.warning);
$("export").disabled = !ok;
}
function drawProgram(s) {
const lines = session.program().split("\n");
const bad = new Map();
for (const e of s.errors) if (e.line) bad.set(e.line, `${e.code}: ${e.message}`);
const pending = s.statements.filter((x) => !x.complete).map((x) => x.line);
const ol = $("program");
ol.replaceChildren(...lines.map((l, i) => {
const li = document.createElement("li");
li.textContent = l || " ";
li.value = i + 1;
if (bad.has(i + 1)) { li.className = "bad"; li.title = bad.get(i + 1); }
else if (pending.includes(i + 1)) li.className = "pending";
return li;
}));
ol.scrollTop = ol.scrollHeight;
}
function drawProblems(s) {
const ul = $("problems");
const shown = s.errors.length ? s.errors : [];
ul.replaceChildren(...shown.map((e) => {
const li = document.createElement("li");
li.dataset.code = e.code;
li.className = e.warning ? "warn" : "bad";
li.textContent = `${e.line ? `line ${e.line}` : "the app"}: ${e.message} (${e.code})`;
return li;
}));
if (!shown.length) {
const li = document.createElement("li");
li.className = "ok";
li.textContent = s.finished ? "None: every line was accepted." : s.tree ? "None so far." : "Waiting for the first line.";
ul.append(li);
}
}
// ---------------------------------------------------------------------------- the sources
let disconnect = null;
$("describe").addEventListener("submit", async (e) => {
e.preventDefault();
const prompt = $("prompt").value.trim();
const s = sourceNow();
if (s === "own") {
const endpoint = $("own-endpoint").value.trim();
const model = $("own-model").value.trim();
let u;
try { u = new URL(endpoint); } catch { return says("Give your model's address: the full URL of its chat/completions route."); }
says(SAYS.own(u.host));
// The key is read here, at the call, and handed to the call alone.
await build(ownModel({ endpoint, model, key: $("own-key").value, system: guide(), prompt }), "own");
} else if (s === "agent") {
connectAgent();
} else {
const name = $("recording").value;
says(SAYS.replay(name));
const rec = await (await fetch(`${root.dataset.recorded}wai-app-${name}.json`, { cache: "no-store" })).json();
$("prompt").value = rec.prompt;
await build(replay(rec, { speed: Number(root.dataset.speed || 1) }), "replay");
}
});
function connectAgent() {
if (disconnect) disconnect();
const url = $("agent-bridge").value.trim();
const code = $("agent-code").value.trim();
try {
says(SAYS.agent(new URL(url).host));
disconnect = bridge({
url, code,
onProgram: (iter) => build(iter, "agent"),
onState: (st) => { $("agent-state").textContent = st === "connected" ? "Connected: ask your agent to build the app." : st === "retrying" ? "Not connected: is `wai mcp --toolset app` running, and the code right?" : "Closed."; },
});
} catch (e) {
says(String(e.message || e));
}
}
// ---------------------------------------------------------------------------- the export
// The program in its canonical form, as a file: `wai create <dir> --from app.oui` makes the
// project of it, on the person's machine, with the one writer every WAI project has.
$("export").addEventListener("click", () => {
const status = $("export-status");
try {
const text = session.canonical();
const title = (test.last && test.last.tree && test.last.tree.props && test.last.tree.props.title && test.last.tree.props.title.value) || "wai-app";
const dir = title.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-|-$/g, "") || "wai-app";
const a = $("download");
if (a.href) URL.revokeObjectURL(a.href);
a.href = URL.createObjectURL(new Blob([text], { type: "text/plain" }));
a.download = "app.oui";
a.textContent = `app.oui (${text.split("\n").filter(Boolean).length} lines)`;
a.hidden = false;
const command = `wai create ${dir} --from app.oui`;
$("export-command").textContent = command;
test.exported = { file: "app.oui", text, command };
status.textContent = "Then, where you saved it: run the command below, and `wai dev` in the project it makes. The project is Apache-2.0, and no key is in it.";
a.click();
} catch (e) {
status.textContent = `Not exported: ${e.message || e}`;
}
});

59
wai/wai-app/web/index.html Normal file
View file

@ -0,0 +1,59 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Create</title>
<link rel="icon" href="data:,">
<link rel="stylesheet" href="./ui.css">
<style>body { margin: 0; padding: 16px; background: var(--wai-bg); } h1 { font: 600 1.3rem system-ui, sans-serif; color: var(--wai-fg); margin: 0 0 12px; }</style>
</head>
<body>
<h1>Create</h1>
<!-- The Create page, as wai-app serves it for development and its end-to-end test; the WAI
site's page carries the same element ids. The room service is /room-api on this page's
origin, the relay /moq. -->
<div id="create" class="create" data-service="/room-api" data-frame="./room-frame.html" data-runtime="./runtime.json" data-recorded="./recorded/">
<form id="describe">
<label for="prompt">Describe your app</label>
<textarea id="prompt">A show for my channel, with chat and a live poll</textarea>
<fieldset>
<legend>The model that writes it</legend>
<label><input type="radio" name="source" value="own"> Your own model</label>
<label><input type="radio" name="source" value="agent"> Your coding agent</label>
<label><input type="radio" name="source" value="replay" checked> A recorded stream</label>
</fieldset>
<div id="own" class="source-fields" hidden>
<label>Its chat/completions address <input id="own-endpoint" type="url" placeholder="http://127.0.0.1:8000/v1/chat/completions" autocomplete="off"></label>
<label>Model <input id="own-model" autocomplete="off"></label>
<label>Your key (kept in this tab's memory alone) <input id="own-key" type="password" autocomplete="off"></label>
</div>
<div id="agent" class="source-fields" hidden>
<label>The bridge <code>wai mcp --toolset app</code> printed <input id="agent-bridge" placeholder="http://127.0.0.1:port" autocomplete="off"></label>
<label>Its pairing code <input id="agent-code" autocomplete="off"></label>
<span id="agent-state" role="status"></span>
</div>
<div id="replay" class="source-fields">
<label>Recording <select id="recording"><option value="show">A show with chat and a live poll</option><option value="room">A live room with chat</option><option value="watch-refused">A watch party asked for chat (refused lines)</option></select></label>
</div>
<div><button id="build" class="wai-btn wai-primary" type="submit">Build it</button></div>
</form>
<p id="model-says" role="status" aria-live="polite"></p>
<div class="create-grid">
<section id="app-pane" aria-label="Your app"><div id="app" class="wai-app"></div></section>
<aside>
<h2>The program, as it arrives</h2>
<ol id="program"></ol>
<h2>Problems</h2>
<ul id="problems"></ul>
<h2>Export</h2>
<button id="export" class="wai-btn" type="button" disabled>Export app.oui</button>
<a id="download" hidden></a>
<p id="export-status" class="wai-note"></p>
<pre id="export-command" class="wai-note"></pre>
</aside>
</div>
</div>
<script type="module" src="./create.js"></script>
</body>
</html>

113
wai/wai-app/web/model.js Normal file
View file

@ -0,0 +1,113 @@
// Where a program comes from, each an async iterator of text chunks:
//
// ownModel the visitor's own model: an endpoint that streams the common
// `chat/completions` server-sent-events format (a model server on their own
// machine, or a provider they hold a key for), called from this page with
// their key. The key is held by the caller in memory and sent to that
// endpoint alone; nothing here stores it.
// bridge the visitor's coding agent, through the bridge `wai mcp --toolset app` runs on 127.0.0.1:
// each program the agent streams, read with the pairing code.
// replay a recorded stream: chunks and the pauses between them.
//
// No source calls a model of this site's own: there is none.
// Lines of a server-sent-events body, as they arrive.
async function* sseLines(body) {
const dec = new TextDecoder();
let buf = "";
const reader = body.getReader();
for (;;) {
const { value, done } = await reader.read();
if (done) break;
buf += dec.decode(value, { stream: true });
let nl;
while ((nl = buf.indexOf("\n")) >= 0) {
const line = buf.slice(0, nl).replace(/\r$/, "");
buf = buf.slice(nl + 1);
yield line;
}
}
if (buf) yield buf;
}
// The text of each `data:` event of a `chat/completions` stream: `choices[0].delta.content`.
export async function* completionChunks(lines) {
for await (const line of lines) {
if (!line.startsWith("data:")) continue;
const data = line.slice(5).trim();
if (!data || data === "[DONE]") continue;
let j;
try { j = JSON.parse(data); } catch { continue; }
if (j.error) throw new Error(`the model answered: ${j.error.message || JSON.stringify(j.error)}`);
const c = j.choices && j.choices[0] && j.choices[0].delta && j.choices[0].delta.content;
if (typeof c === "string" && c) yield c;
}
}
// The visitor's own model. `endpoint` is the full URL of its `chat/completions` route.
export async function* ownModel({ endpoint, model, key, system, prompt, signal }) {
const u = new URL(endpoint);
if (u.protocol !== "https:" && !(u.protocol === "http:" && ["127.0.0.1", "localhost", "[::1]"].includes(u.hostname))) {
throw new Error("the endpoint must be https, or http on this machine (127.0.0.1)");
}
const headers = { "content-type": "application/json" };
if (key) headers.authorization = `Bearer ${key}`;
const r = await fetch(u.toString(), {
method: "POST",
headers,
signal,
credentials: "omit",
referrerPolicy: "no-referrer",
cache: "no-store",
body: JSON.stringify({ model, stream: true, messages: [{ role: "system", content: system }, { role: "user", content: prompt }] }),
});
if (!r.ok || !r.body) throw new Error(`the model's endpoint answered ${r.status}`);
yield* completionChunks(sseLines(r.body));
}
// The coding agent's bridge: `onProgram(iterator)` is called for each program the agent
// streams. Returns a function that disconnects.
export function bridge({ url, code, onProgram, onState }) {
const u = new URL(url);
if (!["127.0.0.1", "localhost", "[::1]"].includes(u.hostname)) throw new Error("the bridge is on this machine: http://127.0.0.1:<port>");
const es = new EventSource(`${u.origin}/create/stream?code=${encodeURIComponent(code)}`);
let queue = null;
es.addEventListener("hello", () => onState && onState("connected"));
es.addEventListener("start", () => {
if (queue) queue.end();
queue = channel();
onProgram(queue.iter);
});
es.addEventListener("chunk", (e) => { if (queue) queue.put(JSON.parse(e.data).text); });
es.addEventListener("end", () => { if (queue) { queue.end(); queue = null; } });
es.onerror = () => onState && onState(es.readyState === EventSource.CLOSED ? "closed" : "retrying");
return () => { es.close(); if (queue) queue.end(); };
}
function channel() {
const items = [];
let wake = null;
let done = false;
return {
put(x) { items.push(x); if (wake) { wake(); wake = null; } },
end() { done = true; if (wake) { wake(); wake = null; } },
iter: (async function* () {
for (;;) {
if (items.length) { yield items.shift(); continue; }
if (done) return;
await new Promise((r) => { wake = r; });
}
})(),
};
}
// A recorded stream: `{ chunks: [...], pause_ms }` (or `pauses: [...]`), replayed with its
// pauses, `speed` times faster.
export async function* replay(rec, { speed = 1, signal } = {}) {
for (let i = 0; i < rec.chunks.length; i++) {
if (signal && signal.aborted) return;
const ms = rec.pauses ? rec.pauses[i] || 0 : rec.pause_ms || 0;
if (ms > 0) await new Promise((r) => setTimeout(r, ms / speed));
yield rec.chunks[i];
}
}

41
wai/wai-app/web/recorded/wai-app-room.json Normal file
View file

@ -0,0 +1,41 @@
{
"prompt": "A video room for my team with a chat beside it",
"note": "A stream written in the shape a model streams it (chunks of 1 to 9 characters, cut mid-token), not a recording of a model.",
"pause_ms": 25,
"chunks": [
"root = ",
"App(",
"[room",
", r",
"eceipts],",
" \"Team",
" r",
"oom\")\nr",
"o",
"om = Roo",
"m([",
"pl",
"ayer",
", c",
"hat, p",
"eople], ",
"\"",
"S",
"tand-up\")",
"\npl",
"ayer = ",
"Player()",
"\nch",
"at = ",
"Chat(",
")\n",
"people ",
"= Parti",
"cipants",
"()",
"\nrecei",
"pts =",
" Receipt",
"s()\n"
]
}

85
wai/wai-app/web/recorded/wai-app-show.json Normal file
View file

@ -0,0 +1,85 @@
{
"prompt": "A show for my channel, with chat and a live poll",
"note": "A stream written in the shape a model streams it (chunks of 1 to 9 characters, cut mid-token), not a recording of a model.",
"pause_ms": 25,
"chunks": [
"Here is ",
"your s",
"how.\n`",
"``\nr",
"oot = ",
"App([show",
", re",
"cei",
"pts",
", ener",
"gy], \"Fri",
"da",
"y N",
"ight Live",
"\")\n",
"show = S",
"ho",
"w([",
"player, ",
"chat, pol",
"l",
", people]",
", \"L",
"ive ",
"from the",
" studio\",",
" 1)\np",
"laye",
"r = Pl",
"ayer(\"On ",
"stage\"",
")\ncha",
"t = Chat",
"(\"Talk to",
" the ho",
"st",
"s\"",
")\npo",
"ll = Pol",
"l(",
"\"Whi",
"ch song n",
"e",
"xt",
"?\", [",
"\"The",
" quiet",
" one",
"\", \"Th",
"e loud on",
"e\", \"",
"Somethin",
"g new\"]",
", 20)\npeo",
"ple",
" = Part",
"i",
"cipants(\"",
"In the ",
"room\")\nr",
"ecei",
"pts ",
"=",
" R",
"ece",
"ipts(\"",
"This",
" p",
"age's rec",
"eipts\")\n",
"en",
"er",
"gy = Ener",
"gyMeter(",
"\"What it ",
"cos",
"t\"",
")\n```\n"
]
}

51
wai/wai-app/web/recorded/wai-app-watch-refused.json Normal file
View file

@ -0,0 +1,51 @@
{
"prompt": "A watch party for my channel with chat and a live poll",
"note": "A stream written in the shape a model streams it (chunks of 1 to 9 characters, cut mid-token), not a recording of a model.",
"pause_ms": 25,
"chunks": [
"root = Ap",
"p([part",
"y, note",
"], \"Mo",
"vie ",
"ni",
"ght\")\np",
"a",
"rty",
" =",
" Watch",
"To",
"geth",
"er(\"fil",
"m.mp4\", ",
"[chat, p",
"oll",
", peop",
"le], \"W",
"atch ",
"par",
"t",
"y\")",
"\nch",
"at =",
" Chat(\"",
"Talk",
"\")",
"\npoll =",
" Poll(\"",
"Again?",
"\", [",
"\"Yes\", \"N",
"o\"",
"])\npeop",
"le = Par",
"ticipants",
"()\nn",
"ote",
" = Tex",
"t(\"Everyo",
"ne w",
"atches in",
" sync.\")\n"
]
}

112
wai/wai-app/web/renderer.js Normal file
View file

@ -0,0 +1,112 @@
// The streaming renderer: each snapshot of a program (joule-ui's line language, a tree
// every widget of which the registry accepted) reconciled into the page by key. A widget
// keeps its instance, and so its state (a room joined, a chat typed), across snapshots
// while its key and its widget stay the same; only its props are updated. A widget that
// appears is created (and marked, so the page shows it arriving); one that is gone is
// destroyed.
//
// A component is `(ctx, props, node) => ({ el, slot?, update?(props), destroy?() })`:
// `el` its element, `slot` where its children go (it holds nothing else).
export class Renderer {
constructor(container, components, ctx) {
this.container = container;
this.components = components;
this.ctx = ctx;
this.inst = new Map();
this.created = 0;
this.renders = 0;
}
render(snap) {
this.renders++;
const tree = snap && snap.tree;
if (!tree) {
for (const [k, i] of [...this.inst]) this.destroy(k, i);
this.container.replaceChildren();
return;
}
const nodes = snap.nodes || [];
let idx = 0;
const seen = new Set();
const walk = (w) => {
const node = nodes[idx++] || { key: `#${idx}`, line: 0, statement: "" };
const key = node.key;
seen.add(key);
let inst = this.inst.get(key);
if (inst && inst.name !== w.name) {
this.destroy(key, inst);
inst = null;
}
const json = JSON.stringify(w.props || {});
const props = plainProps(w.props);
if (!inst) {
const make = this.components[w.name] || unknown(w.name);
const comp = make(this.ctx, props, node);
comp.el.classList.add("wai-w", "wai-new");
comp.el.dataset.key = key;
comp.el.dataset.widget = w.name;
comp.el.dataset.line = String(node.line);
setTimeout(() => comp.el.classList.remove("wai-new"), 1400);
inst = { name: w.name, comp, json };
this.inst.set(key, inst);
this.created++;
} else if (inst.json !== json) {
if (inst.comp.update) inst.comp.update(props);
inst.json = json;
}
inst.comp.el.dataset.line = String(node.line);
const kids = (w.children || []).map(walk);
if (inst.comp.slot) place(inst.comp.slot, kids.map((k) => k.comp.el));
return inst;
};
const root = walk(tree);
for (const [k, i] of [...this.inst]) if (!seen.has(k)) this.destroy(k, i);
if (this.container.children.length !== 1 || this.container.firstElementChild !== root.comp.el) this.container.replaceChildren(root.comp.el);
}
destroy(key, inst) {
try { if (inst.comp.destroy) inst.comp.destroy(); } catch { /* the component's own fault */ }
inst.comp.el.remove();
this.inst.delete(key);
}
// The widgets on the page now, by key.
keys() { return [...this.inst.keys()]; }
}
// A prop value in joule-ui's wire form (`{type, value}`) as a plain value: text, number,
// boolean, null, an array, an object, or an action `{ action, payload }`.
export function plain(v) {
if (!v || typeof v !== "object") return v;
switch (v.type) {
case "null": return null;
case "list": return (v.value || []).map(plain);
case "object": return Object.fromEntries(Object.entries(v.value || {}).map(([k, x]) => [k, plain(x)]));
case "action": return { action: v.value.id, payload: v.value.payload == null ? null : plain(v.value.payload) };
default: return v.value;
}
}
export function plainProps(props) {
return Object.fromEntries(Object.entries(props || {}).map(([k, v]) => [k, plain(v)]));
}
// Put `els` in `slot` in order, moving only what is out of place (an iframe moved in the
// document reloads, so nothing in place is touched).
export function place(slot, els) {
for (const c of [...slot.children]) if (!els.includes(c)) c.remove();
for (let i = 0; i < els.length; i++) {
const at = slot.children[i];
if (at !== els[i]) slot.insertBefore(els[i], at || null);
}
}
function unknown(name) {
return () => {
const el = document.createElement("div");
el.className = "wai-unknown";
el.textContent = `(${name}: no component for this widget on this page)`;
return { el };
};
}

199
wai/wai-app/web/room-frame.html Normal file
View file

@ -0,0 +1,199 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>The room</title>
<!-- A frame of the page that holds a room widget, not a page of its own: not indexed. -->
<meta name="robots" content="noindex">
<link rel="icon" href="data:,">
<!-- The room client of joule-moq-web in a frame of the page that holds a WAI room widget
(wai-app's components.js): the same markup as the client's own page, every element it
looks up present, and shown in part. The widget drives the client through its interface
(roomTest) from the page around it, which seals every action; the frame shows the room's
video (view=stage), the file a watch room watches (view=watch), or nothing (view=none). -->
<style>
:root { --bg: #f6f7f9; --fg: #15171a; --muted: #5d6470; --panel: #ffffff; --line: #d9dde3; --human: #1f6f43; --machine: #8a4b00; }
@media (prefers-color-scheme: dark) { :root { --bg: #111317; --fg: #e8eaee; --muted: #9aa2ae; --panel: #1a1d23; --line: #2c313a; --human: #6fd39b; --machine: #f0a957; } }
* { box-sizing: border-box; }
[hidden] { display: none !important; }
body { margin: 0; padding: 16px; background: var(--bg); color: var(--fg); font: 14px/1.45 system-ui, sans-serif; }
header { display: flex; gap: 12px; align-items: baseline; flex-wrap: wrap; margin-bottom: 12px; }
h1 { font-size: 16px; margin: 0; }
#who { color: var(--muted); font-family: ui-monospace, monospace; }
main { display: grid; grid-template-columns: minmax(0, 2fr) minmax(0, 1fr); gap: 12px; }
@media (max-width: 760px) { main { grid-template-columns: minmax(0, 1fr); } }
section { background: var(--panel); border: 1px solid var(--line); border-radius: 6px; padding: 10px; min-width: 0; }
canvas { width: 100%; height: auto; display: block; background: #101218; border-radius: 4px; }
#caption { min-height: 2.6em; margin-top: 8px; display: flex; gap: 8px; align-items: baseline; flex-wrap: wrap; }
#caption .origin { font-size: 12px; padding: 1px 6px; border-radius: 3px; border: 1px solid currentColor; }
#caption .origin-human { color: var(--human); }
#caption .origin-machine, #caption .origin-machine-reviewed { color: var(--machine); }
#timeline { list-style: none; margin: 0; padding: 0; max-height: 280px; overflow: auto; font-family: ui-monospace, monospace; font-size: 12px; }
#timeline li { padding: 2px 0; border-bottom: 1px solid var(--line); overflow-wrap: anywhere; }
#log { white-space: pre-wrap; font-family: ui-monospace, monospace; font-size: 11px; color: var(--muted); max-height: 220px; overflow: auto; margin: 0; overflow-wrap: anywhere; }
h2 { font-size: 13px; margin: 0 0 6px; color: var(--muted); font-weight: 600; }
.controls { display: flex; gap: 8px; align-items: center; flex-wrap: wrap; margin-bottom: 12px; }
#status { color: var(--muted); }
/* Co-performance: the floor, what's next, the audience and fidelity. */
.co-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: 12px; }
.co-grid h3 { font-size: 12px; margin: 0 0 4px; color: var(--muted); font-weight: 600; }
.co-grid ul, .co-grid ol { margin: 0; padding-left: 18px; font-size: 13px; }
.co-grid li { padding: 1px 0; overflow-wrap: anywhere; }
#co-now { margin: 0 0 6px; font-weight: 600; }
#co-marks, #co-left { color: var(--muted); font-size: 12px; margin: 4px 0 0; }
#co-intents li[data-outcome="upcoming"] { color: var(--machine); }
#co-intents li[data-outcome="kept"] { color: var(--human); }
#co-lead { margin-top: 8px; }
#co-lead input[type="number"] { width: 4.5em; }
#co-input { margin-top: 6px; }
#co-options { display: flex; gap: 6px; flex-wrap: wrap; margin-bottom: 6px; }
table.binding { border-collapse: collapse; font-family: ui-monospace, monospace; font-size: 12px; width: 100%; }
table.binding th, table.binding td { text-align: left; padding: 2px 8px 2px 0; border-bottom: 1px solid var(--line); white-space: nowrap; }
table.binding tr[data-state="verified"] td:nth-child(2) { color: var(--human); }
table.binding tr[data-state="refused"] td:nth-child(2), table.binding tr[data-state="unbound"] td:nth-child(2) { color: var(--machine); }
body.embed { padding: 0; background: transparent; }
body.embed > header, body.embed #room-app, body.embed #dvr, body.embed #dvr-canvas, body.embed #remote,
body.embed #co, body.embed #binding-debug, body.embed main > section:has(#timeline), body.embed main > section:has(#log) { display: none !important; }
body.embed main { display: block; }
body.embed main > section { border: 0; padding: 0; background: transparent; }
body.embed #cowatch .controls > :not(#cw-sync) { display: none !important; }
body.embed[data-view="none"] main { display: none; }
body.embed[data-view="watch"] main > section:has(#canvas) { display: none; }
body.embed[data-view="stage"] #cowatch { display: none !important; }
</style>
</head>
<body class="embed">
<header><h1>MoQ room client</h1><span id="who">loading</span></header>
<!-- An end-to-end encrypted room: its epoch and the members of its group. -->
<div id="e2ee" class="controls" hidden><strong>End-to-end encrypted</strong> <span id="e2ee-state"></span></div>
<!-- With a room service (?service=<url>): enter as a publisher or a viewer. -->
<div id="room-app" class="controls">
<button id="join" type="button">Join with camera and microphone</button>
<button id="watch" type="button">Watch</button>
<button id="leave" type="button" hidden>Leave</button>
<span id="status" role="status" aria-live="polite"></span>
<form id="caption-form" hidden><label for="caption-text">Caption</label> <input id="caption-text" maxlength="200" autocomplete="off"> <button type="submit">Send</button></form>
</div>
<main>
<section>
<h2>The room, composed</h2>
<canvas id="canvas" width="640" height="360"></canvas>
<div id="caption" aria-live="polite"></div>
<div id="dvr" class="controls" style="margin: 8px 0 0">
<label for="seek-s">Seconds back</label>
<input id="seek-s" type="number" min="1" max="900" value="10" style="width: 5em">
<button id="seek" type="button" disabled>Seek back</button>
<button id="live" type="button" hidden>Back to live</button>
<span id="dvr-status" role="status" aria-live="polite"></span>
</div>
<canvas id="dvr-canvas" width="640" height="360" hidden style="margin-top: 8px" aria-label="The participant's earlier video, from the relay's window"></canvas>
</section>
<!-- A watch room (WAI co-watch): the source the room watches, its shared controls and how
closely the room keeps together. -->
<section id="cowatch" hidden style="grid-column: 1 / -1">
<h2>Watching together</h2>
<video id="cowatch-media" playsinline style="width: 100%; max-height: 360px; background: #101218; border-radius: 4px"></video>
<div class="controls" style="margin: 8px 0 0">
<button id="cw-play" type="button">Play</button>
<button id="cw-pause" type="button">Pause</button>
<button id="cw-back" type="button">Back 10 s</button>
<button id="cw-fwd" type="button">Ahead 10 s</button>
<label for="cw-rate">Speed</label>
<select id="cw-rate"><option value="750000">0.75×</option><option value="1000000" selected>1×</option><option value="1250000">1.25×</option><option value="1500000">1.5×</option></select>
<span id="cw-sync" role="status" aria-live="polite"></span>
</div>
</section>
<!-- A remote desktop room (WAI remote-desktop): the host's screen, what it captures and who
holds the floor. -->
<section id="remote" hidden style="grid-column: 1 / -1" aria-label="Remote desktop">
<h2>Remote desktop</h2>
<canvas id="remote-canvas" width="1280" height="720" aria-label="The host's screen"></canvas>
<div class="controls" style="margin: 8px 0 0">
<button id="remote-request" type="button" hidden>Ask for control</button>
<button id="remote-release" type="button" hidden>Give control back</button>
<span id="remote-state" role="status" aria-live="polite"></span>
</div>
</section>
<section>
<h2>Events</h2>
<ul id="timeline"></ul>
</section>
<!-- A room with co-performance (WAI co-performance): see co.js. -->
<section id="co" hidden style="grid-column: 1 / -1" aria-label="Co-performance">
<h2>Co-performance</h2>
<div class="co-grid">
<div>
<h3>The floor</h3>
<p id="co-now" role="status" aria-live="polite"></p>
<ol id="co-turns"></ol>
<p id="co-marks"></p>
<div class="controls" style="margin: 6px 0 0"><button id="co-pass" type="button" hidden>Pass the floor</button></div>
</div>
<div>
<h3>What's next</h3>
<ul id="co-intents" aria-live="polite"></ul>
</div>
<div>
<h3>The audience</h3>
<p id="co-question"></p>
<form id="co-input" hidden>
<div id="co-options"></div>
<div id="co-prompt-row" hidden><label for="co-prompt">Your prompt</label> <input id="co-prompt" maxlength="200" autocomplete="off"> <button type="submit">Send</button></div>
<p id="co-left"></p>
</form>
<ul id="co-selections"></ul>
</div>
<div>
<h3>Fidelity</h3>
<ul id="co-fidelity"></ul>
</div>
</div>
<div id="co-lead" class="controls" hidden>
<label for="co-turn-s">Turn (s)</label> <input id="co-turn-s" type="number" min="1" max="60" value="4">
<label for="co-turns-n">Turns</label> <input id="co-turns-n" type="number" min="1" max="64" value="4">
<label for="co-mode">Out of turn</label> <select id="co-mode"><option value="mark">marked</option><option value="refuse">refused</option></select>
<button id="co-start" type="button">Start turns</button>
<label for="co-q">Ask</label> <input id="co-q" maxlength="200" placeholder="Which way next?">
<label for="co-options-n">Options</label> <input id="co-options-n" type="number" min="2" max="32" value="3">
<label for="co-window-s">Open for (s)</label> <input id="co-window-s" type="number" min="2" max="300" value="8">
<button id="co-vote" type="button">Open a vote</button>
<button id="co-prompts" type="button">Ask for prompts</button>
<button id="co-fid" type="button">State fidelity</button>
</div>
</section>
<!-- With ?debug=binding: the state of the frame each tile shows (WAI browser-sink §10). -->
<section id="binding-debug" style="grid-column: 1 / -1" hidden>
<h2>Frames on screen</h2>
<div style="overflow-x: auto"><table class="binding">
<thead><tr><th>Tile</th><th>State</th><th>Tier</th><th>Digest</th><th>Receipt</th><th>Chain</th><th>Provenance</th><th>Route</th><th>Power efficient</th><th>Energy</th></tr></thead>
<tbody></tbody>
</table></div>
</section>
<section style="grid-column: 1 / -1">
<h2>Log</h2>
<pre id="log"></pre>
</section>
</main>
<button id="sound" type="button" hidden style="margin: 6px 0">Turn the room's sound on</button>
<script>
document.body.dataset.view = new URLSearchParams(location.search).get("view") || "none";
// The room's audio plays in this frame, and a browser starts a frame's audio only after a
// click in the frame itself: every audio context the client makes is kept here, and while
// one waits for that click, this button asks for it.
const contexts = [];
if (window.AudioContext) {
const Real = window.AudioContext;
window.AudioContext = class extends Real {
constructor(...a) { super(...a); contexts.push(this); }
};
}
const sound = document.getElementById("sound");
sound.addEventListener("click", () => { for (const c of contexts) c.resume(); });
setInterval(() => { sound.hidden = !contexts.some((c) => c.state === "suspended"); }, 500);
</script>
<script type="module" src="../../../joule-code/joule-code-rs/crates/joule-moq-web/web/room/app.js"></script>
</body>
</html>

147
wai/wai-app/web/runtime.mjs Normal file
View file

@ -0,0 +1,147 @@
#!/usr/bin/env node
// Lay out the Create runtime from this repository's working tree, as the WAI site serves it
// and as an exported project carries it under vendor/:
//
// <out>/create/ wai-app's page files, each named wai-app-<name> (the widgets, the
// renderer, the session, the room frame, the styles, Create's own page
// and its recordings), and create/pkg/ (wai-app's WebAssembly build, which
// this script does not make)
// <out>/room/ joule-moq-web's room client, each file named wai-app-room-<name>.js
// (app.js as wai-app-room-client.js), its imports rewritten to those names
// and to the WebAssembly packages room/moq/pkg/wai_app_room_moq.js,
// room/moq-e2ee/pkg/wai_app_room_moq_e2ee.js and room/live/pkg/wai_app_room_live.js
// (also not made here)
// <out>/create/wai-app-runtime.json every file an app needs, by its path under <out>,
// with its SHA-256: what an export copies under vendor/
//
// Every served name is unique across the WAI site, whose deploy fingerprints assets keyed
// by basename.
//
// node wai/wai-app/web/runtime.mjs <out>
//
// The room client's files are those app.js reaches by its imports. A relative path string
// in any of them that this script does not map fails the layout, so a new import is never
// served unrenamed. Run it after the WebAssembly packages are in place: runtime.json lists
// what is there.
import fs from "node:fs";
import path from "node:path";
import crypto from "node:crypto";
import { fileURLToPath } from "node:url";
const HERE = path.dirname(fileURLToPath(import.meta.url));
const REPO = path.resolve(HERE, "../../..");
const WEB = path.join(REPO, "joule-code/joule-code-rs/crates/joule-moq-web/web");
const out = process.argv[2];
if (!out) {
console.error("usage: node runtime.mjs <out>");
process.exit(2);
}
const OUT = path.resolve(out);
// wai-app's own files by their served names: unique across the site, which fingerprints its
// assets keyed by basename, and the same in an export's vendor/. Which of them an app needs.
const CREATE_FILES = {
"app.js": "wai-app-app.js",
"renderer.js": "wai-app-renderer.js",
"components.js": "wai-app-components.js",
"session.js": "wai-app-session.js",
"ui.css": "wai-app.css",
"room-frame.html": "wai-app-room-frame.html",
"create.js": "wai-app-create.js",
"model.js": "wai-app-model.js",
"index.html": "index.html",
"runtime.json": "wai-app-runtime.json",
"pkg/wai_app.js": "pkg/wai_app.js",
};
const APP_FILES = ["wai-app-app.js", "wai-app-renderer.js", "wai-app-components.js", "wai-app-session.js", "wai-app.css", "wai-app-room-frame.html", "pkg/wai_app.js", "pkg/wai_app_bg.wasm"];
const FRAME_CLIENT = "../../../joule-code/joule-code-rs/crates/joule-moq-web/web/room/app.js";
// The room client's served name for each file of joule-moq-web/web it is made of.
function servedName(rel) {
const fixed = {
"room/app.js": "room/wai-app-room-client.js",
"pkg/joule_moq_web.js": "room/moq/pkg/wai_app_room_moq.js",
"pkg/joule_moq_web_e2ee.js": "room/moq-e2ee/pkg/wai_app_room_moq_e2ee.js",
"pkg/wai_live_web.js": "room/live/pkg/wai_app_room_live.js",
};
if (fixed[rel]) return fixed[rel];
const m = rel.match(/^(room|moq)\/([a-z0-9-]+)\.js$/);
if (m) return `room/wai-app-room-${m[2]}.js`;
return null;
}
const SPEC = /(["'`])(\.{1,2}\/[^"'`\s]*)\1/g;
const sha256 = (b) => crypto.createHash("sha256").update(b).digest("hex");
function layRoomClient() {
const todo = ["room/app.js"];
const done = new Set();
while (todo.length) {
const rel = todo.pop();
if (done.has(rel)) continue;
done.add(rel);
const served = servedName(rel);
const text = fs.readFileSync(path.join(WEB, rel), "utf8");
const rewritten = text.replace(SPEC, (all, q, spec) => {
const target = path.posix.normalize(path.posix.join(path.posix.dirname(rel), spec));
const to = servedName(target);
if (!to) throw new Error(`runtime: ${rel} names ${spec} (${target}), which the room client's layout does not map`);
if (!target.startsWith("pkg/")) todo.push(target);
let r = path.posix.relative(path.posix.dirname(served), to);
if (!r.startsWith(".")) r = `./${r}`;
return `${q}${r}${q}`;
});
const header = `// joule-moq-web/web/${rel} (Apache-2.0), sha256 ${sha256(text)}; its imports renamed by wai/wai-app/web/runtime.mjs.\n`;
fs.mkdirSync(path.join(OUT, path.dirname(served)), { recursive: true });
fs.writeFileSync(path.join(OUT, served), header + rewritten);
}
return [...done].sort();
}
// wai-app's files under their served names, each relative path in them (an import, a
// stylesheet, a frame, a data attribute) rewritten to those names; one the table does not
// map fails the layout. The room frame loads the room client laid out beside it.
function layCreate() {
const dir = path.join(OUT, "create");
fs.mkdirSync(dir, { recursive: true });
for (const [src, served] of Object.entries(CREATE_FILES)) {
if (src === "runtime.json" || src.startsWith("pkg/")) continue;
let text = fs.readFileSync(path.join(HERE, src), "utf8");
if (src === "room-frame.html") {
if (!text.includes(`src="${FRAME_CLIENT}"`)) throw new Error("runtime: room-frame.html does not load the room client by its source path");
text = text.replace(`src="${FRAME_CLIENT}"`, 'src="../room/wai-app-room-client.js"');
}
text = text.replace(/(["'`])\.\/([a-z0-9_./-]+)\1/gi, (all, q, rel) => {
if (rel === "recorded/") return all;
const to = CREATE_FILES[rel];
if (!to) throw new Error(`runtime: ${src} names ./${rel}, which the layout does not map`);
return `${q}./${to}${q}`;
});
fs.writeFileSync(path.join(dir, served), text);
}
fs.mkdirSync(path.join(dir, "recorded"), { recursive: true });
for (const f of fs.readdirSync(path.join(HERE, "recorded"))) fs.copyFileSync(path.join(HERE, "recorded", f), path.join(dir, "recorded", f));
}
function walk(d, base, acc) {
if (!fs.existsSync(d)) return acc;
for (const e of fs.readdirSync(d, { withFileTypes: true }).sort((a, b) => (a.name < b.name ? -1 : 1))) {
const p = path.join(d, e.name);
if (e.isDirectory()) walk(p, base, acc);
else acc.push(path.relative(base, p).split(path.sep).join("/"));
}
return acc;
}
const room = layRoomClient();
layCreate();
const files = [];
for (const f of APP_FILES) {
const p = path.join(OUT, "create", f);
if (!fs.existsSync(p)) throw new Error(`runtime: create/${f} is missing (build wai-app's WebAssembly package into ${path.join(OUT, "create/pkg")} first)`);
files.push(`create/${f}`);
}
for (const f of walk(path.join(OUT, "room"), OUT, [])) files.push(f);
const listed = files.map((f) => ({ path: f, sha256: sha256(fs.readFileSync(path.join(OUT, f))) }));
fs.writeFileSync(path.join(OUT, "create/wai-app-runtime.json"), JSON.stringify({ note: "The runtime an app made with Create carries under vendor/, by path, with each file's SHA-256.", files: listed }, null, 1) + "\n");
console.log(`runtime: ${room.length} room client file(s), ${listed.length} runtime file(s) listed in create/wai-app-runtime.json`);

58
wai/wai-app/web/session.js Normal file
View file

@ -0,0 +1,58 @@
// The page's session: wai-app's WebAssembly build, the program as it streams, and the log
// every action a person takes on the page is sealed into (joule-ui SPEC §6, §7).
//
// `fire(action, inputs, handler)` is the interaction lifecycle around an asynchronous
// handler: authorized first (a denied action's handler never runs), the handler run and
// timed on the page's clock, then sealed as the next link of the log under the page's own
// key. A page has no energy meter: the figure is the estimator's, from the handler's wall
// time, and the log says so (`Estimator`).
import init, { Session, verifyLog, registry, guide, check } from "./pkg/wai_app.js";
export { verifyLog, registry, guide, check };
export async function openSession() {
await init();
const seed = crypto.getRandomValues(new Uint8Array(32));
const s = new Session(seed);
seed.fill(0);
return new PageSession(s);
}
export class PageSession {
constructor(wasm) {
this.w = wasm;
this.listeners = new Set();
this.actor = wasm.actor();
}
push(chunk) { return JSON.parse(this.w.push(chunk)); }
finish() { return JSON.parse(this.w.finish()); }
reset() { this.w.reset(); }
program() { return this.w.program(); }
canonical() { return this.w.canonical(); }
log() { return JSON.parse(this.w.log()); }
logRaw() { return this.w.logRaw(); }
verify() { return JSON.parse(this.w.verify()); }
totalUj() { return this.w.totalUj(); }
on(f) { this.listeners.add(f); return () => this.listeners.delete(f); }
note(entry) { for (const f of this.listeners) { try { f(entry); } catch { /* a view's own fault */ } } }
async fire(action, inputs, handler) {
const b = JSON.parse(this.w.beginAction(action, JSON.stringify(inputs === undefined ? null : inputs)));
if (b.sealed) {
this.note(b.sealed);
throw new Error(`${action} was denied: ${b.sealed.reason}`);
}
const t0 = performance.now();
let out;
let err = null;
try { out = await handler(); } catch (e) { err = e; }
const wall = performance.now() - t0;
const output = err ? { error: String((err && err.message) || err) } : out === undefined ? null : out;
let text;
try { text = JSON.stringify(output); } catch { text = JSON.stringify(String(output)); }
const entry = JSON.parse(this.w.finishAction(b.pending, text === undefined ? "null" : text, wall));
this.note(entry);
if (err) throw err;
return out;
}
}

112
wai/wai-app/web/ui.css Normal file
View file

@ -0,0 +1,112 @@
/* The WAI widgets (components.js) and Create (create.js). Light and dark, from the
visitor's preference; every colour a token on the root. */
:root {
--wai-bg: #f6f7f9;
--wai-panel: #ffffff;
--wai-fg: #15171a;
--wai-muted: #5d6470;
--wai-line: #d9dde3;
--wai-accent: #1f5fd1;
--wai-accent-fg: #ffffff;
--wai-ok: #1f6f43;
--wai-bad: #a3271f;
--wai-warn: #8a4b00;
--wai-new: #fff3c4;
}
@media (prefers-color-scheme: dark) {
:root:not([data-theme="light"]) {
--wai-bg: #111317;
--wai-panel: #1a1d23;
--wai-fg: #e8eaee;
--wai-muted: #9aa2ae;
--wai-line: #2c313a;
--wai-accent: #6f9cff;
--wai-accent-fg: #0b0d10;
--wai-ok: #6fd39b;
--wai-bad: #ff8a80;
--wai-warn: #f0a957;
--wai-new: #3a3420;
}
}
:root[data-theme="dark"] {
--wai-bg: #111317; --wai-panel: #1a1d23; --wai-fg: #e8eaee; --wai-muted: #9aa2ae; --wai-line: #2c313a;
--wai-accent: #6f9cff; --wai-accent-fg: #0b0d10; --wai-ok: #6fd39b; --wai-bad: #ff8a80; --wai-warn: #f0a957; --wai-new: #3a3420;
}
.wai-app { color: var(--wai-fg); font: 15px/1.5 system-ui, sans-serif; }
body:has(> .wai-app) { margin: 0; padding: 16px; background: var(--wai-bg); }
.wai-app *, .create * { box-sizing: border-box; }
[hidden] { display: none !important; }
.wai-w { transition: background-color 1.2s ease; border-radius: 8px; }
.wai-new { background-color: var(--wai-new); }
.wai-app-shell { display: flex; flex-direction: column; gap: 12px; }
.wai-app-title { font-size: 1.4rem; margin: 0; letter-spacing: -0.01em; }
.wai-app-title:empty { display: none; }
.wai-app-children { display: flex; flex-direction: column; gap: 12px; }
.wai-text { margin: 0; color: var(--wai-muted); }
.wai-room { background: var(--wai-panel); border: 1px solid var(--wai-line); border-radius: 10px; padding: 12px; display: flex; flex-direction: column; gap: 10px; min-width: 0; }
.wai-room-head { display: flex; align-items: baseline; gap: 10px; flex-wrap: wrap; }
.wai-room-title { font-size: 1.1rem; margin: 0; }
.wai-kind { font-size: 12px; color: var(--wai-muted); border: 1px solid var(--wai-line); border-radius: 999px; padding: 1px 8px; }
.wai-room-controls, .wai-watch-controls { display: flex; gap: 8px; flex-wrap: wrap; align-items: center; }
.wai-room-status { color: var(--wai-muted); font-size: 14px; }
.wai-frame { width: 100%; border: 0; display: block; min-height: 0; }
.wai-room-stage[data-view="none"] .wai-frame { min-height: 0; }
.wai-room-children { display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: 12px; }
.wai-room-children > .wai-player, .wai-room-children > .wai-text { grid-column: 1 / -1; }
.wai-btn { font: inherit; padding: 6px 12px; border-radius: 8px; border: 1px solid var(--wai-line); background: var(--wai-panel); color: var(--wai-fg); cursor: pointer; }
.wai-btn:disabled { opacity: 0.5; cursor: default; }
.wai-primary { background: var(--wai-accent); color: var(--wai-accent-fg); border-color: var(--wai-accent); }
.wai-input { font: inherit; padding: 6px 8px; border-radius: 8px; border: 1px solid var(--wai-line); background: var(--wai-bg); color: var(--wai-fg); flex: 1; min-width: 0; }
.wai-h { font-size: 13px; margin: 0 0 6px; color: var(--wai-muted); font-weight: 600; }
.wai-note { font-size: 12px; color: var(--wai-muted); margin: 6px 0 0; }
.wai-player { font-size: 13px; color: var(--wai-muted); }
.wai-chat, .wai-poll, .wai-participants, .wai-receipts, .wai-energymeter { border: 1px solid var(--wai-line); border-radius: 8px; padding: 10px; min-width: 0; background: var(--wai-panel); }
.wai-chat-list { list-style: none; margin: 0 0 8px; padding: 0; max-height: 200px; overflow: auto; font-size: 14px; }
.wai-chat-list li { padding: 2px 0; overflow-wrap: anywhere; }
.wai-chat-form { display: flex; gap: 6px; }
.wai-poll-options { display: flex; flex-wrap: wrap; gap: 6px; margin: 6px 0; }
.wai-poll-result { margin: 6px 0 0; font-size: 14px; }
.wai-poll[data-state="closed"] .wai-poll-result { color: var(--wai-ok); }
.wai-people { margin: 0; padding-left: 18px; font-family: ui-monospace, monospace; font-size: 13px; }
.wai-verdict { margin: 0 0 6px; font-size: 13px; }
.wai-verdict[data-ok="true"] { color: var(--wai-ok); }
.wai-verdict[data-ok="false"] { color: var(--wai-bad); }
.wai-scroll { overflow-x: auto; }
.wai-log { border-collapse: collapse; font-size: 12px; width: 100%; }
.wai-log th, .wai-log td { text-align: left; padding: 2px 8px 2px 0; border-bottom: 1px solid var(--wai-line); white-space: nowrap; }
.wai-mono { font-family: ui-monospace, monospace; }
.wai-energy { margin: 0; padding-left: 18px; font-size: 13px; }
.wai-energy-total { margin: 0; font-weight: 600; }
.wai-problems { color: var(--wai-bad); white-space: pre-wrap; }
.wai-unknown { color: var(--wai-warn); font-size: 13px; }
/* Create */
.create { color: var(--wai-fg); font: 15px/1.5 system-ui, sans-serif; display: flex; flex-direction: column; gap: 14px; }
.create form { display: flex; flex-direction: column; gap: 10px; background: var(--wai-panel); border: 1px solid var(--wai-line); border-radius: 10px; padding: 12px; }
.create textarea { font: inherit; width: 100%; min-height: 4.5em; padding: 8px; border-radius: 8px; border: 1px solid var(--wai-line); background: var(--wai-bg); color: var(--wai-fg); }
.create fieldset { border: 0; padding: 0; margin: 0; display: flex; gap: 14px; flex-wrap: wrap; }
.create legend { font-size: 13px; color: var(--wai-muted); margin-bottom: 4px; }
.create .source-fields { display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 8px; }
.create .source-fields label { display: flex; flex-direction: column; font-size: 13px; color: var(--wai-muted); gap: 2px; }
.create .source-fields input, .create .source-fields select { font: inherit; padding: 6px 8px; border-radius: 8px; border: 1px solid var(--wai-line); background: var(--wai-bg); color: var(--wai-fg); }
#model-says { margin: 0; font-size: 14px; color: var(--wai-muted); }
.create-grid { display: grid; grid-template-columns: minmax(0, 3fr) minmax(0, 2fr); gap: 14px; align-items: start; }
@media (max-width: 860px) { .create-grid { grid-template-columns: minmax(0, 1fr); } }
.create aside { background: var(--wai-panel); border: 1px solid var(--wai-line); border-radius: 10px; padding: 12px; min-width: 0; }
.create aside h2 { font-size: 13px; color: var(--wai-muted); margin: 10px 0 6px; font-weight: 600; }
.create aside h2:first-child { margin-top: 0; }
#program { font-family: ui-monospace, monospace; font-size: 12px; margin: 0; padding-left: 3.2em; max-height: 320px; overflow: auto; }
#program li { white-space: pre-wrap; overflow-wrap: anywhere; }
#program li.bad { color: var(--wai-bad); }
#program li.pending { color: var(--wai-muted); }
#problems { margin: 0; padding-left: 18px; font-size: 13px; }
#problems li.bad { color: var(--wai-bad); }
#problems li.warn { color: var(--wai-warn); }
#problems li.ok { color: var(--wai-ok); }
#app:empty::before { content: "The app appears here, line by line, as the model writes it."; color: var(--wai-muted); font-size: 14px; }

105
wai/wai-app/web/web.test.mjs Normal file
View file

@ -0,0 +1,105 @@
// The page modules that need no browser: the renderer's reconciliation (on a small stand-in
// for the document), the prop values' plain form, the ZIP writer, and the model stream
// readers.
//
// node --test wai/wai-app/web/web.test.mjs
import test from "node:test";
import assert from "node:assert/strict";
import zlib from "node:zlib";
// A stand-in for the few DOM operations the renderer makes.
class El {
constructor(tag) { this.tag = tag; this.children = []; this.parent = null; this.dataset = {}; this.classList = new Set(); this.classList.remove = Set.prototype.delete; }
get firstElementChild() { return this.children[0] || null; }
remove() { if (this.parent) { this.parent.children.splice(this.parent.children.indexOf(this), 1); this.parent = null; } }
insertBefore(el, ref) {
el.remove();
const i = ref ? this.children.indexOf(ref) : this.children.length;
this.children.splice(i < 0 ? this.children.length : i, 0, el);
el.parent = this;
this.moves = (this.moves || 0) + 1;
}
replaceChildren(...els) { for (const c of [...this.children]) c.remove(); for (const e of els) this.insertBefore(e, null); }
}
globalThis.setTimeout = ((orig) => (f, ms) => orig(f, ms))(globalThis.setTimeout);
const { Renderer, plain, plainProps } = await import("./renderer.js");
const { completionChunks, replay } = await import("./model.js");
const T = (v) => ({ type: "text", value: v });
function comps(log) {
const make = (name, withSlot) => (ctx, props, node) => {
const el = new El(name);
const c = { el, props, update(p) { log.push(`update ${node.key}`); c.props = p; }, destroy() { log.push(`destroy ${node.key}`); } };
if (withSlot) { c.slot = new El("slot"); el.children.push(c.slot); }
log.push(`create ${node.key}`);
return c;
};
return { App: make("App", true), Room: make("Room", true), Chat: make("Chat"), Text: make("Text") };
}
const snap = (tree, keys) => ({ tree, nodes: keys.map((key, i) => ({ key, line: i + 1, statement: key })) });
test("a widget keeps its instance across snapshots while its key and widget stay", () => {
const log = [];
const root = new El("root");
const r = new Renderer(root, comps(log), {});
const app = (kids, title) => ({ name: "App", props: { title: T(title) }, children: kids });
r.render(snap(app([], "A"), ["root"]));
const room0 = { name: "Room", props: {}, children: [] };
r.render(snap(app([room0], "A"), ["root", "room"]));
const roomEl = r.inst.get("room").comp.el;
r.render(snap(app([{ name: "Room", props: {}, children: [{ name: "Chat", props: {} }] }], "A b"), ["root", "room", "chat"]));
assert.equal(r.inst.get("room").comp.el, roomEl, "the room is the same instance");
assert.deepEqual(log, ["create root", "create room", "update root", "create chat"]);
// The room's element was placed once and never moved again (a frame moved reloads).
assert.equal(r.inst.get("root").comp.slot.moves, 1);
// A widget gone is destroyed; one whose widget changed under its key is made again.
r.render(snap(app([{ name: "Text", props: { text: T("x") } }], "A b"), ["root", "room"]));
assert.deepEqual(log.slice(4), ["destroy room", "create room", "destroy chat"]);
r.render({ tree: null, nodes: [] });
assert.equal(root.children.length, 0);
});
test("children are placed in order, inserting without moving what is in place", () => {
const log = [];
const root = new El("root");
const r = new Renderer(root, comps(log), {});
const kid = (n) => ({ name: "Text", props: { text: T(n) } });
r.render(snap({ name: "App", props: {}, children: [kid("a"), kid("c")] }, ["root", "a", "c"]));
const slot = r.inst.get("root").comp.slot;
const c = r.inst.get("c").comp.el;
slot.moves = 0;
r.render(snap({ name: "App", props: {}, children: [kid("a"), kid("b"), kid("c")] }, ["root", "a", "b", "c"]));
assert.deepEqual(slot.children.map((e) => e.tag), ["Text", "Text", "Text"]);
assert.equal(slot.children[2], c);
assert.equal(slot.moves, 1, "only the new widget was inserted");
});
test("prop values in their plain form", () => {
assert.deepEqual(plainProps({ a: T("x"), n: { type: "number", value: 2 }, b: { type: "bool", value: false }, z: { type: "null" } }), { a: "x", n: 2, b: false, z: null });
assert.deepEqual(plain({ type: "list", value: [T("a"), { type: "list", value: [] }] }), ["a", []]);
assert.deepEqual(plain({ type: "object", value: { k: T("v") } }), { k: "v" });
assert.deepEqual(plain({ type: "action", value: { id: "Go", payload: { type: "list", value: [T("a")] } } }), { action: "Go", payload: ["a"] });
});
test("a chat/completions stream gives its content chunks", async () => {
const lines = (async function* () {
yield ": keep-alive";
yield 'data: {"choices":[{"delta":{"role":"assistant"}}]}';
yield 'data: {"choices":[{"delta":{"content":"root = "}}]}';
yield "";
yield 'data: {"choices":[{"delta":{"content":"App()"}}]}';
yield "data: [DONE]";
})();
const got = [];
for await (const c of completionChunks(lines)) got.push(c);
assert.deepEqual(got, ["root = ", "App()"]);
const err = (async function* () { yield 'data: {"error":{"message":"bad key"}}'; })();
await assert.rejects(async () => { for await (const _ of completionChunks(err)) { /* nothing */ } }, /bad key/);
});
test("a recorded stream replays its chunks in order", async () => {
const got = [];
for await (const c of replay({ chunks: ["a", "b", "c"], pause_ms: 1 })) got.push(c);
assert.deepEqual(got, ["a", "b", "c"]);
});

45
wai/wai-rs/Cargo.lock generated
View file

@ -2092,6 +2092,36 @@ dependencies = [
"libc",
]
[[package]]
name = "joule-ui-core"
version = "0.1.0"
dependencies = [
"serde",
"serde_json",
"thiserror 2.0.20",
]
[[package]]
name = "joule-ui-interaction"
version = "0.1.0"
dependencies = [
"blake3",
"ed25519-dalek",
"serde",
"serde_json",
"thiserror 2.0.20",
]
[[package]]
name = "joule-ui-lang"
version = "0.1.0"
dependencies = [
"joule-ui-core",
"serde",
"serde_json",
"sha2 0.10.9",
]
[[package]]
name = "jpeg-encoder"
version = "0.7.1"
@ -4520,6 +4550,7 @@ dependencies = [
"serde",
"serde_json",
"sha2 0.10.9",
"wai-app",
"wai-ops",
"wai-ops-gpu",
"wai-quantum",
@ -4528,6 +4559,20 @@ dependencies = [
"zstd-safe",
]
[[package]]
name = "wai-app"
version = "0.1.0"
dependencies = [
"bs58",
"ed25519-dalek",
"joule-ui-core",
"joule-ui-interaction",
"joule-ui-lang",
"serde",
"serde_json",
"sha2 0.10.9",
]
[[package]]
name = "wai-ops"
version = "0.1.0"

15
wai/wai-rs/Cargo.toml
View file

@ -116,6 +116,9 @@ getrandom = { version = "0.3", optional = true }
# The quantum stack now lives in its own dependency-light crate so it can be used
# without the media/codec chain. `wai` re-exports it unchanged.
wai-quantum = { version = "0.6.0", path = "../wai-quantum", optional = true, default-features = false }
# Create's library (WAI's widgets for joule-ui): `wai create --from app.oui`, `wai app` and
# `wai mcp --toolset app`, in a build with the feature `app`.
wai-app = { path = "../wai-app", optional = true }
# ---- the integer operator set (behind `neural_int`) ----
# `wai-ops/1` (extensions/integer-operators.md): an integer decoder that decodes
@ -714,6 +717,11 @@ cli_tls = ["http", "key_discovery", "dep:rustls", "dep:rustls-platform-verifier"
# development room's test principals. It opens no connection but to a room service on a loopback
# address; no new dependency.
mcp = ["receipt_claims", "live_room", "stream_audit", "ed25519"]
# Apps made with Create (wai/wai-app, a library): `wai create <dir> --from app.oui` (the project,
# its project file written as a template's is), `wai app guide|check`, and `wai mcp`'s `app`
# toolset (the guide, a program checked, streamed into a person's open Create page through a
# bridge on 127.0.0.1, exported). Built into the binary only.
app = ["mcp", "dep:wai-app"]
# Emit a REAL C2PA manifest (not just the assertion shapes). Off by default:
# the SDK is a large dependency chain and most WAI consumers only need the
# binding types, which `provenance` already gives them.
@ -1505,6 +1513,13 @@ name = "cli_mcp"
path = "tests/cli_mcp.rs"
required-features = ["mcp"]
# An app made with Create: `wai create --from app.oui`, `wai app check`, and `wai mcp`'s app
# toolset driven by an MCP client.
[[test]]
name = "cli_app"
path = "tests/cli_app.rs"
required-features = ["app"]
[[test]]
name = "int_kernels_vectors"
path = "tests/int_kernels_vectors.rs"

169
wai/wai-rs/src/bin/wai_cli/dx/app.rs Normal file
View file

@ -0,0 +1,169 @@
//! `wai app`: apps made with Create (a build with the feature `app`, `wai-app` linked in).
//!
//! - `wai app guide` (`app.guide`): the line language and the WAI widgets, as a model writing
//! an app is told them.
//! - `wai app check [<dir> | <app.oui>] [--json]` (`app.check`): a program checked against the
//! WAI registry and WAI's rules, every error with its line and reason code. In a project
//! (a `wai.json` beside the program, as `wai create --from app.oui` writes one), also the
//! room the project file names for `wai dev` against the app's own, and the runtime under
//! `vendor/` file by file against `vendor.json`. Exit 0 when everything holds, 1 when
//! something does not, each failure named.
pub const USAGE: &str = "wai app guide | wai app check [<dir> | <app.oui>] [--json]\n an app made with Create: what a model is told, and the app, its room and its runtime checked";
#[cfg(not(feature = "app"))]
pub fn main(_: &[String]) -> Result<bool, String> {
Err("`wai app` needs a build with the feature `app` (cargo install --path wai/wai-rs --no-default-features --features app)".into())
}
#[cfg(feature = "app")]
pub fn main(args: &[String]) -> Result<bool, String> {
use super::{flag, positional};
use std::path::Path;
match args.first().map(String::as_str) {
Some("guide") => {
print!("{}", wai_app::guide::guide());
Ok(true)
}
Some("check") => {
if let Some(a) = args[1..].iter().find(|a| a.starts_with("--") && *a != "--json") {
return Err(format!("wai app check: {a}: not an option ({USAGE})"));
}
let pos = positional(&args[1..], &[]);
if pos.len() > 1 {
return Err(USAGE.into());
}
let at = Path::new(pos.first().copied().unwrap_or("."));
let report = check(at)?;
let ok = report.failures.is_empty();
if flag(args, "--json") || !std::io::IsTerminal::is_terminal(&std::io::stdout()) {
println!(
"{}",
serde_json::json!({
"wai": "envelope/1", "op": "app.check", "ok": ok,
"result": { "program": report.program, "project": report.project, "room": report.room, "errors": report.errors, "failures": report.failures, "checked": report.checked },
})
);
} else {
println!("wai app check: {}", report.program);
for c in &report.checked {
println!(" ok {c}");
}
for f in &report.failures {
println!(" FAIL {f}");
}
}
Ok(ok)
}
_ => Err(USAGE.into()),
}
}
/// What `wai app check` found.
#[cfg(feature = "app")]
pub(crate) struct Report {
pub program: String,
pub project: Option<String>,
pub room: Option<wai_app::RoomPlan>,
pub errors: Vec<wai_app::ErrorOut>,
/// What held, one line each.
pub checked: Vec<String>,
/// What did not, one line each.
pub failures: Vec<String>,
}
/// Check the app at `at`: a project directory, or a program file (its directory the project's
/// when a `wai.json` is beside it).
#[cfg(feature = "app")]
pub(crate) fn check(at: &std::path::Path) -> Result<Report, String> {
use super::project;
use wai_app::export::{dev_mode, APP};
let (dir, program_path) = if at.is_dir() {
let p = project::load(at)?;
let app = p.as_ref().and_then(|p| p.app.clone()).unwrap_or_else(|| at.join(APP));
(at.to_path_buf(), app)
} else {
(at.parent().map(|d| if d.as_os_str().is_empty() { std::path::Path::new(".") } else { d }).unwrap_or(std::path::Path::new(".")).to_path_buf(), at.to_path_buf())
};
let text = std::fs::read_to_string(&program_path).map_err(|e| format!("{}: {e}", program_path.display()))?;
let c = wai_app::check(&text);
let mut r = Report { program: program_path.display().to_string(), project: None, room: c.room.clone(), errors: c.errors.clone(), checked: Vec::new(), failures: Vec::new() };
let bad: Vec<&wai_app::ErrorOut> = c.errors.iter().filter(|e| !e.warning).collect();
for e in &bad {
r.failures.push(format!("{} line {}: {}: {}", APP, e.line, e.code, e.message));
}
if bad.is_empty() {
r.checked.push(format!("{APP}: every line accepted by the WAI registry, {} widget(s)", c.nodes.len()));
}
let Some(room) = &c.room else {
r.failures.push(format!("{APP}: the app holds no room (a Room, a WatchTogether or a Show)"));
return Ok(r);
};
let Some(p) = project::load(&dir)? else {
return Ok(r);
};
r.project = Some(dir.join(project::FILE).display().to_string());
// The room `wai dev` runs for the project is the app's.
let want = dev_mode(&room.kind);
match p.dev.mode.as_deref() {
None => r.failures.push(format!("{}: names no room for the app (dev.mode \"{want}\")", project::FILE)),
Some(m) if m != want => r.failures.push(format!("{}: dev.mode is \"{m}\", and the app's room is \"{want}\"", project::FILE)),
Some(_) => r.checked.push(format!("{}: dev.mode \"{want}\", the app's room", project::FILE)),
}
if room.kind == "show" {
match p.dev.audience {
Some(a) if a == room.audience => r.checked.push(format!("{}: dev.audience {a}, the show's", project::FILE)),
got => r.failures.push(format!("{}: dev.audience is {}, and the show's audience is {}", project::FILE, got.map_or("absent".into(), |a| a.to_string()), room.audience)),
}
}
if room.kind == "watch" {
let source = room.source.as_deref().unwrap_or_default();
let want_path = dir.join("media").join(source);
match &p.dev.watch {
Some(w) if *w == want_path => {
if w.is_file() {
r.checked.push(format!("{}: dev.watch media/{source}, present", project::FILE));
} else {
r.failures.push(format!("media/{source} is not there: put the file the room watches at it"));
}
}
got => r.failures.push(format!("{}: dev.watch is {}, and the app watches media/{source}", project::FILE, got.as_ref().map_or("absent".into(), |w| w.display().to_string()))),
}
}
// The runtime under vendor/, file by file.
let vendor = dir.join("vendor.json");
match std::fs::read_to_string(&vendor).map_err(|e| e.to_string()).and_then(|t| serde_json::from_str::<serde_json::Value>(&t).map_err(|e| e.to_string())) {
Err(e) => r.failures.push(format!("vendor.json: {e}")),
Ok(v) => {
let files = v["files"].as_array().cloned().unwrap_or_default();
let mut good = 0;
for f in &files {
let (Some(path), Some(sum)) = (f["path"].as_str(), f["sha256"].as_str()) else {
r.failures.push("vendor.json: a file without its path or its SHA-256".into());
continue;
};
if path.split('/').any(|s| s.is_empty() || s == "." || s == "..") {
r.failures.push(format!("vendor.json: the path {path} leaves vendor/"));
continue;
}
match std::fs::read(dir.join("vendor").join(path)) {
Err(_) => r.failures.push(format!("vendor/{path} is missing (vendor.json names it)")),
Ok(b) => {
let got = wai_app::runtime::sha256_hex(&b);
if got == sum {
good += 1;
} else {
r.failures.push(format!("vendor/{path}: its SHA-256 is {got}, vendor.json says {sum}"));
}
}
}
}
if files.is_empty() {
r.failures.push("vendor.json lists no runtime file".into());
} else if good == files.len() {
r.checked.push(format!("vendor/: {good} runtime file(s), each the one vendor.json names"));
}
}
}
Ok(r)
}

133
wai/wai-rs/src/bin/wai_cli/dx/create.rs
View file

@ -1,11 +1,19 @@
//! `wai create <dir> --template <name>`: a new project from one of the examples in
//! `wai/examples/`, whose files this binary holds ([`templates`]), so it needs neither the
//! repository nor a network.
//! `wai create` (catalogue `project.create`): a new project, in one of two ways.
//!
//! It writes into a directory that does not exist or is empty, and never over a file. Each
//! example's `test/` directory is the repository's check that the example works, and is not
//! part of the project. Where a template names a generated file (a watch room's sample
//! sound), it is made here.
//! - `wai create <dir> --template <name>`: from one of the examples in `wai/examples/`, whose
//! files this binary holds ([`templates`]), so it needs neither the repository nor a network.
//! Each example's `test/` directory is the repository's check that the example works, and is
//! not part of the project. Where a template names a generated file (a watch room's sample
//! sound), it is made here.
//! - `wai create <dir> --from app.oui` (catalogue `app.export`; a build with `app`): from a
//! program made with Create. The app's files are `wai_app::export`'s, its runtime is copied
//! in from a Create runtime (`--runtime DIR`, `$WAI_APP_RUNTIME`, or `share/wai/app` beside
//! this binary) and checked file by file, and its project file names the program and the
//! room `wai dev` runs for it.
//!
//! Either way the project file is written by [`project::render`], and every project carries
//! the licence its files are under, Apache-2.0, as `LICENSE`. It writes into a directory that
//! does not exist or is empty, and never over a file.
#[path = "templates.rs"]
mod templates;
@ -13,7 +21,10 @@ mod templates;
use super::{flag, opt, positional, project};
use std::path::Path;
pub const USAGE: &str = "wai create <dir> --template <name> | wai create --list [--json] a new project from one of the examples";
pub const USAGE: &str = "wai create <dir> --template <name> | wai create <dir> --from app.oui [--runtime DIR] | wai create --list [--json]\n a new project from one of the examples, or from an app made with Create";
/// The licence every project `wai create` writes is under, and the file that says so.
const LICENSE: (&str, &[u8]) = ("LICENSE", include_bytes!("../../../../../LICENSE"));
/// A file of a template: its path in the project, and its bytes.
pub type File = (&'static str, &'static [u8]);
@ -33,7 +44,7 @@ pub fn main(args: &[String]) -> Result<bool, String> {
if flag(args, "--json") {
let v: Vec<serde_json::Value> = templates::TEMPLATES
.iter()
.map(|t| serde_json::json!({ "template": t.name, "title": t.title, "files": t.files.iter().map(|f| f.0).chain(t.generated.iter().map(|g| g.0)).collect::<Vec<_>>() }))
.map(|t| serde_json::json!({ "template": t.name, "title": t.title, "files": t.files.iter().map(|f| f.0).chain(t.generated.iter().map(|g| g.0)).chain([LICENSE.0]).collect::<Vec<_>>() }))
.collect();
println!("{}", serde_json::to_string_pretty(&v).expect("json"));
} else {
@ -43,9 +54,24 @@ pub fn main(args: &[String]) -> Result<bool, String> {
}
return Ok(true);
}
if let Some(a) = args.iter().find(|a| a.starts_with("--") && *a != "--template") {
if let Some(a) = args.iter().find(|a| a.starts_with("--") && !["--template", "--from", "--runtime"].contains(&a.as_str())) {
return Err(format!("wai create: {a}: not an option ({USAGE})"));
}
if let Some(program) = opt(args, "--from") {
let pos = positional(args, &["--from", "--runtime"]);
let (Some(dir), 1, None) = (pos.first(), pos.len(), opt(args, "--template")) else {
return Err(format!("wai create <dir> --from app.oui [--runtime DIR]\n{USAGE}"));
};
return from_app(Path::new(program), Path::new(dir), opt(args, "--runtime").map(Path::new)).map(|written| {
println!("wai create: an app made with Create, in {dir}");
for f in written.iter().filter(|f| !f.starts_with("vendor/")) {
println!(" {f}");
}
println!(" vendor/ ({} runtime files)", written.iter().filter(|f| f.starts_with("vendor/")).count());
println!("Next:\n cd {dir}\n wai app check the app, its room and its runtime\n wai dev then open the link it prints");
true
});
}
let pos = positional(args, &["--template"]);
let (Some(dir), Some(name), 1) = (pos.first(), opt(args, "--template"), pos.len()) else {
return Err(format!("{USAGE}\ntemplates: {}", names()));
@ -71,16 +97,25 @@ fn names() -> String {
/// The template's files in `dir`, which must not exist or must be empty. A project file is
/// written where the template carries none.
fn write(t: &Template, dir: &Path) -> Result<(), String> {
let mut files: Vec<(String, Vec<u8>)> = t.files.iter().map(|(p, b)| (p.to_string(), b.to_vec())).collect();
files.extend(t.generated.iter().map(|(p, f)| (p.to_string(), f())));
if !files.iter().any(|(p, _)| p == project::FILE) {
files.push((project::FILE.into(), project::minimal(t.name).into_bytes()));
}
write_project(dir, files)
}
/// A project's files in `dir`, which must not exist or must be empty, with the licence every
/// project carries; then its project file, read back. The one place `wai create` writes.
fn write_project(dir: &Path, mut files: Vec<(String, Vec<u8>)>) -> Result<(), String> {
if dir.exists() && std::fs::read_dir(dir).map_err(|e| format!("{}: {e}", dir.display()))?.next().is_some() {
return Err(format!("{}: not empty; name a new directory", dir.display()));
}
let mut files: Vec<(&str, Vec<u8>)> = t.files.iter().map(|(p, b)| (*p, b.to_vec())).collect();
files.extend(t.generated.iter().map(|(p, f)| (*p, f())));
if !files.iter().any(|(p, _)| *p == project::FILE) {
files.push((project::FILE, project::minimal(t.name).into_bytes()));
if !files.iter().any(|(p, _)| p == LICENSE.0) {
files.push((LICENSE.0.into(), LICENSE.1.to_vec()));
}
for (rel, bytes) in &files {
if rel.split('/').any(|s| s.is_empty() || s == "." || s == "..") {
if rel.split('/').any(|s| s.is_empty() || s == "." || s == "..") || rel.starts_with('/') {
return Err(format!("{rel}: not a path a template may write"));
}
let p = dir.join(rel);
@ -90,11 +125,73 @@ fn write(t: &Template, dir: &Path) -> Result<(), String> {
let mut f = std::fs::OpenOptions::new().write(true).create_new(true).open(&p).map_err(|e| format!("{}: {e}", p.display()))?;
std::io::Write::write_all(&mut f, bytes).map_err(|e| format!("{}: {e}", p.display()))?;
}
// The project file the template wrote must read: a template is checked as it is made.
// The project file must read: a project is checked as it is made.
project::load(dir)?.ok_or("the project file is missing")?;
Ok(())
}
/// The Create runtime: the flag, `$WAI_APP_RUNTIME`, or `share/wai/app` beside this binary's
/// directory (an installed archive's layout), checked for its listing.
#[cfg(feature = "app")]
pub(super) fn find_runtime(flag_value: Option<&Path>) -> Result<std::path::PathBuf, String> {
use std::path::PathBuf;
let exe_dir = std::env::current_exe().ok().and_then(|e| std::fs::canonicalize(e).ok()).and_then(|e| e.parent().map(Path::to_path_buf));
let candidates: Vec<PathBuf> = match flag_value.map(Path::to_path_buf).or_else(|| std::env::var_os("WAI_APP_RUNTIME").map(PathBuf::from)) {
Some(p) => vec![p],
None => exe_dir.map(|d| vec![d.join("../share/wai/app"), d.join("share/wai/app")]).unwrap_or_default(),
};
candidates.iter().find(|c| c.join(wai_app::runtime::LISTING).is_file()).cloned().ok_or_else(|| {
format!(
"the Create runtime is not found (looked for {} in {}): give --runtime DIR, a runtime laid out by wai/wai-app/web/runtime.mjs",
wai_app::runtime::LISTING,
if candidates.is_empty() { "nowhere".to_string() } else { candidates.iter().map(|c| c.display().to_string()).collect::<Vec<_>>().join(", ") }
)
})
}
/// The project for a program made with Create (`app.export`) in `dir`: the app's files and its
/// runtime, its project file, its licence. The files written, by path.
#[cfg(feature = "app")]
pub(super) fn export_app(program: &str, dir: &Path, runtime: Option<&Path>) -> Result<Vec<String>, String> {
use wai_app::export::Contents;
let rt_dir = find_runtime(runtime)?;
let rt = wai_app::runtime::read(&rt_dir)?;
let listed: Vec<wai_app::export::RuntimeFile> = rt.iter().map(|(f, _)| f.clone()).collect();
let e = wai_app::export::export(program, &listed).map_err(|e| e.to_string())?;
let mut files: Vec<(String, Vec<u8>)> = Vec::new();
for f in e.files {
let bytes = match f.contents {
Contents::Text(t) => t.into_bytes(),
Contents::Runtime(from) => rt.iter().find(|(r, _)| r.path == from).map(|(_, b)| b.clone()).ok_or(format!("{from}: not in the runtime"))?,
};
files.push((f.path, bytes));
}
let p = &e.project;
let mut dev: Vec<(&str, serde_json::Value)> = vec![("site", ".".into()), ("mode", p.mode.clone().into())];
if let Some(w) = &p.watch {
dev.push(("watch", w.clone().into()));
}
if let Some(a) = p.audience {
dev.push(("audience", a.into()));
}
let top: Vec<(&str, serde_json::Value)> = vec![("title", p.title.clone().into()), ("summary", p.summary.clone().into()), ("app", wai_app::export::APP.into())];
files.push((project::FILE.into(), project::render(&top, &dev).into_bytes()));
let names: Vec<String> = files.iter().map(|(p, _)| p.clone()).chain([LICENSE.0.to_string()]).collect();
write_project(dir, files)?;
Ok(names)
}
#[cfg(feature = "app")]
fn from_app(program: &Path, dir: &Path, runtime: Option<&Path>) -> Result<Vec<String>, String> {
let text = std::fs::read_to_string(program).map_err(|e| format!("{}: {e}", program.display()))?;
export_app(&text, dir, runtime)
}
#[cfg(not(feature = "app"))]
fn from_app(_: &Path, _: &Path, _: Option<&Path>) -> Result<Vec<String>, String> {
Err("`wai create --from` needs a build with the feature `app` (cargo install --path wai/wai-rs --no-default-features --features app)".into())
}
/// A sample sound for a watch room: twenty seconds of a tone that steps in pitch each
/// second, 8 kHz, 16-bit, mono, so any stretch of it is told from another by ear.
pub fn tone_wav() -> Vec<u8> {
@ -140,6 +237,10 @@ mod tests {
let p = project::load(&d).unwrap().unwrap();
assert_eq!(p.template.as_deref(), Some(t.name), "{}: its project file names it", t.name);
assert!(d.join("README.md").is_file(), "{}: a README", t.name);
// Every project states its licence: the file, and a line of its README.
assert_eq!(std::fs::read(d.join("LICENSE")).unwrap(), LICENSE.1, "{}: LICENSE", t.name);
assert!(std::str::from_utf8(LICENSE.1).unwrap().contains("Apache License"));
assert!(std::fs::read_to_string(d.join("README.md")).unwrap().contains("Apache-2.0"), "{}: its README names its licence", t.name);
let _ = std::fs::remove_dir_all(&d);
}
}

61
wai/wai-rs/src/bin/wai_cli/dx/dev.rs
View file

@ -23,7 +23,7 @@ use std::process::{Child, Command, Stdio};
use std::sync::Arc;
use std::time::{Duration, Instant};
pub const USAGE: &str = "wai dev [--port P] [--dir D] [--site DIR] [--watch FILE [--watch-ms MS]] [--call] [--remote]\n [--messaging] [--e2ee] [--publishers N] [--check [--json]] [--for S]\n a local relay, room service and room page on one origin (`wai dev --help`)";
pub const USAGE: &str = "wai dev [--port P] [--dir D] [--site DIR] [--watch FILE [--watch-ms MS]] [--call] [--remote]\n [--show [--audience N]] [--messaging] [--e2ee] [--publishers N] [--check [--json]] [--for S]\n a local relay, room service and room page on one origin (`wai dev --help`)";
const HELP: &str = "wai dev — a relay, a room service and the room page on one origin, on this machine
@ -35,6 +35,9 @@ const HELP: &str = "wai dev — a relay, a room service and the room page on one
.ogg, .mp3); --watch-ms MS its duration, read from a WAV file's header
--call the room is a call: /room/call.html places one, rings, answers
--remote the room is one host's shared screen; prints the host's invitation
--show the room is a show: hosts on camera, an audience that votes in polls
the room's clock opens and signs (co-performance); --audience N each
viewer's inputs to a poll (1)
--messaging a conversation beside the room, at /msg-api/: /room/messages.html
--e2ee every room end-to-end encrypted
--check start, check every piece through the origin, report, stop
@ -63,6 +66,8 @@ enum Mode {
Watch { file: PathBuf, ms: u64 },
Call,
Remote,
/// A room with co-performance: its clock leads polls the audience answers.
Show { audience: u8 },
}
#[derive(Debug)]
@ -94,8 +99,8 @@ pub fn main(args: &[String]) -> Result<bool, String> {
}
fn options(args: &[String]) -> Result<Opts, String> {
let known_values = ["--port", "--dir", "--site", "--publishers", "--watch", "--watch-ms", "--for", "--web", "--relay", "--room"];
let known_flags = ["--call", "--remote", "--messaging", "--e2ee", "--check", "--json"];
let known_values = ["--port", "--dir", "--site", "--publishers", "--watch", "--watch-ms", "--for", "--web", "--relay", "--room", "--audience"];
let known_flags = ["--call", "--remote", "--show", "--messaging", "--e2ee", "--check", "--json"];
let mut i = 0;
while i < args.len() {
let a = args[i].as_str();
@ -129,6 +134,8 @@ fn options(args: &[String]) -> Result<Opts, String> {
"call"
} else if flag(args, "--remote") {
"remote"
} else if flag(args, "--show") || opt(args, "--audience").is_some() {
"show"
} else if opt(args, "--watch").is_some() {
"watch"
} else {
@ -138,6 +145,13 @@ fn options(args: &[String]) -> Result<Opts, String> {
"room" => Mode::Room,
"call" => Mode::Call,
"remote" => Mode::Remote,
"show" => {
let audience = match num("--audience")? {
Some(n) => u8::try_from(n).ok().filter(|n| *n >= 1).ok_or("--audience: 1 to 255")?,
None => d.audience.unwrap_or(1),
};
Mode::Show { audience }
}
_ => {
let file = watch.ok_or("a watch room needs --watch FILE")?;
let file = std::fs::canonicalize(&file).map_err(|e| format!("--watch {}: {e}", file.display()))?;
@ -313,6 +327,12 @@ fn relay_toml(o: &Opts, ws: u16, lines: &str) -> String {
Mode::Watch { .. } => &[],
_ => &["audio", "video"],
};
// A show's relay holds the floor and the audience's inputs, honouring the room's clock
// (co-performance §6): the clock's key is in the room service's relay lines.
let floor = match (&o.mode, clock_of(lines)) {
(Mode::Show { .. }, Some(clock)) => format!("[floor]\nroom_keys = [\"{clock}\"]\n\n"),
_ => String::new(),
};
let mut t = format!(
"# Written by `wai dev` on each start: the relay on loopback, its WebSocket binding alone.\nwebsocket_listen = \"127.0.0.1:{ws}\"\npath = \"{RELAY_PATH}\"\n\n[tls]\ndev_names = [\"localhost\", \"127.0.0.1\"]\n\n[receipts]\nreceipts_track_suffix = \"_receipts\"\n\n"
);
@ -321,7 +341,12 @@ fn relay_toml(o: &Opts, ws: u16, lines: &str) -> String {
t += &format!("[[receipts.door]]\nprefix = [\"*\", \"*\", \"{k}\"]\ntrust_field = 1\nhold_ms = 2000\n\n");
}
t += &format!("[summary]\nout_path = {}\n\n[auth]\nrequired = true\n\n", q(&o.dir.join("sessions.jsonl")));
t + lines
t + &floor + lines
}
/// The room's clock key in a room service's relay lines (`# clock <hex>`).
fn clock_of(lines: &str) -> Option<&str> {
lines.lines().find_map(|l| l.strip_prefix("# clock")).map(str::trim).filter(|k| k.len() == 64 && k.bytes().all(|b| b.is_ascii_hexdigit()))
}
fn room_toml(o: &Opts, port: u16, ws: u16) -> String {
@ -339,6 +364,10 @@ fn room_toml(o: &Opts, port: u16, ws: u16) -> String {
match o.mode {
Mode::Call => t += "\n[call]\nring_ms = 30000\n",
Mode::Remote => t += "\n[remote]\nwidth = 1280\nheight = 720\nfps = 15\nallow = \"all\"\ncontrol = 3\n",
// Each viewer's inputs to one poll; the first host to join leads (opens the polls).
Mode::Show { audience } => {
t += &format!("\n[co]\naudience_input = {audience}\nlatency_budget_ms = 0\nlead = \"first\"\nai = false\nfidelity_every_s = 0\ncommit_after_ms = 1500\nlead_ms = 300\n")
}
_ => {}
}
t
@ -491,6 +520,7 @@ fn announce(o: &Opts, base: &str) {
}
Mode::Watch { file, ms } => println!(" watch room {base}/room/?service={ROOM_API} {} ({} s), in sync for everyone in it", file.display(), ms / 1000),
Mode::Room => println!(" the room {base}/room/?service={ROOM_API} open it in two tabs and press Join in each"),
Mode::Show { audience } => println!(" the show {base}/room/?service={ROOM_API} hosts join, viewers watch; each viewer has {audience} input(s) to a poll"),
}
if o.messaging {
println!(" messages {base}/room/messages.html?service={MSG_API}");
@ -538,7 +568,7 @@ fn check(o: &Opts, origin: SocketAddr) -> Vec<Check> {
let v: Value = serde_json::from_slice(&b).map_err(|_| format!("{s}: not JSON"))?;
if s == 200 && v["descriptor"].is_string() { Ok("descriptor and grants".into()) } else { Err(format!("{s}")) }
}));
if matches!(o.mode, Mode::Room | Mode::Watch { .. }) && !o.e2ee {
if matches!(o.mode, Mode::Room | Mode::Watch { .. } | Mode::Show { .. }) && !o.e2ee {
let body = json!({ "principal": CHECK_PRINCIPAL, "role": "viewer" }).to_string();
c("a viewer's grant", http(origin, "POST", &format!("{ROOM_API}/grant"), &[("content-type", "application/json")], body.as_bytes()).map_err(|e| e.to_string()).and_then(|(s, b)| {
let v: Value = serde_json::from_slice(&b).map_err(|_| format!("{s}: not JSON"))?;
@ -743,7 +773,8 @@ fn handle(mut c: TcpStream, f: &Front) -> std::io::Result<()> {
}
/// A URL path as a relative file path: percent-decoded, and refused when a segment is
/// empty past the first, `.` or `..`, or holds a backslash, a NUL or a control byte.
/// empty past the first, `.` or `..`, starts with a dot (the project's own `.wai/`, where the
/// room's keys are, is never served), or holds a backslash, a NUL or a control byte.
fn clean(path: &str) -> Option<PathBuf> {
let b = path.as_bytes();
let mut d = Vec::with_capacity(b.len());
@ -769,7 +800,7 @@ fn clean(path: &str) -> Option<PathBuf> {
}
return None;
}
if *seg == "." || *seg == ".." || seg.bytes().any(|c| c == b'\\' || c < 0x20 || c == 0x7f) {
if seg.starts_with('.') || seg.bytes().any(|c| c == b'\\' || c < 0x20 || c == 0x7f) {
return None;
}
out.push(seg);
@ -791,7 +822,7 @@ fn mime(p: &Path) -> &'static str {
"mp4" | "m4s" => "video/mp4",
"m3u8" => "application/vnd.apple.mpegurl",
"mpd" => "application/dash+xml",
"txt" | "md" => "text/plain; charset=utf-8",
"txt" | "md" | "oui" => "text/plain; charset=utf-8",
_ => "application/octet-stream",
}
}
@ -908,6 +939,10 @@ mod tests {
std::fs::create_dir_all(web.join("room")).unwrap();
std::fs::write(web.join("room/index.html"), "<script src=app.js></script>").unwrap();
std::fs::write(dir.join("secret.txt"), "never served").unwrap();
// A project served at / keeps the room's keys in .wai/: never served.
std::fs::create_dir_all(web.join(".wai/dev")).unwrap();
std::fs::write(web.join(".wai/dev/room-keys.toml"), "never served").unwrap();
std::fs::write(web.join("room/.env"), "never served").unwrap();
let f = Arc::new(Front { site: None, web, services: vec![(ROOM_API.into(), svc)], relay, home: "/room/?service=/room-api".into() });
let l = TcpListener::bind("127.0.0.1:0").unwrap();
let a = l.local_addr().unwrap();
@ -961,7 +996,7 @@ mod tests {
assert_eq!((s, String::from_utf8_lossy(&b).contains("app.js")), (200, true));
assert_eq!(get(a, "/room").unwrap().0, 301);
assert_eq!(get(a, "/").unwrap().0, 302);
for bad in ["/../secret.txt", "/room/%2e%2e/%2e%2e/secret.txt", "/room//index.html", "/room/%5c..%5csecret.txt"] {
for bad in ["/../secret.txt", "/room/%2e%2e/%2e%2e/secret.txt", "/room//index.html", "/room/%5c..%5csecret.txt", "/.wai/dev/room-keys.toml", "/room/.env", "/%2ewai/dev/room-keys.toml"] {
let (s, b) = get(a, bad).unwrap();
assert!(s == 400 || s == 404, "{bad}: {s}");
assert!(!String::from_utf8_lossy(&b).contains("never served"), "{bad}");
@ -1003,5 +1038,13 @@ mod tests {
assert!(r.contains("public_prefix = \"/room-api\"") && r.contains("relay_url = \"ws://127.0.0.1:9001/moq\"") && r.contains("max_publishers = 3"));
let t = relay_toml(&o, 9001, "[[auth.authorities]]\n");
assert!(t.contains("websocket_listen = \"127.0.0.1:9001\"") && !t.contains("quic_listen") && t.contains("required = true") && t.contains("\"video\""));
assert!(!r.contains("[co]") && !t.contains("[floor]"));
// A show: the room's co-performance, and the relay holding the floor to the room's clock.
let clock = "ab".repeat(32);
let show = Opts { mode: Mode::Show { audience: 2 }, ..o };
let r = room_toml(&show, 9000, 9001);
assert!(r.contains("\n[co]\naudience_input = 2\n") && r.contains("lead = \"first\""), "{r}");
let t = relay_toml(&show, 9001, &format!("# authority {}\n# clock {clock}\n[[auth.authorities]]\n", "cd".repeat(32)));
assert!(t.contains(&format!("[floor]\nroom_keys = [\"{clock}\"]\n")) && t.contains("\"video\""), "{t}");
}
}

9
wai/wai-rs/src/bin/wai_cli/dx/mod.rs
View file

@ -1,9 +1,12 @@
//! Every verb kept in a module of its own, one verb a module, every verb in [`VERBS`]:
//!
//! - `hello`: the four steps of WAI (name, dispatch, rebuild, receipt) on a bundled sample;
//! - `create`: a new project from one of the examples (`wai/examples/`);
//! - `create`: a new project from one of the examples (`wai/examples/`), or from an app made
//! with Create (`--from app.oui`, a build with `app`);
//! - `dev`: a relay, a room service and the room page on one origin, on this machine;
//! - `verify`: the receipts under a directory, checked offline;
//! - `app` (`app.guide`, `app.check`; a build with `app`): an app made with Create, its guide
//! and its check;
//! - `mcp` (`agent.mcp`): WAI's operations as tools for a coding agent, a Model Context
//! Protocol server on standard input and output, in toolsets (`../mcp/`);
//! - `setup-agent` (`agent.setup`): a project made ready for coding agents (`../setup_agent.rs`).
@ -13,6 +16,7 @@
pub mod project;
mod app;
mod create;
mod dev;
mod hello;
@ -50,11 +54,12 @@ pub const VERBS: &[Verb] = &[
Verb { name: "create", usage: create::USAGE, run: Run::Checks(create::main) },
Verb { name: "dev", usage: dev::USAGE, run: Run::Checks(dev::main) },
Verb { name: "verify", usage: verify::USAGE, run: Run::Checks(verify::main) },
Verb { name: "app", usage: app::USAGE, run: Run::Checks(app::main) },
Verb { name: MCP, usage: MCP_USAGE, run: Run::Checks(mcp_main) },
Verb { name: "setup-agent", usage: setup_agent::USAGE, run: Run::Status(setup_agent::main) },
];
const MCP_USAGE: &str = "wai mcp [--toolset core,dev] [--read-only] [--project DIR] [--spec-root DIR] [--room-bin BIN]\n WAI's operations as tools for a coding agent: a Model Context Protocol server on standard\n input and output (a build with `mcp`)";
const MCP_USAGE: &str = "wai mcp [--toolset core,dev,app] [--read-only] [--project DIR] [--spec-root DIR] [--room-bin BIN]\n WAI's operations as tools for a coding agent: a Model Context Protocol server on standard\n input and output (a build with `mcp`)";
#[cfg(feature = "mcp")]
fn mcp_main(args: &[String]) -> Result<bool, String> {

76
wai/wai-rs/src/bin/wai_cli/dx/project.rs
View file

@ -1,11 +1,14 @@
//! The project file, `wai.json`: what a template wrote, and what `wai dev` runs in the
//! project's directory. This module is the one place it is named, read and written.
//! The project file, `wai.json`: what a template wrote, or what `wai create --from app.oui`
//! wrote for an app made with Create, and what `wai dev` runs in the project's directory. This
//! module is the one place it is named, read and written ([`render`]).
//!
//! ```json
//! { "template": "video-room",
//! "title": "A video room in one page",
//! "app": "app.oui",
//! "dev": { "site": ".", "page": "/", "mode": "room", "publishers": 4,
//! "watch": "tone.wav", "watch_ms": 20000, "messaging": false, "e2ee": false } }
//! "watch": "tone.wav", "watch_ms": 20000, "messaging": false, "e2ee": false,
//! "audience": 2 } }
//! ```
//!
//! Every member is optional. `dev` gives `wai dev` its defaults; a flag on the command line
@ -26,7 +29,8 @@ pub struct Dev {
/// The page `wai dev` names first when it is ready (a path on the front, `/` by default
/// with a site, else the mode's page).
pub page: Option<String>,
/// `room` (the default), `watch`, `call` or `remote`.
/// `room` (the default), `watch`, `call`, `remote` or `show` (a room with co-performance:
/// hosts on camera and an audience that votes).
pub mode: Option<String>,
pub publishers: Option<u32>,
/// A watch room's source, and its duration where it is not a WAV file.
@ -35,6 +39,8 @@ pub struct Dev {
/// A conversation beside the room.
pub messaging: Option<bool>,
pub e2ee: Option<bool>,
/// A show's inputs per viewer per poll, 1 to 255.
pub audience: Option<u8>,
}
/// A project file as read.
@ -45,6 +51,8 @@ pub struct Project {
pub template: Option<String>,
pub title: Option<String>,
pub summary: Option<String>,
/// A program made with Create (`app.oui`), which the project's page renders.
pub app: Option<PathBuf>,
pub dev: Dev,
}
@ -63,7 +71,7 @@ pub fn load(dir: &Path) -> Result<Option<Project>, String> {
pub fn parse(root: &Path, text: &str) -> Result<Project, String> {
let v: Value = serde_json::from_str(text).map_err(|e| format!("not JSON: {e}"))?;
let top = v.as_object().ok_or("not a JSON object")?;
only(top, &["template", "title", "summary", "dev"], "")?;
only(top, &["template", "title", "summary", "app", "dev"], "")?;
let s = |m: &Map<String, Value>, k: &str| -> Result<Option<String>, String> {
match m.get(k) {
None => Ok(None),
@ -71,10 +79,17 @@ pub fn parse(root: &Path, text: &str) -> Result<Project, String> {
Some(_) => Err(format!("{k}: a string")),
}
};
let mut p = Project { root: root.to_path_buf(), template: s(top, "template")?, title: s(top, "title")?, summary: s(top, "summary")?, dev: Dev::default() };
let mut p = Project {
root: root.to_path_buf(),
template: s(top, "template")?,
title: s(top, "title")?,
summary: s(top, "summary")?,
app: s(top, "app")?.map(|a| root.join(a)),
dev: Dev::default(),
};
if let Some(d) = top.get("dev") {
let d = d.as_object().ok_or("dev: an object")?;
only(d, &["site", "page", "mode", "publishers", "watch", "watch_ms", "messaging", "e2ee"], "dev.")?;
only(d, &["site", "page", "mode", "publishers", "watch", "watch_ms", "messaging", "e2ee", "audience"], "dev.")?;
let path = |k: &str| -> Result<Option<PathBuf>, String> { Ok(s(d, k).map_err(|e| format!("dev.{e}"))?.map(|x| root.join(x))) };
let num = |k: &str| -> Result<Option<u64>, String> {
match d.get(k) {
@ -97,20 +112,39 @@ pub fn parse(root: &Path, text: &str) -> Result<Project, String> {
watch_ms: num("watch_ms")?,
messaging: boolean("messaging")?,
e2ee: boolean("e2ee")?,
audience: num("audience")?.map(|n| u8::try_from(n).ok().filter(|n| *n >= 1).ok_or_else(|| "dev.audience: 1 to 255".to_string())).transpose()?,
};
if let Some(m) = &p.dev.mode
&& !["room", "watch", "call", "remote"].contains(&m.as_str())
&& !MODES.contains(&m.as_str())
{
return Err(format!("dev.mode: room, watch, call or remote, not {m}"));
return Err(format!("dev.mode: room, watch, call, remote or show, not {m}"));
}
if p.dev.audience.is_some() && p.dev.mode.as_deref() != Some("show") {
return Err("dev.audience: a show's alone (dev.mode \"show\")".into());
}
}
Ok(p)
}
/// The modes `wai dev` runs.
pub const MODES: &[&str] = &["room", "watch", "call", "remote", "show"];
/// The text of a project file naming `template`, for `wai create` to write when the
/// template carries none of its own.
pub fn minimal(template: &str) -> String {
format!("{{\n \"template\": {}\n}}\n", Value::from(template))
render(&[("template", Value::from(template))], &[])
}
/// A project file's text, in the layout every template's own has: its members in the order
/// given, one a line, and the `dev` block (when it has members) on one line, last. Every
/// project file `wai create` writes is this function's.
pub fn render(top: &[(&str, Value)], dev: &[(&str, Value)]) -> String {
let mut lines: Vec<String> = top.iter().map(|(k, v)| format!(" {}: {v}", Value::from(*k))).collect();
if !dev.is_empty() {
let inner: Vec<String> = dev.iter().map(|(k, v)| format!("{}: {v}", Value::from(*k))).collect();
lines.push(format!(" \"dev\": {{ {} }}", inner.join(", ")));
}
format!("{{\n{}\n}}\n", lines.join(",\n"))
}
fn only(m: &Map<String, Value>, known: &[&str], at: &str) -> Result<(), String> {
@ -146,4 +180,26 @@ mod tests {
fn the_minimal_file_reads_back() {
assert_eq!(parse(Path::new("/p"), &minimal("video-room")).unwrap().template.as_deref(), Some("video-room"));
}
/// The writer lays a file out as the templates' own files are laid out, byte for byte.
#[test]
fn the_writer_writes_a_templates_file_as_it_is() {
let ex = |t: &str| std::fs::read_to_string(Path::new(env!("CARGO_MANIFEST_DIR")).join("../examples").join(t).join(FILE)).unwrap();
for t in ["watch-together", "messaging", "android-app"] {
let p = parse(Path::new("/p"), &ex(t)).unwrap();
let v: Value = serde_json::from_str(&ex(t)).unwrap();
let top: Vec<(&str, Value)> = ["template", "title", "summary"].iter().filter_map(|k| v.get(*k).map(|x| (*k, x.clone()))).collect();
let dev: Vec<(&str, Value)> = v.get("dev").and_then(Value::as_object).map(|d| d.iter().map(|(k, x)| (k.as_str(), x.clone())).collect()).unwrap_or_default();
assert_eq!(render(&top, &dev), ex(t), "{t}");
assert_eq!(p.template.as_deref(), Some(t));
}
}
#[test]
fn an_apps_file_names_its_program_and_a_show_its_audience() {
let p = parse(Path::new("/p"), &render(&[("title", "x".into()), ("app", "app.oui".into())], &[("site", ".".into()), ("mode", "show".into()), ("audience", 2.into())])).unwrap();
assert_eq!((p.app.as_deref(), p.dev.mode.as_deref(), p.dev.audience), (Some(Path::new("/p/app.oui")), Some("show"), Some(2)));
assert!(parse(Path::new("/p"), r#"{"dev":{"mode":"room","audience":2}}"#).unwrap_err().contains("show"));
assert!(parse(Path::new("/p"), r#"{"dev":{"mode":"show","audience":0}}"#).unwrap_err().contains("1 to 255"));
}
}

64
wai/wai-rs/src/bin/wai_cli/mcp/app.rs Normal file
View file

@ -0,0 +1,64 @@
//! The `app` toolset: Create's operations (`app.guide`, `app.check`, `app.stream`,
//! `app.export`) through `wai-app`. A program streams into the person's open Create page
//! through the bridge this server runs on 127.0.0.1 (`wai_app::bridge`), which a page reads only
//! with the pairing code the bridge makes at start; an export is `wai create --from app.oui`'s,
//! inside the project directory.
use super::tools::str_arg;
use super::Server;
use serde_json::{json, Value};
use std::path::Path;
fn bridge_facts(server: &Server) -> Value {
match &server.bridge {
Some(b) => json!({"bridge": format!("http://127.0.0.1:{}", b.port), "code": b.code, "pages_connected": b.listening()}),
None => json!({"bridge": null, "note": "this server runs no bridge (it was started read-only)"}),
}
}
pub(crate) fn guide(server: &mut Server, _: &Value) -> Result<Value, String> {
let mut v = json!({"guide": wai_app::guide::guide()});
if let (Some(o), Value::Object(b)) = (v.as_object_mut(), bridge_facts(server)) {
o.extend(b);
}
if let Some(b) = &server.bridge {
v["connect"] = json!(format!(
"In Create, choose \"Your coding agent\" and enter the bridge http://127.0.0.1:{} and the code {}.",
b.port, b.code
));
}
Ok(v)
}
pub(crate) fn check(_: &mut Server, args: &Value) -> Result<Value, String> {
let c = wai_app::check(str_arg(args, "program")?);
let bad: Vec<String> = c.errors.iter().filter(|e| !e.warning).map(|e| format!("line {}: {}: {}", e.line, e.code, e.message)).collect();
let mut v = serde_json::to_value(&c).map_err(|e| e.to_string())?;
v["ok"] = json!(bad.is_empty() && c.room.is_some());
v["problems"] = json!(bad);
Ok(v)
}
pub(crate) fn stream(server: &mut Server, args: &Value) -> Result<Value, String> {
let program = str_arg(args, "program")?;
let b = server.bridge.as_ref().ok_or("this server runs no bridge (it was started read-only)")?;
if b.listening() == 0 {
return Err(format!(
"No Create page is connected. Ask the person to open Create, choose \"Your coding agent\", and enter the bridge http://127.0.0.1:{} and the code {}.",
b.port, b.code
));
}
let chunk = args.get("chunk").and_then(Value::as_u64).unwrap_or(24).clamp(1, 4096) as usize;
let pace = args.get("pace_ms").and_then(Value::as_u64).unwrap_or(30).min(2000);
let (chunks, delivered) = b.stream(program, chunk, pace);
let c = wai_app::check(program);
Ok(json!({"delivered": delivered, "chunks": chunks, "errors": c.errors}))
}
pub(crate) fn export(server: &mut Server, args: &Value) -> Result<Value, String> {
let program = str_arg(args, "program")?;
let dir = server.inside_project(Path::new(str_arg(args, "dir")?))?;
let runtime = args.get("runtime_dir").and_then(Value::as_str).map(Path::new);
let files = crate::dx::create::export_app(program, &dir, runtime)?;
Ok(json!({"dir": dir.display().to_string(), "files": files, "next": "wai app check, then wai dev, in that directory"}))
}

37
wai/wai-rs/src/bin/wai_cli/mcp/mod.rs
View file

@ -13,7 +13,9 @@
//! conformance corpora, search the specification's text;
//! - `dev`: a development room on loopback (`dev_room.rs`): start and stop it, mint grants for
//! test principals, join and publish as them, and read the session, each on the target
//! `local` alone.
//! `local` alone;
//! - `app` (a build with `app`; `app.rs`): Create's guide, a program checked, streamed into the
//! person's open Create page through a bridge on 127.0.0.1, and exported as a project.
//!
//! Each answers with a structured result. Nothing leaves the machine, and the only tool that
//! takes a service's address refuses every one that is not on this machine.
@ -36,6 +38,9 @@ use serde_json::{json, Value};
use std::io::{BufRead, Write};
use std::path::PathBuf;
#[cfg(feature = "app")]
#[path = "app.rs"]
mod app;
#[path = "dev_room.rs"]
mod dev_room;
#[path = "tools.rs"]
@ -58,6 +63,11 @@ corpora through their independent verifiers; and search the specification's text
s.push_str(" The development tools, each marked DEVELOPMENT ONLY, start a relay and a room service on loopback with keys \
held in memory, mint grants for test principals, join and publish as them, and read the room's session for verification; they act \
on the target local alone and refuse any service that is not on this machine.");
}
if toolsets.contains(&tools::Toolset::App) {
s.push_str(" Create's tools give the line language and the WAI widgets a WAI app is written in (read app_guide first), \
check a program, stream it into the person's open Create page through a bridge on 127.0.0.1 that the page reads with a pairing \
code, and export it as a project.");
}
s.push_str(" No tool sends anything off the machine.");
s
@ -75,6 +85,10 @@ pub(crate) struct Server {
pub(crate) dev: Option<dev_room::Dev>,
pub(crate) principals: std::collections::BTreeMap<String, dev_room::Principal>,
pub(crate) client: Option<String>,
/// The bridge Create pages read programs from, while the `app` toolset is offered and the
/// server is not read-only.
#[cfg(feature = "app")]
pub(crate) bridge: Option<wai_app::bridge::Bridge>,
}
impl Server {
@ -90,7 +104,19 @@ impl Server {
},
};
let room_bin = room_bin.or_else(|| std::env::var_os("WAI_ROOM_BIN").map(PathBuf::from));
Server { toolsets, read_only, project, spec_root, spec_root_from, room_bin, dev: None, principals: Default::default(), client: None }
Server {
toolsets,
read_only,
project,
spec_root,
spec_root_from,
room_bin,
dev: None,
principals: Default::default(),
client: None,
#[cfg(feature = "app")]
bridge: None,
}
}
/// `p` (relative to the project directory) as a path to write, when it is inside the
@ -218,7 +244,14 @@ pub fn main(args: &[String]) -> Result<bool, String> {
toolsets.push(tools::Toolset::Core);
}
let project = project.map_or_else(std::env::current_dir, Ok).and_then(|p| p.canonicalize()).map_err(|e| format!("the project directory: {e}"))?;
#[cfg_attr(not(feature = "app"), allow(unused_mut))]
let mut server = Server::new(toolsets, spec_root, room_bin, project, read_only);
#[cfg(feature = "app")]
if server.toolsets.contains(&tools::Toolset::App) && !server.read_only {
let b = wai_app::bridge::Bridge::start(0).map_err(|e| format!("the bridge for Create pages: {e}"))?;
eprintln!("wai mcp: the bridge for Create pages is http://127.0.0.1:{}, pairing code {}", b.port, b.code);
server.bridge = Some(b);
}
let stdin = std::io::stdin();
let mut out = std::io::stdout().lock();
for line in stdin.lock().lines() {

60
wai/wai-rs/src/bin/wai_cli/mcp/tools.rs
View file

@ -3,6 +3,8 @@
//! hints and the function that runs it. A tool's name is its operation's, with `_` for the dot
//! (`wai::ops::mcp_tool_name`); nothing else names it.
#[cfg(feature = "app")]
use super::app;
use super::dev_room;
use super::Server;
use serde_json::{json, Map, Value};
@ -16,15 +18,26 @@ pub(crate) enum Toolset {
Core,
/// The development room on loopback, on the target `local` alone.
Dev,
/// Create's: the guide, a program checked, streamed into the person's open Create page, and
/// exported as a project (a build with `app`).
#[cfg_attr(not(feature = "app"), allow(dead_code))]
App,
}
impl Toolset {
pub(crate) const ALL: &'static [Toolset] = &[Toolset::Core, Toolset::Dev];
/// The toolsets this build serves.
pub(crate) const ALL: &'static [Toolset] = &[
Toolset::Core,
Toolset::Dev,
#[cfg(feature = "app")]
Toolset::App,
];
pub(crate) fn name(self) -> &'static str {
match self {
Toolset::Core => "core",
Toolset::Dev => "dev",
Toolset::App => "app",
}
}
@ -44,6 +57,9 @@ pub(crate) enum Effect {
Writes,
/// Starts or drives the development room on loopback.
DevRoom,
/// Sends to the person's open Create page through the bridge on loopback.
#[cfg_attr(not(feature = "app"), allow(dead_code))]
Bridge,
}
pub(crate) struct Tool {
@ -94,6 +110,18 @@ pub(crate) const TOOLS: &[Tool] = &[
description: "DEVELOPMENT ONLY. On the target local: close a test principal's session in the development room and wait for the clock's leave." },
Tool { op: "room.session", toolset: Toolset::Dev, title: "Read the development room's session", effect: Effect::DevRoom, schema: s_target, run: dev_room::session,
description: "DEVELOPMENT ONLY. On the target local: the development room as a verifier is given it (descriptor, grants, key-log head and the clock's event chain), ready for receipt_verify." },
#[cfg(feature = "app")]
Tool { op: "app.guide", toolset: Toolset::App, title: "The guide to writing a WAI app", effect: Effect::Reads, schema: s_none, run: app::guide,
description: "The line language and every WAI widget, as a model writing a WAI app is told them, and the bridge and pairing code the person's Create page connects to. Read it before writing a program." },
#[cfg(feature = "app")]
Tool { op: "app.check", toolset: Toolset::App, title: "Check a WAI app program", effect: Effect::Reads, schema: s_program, run: app::check,
description: "Check a WAI app program against the WAI registry and WAI's rules: the widget tree it renders, every error with its line and reason code, and the room it needs. Fix each error at its line and check again." },
#[cfg(feature = "app")]
Tool { op: "app.stream", toolset: Toolset::App, title: "Stream a program into the person's Create page", effect: Effect::Bridge, schema: s_app_stream, run: app::stream,
description: "Stream a program into the person's open Create page, connected to this server's bridge on 127.0.0.1, which renders the app as it arrives and lets them try its room." },
#[cfg(feature = "app")]
Tool { op: "app.export", toolset: Toolset::App, title: "Export an app as a project", effect: Effect::Writes, schema: s_app_export, run: app::export,
description: "Write the app's starter project into a new or empty directory inside the project directory, as `wai create <dir> --from app.oui` does: the program, wai.json naming its room, the page and its runtime, a README, an AGENTS.md and the licence." },
];
/// The tools a server offers: those of its toolsets, or with `--read-only` those of them that
@ -266,6 +294,36 @@ fn s_room_join() -> Value {
fn s_room_principal() -> Value {
obj(json!({"target": target_prop(), "principal": {"type": "string", "description": "The test principal's name."}}), &["principal"])
}
#[cfg(feature = "app")]
fn program_prop() -> Value {
json!({"type": "string", "description": "The app as a program in joule-ui's line language: one widget a line, `root = App([...], \"Title\")` first (see app_guide)."})
}
#[cfg(feature = "app")]
fn s_program() -> Value {
obj(json!({"program": program_prop()}), &["program"])
}
#[cfg(feature = "app")]
fn s_app_stream() -> Value {
obj(
json!({
"program": program_prop(),
"chunk": {"type": "integer", "minimum": 1, "maximum": 4096, "description": "Characters a chunk (24 by default)."},
"pace_ms": {"type": "integer", "minimum": 0, "maximum": 2000, "description": "Milliseconds between chunks (30 by default)."}
}),
&["program"],
)
}
#[cfg(feature = "app")]
fn s_app_export() -> Value {
obj(
json!({
"program": program_prop(),
"dir": {"type": "string", "description": "The new project's directory, inside the project directory: absent or empty."},
"runtime_dir": {"type": "string", "description": "A Create runtime (laid out by wai/wai-app/web/runtime.mjs); default $WAI_APP_RUNTIME or share/wai/app beside wai."}
}),
&["program", "dir"],
)
}
// ------------------------------------------------------------------------------- helpers

14
wai/wai-rs/src/bin/wai_cli/setup_agent.rs
View file

@ -34,7 +34,7 @@ use std::io::{BufRead, IsTerminal, Write};
use std::path::{Path, PathBuf};
/// The command's usage, for `wai`'s own.
pub const USAGE: &str = "wai setup-agent [--dir D] [--dry-run] [--confirm] [--json] [--all] [--only FILE,...] [--toolset core,dev]\n write AGENTS.md (and an import of it in a CLAUDE.md already there) and register `wai mcp`\n in the project's agent configuration";
pub const USAGE: &str = "wai setup-agent [--dir D] [--dry-run] [--confirm] [--json] [--all] [--only FILE,...] [--toolset core,dev,app]\n write AGENTS.md (and an import of it in a CLAUDE.md already there) and register `wai mcp`\n in the project's agent configuration";
/// The operation, as the catalogue names it.
const OP: &str = "agent.setup";
@ -107,8 +107,13 @@ fn usage() -> &'static str {
"usage: wai setup-agent [--dir <project>] [--dry-run] [--confirm] [--json] [--all] [--only <file>[,<file>...]] [--toolset <set>[,<set>...]] [--command <program>]"
}
/// The toolsets `wai mcp` serves, for `--toolset`.
const TOOLSETS: &[&str] = &["core", "dev"];
/// The toolsets this build's `wai mcp` serves, for `--toolset`.
const TOOLSETS: &[&str] = &[
"core",
"dev",
#[cfg(feature = "app")]
"app",
];
fn parse(args: &[String]) -> Result<Opts, String> {
let mut o = Opts { dir: PathBuf::from("."), dry_run: false, confirm: false, json: false, all: false, only: None, toolsets: None, command: "wai".into() };
@ -174,7 +179,7 @@ receipts, claims and live-room records are signed so that a verifier can check t
tools, each named by its operation (`receipt.verify` is the tool `receipt_verify`). The `core` toolset inspects envelopes \
and packages, reads the capability registry, verifies receipts, claims, live-room sessions and audit bundles, packages \
media, lists and runs conformance corpora, and searches the specification. The `dev` toolset (`{command} {mcp} --toolset core,dev`) \
runs a development room on loopback.\n\
runs a development room on loopback{app}.\n\
- Verify with the tools. Do not judge a signature, a receipt or a session yourself: quote the verifier's verdict and its reason code.\n\
- The specification's text is the authority. Search it, and cite a rule by its section (`SPEC.md` §N, or the extension and its section).\n\
- The development tools are for development only. They run on 127.0.0.1 with keys held in memory and refuse any other \
@ -184,6 +189,7 @@ service; never point a deployment at them.\n\
`{command} verify`, `{command} dev`, `{command} registry export --check`.\n\
{END}\n",
mcp = super::MCP,
app = if TOOLSETS.contains(&"app") { ", and the `app` toolset writes, checks and exports apps made with Create" } else { "" },
)
}

330
wai/wai-rs/tests/cli_app.rs Normal file
View file

@ -0,0 +1,330 @@
//! An app made with Create, through `wai` built with the feature `app`:
//!
//! - `wai create <dir> --from app.oui` writes an ordinary WAI project: the app's files and its
//! runtime under `vendor/`, the project file laid out as every template's is and naming the
//! program and the room `wai dev` runs for it, and the licence; a template's project and an
//! exported one share their shape;
//! - `wai app check` passes on it, and refuses a project whose room is not its app's, whose
//! runtime changed, whose program has a refused line, or that names no room;
//! - `wai create --from` refuses a program with errors, one with no room, and a directory with
//! anything in it;
//! - `wai mcp --toolset app`: the tools are Create's operations (`app_guide`, `app_check`,
//! `app_stream`, `app_export`); a program streams through the bridge to a page connected with
//! the pairing code and to no page without it; an export is written inside the project
//! directory alone; `--read-only` runs no bridge and offers no tool that streams or writes.
#![cfg(feature = "app")]
use serde_json::{json, Value};
use sha2::{Digest, Sha256};
use std::io::{BufRead, BufReader, Read, Write};
use std::path::{Path, PathBuf};
use std::process::{Child, ChildStdin, ChildStdout, Command, Output, Stdio};
const SHOW: &str = "root = App([show], \"Friday night\")\nshow = Show([player, chat, poll, people], \"Live from the studio\", 2)\nplayer = Player()\nchat = Chat(\"Talk\")\npoll = Poll(\"Which song next?\", [\"The first one\", \"The quiet one\"])\npeople = Participants()\n";
fn scratch(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("wai-cli-app-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d.canonicalize().unwrap()
}
fn hex(b: &[u8]) -> String {
b.iter().map(|x| format!("{x:02x}")).collect()
}
/// A Create runtime as `web/runtime.mjs` lays one out: a few files and their listing.
fn runtime(d: &Path) -> PathBuf {
let rt = d.join("runtime");
let files = [("create/wai-app-app.js", "export {};\n"), ("create/pkg/wai_app_bg.wasm", "\0asm"), ("room/wai-app-room-client.js", "// the room client\n")];
let mut listed = Vec::new();
for (p, text) in files {
let f = rt.join(p);
std::fs::create_dir_all(f.parent().unwrap()).unwrap();
std::fs::write(&f, text).unwrap();
listed.push(json!({"path": p, "sha256": hex(&Sha256::digest(text.as_bytes()))}));
}
std::fs::write(rt.join("create/wai-app-runtime.json"), json!({"files": listed}).to_string()).unwrap();
rt
}
fn wai(dir: &Path, args: &[&str]) -> Output {
Command::new(env!("CARGO_BIN_EXE_wai")).current_dir(dir).args(args).output().expect("run wai")
}
fn text(o: &Output) -> String {
format!("{}{}", String::from_utf8_lossy(&o.stdout), String::from_utf8_lossy(&o.stderr))
}
/// `wai app check` in `dir`: its exit status and its JSON.
fn app_check(dir: &Path) -> (i32, Value) {
let o = wai(dir, &["app", "check", "--json"]);
let v: Value = serde_json::from_slice(&o.stdout).unwrap_or_else(|e| panic!("{e}: {}", text(&o)));
assert_eq!((v["wai"].as_str(), v["op"].as_str()), (Some("envelope/1"), Some("app.check")), "{v}");
(o.status.code().unwrap(), v)
}
fn files_of(dir: &Path) -> Vec<String> {
fn walk(root: &Path, p: &Path, out: &mut Vec<String>) {
for e in std::fs::read_dir(p).unwrap().flatten() {
let q = e.path();
if q.is_dir() {
walk(root, &q, out);
} else {
out.push(q.strip_prefix(root).unwrap().to_string_lossy().replace('\\', "/"));
}
}
}
let mut out = Vec::new();
walk(dir, dir, &mut out);
out.sort();
out
}
#[test]
fn an_export_is_an_ordinary_project_and_its_check_holds() {
let d = scratch("export");
let rt = runtime(&d);
std::fs::write(d.join("app.oui"), SHOW).unwrap();
let o = wai(&d, &["create", "show", "--from", "app.oui", "--runtime", rt.to_str().unwrap()]);
assert!(o.status.success(), "{}", text(&o));
let p = d.join("show");
let mut want: Vec<String> = ["app.oui", "index.html", "README.md", "AGENTS.md", ".gitignore", "vendor.json", "wai.json", "LICENSE"].iter().map(|s| s.to_string()).collect();
want.extend(["vendor/create/wai-app-app.js", "vendor/create/pkg/wai_app_bg.wasm", "vendor/room/wai-app-room-client.js"].iter().map(|s| s.to_string()));
want.sort();
assert_eq!(files_of(&p), want);
// The project file: the template's layout, naming the program and the show's room.
let wj = std::fs::read_to_string(p.join("wai.json")).unwrap();
assert_eq!(
wj,
"{\n \"title\": \"Friday night\",\n \"summary\": \"A WAI app made with Create: a show, with hosts on camera and an audience that watches and votes.\",\n \"app\": \"app.oui\",\n \"dev\": { \"site\": \".\", \"mode\": \"show\", \"audience\": 2 }\n}\n"
);
// A template's project has the same shape: its project file read by the same reader and
// laid out the same way, a README and the licence.
let o = wai(&d, &["create", "room", "--template", "video-room"]);
assert!(o.status.success(), "{}", text(&o));
let t = std::fs::read_to_string(d.join("room/wai.json")).unwrap();
for f in [&wj, &t] {
let lines: Vec<&str> = f.lines().collect();
assert!(lines.first() == Some(&"{") && lines.last() == Some(&"}") && lines[1..lines.len() - 1].iter().all(|l| l.starts_with(" \"")), "{f}");
}
assert_eq!(std::fs::read(d.join("room/LICENSE")).unwrap(), std::fs::read(p.join("LICENSE")).unwrap());
for r in [p.join("README.md"), d.join("room/README.md")] {
assert!(std::fs::read_to_string(&r).unwrap().contains("Apache-2.0"), "{}", r.display());
}
// The program in the project is the canonical form of the one given.
assert!(std::fs::read_to_string(p.join("app.oui")).unwrap().starts_with("root = App("));
let (code, v) = app_check(&p);
assert_eq!(code, 0, "{v}");
assert_eq!(v["result"]["room"]["kind"], "show");
assert!(v["result"]["failures"].as_array().unwrap().is_empty(), "{v}");
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn the_check_refuses_a_project_that_is_not_whole() {
let d = scratch("refused");
let rt = runtime(&d);
std::fs::write(d.join("app.oui"), SHOW).unwrap();
assert!(wai(&d, &["create", "good", "--from", "app.oui", "--runtime", rt.to_str().unwrap()]).status.success());
let good = d.join("good");
let cases: [Case; 5] = [
("whose room is not its app's", |p| edit(&p.join("wai.json"), "\"mode\": \"show\", \"audience\": 2", "\"mode\": \"room\""), "dev.mode is \"room\""),
("whose show has another audience", |p| edit(&p.join("wai.json"), "\"audience\": 2", "\"audience\": 7"), "dev.audience is 7"),
("that names no room", |p| edit(&p.join("wai.json"), ", \"mode\": \"show\", \"audience\": 2", ""), "names no room"),
("with a runtime file changed", |p| std::fs::write(p.join("vendor/create/wai-app-app.js"), "export {}; // changed\n").unwrap(), "vendor/create/wai-app-app.js: its SHA-256"),
("with a line the registry refuses", |p| edit(&p.join("app.oui"), "Participants()", "Hologram()"), "unknown-component"),
];
for (what, mutate, expect) in cases {
let b = d.join("broken");
let _ = std::fs::remove_dir_all(&b);
let o = Command::new("cp").args(["-R"]).arg(&good).arg(&b).output().unwrap();
assert!(o.status.success());
mutate(&b);
let (code, v) = app_check(&b);
let failures: String = v["result"]["failures"].as_array().unwrap().iter().map(|f| f.as_str().unwrap()).collect::<Vec<_>>().join("\n");
assert!(code == 1 && failures.contains(expect), "a project {what}: {code} {v}");
}
let _ = std::fs::remove_dir_all(&d);
}
/// A project broken one way: what is wrong with it, how to break it, and what the check says.
type Case = (&'static str, fn(&Path), &'static str);
fn edit(p: &Path, from: &str, to: &str) {
let t = std::fs::read_to_string(p).unwrap();
assert_eq!(t.matches(from).count(), 1, "{from} in {}: {t}", p.display());
std::fs::write(p, t.replace(from, to)).unwrap();
}
#[test]
fn a_watch_room_names_its_file_and_its_check_wants_it() {
let d = scratch("watch");
let rt = runtime(&d);
std::fs::write(d.join("w.oui"), "root = App([w], \"Film night\")\nw = WatchTogether(\"film.wav\", [Participants()])\n").unwrap();
assert!(wai(&d, &["create", "w", "--from", "w.oui", "--runtime", rt.to_str().unwrap()]).status.success());
let p = d.join("w");
assert!(std::fs::read_to_string(p.join("wai.json")).unwrap().contains("\"dev\": { \"site\": \".\", \"mode\": \"watch\", \"watch\": \"media/film.wav\" }"));
assert!(p.join("media/README.md").is_file());
let (code, v) = app_check(&p);
assert!(code == 1 && v["result"]["failures"].to_string().contains("media/film.wav is not there"), "{v}");
std::fs::write(p.join("media/film.wav"), b"RIFF").unwrap();
let (code, v) = app_check(&p);
assert_eq!(code, 0, "{v}");
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn what_is_not_exported() {
let d = scratch("not");
let rt = runtime(&d);
let rt = rt.to_str().unwrap();
std::fs::write(d.join("bad.oui"), "root = App([r])\nr = Room([Poll(\"q\", [\"a\", \"b\"])])\n").unwrap();
std::fs::write(d.join("noroom.oui"), "root = App([t])\nt = Text(\"no room\")\n").unwrap();
std::fs::write(d.join("app.oui"), SHOW).unwrap();
for (prog, why) in [("bad.oui", "error"), ("noroom.oui", "no room")] {
let o = wai(&d, &["create", "x", "--from", prog, "--runtime", rt]);
assert!(o.status.code() == Some(2) && text(&o).contains(why), "{prog}: {}", text(&o));
assert!(!d.join("x").exists());
}
std::fs::create_dir(d.join("full")).unwrap();
std::fs::write(d.join("full/mine.txt"), "mine").unwrap();
let o = wai(&d, &["create", "full", "--from", "app.oui", "--runtime", rt]);
assert!(o.status.code() == Some(2) && text(&o).contains("not empty"), "{}", text(&o));
let o = wai(&d, &["create", "y", "--from", "app.oui", "--runtime", d.join("nowhere").to_str().unwrap()]);
assert!(o.status.code() == Some(2) && text(&o).contains("runtime"), "{}", text(&o));
let _ = std::fs::remove_dir_all(&d);
}
// ------------------------------------------------------------------------------- the toolset
struct Client {
child: Child,
stdin: ChildStdin,
out: BufReader<ChildStdout>,
next: i64,
bridge: Option<(u16, String)>,
}
impl Client {
fn start(project: &Path, extra: &[&str]) -> Client {
let mut child = Command::new(env!("CARGO_BIN_EXE_wai"))
.args(["mcp", "--project"])
.arg(project)
.args(extra)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.unwrap();
let stdin = child.stdin.take().unwrap();
let out = BufReader::new(child.stdout.take().unwrap());
let mut c = Client { child, stdin, out, next: 1, bridge: None };
let init = c.request("initialize", json!({"protocolVersion": "2025-06-18", "capabilities": {}, "clientInfo": {"name": "cli_app test", "version": "1"}}));
assert_eq!(init["result"]["serverInfo"]["name"], "wai", "{init}");
c
}
/// The bridge's address and pairing code, as the server says them at start.
fn read_bridge(&mut self) {
let mut err = BufReader::new(self.child.stderr.take().unwrap());
let mut first = String::new();
err.read_line(&mut first).unwrap();
let port: u16 = first.split("http://127.0.0.1:").nth(1).and_then(|s| s.split(',').next()).and_then(|s| s.parse().ok()).expect(&first);
let code = first.trim().rsplit(' ').next().unwrap().to_string();
self.bridge = Some((port, code));
}
fn request(&mut self, method: &str, params: Value) -> Value {
let id = self.next;
self.next += 1;
writeln!(self.stdin, "{}", json!({"jsonrpc": "2.0", "id": id, "method": method, "params": params})).unwrap();
self.stdin.flush().unwrap();
let mut l = String::new();
assert!(self.out.read_line(&mut l).unwrap() > 0, "the server closed its output");
serde_json::from_str(&l).unwrap()
}
fn call(&mut self, name: &str, args: Value) -> Result<Value, String> {
let v = self.request("tools/call", json!({"name": name, "arguments": args}));
let r = &v["result"];
assert!(r.is_object(), "{name}: {v}");
if r["isError"] == json!(true) {
return Err(r["content"][0]["text"].as_str().unwrap_or("").to_string());
}
Ok(r["structuredContent"].clone())
}
fn names(&mut self) -> Vec<String> {
let l = self.request("tools/list", json!({}));
l["result"]["tools"].as_array().unwrap().iter().map(|t| t["name"].as_str().unwrap().to_string()).collect()
}
}
impl Drop for Client {
fn drop(&mut self) {
let _ = self.child.kill();
let _ = self.child.wait();
}
}
#[test]
fn create_through_the_app_toolset() {
let d = scratch("mcp");
let rt = runtime(&d);
let mut c = Client::start(&d, &["--toolset", "app"]);
c.read_bridge();
let (port, code) = c.bridge.clone().unwrap();
assert_eq!(c.names(), ["app_guide", "app_check", "app_stream", "app_export"]);
let g = c.call("app_guide", json!({})).unwrap();
assert!(g["guide"].as_str().unwrap().contains("Poll(") && g["code"] == code.as_str(), "{g}");
let ok = c.call("app_check", json!({"program": SHOW})).unwrap();
assert_eq!((ok["ok"].as_bool(), ok["room"]["kind"].as_str()), (Some(true), Some("show")), "{ok}");
let refused = c.call("app_check", json!({"program": "root = App([w])\nw = WatchTogether(\"f.mp4\", [Chat()])\n"})).unwrap();
assert!(refused["ok"] == false && refused["problems"][0].as_str().unwrap().starts_with("line 2: disallowed-child"), "{refused}");
// No page is connected: the stream says how to connect one.
assert!(c.call("app_stream", json!({"program": SHOW})).unwrap_err().contains(&code));
// A page without the code is refused; one with it reads the stream.
let get = |path: &str| {
let mut s = std::net::TcpStream::connect(("127.0.0.1", port)).unwrap();
write!(s, "GET {path} HTTP/1.1\r\nHost: 127.0.0.1\r\n\r\n").unwrap();
s
};
let mut resp = String::new();
get("/create/stream?code=wrong").read_to_string(&mut resp).unwrap();
assert!(resp.starts_with("HTTP/1.1 403"), "{resp}");
let mut page = BufReader::new(get(&format!("/create/stream?code={code}")));
let mut head = String::new();
page.read_line(&mut head).unwrap();
assert!(head.starts_with("HTTP/1.1 200"), "{head}");
let mut s = Err(String::new());
for _ in 0..100 {
s = c.call("app_stream", json!({"program": SHOW, "chunk": 40, "pace_ms": 0}));
if s.is_ok() {
break;
}
std::thread::sleep(std::time::Duration::from_millis(20));
}
let s = s.unwrap();
assert_eq!(s["delivered"], 1, "{s}");
let mut seen = String::new();
while !seen.contains("event: end") {
let mut l = String::new();
page.read_line(&mut l).unwrap();
seen.push_str(&l);
}
assert!(seen.contains("event: start") && seen.contains("Friday night"), "{seen}");
// An export, inside the project directory alone.
let e = c.call("app_export", json!({"program": SHOW, "dir": "show", "runtime_dir": rt})).unwrap();
assert!(e["files"].as_array().unwrap().iter().any(|f| f == "wai.json"), "{e}");
assert_eq!(app_check(&d.join("show")).0, 0);
let outside = std::env::temp_dir().join(format!("wai-cli-app-outside-{}", std::process::id()));
let err = c.call("app_export", json!({"program": SHOW, "dir": outside, "runtime_dir": rt})).unwrap_err();
assert!(err.contains("outside the project directory") && !outside.exists(), "{err}");
// Read-only: no bridge, and nothing that streams or writes.
let mut ro = Client::start(&d, &["--toolset", "core,app", "--read-only"]);
let names = ro.names();
assert!(names.contains(&"app_check".to_string()) && !names.iter().any(|n| n == "app_stream" || n == "app_export"), "{names:?}");
assert!(ro.call("app_guide", json!({})).unwrap()["bridge"].is_null());
let _ = std::fs::remove_dir_all(&d);
}