open-standards/aac/aac-rs/tools/ai

The sentence helper's model

Express (aac-web) can download one on-device language model, on request, and use it to suggest whole sentences from the words a person chose, or replies to what was said to them (crates/aac-web/src/ai_helper.rs). It runs in the browser on the graphics chip (WebGPU), in a worker, with Google's LiteRT-LM web runtime. Nothing typed is sent anywhere, and it works offline once downloaded.

Piece	Where	Pinned by
Model: Gemma 4 E2B, Google AI Edge's web build (gemma-4-E2B-it-web.litertlm, 2,008,432,640 bytes, Apache-2.0)	/app/ai/ (this tool's output)	Hugging Face revision, size and SHA-256 in build_ai.py
Runtime: @litert-lm/core 0.17.1 (four WebAssembly builds) and the app's worker	/app/vendor/litert-lm-<version>-<rev>/	npm integrity in third_party/litert-lm/vendor.sh; each file's size and SHA-256 in ai_helper.rs RUNTIME_FILES
Languages, their status, and the examples the model is shown	languages.json → the catalogue	this repository

Build and publish

python3 tools/ai/build_ai.py                 # download the pinned model once (into .cache/), split, write the catalogue
python3 tools/ai/build_ai.py --source FILE   # or use a local copy (its size and SHA-256 are checked)
python3 tools/ai/check_packs.py ../../../aac-ai-packs

The output goes beside the open-standards checkout, in aac-ai-packs/ (outside git; $AAC_AI_DIR or --out override). It holds:

• index.json: a pointer to the current catalogue, served no-store.
• index.<sha12>.json: the catalogue (schema aac-ai-index/1). It lists the model's parts by size and SHA-256, and each tested language's status, note and examples. Its committed copy is data/ai/index.json.
• gemma-4-e2b-it-web/part-NN.<sha12>.bin: the model file in 64 MiB parts. Together the parts are the file, unchanged. Each part is small enough for a tablet's model store to hash in memory, and for Cloudflare to cache.
• LICENSES.md and the licence text.

The site builds (prerender-static.sh here, and aac-aurasense-web's scripts/build-app.sh) vendor the runtime and copy the parts to /app/ai/, after check_packs.py has matched them to the committed catalogue. Set AAC_AI=none to build without them (CI does); a deploy of such a build removes the model from the site.

In the browser

• Download. Settings → Sentence helper. The person sees the size and whether they are on Wi-Fi before anything starts.
• Storage. The worker and the one WebAssembly build this browser runs are stored in the model store (Cache API aac-models-v1), with every part of the model. Each file is SHA-256 checked before it is kept.
• Loading. The worker streams the parts, one after another, from the model store into GPU memory (LiteRT-LM's GPU Artisan backend). The model never sits in JavaScript memory whole.
• When it runs. Only where WebGPU has a hardware adapter that is not refused (ai_helper::fit). Never on the processor.
• Checking it. After download it is loaded once and asked one question. The first load compiles the graphics programs, once. A device that cannot run it says so then, rather than at the person's first request.
• Memory. About 1.8 GB of graphics memory while it is in use. It is let go after 10 minutes without a request, and loaded again (seconds) on the next one.

Measured on 2026-09-30 and 2026-10-01:

	MacBook Pro M5 Max, Chrome 154	iPad Air 11-inch (M3), Safari 27.0.1
Runtime build	litertlm_wasm_internal (relaxed SIMD + JSPI)	litertlm_wasm_compat_internal (JSPI)
Download and check, 2.0 GB	8 s from a local server	101 s over Tailscale
First start, once	about 4 s	about 80 s
Later starts	2.7 s	5.9 s
One answer (3 sentences or 4 replies)	0.4–0.7 s	0.8–1.4 s

Languages

A language is offered only after testing. The test is 20 everyday exchanges (eval/ in the evaluation kit), and each answer is read for whether one suggestion says what the person meant, in good grammar. languages.json records the status and a plain note for each tested language; untested languages are not listed, and the app says so.

• Ready: English, Spanish and German (18 of 20 exchanges had a usable sentence), and Hindi (15 of 20).
• Not yet: Swahili (many sentences and replies were not grammatical).

To add a language:

1. Run the 20 exchanges in it with the model.
2. Read the answers, ideally with a native speaker.
3. Add it to languages.json with two sentence examples (one with what the partner said, one without) and two reply examples, written in that language.
4. Rebuild.

The examples are all the model is shown in that language, besides the instructions, which say the language's name.

Updating the runtime or the model

• Runtime. Pin the new @litert-lm/core release in vendor.sh, bump LITERT_LM_VERSION and LITERT_LM_VENDOR_REV in ai_helper.rs, then run vendor.sh and copy the sizes and hashes it prints into RUNTIME_FILES and VENDOR_DIGEST. The model's runtime field in the catalogue must name the new runtime. A model made for another runtime is not offered.
• Model. Change MODEL in build_ai.py (revision, size, SHA-256), test the languages again, rebuild and redeploy. People who have the old model see "A newer version is available"; the update downloads only the parts that changed, and removes the old ones.