3e88f9fbc9
The sentence helper is an optional download (Settings → Sentence helper): Gemma 4 E2B in Google's web build, run by Google's LiteRT-LM web runtime on the graphics chip, in a worker. ✨ beside the message turns the words chosen into three sentences, fitting what the partner said if it was typed in, or suggests four replies when no words are chosen yet. The suggestions take the word predictions' place, at their height, so no cell moves (SPEC §3); the person chooses one and speaks it. Nothing is spoken by itself, and nothing typed leaves the device. - ai_helper.rs: the catalogue (aac-ai-index/1), prompts, whether a device can run it (a WebGPU hardware adapter, never the processor), download, continue, remove and re-check, and suggestions that a later request supersedes. The model's parts (64 MiB, SHA-256 checked) and the one runtime build this browser runs live in the model store; the worker streams the parts from there into GPU memory. - third_party/litert-lm: @litert-lm/core 0.17.1 vendored and pinned (npm integrity, each file's size and SHA-256), with our worker bundled by esbuild 0.28.2; the runtime's CDN default is emptied. - tools/ai: build_ai.py (the pinned model → parts and catalogue), check_packs.py, languages.json (English, Spanish, German and Hindi ready; Swahili not yet), and eval/: the 20 exchanges, a harness that runs the app's own prompts (a test keeps them word for word), and the judged answers. - Measured: Chrome on an M5 Max, 0.4-0.7 s an answer. A real iPad Air (M3) in Safari 27: 101 s to download and check 2 GB, about 80 s for the first start, 5.9 s for later starts, 0.8-1.4 s an answer. Co-Authored-By: Claude Opus 5.5 <[email protected]>
34 lines
2 KiB
Text
34 lines
2 KiB
Text
The sentence helper in the AAC reference app runs an on-device language model
|
|
on Google's LiteRT-LM web runtime. The app serves the runtime from its own
|
|
site, under vendor/litert-lm-<version>-<revision>/, put there by
|
|
third_party/litert-lm/vendor.sh. When someone downloads the sentence helper,
|
|
the app copies the runtime files their browser needs into their model store
|
|
and runs them from there, in a worker on their device. Nothing they type is
|
|
sent anywhere.
|
|
|
|
LiteRT-LM (@litert-lm/core, the version in the directory name), by Google LLC
|
|
(The ODML Authors).
|
|
litertlm_wasm_internal.js and .wasm, litertlm_wasm_asyncify_internal.js and
|
|
.wasm, litertlm_wasm_compat_internal.js and .wasm, and
|
|
litertlm_wasm_compat_asyncify_internal.js and .wasm: unchanged from the npm
|
|
package. They are Google's prebuilt WebAssembly builds of LiteRT-LM, one
|
|
for each combination of WebAssembly features (relaxed SIMD, JavaScript
|
|
Promise Integration). A browser downloads only the one it runs.
|
|
ai-worker.js: the app's own worker (worker.js beside vendor.sh), bundled by
|
|
esbuild with the JavaScript of @litert-lm/core and @litertjs/wasm-utils,
|
|
unchanged except one line: the runtime's default WebAssembly location (a
|
|
public CDN) is emptied, so the runtime can never load anything from another
|
|
site. The app always hands it the WebAssembly itself.
|
|
Licensed under the Apache License, Version 2.0: LICENSE.txt beside this
|
|
file (LiteRT-LM's LICENSE, verbatim).
|
|
Source: https://www.npmjs.com/package/@litert-lm/core
|
|
https://www.npmjs.com/package/@litertjs/wasm-utils
|
|
(package.json "license": "Apache-2.0" in both),
|
|
https://github.com/google-ai-edge/LiteRT-LM (LICENSE read
|
|
2026-10-01 at tag v0.17.1, commit
|
|
5e58e9a0aef7abf7091207a8b1d1063a1c800f08)
|
|
|
|
The language model the runtime runs is not part of this directory and is not
|
|
covered by LICENSE.txt. The app publishes it separately, with its own
|
|
licence and attribution, in the sentence helper's catalogue (Settings >
|
|
Sentence helper).
|