dda1b8af20
Review of 3d242dea (MEDIUM 3, LOW 4-6):
- A file is timed on its own only from its first answer on: Chromium keeps
three to five of a worker's requests on the wire, and the rest waited past
the 30 s stall limit before they were sent, so a slow network with many
files never installed. Before an answer, the whole-install watchdog bounds
it; when nothing at all has moved for 30 s it now cancels every request
still open.
- Every file is fetched to its end even when another failed
(Promise.allSettled), so a retry copies the finished content-hashed files
(a part-downloaded wasm is no longer thrown away because another file
stalled). A file already in this version's precache is left as it is.
- Two registrations of the worker (the site's, and the app's own after WebKit
lost the site's install) never both download a content-hashed file: Web
Locks let one fetch it while the other waits (alive, not stalled), then
copies it.
- Each precache records when it was begun; activate removes only precaches
begun before its own, so it never takes a newer version's files from
another registration's install. The older-version case is documented.
- Lookups of precached files ask this version's precache first, then every
cache: an older runtime copy of a file whose name does not change no
longer wins.
- prerender-static.sh never lists a .br or .gz copy in the precache.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
294 lines
17 KiB
Shell
Executable file
294 lines
17 KiB
Shell
Executable file
#!/usr/bin/env bash
|
|
# ============================================================================
|
|
# prerender-static.sh — build aac.transaction.science as a self-contained
|
|
# static site.
|
|
#
|
|
# AAC ships as a static CSR-hydrate SPA, but it's a Leptos SSR app: the
|
|
# index.html the browser loads is the SSR server's *rendered* output (it carries
|
|
# the hydration island markers and the pre-rendered board). A plain
|
|
# `cargo leptos build` never emits that index.html, and it doesn't copy the
|
|
# PWA assets that live in `static/`. This script closes both gaps and makes the
|
|
# result safe to cache forever:
|
|
#
|
|
# build -> target/site/{pkg/, icons/, symbols/} (cargo-leptos)
|
|
# render -> target/site/index.html (capture SSR `/`)
|
|
# hash -> pkg/aac-web.<sha256:12>.{js,wasm,css} (and rewrite references)
|
|
# vendor -> target/site/vendor/mediapipe-<ver>-<rev>/ (eye gaze, same origin)
|
|
# worker -> target/site/sw.js (static/sw.js + precache list)
|
|
#
|
|
# Output: a self-contained `target/site/` ready to rsync to prod. Deploy it with
|
|
# no_cache_bust — the names are already content-hashed, and the service worker
|
|
# lists them. Idempotent.
|
|
#
|
|
# Hosting under a sub-path, or under another name, is a build-time choice; see
|
|
# AAC_BASE_PATH, AAC_SW_URL, AAC_APP_TITLE and AAC_HIDE_ENERGY in
|
|
# crates/aac-web/src/app.rs. This script builds the root-mounted default.
|
|
# ============================================================================
|
|
set -euo pipefail
|
|
cd "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
|
|
PORT="${AAC_PRERENDER_PORT:-39207}"
|
|
SITE="target/site"
|
|
CACHE="${XDG_CACHE_HOME:-$HOME/.cache}/aac-rs"
|
|
|
|
# wasm-bindgen's CLI must match the crate version exactly (Cargo.toml pins it).
|
|
# Use a private copy of the right version rather than the machine's global one.
|
|
WBG_VER="$(awk '/^name = "wasm-bindgen"$/{getline; gsub(/version = |"/, ""); print; exit}' Cargo.lock)"
|
|
if ! wasm-bindgen --version 2>/dev/null | grep -qx "wasm-bindgen $WBG_VER"; then
|
|
WBG_ROOT="$CACHE/wasm-bindgen-$WBG_VER"
|
|
if [ ! -x "$WBG_ROOT/bin/wasm-bindgen" ]; then
|
|
echo "▸ installing wasm-bindgen-cli $WBG_VER into $WBG_ROOT"
|
|
cargo install --quiet --locked --root "$WBG_ROOT" wasm-bindgen-cli --version "$WBG_VER"
|
|
fi
|
|
export PATH="$WBG_ROOT/bin:$PATH"
|
|
fi
|
|
|
|
echo "▸ cargo leptos build --release"
|
|
rm -rf "$SITE/pkg"
|
|
cargo leptos build --release
|
|
find "$SITE/pkg" -name '*.d.ts' -delete
|
|
# Symbol sprites and their licences (cargo-leptos copies public/; a clean copy
|
|
# drops sprites of earlier builds, whose names carry another hash).
|
|
rm -rf "$SITE/symbols"
|
|
cp -R crates/aac-web/public/symbols "$SITE/symbols"
|
|
# Language packs for word prediction (data/lang: content-hashed, with their
|
|
# licence) under models/lang/, where the app's model store downloads a
|
|
# language's pack once, verifies it and keeps it (aac-web lang_packs.rs).
|
|
rm -rf "$SITE/models/lang"
|
|
mkdir -p "$SITE/models/lang"
|
|
cp data/lang/*.bin data/lang/index.json data/lang/LICENSE.md "$SITE/models/lang/"
|
|
# The app's own words in each language (crates/aac-web/i18n/<code>.ftl) under
|
|
# models/ui/<code>.<first 12 hex of sha256>.ftl, the name the app's index
|
|
# (build.rs) gives it: fetched the first time the app is used in that
|
|
# language, checked, and kept in IndexedDB (aac-web i18n.rs).
|
|
rm -rf "$SITE/models/ui"
|
|
mkdir -p "$SITE/models/ui"
|
|
for f in crates/aac-web/i18n/*.ftl; do
|
|
[ -e "$f" ] || continue
|
|
code="$(basename "$f" .ftl)"
|
|
cp "$f" "$SITE/models/ui/$code.$(shasum -a 256 "$f" | cut -c1-12).ftl"
|
|
done
|
|
# The pinyin index for typing Chinese on the on-screen keyboard (data/pinyin,
|
|
# content-hashed, with its licence): fetched only when Chinese is typed.
|
|
rm -rf "$SITE/models/pinyin"
|
|
mkdir -p "$SITE/models/pinyin"
|
|
cp data/pinyin/*.txt data/pinyin/index.json data/pinyin/LICENSE.md "$SITE/models/pinyin/"
|
|
|
|
echo "▸ starting SSR server on 127.0.0.1:$PORT to prerender /"
|
|
LOG="$(mktemp)"
|
|
LEPTOS_OUTPUT_NAME=aac-web \
|
|
LEPTOS_SITE_ROOT="$SITE" \
|
|
LEPTOS_SITE_PKG_DIR=pkg \
|
|
LEPTOS_SITE_ADDR="127.0.0.1:$PORT" \
|
|
LEPTOS_ENV=PROD \
|
|
target/release/aac-web > "$LOG" 2>&1 &
|
|
SRV=$!
|
|
trap 'kill "$SRV" 2>/dev/null || true' EXIT
|
|
|
|
# Wait for the listener (up to ~30s).
|
|
ready=false
|
|
for _ in $(seq 1 120); do
|
|
if curl -sf "http://127.0.0.1:$PORT/" -o /dev/null 2>/dev/null; then ready=true; break; fi
|
|
sleep 0.25
|
|
done
|
|
if ! $ready; then
|
|
echo "✗ SSR server did not come up; log:" >&2; tail -5 "$LOG" >&2; exit 1
|
|
fi
|
|
|
|
echo "▸ capturing rendered / -> $SITE/index.html"
|
|
curl -sf "http://127.0.0.1:$PORT/" -o "$SITE/index.html"
|
|
|
|
kill "$SRV" 2>/dev/null || true
|
|
trap - EXIT
|
|
|
|
# Sanity-check the capture is the hydrating SSR shell, not an error page.
|
|
test -s "$SITE/index.html"
|
|
grep -q 'mod.hydrate' "$SITE/index.html" || { echo "✗ captured index.html is not the hydrating shell" >&2; exit 1; }
|
|
grep -q '<title>AAC' "$SITE/index.html" || { echo "✗ captured index.html missing expected title" >&2; exit 1; }
|
|
|
|
echo "▸ content-hashing pkg/"
|
|
hash12() { shasum -a 256 "$1" | cut -c1-12; }
|
|
# Portable in-place sed (BSD and GNU disagree about -i): sed_i <args…> <file>
|
|
sed_i() { local f="${!#}"; sed "${@:1:$#-1}" "$f" > "$f.tmp" && mv "$f.tmp" "$f"; }
|
|
P="$SITE/pkg"
|
|
[ -f "$P/aac-web_bg.wasm" ] || mv "$P/aac-web.wasm" "$P/aac-web_bg.wasm"
|
|
W="$(hash12 "$P/aac-web_bg.wasm")"; mv "$P/aac-web_bg.wasm" "$P/aac-web.$W.wasm"
|
|
C="$(hash12 "$P/aac-web.css")"; mv "$P/aac-web.css" "$P/aac-web.$C.css"
|
|
# The glue may name its wasm; point it at the hashed file before hashing the glue.
|
|
sed_i -e "s#aac-web_bg\\.wasm#aac-web.$W.wasm#g" -e "s#aac-web\\.wasm#aac-web.$W.wasm#g" "$P/aac-web.js"
|
|
# wasm-bindgen names the snippet directory after the crate, not its contents,
|
|
# so a snippet keeps its URL from build to build, and Cloudflare gives every .js
|
|
# a 4-hour browser cache whatever Caddy sends. A returning visitor would then
|
|
# pair a new aac-web.js with an old snippet, and the app would not start (seen
|
|
# on 2026-09-30). Name each snippet directory by its contents too, before
|
|
# aac-web.js (whose imports name it) is hashed.
|
|
for d in "$P"/snippets/*/; do
|
|
d="${d%/}"; name="$(basename "$d")"
|
|
S="$(cd "$d" && find . -type f | LC_ALL=C sort | while IFS= read -r f; do printf '%s %s\n' "$f" "$(shasum -a 256 "$f" | cut -c1-64)"; done | shasum -a 256 | cut -c1-12)"
|
|
mv "$d" "$P/snippets/$name-$S"
|
|
# Only where the script loads it (`from "…"`, `import("…")`): the keys of
|
|
# the WebAssembly import object keep the old path, which the .wasm names.
|
|
sed_i -e "s#\(from *[\"']\)\./snippets/$name/#\1./snippets/$name-$S/#g" \
|
|
-e "s#\(import( *[\"']\)\./snippets/$name/#\1./snippets/$name-$S/#g" "$P/aac-web.js"
|
|
grep -q "snippets/$name-$S/" "$P/aac-web.js" || { echo "✗ aac-web.js does not import snippets/$name-$S" >&2; exit 1; }
|
|
if grep -qE "(from *[\"']|import\( *[\"'])\./snippets/$name/" "$P/aac-web.js"; then
|
|
echo "✗ aac-web.js still loads snippets/$name by its old name" >&2; exit 1
|
|
fi
|
|
done
|
|
J="$(hash12 "$P/aac-web.js")"; mv "$P/aac-web.js" "$P/aac-web.$J.js"
|
|
sed_i \
|
|
-e "s#/pkg/aac-web_bg\\.wasm#/pkg/aac-web.$W.wasm#g" \
|
|
-e "s#/pkg/aac-web\\.wasm#/pkg/aac-web.$W.wasm#g" \
|
|
-e "s#/pkg/aac-web\\.js#/pkg/aac-web.$J.js#g" \
|
|
-e "s#/pkg/aac-web\\.css#/pkg/aac-web.$C.css#g" \
|
|
"$SITE/index.html"
|
|
# Leptos's hydration loader builds its URLs from arguments:
|
|
# (root, pkg_path, output_name, wasm_output_name)
|
|
LOADER='("", "pkg", "aac-web", "aac-web")'
|
|
grep -qF "$LOADER" "$SITE/index.html" || { echo "✗ hydration loader call not found — has Leptos changed its loader?" >&2; exit 1; }
|
|
sed_i "s#(\"\", \"pkg\", \"aac-web\", \"aac-web\")#(\"\", \"pkg\", \"aac-web.$J\", \"aac-web.$W\")#" "$SITE/index.html"
|
|
for f in "aac-web.$W.wasm" "aac-web.$J.js" "aac-web.$C.css" "\"aac-web.$J\", \"aac-web.$W\""; do
|
|
grep -q "$f" "$SITE/index.html" || { echo "✗ index.html does not reference $f" >&2; exit 1; }
|
|
done
|
|
|
|
# Eye gaze loads MediaPipe from vendor/mediapipe-<version>-<revision>/ on this
|
|
# origin (crates/aac-web/src/bindings/mediapipe_init.js), so turning on the
|
|
# camera makes no third-party request and the app stays cross-origin isolated.
|
|
rm -rf "$SITE/vendor"
|
|
bash third_party/mediapipe/vendor.sh "$SITE" "$CACHE"
|
|
MP_DIR="$(cd "$SITE/vendor" && echo mediapipe-*)"
|
|
[ -d "$SITE/vendor/$MP_DIR" ] || { echo "✗ vendor.sh left no vendor/mediapipe-* directory" >&2; exit 1; }
|
|
|
|
# On-device voices, downloaded only when the person asks (neural_voice.rs):
|
|
# the Moonshine WASM runtime, vendored and pinned like MediaPipe, and the
|
|
# voice packs, built outside git by tools/voices/build_voices.py, under
|
|
# voices/ (cloned, not copied, on APFS), once tools/voices/check_packs.py
|
|
# has matched them to the committed catalogue. A build without them must say
|
|
# so (AAC_VOICES=none): a deploy would otherwise delete the voices it serves.
|
|
rm -rf "$SITE/voices"
|
|
if [ "${AAC_VOICES:-}" = "none" ]; then
|
|
echo "▸ AAC_VOICES=none: no better voices in this build; the app offers the device's voices only"
|
|
else
|
|
bash third_party/moonshine-wasm/vendor.sh "$SITE" "$CACHE"
|
|
VOICES="${AAC_VOICES_DIR:-$(cd ../../.. && pwd)/aac-voice-packs}"
|
|
python3 tools/voices/check_packs.py "$VOICES" \
|
|
|| { echo "✗ build the packs (tools/voices/build_voices.py), or set AAC_VOICES=none to build without better voices" >&2; exit 1; }
|
|
cp -cR "$VOICES" "$SITE/voices" 2>/dev/null || cp -R "$VOICES" "$SITE/voices"
|
|
# index.json and LICENSES.md keep their names from one catalogue to the next,
|
|
# and the builder stamps every file with one fixed time: give these two this
|
|
# build's own, or rsync's size-and-time check would keep prod's old pointer,
|
|
# to a catalogue its --delete has just removed.
|
|
touch "$SITE/voices/index.json" "$SITE/voices/LICENSES.md"
|
|
fi
|
|
# The sentence helper, downloaded only when the person asks (ai_helper.rs):
|
|
# Google's LiteRT-LM web runtime, vendored and pinned like Moonshine, and the
|
|
# model's parts, built outside git by tools/ai/build_ai.py, under ai/ (cloned,
|
|
# not copied, on APFS), once tools/ai/check_packs.py has matched them to the
|
|
# committed catalogue. A build without them must say so (AAC_AI=none).
|
|
rm -rf "$SITE/ai"
|
|
if [ "${AAC_AI:-}" = "none" ]; then
|
|
echo "▸ AAC_AI=none: no sentence helper in this build"
|
|
else
|
|
bash third_party/litert-lm/vendor.sh "$SITE" "$CACHE"
|
|
AI="${AAC_AI_DIR:-$(cd ../../.. && pwd)/aac-ai-packs}"
|
|
python3 tools/ai/check_packs.py "$AI" \
|
|
|| { echo "✗ build the model parts (tools/ai/build_ai.py), or set AAC_AI=none to build without the sentence helper" >&2; exit 1; }
|
|
cp -cR "$AI" "$SITE/ai" 2>/dev/null || cp -R "$AI" "$SITE/ai"
|
|
touch "$SITE/ai/index.json" "$SITE/ai/LICENSES.md"
|
|
fi
|
|
# The listener, downloaded only when the person asks (listen.rs):
|
|
# onnxruntime-web, vendored and pinned like LiteRT-LM, and the model's files,
|
|
# built outside git by tools/asr/build_asr.py, under asr/ (cloned, not
|
|
# copied, on APFS), once tools/asr/check_packs.py has matched them to the
|
|
# committed catalogue. A build without them must say so (AAC_ASR=none).
|
|
rm -rf "$SITE/asr"
|
|
if [ "${AAC_ASR:-}" = "none" ]; then
|
|
echo "▸ AAC_ASR=none: no listener in this build"
|
|
else
|
|
bash third_party/onnxruntime-web/vendor.sh "$SITE" "$CACHE"
|
|
ASR="${AAC_ASR_DIR:-$(cd ../../.. && pwd)/aac-asr-packs}"
|
|
python3 tools/asr/check_packs.py "$ASR" \
|
|
|| { echo "✗ build the listener's files (tools/asr/build_asr.py), or set AAC_ASR=none to build without the listener" >&2; exit 1; }
|
|
cp -cR "$ASR" "$SITE/asr" 2>/dev/null || cp -R "$ASR" "$SITE/asr"
|
|
touch "$SITE/asr/index.json" "$SITE/asr/LICENSES.md"
|
|
fi
|
|
# First letters, downloaded only when the person asks (abbrev.rs): it runs
|
|
# on the listener's onnxruntime-web (vendored above, or here when the
|
|
# listener is not in this build), with its own worker and Hugging Face's
|
|
# tokenizer library, vendored and pinned like it, and the model's files,
|
|
# built outside git by tools/ae/build_ae.py, under ae/ (cloned, not copied,
|
|
# on APFS), once tools/ae/check_packs.py has matched them to the committed
|
|
# catalogue. A build without them must say so (AAC_AE=none).
|
|
rm -rf "$SITE/ae"
|
|
if [ "${AAC_AE:-}" = "none" ]; then
|
|
echo "▸ AAC_AE=none: no first letters in this build"
|
|
else
|
|
[ "${AAC_ASR:-}" = "none" ] && bash third_party/onnxruntime-web/vendor.sh "$SITE" "$CACHE"
|
|
bash third_party/hf-tokenizers/vendor.sh "$SITE" "$CACHE"
|
|
AE="${AAC_AE_DIR:-$(cd ../../.. && pwd)/aac-ae-packs}"
|
|
python3 tools/ae/check_packs.py "$AE" \
|
|
|| { echo "✗ build first letters' files (tools/ae/build_ae.py), or set AAC_AE=none to build without first letters" >&2; exit 1; }
|
|
cp -cR "$AE" "$SITE/ae" 2>/dev/null || cp -R "$AE" "$SITE/ae"
|
|
touch "$SITE/ae/index.json" "$SITE/ae/LICENSES.md"
|
|
fi
|
|
# The loaders of the sentence helper, the listener and first letters must
|
|
# never reach another site.
|
|
AI_SNIP="$(grep -rl 'ensure_engine' "$SITE/pkg/snippets" || true)"
|
|
[ -n "$AI_SNIP" ] || { echo "✗ the sentence helper's snippet is not in pkg/snippets" >&2; exit 1; }
|
|
if grep -nE 'https?://' $AI_SNIP; then echo "✗ the sentence helper's snippet names another site" >&2; exit 1; fi
|
|
LISTEN_SNIP="$(grep -rl 'listen_once' "$SITE/pkg/snippets" || true)"
|
|
[ -n "$LISTEN_SNIP" ] || { echo "✗ the listener's snippet is not in pkg/snippets" >&2; exit 1; }
|
|
if grep -nE 'https?://' $LISTEN_SNIP; then echo "✗ the listener's snippet names another site" >&2; exit 1; fi
|
|
AE_SNIP="$(grep -rl '__aacLettersEngine' "$SITE/pkg/snippets" || true)"
|
|
[ -n "$AE_SNIP" ] || { echo "✗ first letters' snippet is not in pkg/snippets" >&2; exit 1; }
|
|
if grep -nE 'https?://' $AE_SNIP; then echo "✗ first letters' snippet names another site" >&2; exit 1; fi
|
|
SNIP="$(grep -rl 'initialize_face_landmarker' "$SITE/pkg/snippets" || true)"
|
|
[ -n "$SNIP" ] || { echo "✗ eye-tracking snippet not found in pkg/snippets" >&2; exit 1; }
|
|
if grep -nE 'https?://' $SNIP; then echo "✗ eye-tracking snippet loads from a third party" >&2; exit 1; fi
|
|
|
|
echo "▸ bundling PWA assets and the service worker"
|
|
cp crates/aac-web/static/manifest.json "$SITE/manifest.json"
|
|
# Precache: the shell, the manifest, every file in pkg/ (including wasm-bindgen
|
|
# snippets such as the eye-tracking loader), the icons, and every symbol sprite
|
|
# the prerendered board draws (the default set and the sets it falls back to),
|
|
# so symbols show offline from the first visit, and their LICENSES.md. Other sprites are cached by the
|
|
# worker the first time they are drawn.
|
|
PRECACHE=("/" "/manifest.json")
|
|
# (Never a .br or .gz copy of a file: the server sends it in the file's place.)
|
|
while IFS= read -r f; do PRECACHE+=("/${f#"$SITE"/}"); done < <(find "$SITE/pkg" "$SITE/icons" -type f ! -name '*.br' ! -name '*.gz' | LC_ALL=C sort)
|
|
DRAWN="$(grep -oE '/symbols/[a-z]+\.[0-9a-f]{12}\.svg' "$SITE/index.html" | LC_ALL=C sort -u || true)"
|
|
[ -n "$DRAWN" ] || { echo "✗ the prerendered board draws no symbols" >&2; exit 1; }
|
|
for u in $DRAWN; do
|
|
[ -f "$SITE$u" ] || { echo "✗ index.html draws $u, which the build does not ship" >&2; exit 1; }
|
|
PRECACHE+=("$u")
|
|
done
|
|
# Settings links the symbols' licences; the worker serves it offline too.
|
|
[ -f "$SITE/symbols/LICENSES.md" ] || { echo "✗ the build does not ship symbols/LICENSES.md" >&2; exit 1; }
|
|
PRECACHE+=("/symbols/LICENSES.md")
|
|
SYMBOL_FILES="[$(cd "$SITE/symbols" && printf '"%s",' *.svg | sed 's/,$//')]"
|
|
VERSION="$(for u in "${PRECACHE[@]}"; do
|
|
if [ "$u" = "/" ]; then f="$SITE/index.html"; else f="$SITE$u"; fi
|
|
printf '%s %s\n' "$u" "$(shasum -a 256 "$f" | cut -c1-64)"
|
|
done | shasum -a 256 | cut -c1-12)"
|
|
LIST="[$(printf '"%s",' "${PRECACHE[@]}" | sed 's/,$//')]"
|
|
# The worker keeps only the current vendor directory in its vendor cache.
|
|
sed -e "s#__VERSION__#$VERSION#" -e "s#__PRECACHE_URLS__#$LIST#" -e "s#__VENDOR_DIRS__#[\"$MP_DIR/\"]#" \
|
|
-e "s#__SYMBOL_FILES__#$SYMBOL_FILES#" \
|
|
crates/aac-web/static/sw.js > "$SITE/sw.js"
|
|
grep -q "__" "$SITE/sw.js" && grep -nE "__[A-Z_]+__" "$SITE/sw.js" && { echo "✗ sw.js has an unfilled placeholder" >&2; exit 1; }
|
|
|
|
# The optional non-commercial symbol sets are never in the standard's own build
|
|
# (aac-web's nc-symbols feature is off): no symbol pack anywhere (by place,
|
|
# name or a pack's magic bytes, .gz/.br sidecars read decompressed), none in
|
|
# the worker's precache list, no file in any folder named for them, and no
|
|
# file of the app's own naming them in its content, sidecars decompressed.
|
|
# The third-party data copied in beside it (voices/, ai/, asr/, ae/, vendor/,
|
|
# and the word lists in models/lang/: dictionaries hold such words as Italian
|
|
# "sclera", the eye) is left out of that content scan only. The same checks as a
|
|
# deployment's (aac-aurasense-web scripts/check-nc-off.mjs); planted-file
|
|
# tests: node --test tools/symbols/check-no-nc.test.mjs.
|
|
command -v node >/dev/null || { echo "✗ node is needed to check the build for the optional symbol sets" >&2; exit 1; }
|
|
node tools/symbols/check-no-nc.mjs "$SITE"
|
|
|
|
echo "✓ self-contained dist ready at $SITE/ (service worker $VERSION, ${#PRECACHE[@]} files precached)"
|
|
ls -1 "$SITE" "$SITE/pkg"
|