27118aadc8
- The microphone is never left open. The wait for words (30 s) starts again whenever a sound with no words in it ends (the worker now says so), listening ends after 90 s in all, and it stops and lets the microphone go when the page is hidden or closed or the microphone goes away. No sound is kept once listening ends. - Offered only where JSPI is (Chrome, Edge, Safari on a current iPad): the model is then read one tensor at a time. The asyncify build, and the path that read all 1.2 GB first, are gone; fit() says plainly why a browser without JSPI cannot run it. Vendored with ONNX Runtime's ThirdPartyNotices.txt (pinned like its LICENSE); revision aac2. - Closing the sentence helper no longer cuts short the listener's first run after a download: unload spares it, and a stop from outside counts as interrupted (start it again, nothing downloaded again), not as failing on this device; an update keeps the one that worked. - A model the catalogue lists under a new name is followed without a download; one it no longer lists is offered as an update. - Muting while the app speaks goes by the audio clock (each block carries its time), and adds the output's delay to the tail. - After a failed run (the graphics chip lost), the worker is dropped, so the next 🎤 starts afresh. Busy says what is busy; the runtime's own error text never reaches the person. - Test hooks: the listener's timers, and whether each model is loaded. Languages: offered only as we measured them on this app's runtime, ≤ 15 in 100. tools/asr/eval runs the first 25 FLEURS test clips of each candidate through the app's own worker on WebGPU (the VAD and the 500 ms rule as shipped) and counts the errors; every clip is in eval/results-2026-10-01.json. Ready: Spanish 7.5, Arabic 9.2, Korean 10.7 (characters), French 12.4, English 12.6, Russian 14.3. Not yet: Ukrainian, Italian, Turkish, Polish, Japanese, Dutch, German, Portuguese, Hindi, Vietnamese, mostly because the 500 ms rule ends listening at a pause between sentences: with a 1 s rule (measured, not shipped) 12 would pass. NVIDIA's rates are kept for reference. Co-Authored-By: Claude Opus 5.5 <[email protected]>
33 lines
1.9 KiB
Text
33 lines
1.9 KiB
Text
The listener in the AAC reference app ("Hearing your partner") turns what
|
|
someone says to the person into text, on their device, with onnxruntime-web.
|
|
The app serves the runtime from its own site, under
|
|
vendor/onnxruntime-web-<version>-<revision>/, put there by
|
|
third_party/onnxruntime-web/vendor.sh. When someone downloads the listener,
|
|
the app copies the runtime files their browser needs into their model store
|
|
and runs them from there, in a worker on their device, on its graphics chip.
|
|
Nothing heard is recorded or sent anywhere.
|
|
|
|
ONNX Runtime Web (onnxruntime-web, the version in the directory name), by
|
|
Microsoft Corporation.
|
|
ort.jspi.bundle.min.mjs and ort-wasm-simd-threaded.jspi.wasm: unchanged
|
|
from the npm package. They are Microsoft's prebuilt WebAssembly build of
|
|
ONNX Runtime with its WebGPU execution provider, for browsers with
|
|
JavaScript Promise Integration (the listener is offered only in those).
|
|
Licensed under the MIT License: LICENSE.txt beside this file (ONNX
|
|
Runtime's LICENSE, verbatim). ThirdPartyNotices.txt beside it is ONNX
|
|
Runtime's own notices for the third-party code it includes, verbatim, at
|
|
the same release.
|
|
Source: https://www.npmjs.com/package/onnxruntime-web
|
|
(package.json "license": "MIT"),
|
|
https://github.com/microsoft/onnxruntime (LICENSE read 2026-10-01
|
|
at tag v1.30.0, commit f2c39fe2f838cf35ce7da92824f5a5e3ee6e88a7)
|
|
|
|
asr-worker.js is the app's own worker (worker.js beside vendor.sh), part of
|
|
the AAC reference app and under its licence (Apache-2.0). It drives the
|
|
runtime; it is not part of ONNX Runtime.
|
|
|
|
The speech model the runtime runs (NVIDIA Nemotron 3.5 ASR Streaming 0.6B,
|
|
OpenMDW-1.1) and the voice activity detector (Silero VAD, MIT) are not part of
|
|
this directory and are not covered by LICENSE.txt. The app publishes them
|
|
separately, with their own licences and attribution, in the listener's
|
|
catalogue (Settings > Hearing your partner).
|