ace56fe36f
- Listening's timers (30 s for words, 90 s in all) run from the press, so a permission prompt nobody answers, or audio that never starts, ends it too, and no wait holds the sentence helper; a page already hidden does not open the microphone, and one hidden while the browser asked lets it go. - A finish that fails drops the worker, as a failed run does; a failed first-run check lets the model go (it kept 1.3 GB loaded). - Test hooks: shorter timers, the app counted as speaking, the worker's events counted, a failed run on request. - Every locale of a language uses the prompt slot that was measured for it. FLEURS has one accent per language, so the model's en-GB, es-US, pt-PT and fr-CA slots cannot be measured, and are not used. - eval/score.py counts the clips the end-of-speech rule cut at a pause against the run with no end at all (--no-end), the one method used for the notes and the README: with 1 s, Hindi 3 and Portuguese 1; with 500 ms, 43 of 400 (Vietnamese 16, Hindi 5, Portuguese 5, Italian 4, German 3). Co-Authored-By: Claude Opus 5.5 <[email protected]>
354 lines
18 KiB
JSON
354 lines
18 KiB
JSON
{
|
|
"about": "The languages the listener (Hearing your partner) is offered in. A language is ready only when we measured it on this app's own runtime at most 15 errors in 100 words (characters for Chinese, Japanese and Korean): the first 25 FLEURS test clips, through the app's own worker on WebGPU, with its voice activity detector and 1 s end-of-speech rule (tools/asr/eval). NVIDIA's published rate is kept beside ours, for reference. tools/asr/build_asr.py copies this into the catalogue; the model is told the language by its prompt slot: the one measured for the language, for every locale of it.",
|
|
"languages": {
|
|
"am": {
|
|
"status": "not-yet",
|
|
"name": "Amharic",
|
|
"note": "The speech model was not trained to write Amharic down."
|
|
},
|
|
"ar": {
|
|
"status": "ready",
|
|
"name": "Arabic",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 7,
|
|
"measure": "wer",
|
|
"published": 12.55,
|
|
"measured": 8.4,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of ar_eg (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 8 words in 100 came out wrong (NVIDIA's own test: about 13)."
|
|
},
|
|
"bn": {
|
|
"status": "not-yet",
|
|
"name": "Bengali",
|
|
"note": "The speech model was not trained to write Bengali down."
|
|
},
|
|
"cs": {
|
|
"status": "not-yet",
|
|
"name": "Czech",
|
|
"slot": 22,
|
|
"measure": "wer",
|
|
"published": 22.41,
|
|
"source": "NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In NVIDIA's own test on read speech, about 22 words in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"da": {
|
|
"status": "not-yet",
|
|
"name": "Danish",
|
|
"slot": 25,
|
|
"measure": "wer",
|
|
"published": 30.0,
|
|
"source": "NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In NVIDIA's own test on read speech, about 30 words in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"de": {
|
|
"status": "ready",
|
|
"name": "German",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 9,
|
|
"measure": "wer",
|
|
"published": 8.83,
|
|
"measured": 12.91,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of de_de (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 13 words in 100 came out wrong (NVIDIA's own test: about 9)."
|
|
},
|
|
"el": {
|
|
"status": "not-yet",
|
|
"name": "Greek",
|
|
"note": "NVIDIA trained the speech model on it only as a start for more training; it does not write Greek down well yet."
|
|
},
|
|
"en": {
|
|
"status": "ready",
|
|
"name": "English",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 0,
|
|
"measure": "wer",
|
|
"published": 8.27,
|
|
"measured": 9.78,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of en_us (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 10 words in 100 came out wrong (NVIDIA's own test: about 8)."
|
|
},
|
|
"es": {
|
|
"status": "ready",
|
|
"name": "Spanish",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 2,
|
|
"measure": "wer",
|
|
"published": 4.39,
|
|
"measured": 4.47,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of es_419 (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 4 words in 100 came out wrong (NVIDIA's own test: about 4)."
|
|
},
|
|
"fa": {
|
|
"status": "not-yet",
|
|
"name": "Persian",
|
|
"note": "The speech model was not trained to write Persian down."
|
|
},
|
|
"fi": {
|
|
"status": "not-yet",
|
|
"name": "Finnish",
|
|
"slot": 26,
|
|
"measure": "wer",
|
|
"published": 20.05,
|
|
"source": "NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In NVIDIA's own test on read speech, about 20 words in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"fr": {
|
|
"status": "ready",
|
|
"name": "French",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 8,
|
|
"measure": "wer",
|
|
"published": 9.79,
|
|
"measured": 11.67,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of fr_fr (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 12 words in 100 came out wrong (NVIDIA's own test: about 10)."
|
|
},
|
|
"gu": {
|
|
"status": "not-yet",
|
|
"name": "Gujarati",
|
|
"note": "The speech model was not trained to write Gujarati down."
|
|
},
|
|
"ha": {
|
|
"status": "not-yet",
|
|
"name": "Hausa",
|
|
"note": "The speech model was not trained to write Hausa down."
|
|
},
|
|
"he": {
|
|
"status": "not-yet",
|
|
"name": "Hebrew",
|
|
"note": "NVIDIA trained the speech model on it only as a start for more training; it does not write Hebrew down well yet."
|
|
},
|
|
"hi": {
|
|
"status": "not-yet",
|
|
"name": "Hindi",
|
|
"slot": 6,
|
|
"measure": "wer",
|
|
"published": 7.41,
|
|
"measured": 17.51,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of hi_in (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 18 words in 100 came out wrong: too many to rely on yet, partly because it stopped listening at a long pause between sentences (3 of 25)."
|
|
},
|
|
"hu": {
|
|
"status": "not-yet",
|
|
"name": "Hungarian",
|
|
"slot": 23,
|
|
"measure": "wer",
|
|
"published": 30.92,
|
|
"source": "NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In NVIDIA's own test on read speech, about 31 words in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"id": {
|
|
"status": "not-yet",
|
|
"name": "Indonesian",
|
|
"note": "The speech model was not trained to write Indonesian down."
|
|
},
|
|
"it": {
|
|
"status": "ready",
|
|
"name": "Italian",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 15,
|
|
"measure": "wer",
|
|
"published": 4.83,
|
|
"measured": 6.89,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of it_it (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 7 words in 100 came out wrong (NVIDIA's own test: about 5)."
|
|
},
|
|
"ja": {
|
|
"status": "ready",
|
|
"name": "Japanese",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 10,
|
|
"measure": "cer",
|
|
"published": 12.22,
|
|
"measured": 14.61,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of ja_jp (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 15 characters in 100 came out wrong (NVIDIA's own test: about 12)."
|
|
},
|
|
"kn": {
|
|
"status": "not-yet",
|
|
"name": "Kannada",
|
|
"note": "The speech model was not trained to write Kannada down."
|
|
},
|
|
"ko": {
|
|
"status": "ready",
|
|
"name": "Korean",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 14,
|
|
"measure": "cer",
|
|
"published": 7.27,
|
|
"measured": 10.74,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of ko_kr (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 11 characters in 100 came out wrong (NVIDIA's own test: about 7)."
|
|
},
|
|
"ml": {
|
|
"status": "not-yet",
|
|
"name": "Malayalam",
|
|
"note": "The speech model was not trained to write Malayalam down."
|
|
},
|
|
"mr": {
|
|
"status": "not-yet",
|
|
"name": "Marathi",
|
|
"note": "The speech model was not trained to write Marathi down."
|
|
},
|
|
"ms": {
|
|
"status": "not-yet",
|
|
"name": "Malay",
|
|
"note": "The speech model was not trained to write Malay down."
|
|
},
|
|
"nb": {
|
|
"status": "not-yet",
|
|
"name": "Norwegian",
|
|
"slot": 103,
|
|
"measure": "wer",
|
|
"published": 18.9,
|
|
"source": "NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In NVIDIA's own test on read speech, about 19 words in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"nl": {
|
|
"status": "not-yet",
|
|
"name": "Dutch",
|
|
"slot": 16,
|
|
"measure": "wer",
|
|
"published": 12.17,
|
|
"measured": 16.23,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of nl_nl (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 16 words in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"pa": {
|
|
"status": "not-yet",
|
|
"name": "Punjabi",
|
|
"note": "The speech model was not trained to write Punjabi down."
|
|
},
|
|
"pl": {
|
|
"status": "not-yet",
|
|
"name": "Polish",
|
|
"slot": 17,
|
|
"measure": "wer",
|
|
"published": 17.48,
|
|
"measured": 17.36,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of pl_pl (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 17 words in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"pt": {
|
|
"status": "ready",
|
|
"name": "Portuguese",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 12,
|
|
"measure": "wer",
|
|
"published": 5.81,
|
|
"measured": 11.43,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of pt_br (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 11 words in 100 came out wrong (NVIDIA's own test: about 6)."
|
|
},
|
|
"ro": {
|
|
"status": "not-yet",
|
|
"name": "Romanian",
|
|
"slot": 20,
|
|
"measure": "wer",
|
|
"published": 29.04,
|
|
"source": "NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In NVIDIA's own test on read speech, about 29 words in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"ru": {
|
|
"status": "ready",
|
|
"name": "Russian",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 11,
|
|
"measure": "wer",
|
|
"published": 9.87,
|
|
"measured": 14.32,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of ru_ru (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 14 words in 100 came out wrong (NVIDIA's own test: about 10)."
|
|
},
|
|
"sv": {
|
|
"status": "not-yet",
|
|
"name": "Swedish",
|
|
"slot": 24,
|
|
"measure": "wer",
|
|
"published": 23.63,
|
|
"source": "NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In NVIDIA's own test on read speech, about 24 words in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"sw": {
|
|
"status": "not-yet",
|
|
"name": "Swahili",
|
|
"note": "The speech model was not trained to write Swahili down."
|
|
},
|
|
"ta": {
|
|
"status": "not-yet",
|
|
"name": "Tamil",
|
|
"note": "The speech model was not trained to write Tamil down."
|
|
},
|
|
"te": {
|
|
"status": "not-yet",
|
|
"name": "Telugu",
|
|
"note": "The speech model was not trained to write Telugu down."
|
|
},
|
|
"th": {
|
|
"status": "not-yet",
|
|
"name": "Thai",
|
|
"note": "NVIDIA trained the speech model on it only as a start for more training; it does not write Thai down well yet."
|
|
},
|
|
"tl": {
|
|
"status": "not-yet",
|
|
"name": "Filipino",
|
|
"note": "The speech model was not trained to write Filipino down."
|
|
},
|
|
"tr": {
|
|
"status": "not-yet",
|
|
"name": "Turkish",
|
|
"slot": 18,
|
|
"measure": "wer",
|
|
"published": 12.05,
|
|
"measured": 16.19,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of tr_tr (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 16 words in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"uk": {
|
|
"status": "ready",
|
|
"name": "Ukrainian",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 19,
|
|
"measure": "wer",
|
|
"published": 14.55,
|
|
"measured": 13.8,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of uk_ua (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 14 words in 100 came out wrong (NVIDIA's own test: about 15)."
|
|
},
|
|
"ur": {
|
|
"status": "not-yet",
|
|
"name": "Urdu",
|
|
"note": "The speech model was not trained to write Urdu down."
|
|
},
|
|
"vi": {
|
|
"status": "ready",
|
|
"name": "Vietnamese",
|
|
"model": "nemotron-3.5-asr-streaming-0.6b",
|
|
"slot": 33,
|
|
"measure": "wer",
|
|
"published": 12.29,
|
|
"measured": 14.11,
|
|
"source": "ours (what decides): the first 25 FLEURS test clips of vi_vn (google/fleurs@70bb2e84, CC BY 4.0) through this app's worker on onnxruntime-web 1.30.0 WebGPU (Chrome, Apple GPU), with the VAD and the 1 s end-of-speech rule as in the app, 2026-10-01 (tools/asr/eval/results-2026-10-01.json); for reference, NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In our test of 25 read-aloud passages, on this app, about 14 words in 100 came out wrong (NVIDIA's own test: about 12)."
|
|
},
|
|
"yo": {
|
|
"status": "not-yet",
|
|
"name": "Yoruba",
|
|
"note": "The speech model was not trained to write Yoruba down."
|
|
},
|
|
"zh": {
|
|
"status": "not-yet",
|
|
"name": "Chinese",
|
|
"slot": 4,
|
|
"measure": "cer",
|
|
"published": 20.03,
|
|
"source": "NVIDIA's model card (nvidia/nemotron-3.5-asr-streaming-0.6b, revision ea30d66d): FLEURS test, 320 ms chunks, language given",
|
|
"note": "In NVIDIA's own test on read speech, about 20 characters in 100 came out wrong: too many to rely on yet."
|
|
},
|
|
"zu": {
|
|
"status": "not-yet",
|
|
"name": "Zulu",
|
|
"note": "The speech model was not trained to write Zulu down."
|
|
}
|
|
}
|
|
}
|