feat: Apple Speech (SpeechAnalyzer) backend (#121)

[kiki830621]

Adds apple-speech — the OS-native backend (Speech.framework's SpeechAnalyzer / SpeechTranscriber, macOS 26+). It is the only backend in the pool with no third-party dependency and no HuggingFace weight pin: the recognizer ships with the operating system, so its version IS the OS version.

That claim is about provenance, not network traffic — a locale whose asset is absent is still downloaded from Apple on first use, as ja_JP was here. An earlier draft of this PR said "no weight download" in three places; that was a claim exceeding its evidence and has been corrected.

Refs #121

Measured results

The point of the issue was comparison, so here are numbers rather than a wiring claim. M5 Max, macOS 27, one corpus per language — indicative, not a certification; the grid row stays verified: false until a full sweep.

corpus	metric	apple-speech	whisperkit large-v3-turbo
cv-zhtw-2 (zh)	CER	13.9 % @ 63–65× realtime	12.5 % @ 7.1×
fleurs-ja-1 (ja)	CER	10.9 % @ 85× realtime	10.6 % @ 6.9×
librispeech-testclean-1 (en)	WER	4.3 % @ 58× realtime	not yet measured
librispeech-devclean-1 (en)	WER	4.1 % @ 45× realtime	not yet measured

Apple lands within ~1.4 pp of Whisper large-v3-turbo on zh and ~0.3 pp on ja, at roughly 9–12× the speed, with nothing to download or hold resident. peak-GB reads 0.00 because recognition runs in Apple's out-of-process daemon and the in-process sampler cannot see it — that column is not comparable for this backend (filed as #138).

Live probe evidence

Check	Result
Supported locales	45, of which 11 pre-installed (en_*×9, zh_CN, zh_TW)
Determinism knob	none — Speech.swiftinterface has zero hits for temperature/greedy/beam/sampling/seed → records flag-not-consumed (#118)
Per-run confidence	readable and discriminating (0.985 / 0.790 / 0.563 / 0.513 / 0.180)

The misleading error, isolated

A missing locale asset surfaces as SFSpeechErrorDomain Code=3 "Audio format is not supported". It is not a format problem. Two-step proof: three corpora byte-identical in format (1 ch / 16 kHz / Int16) behaved differently, and the same ja file transcribed fine under en_US. Installing the ja asset fixed it immediately. The engine checks installedLocales and never diagnoses availability from the error text — following that message leads to a resampling bug that does not exist.

Defects found during verification

Verification ran three independent lenses plus a cross-model (Codex) leg. Six defects surfaced, four capable of producing plausible-but-wrong output rather than an error. All are fixed in the second commit.

1. zh-Hans selected the Traditional model. Subtags were read by position, so the script subtag was taken for a region, matched nothing, and fell through to the zh preference — a user asking for Simplified silently got Traditional. Confirmed by probe, then fixed by classifying subtags by shape (BCP-47: script = 4 alpha, region = 2 alpha / 3 digits).
2. Provenance recorded the requested language, not the one that ran. pt-BR served by pt_PT was labelled pt-BR. Now records the resolved locale — hyphenated, because LanguageResolver.baseSubtag splits on - only, so Apple's zh_TW would have silently disabled the D7 Traditional/Simplified fold for this backend.
3. Recognized speech could vanish silently. A result with unusable timing was dropped. The WER denominator is the reference word count and does not move, so a dropped correct word adds a deletion while a dropped garbage word removes an insertion — the measured rate shifts in an unsignposted direction. Now fails the run.
4. Asset downloaded before the audio file was validated — a path typo could cost hundreds of MB before reporting "no such file".
5. A nil installation request fell through to the very "Audio format is not supported" the preflight exists to prevent. installedLocales is now re-checked afterwards.
6. Orphaned collector Task — neither awaited nor cancelled when the analyzer threw, pinning the transcriber across a benchmark sweep.

Plus, separately: --backend apple-speech was silently substituted. Router's membership list omitted the backend, so the override was discarded and another backend's output was written under the user's chosen name — behind an unavailable warning that was both false (isAvailable() returns true) and invisible without --explain. Measured, same command, before → after:

before:  你给我听好了…五股泰山轻轨…     ← mlx-audio Whisper, Simplified
after:   你給我聽好了…五穀泰山輕軌…     ← Apple Speech zh_TW, Traditional

RED evidence for the added test: the router threw no ASR backend is available: whisperkit, …, apple-speech — listing apple-speech in the sentence denying it exists.

Design decisions worth review

• Language is required, not guessed. SpeechTranscriber has no auto-detect mode; a ja file decoded under en_US yields measured garbage, and a benchmark harness must not admit a silent guess into its evidence base.
• Confidence is derived, not reported — the minimum over Apple's per-run values, so a confident run cannot mask a weak one inside the same cue. Documented as a minimum observed run confidence, not a calibrated per-cue probability: more runs means more chances to hit a low one, so long cues score systematically lower. NaN/inf/out-of-range now read as nil.
• verified: false, estMemoryGB an explicit unmeasured placeholder. The numbers above are ad-hoc runs, not a bestasr benchmark sweep over the canonical corpora, and Apple publishes no model footprint.
• languages is the 25 probed base subtags, not "multi" — recommend/transcribe 的 --language auto 未偵測音訊語言，非英文內容被推薦英文專用後端 #105 is the standing lesson. (mul in the list is ISO 639-2 "multiple languages", Apple's real mul_IN subtag, not this field's sentinel.)

Tests

421 tests / 87 suites green (branch baseline 389 → +32). The suite is hermetic — no real transcription, no download — so it certifies wiring, not recognition quality; that evidence is the tables above.

Non-vacuousness was checked by mutation: deleting the script-resolution branch failed exactly the script test (2 assertions) and nothing else.

Known limitations

• Accuracy is indicative only — one corpus per language, verified: false until a full sweep.
• Two liveness gaps remain unfixed and are filed as apple-speech: collector subscription race and unbounded wait on results #137: the collector may not be subscribed before audio is fed (latent race, not reproduced in any probe), and collector.value has no upper bound.
• Router warnings never reach the user without --explain #136 (router warnings invisible without --explain) is pre-existing and is what made defect 7 above silent.
• peak-GB is not comparable across in-process and out-of-process backends #138 — peak-GB is not comparable for out-of-process backends.

[kiki830621]

Post-fix verification (real binary, not the test suite)

The zh-Hans fix, end-to-end

Same audio, same backend, only the tag differs. Before the fix both returned Traditional, because Hans was read as a region, matched nothing, and fell through to the zh preference:

$ bestasr transcribe cv-zhtw-2.wav --backend apple-speech --language zh-Hans
你给我听好了，以后只有我才有资格让你流泪。…五谷泰山轻轨…法国拿破仑三世。

$ bestasr transcribe cv-zhtw-2.wav --backend apple-speech --language zh-Hant
你給我聽好了，以後隻有我才有資格讓你流淚。…五穀泰山輕軌…法國拿破侖三世。

给/給, 听/聽, 资/資, 轻轨/輕軌, 仑/侖 — the two scripts now diverge exactly as requested.

Incidentally the Simplified model gets 只有 right where the Traditional one writes 隻有, which is a nice reminder that the region/script choice is not cosmetic for measured CER.

Three languages on corpora not used during development

Re-run after all six verify-round fixes, on fresh corpora, to confirm nothing regressed through them:

corpus	metric	result
cv-zhtw-3 (zh)	CER	19.7 % @ 70× realtime
fleurs-ja-2 (ja)	CER	10.9 % @ 103× realtime
librispeech-testclean-2 (en)	WER	3.7 % @ 67× realtime

Metric selection still resolves correctly (CER for zh/ja, WER for en) after the change that records the resolved locale rather than the requested tag — the specific regression risk there was that Apple's underscored zh_TW would defeat LanguageResolver.baseSubtag.

Note cv-zhtw-3 at 19.7 % is well above cv-zhtw-2's 13.9 %: per-corpus variance is large at this sample size, which is why the row stays verified: false until a proper sweep.

CI

Green (2m50s). Worth recording why, since it was in question: the workflow runs xcode-select -s "$NEWEST" and the macos-15 runner carries Xcode 26 (log shows Swift 6.2.4). The build floor genuinely rose to the macOS 26 SDK — @available gates the runtime, not the type checker — so that requirement is now stated in README Requirements rather than left implicit.

[kiki830621]

Requirements audit against #121

The issue deferred five decisions to diagnose. Status of each:

#	Decision	Outcome
1	Which API?	New SpeechAnalyzer + SpeechTranscriber only — the issue's stated preference. The old SFSpeechRecognizer is deliberately not wired: it defaults to requiresOnDeviceRecognition = false, so measuring it would put Apple-server numbers in a local-ASR ranking. Verified against the SDK: Speech.swiftinterface has zero hits for onDevice|server|cloud|network|remote, so the new API has no such path to guard.
2	Deployment-target conflict	@available(macOS 26, *) + isAvailable() false below — exactly the issue's preferred option, and it fits the Engine contract that availability detection is graceful. Consequence the issue did not anticipate: @available gates the runtime, not the type checker, so the package now needs Xcode 26 / the macOS 26 SDK to build while still running on macOS 14. Now stated in README Requirements; CI is green only because the workflow selects the newest Xcode (log shows Swift 6.2.4 = Xcode 26).
3	Are en/ja/zh all supported?	Yes, measured, not assumed. supportedLocales → 45 locales covering all three; each transcribed end-to-end and benchmarked.
4	How to express the grid row	speechanalyzer / system / default, each choice justified in-comment. languages is the 25 probed base subtags rather than "multi" (#105).
5	Where does the asset download count?	Satisfied by existing design, verified not assumed. BenchmarkRunner.swift:239-242 performs a warm-up transcribe before the timed run, so installAssetIfNeeded lands in warm-up and is excluded from RTF — the same treatment WhisperKit's model download gets, so cold-start remains comparable. Measured warmup_seconds of 0.58–1.4 s on this machine reflects load-only, the assets already being installed.

One thing the issue asked for that is not visible yet

The issue expects decode_deterministic to record an honest value for a backend that cannot consume the flag. Inspecting the 8 apple-speech rows now in the local store:

model_id:             'apple-speech|speechanalyzer|system|default'
decode_deterministic: <absent>

Absent, not flag-not-consumed — and that is a merge-order artifact, not a defect in this PR. main still carries the pre-#118 inline whitelist (CommandCore.swift:598-600), which writes nil for every non-whisper backend. #130 replaces it with DecodeDeterminism.forBackend, whose flagConsumingBackends is [whisperKit, whisperCpp] — apple-speech is absent from it, so it resolves to .flagNotConsumed automatically with no further change needed here.

Checked for interaction: #130 and #135 both touch DataModels.swift, CommandCore.swift and CHANGELOG.md, but git merge-tree produces no conflict markers in either order.

Verification legs actually run

Two, not the three planned. The cross-model (Codex, gpt-5.6-sol xhigh) leg found six defects — four of the plausible-but-wrong class — all fixed and covered. My own independent pass found the Router substitution and the baseSubtag underscore trap. The three-lens leg failed operationally: the agents completed but never returned their reports across two requests, so their findings are simply missing rather than clean. This requirements audit was done by hand to replace one of them; correctness and security were not re-run independently beyond Codex and my own reading.

Codex was also wrong twice and was not taken at face value: its case-sensitivity concerns (en-GB, pt-BR) do not reproduce, because LanguageResolver.resolve already lowercases, and its "no tests were added" finding was an artifact of my sending it only the Sources/ diff.