Switch to mlx-whisper for Apple Silicon acceleration

[goyamegh]

Summary

• Switch from faster-whisper to mlx-whisper for native Apple Silicon (MLX/Metal) GPU inference — 2-4x faster transcription on M-series Macs
• Eliminate temp WAV file I/O — audio buffer is passed directly as a numpy array, saving ~100-200ms per transcription
• Add chunked transcription — audio is transcribed every 5s during recording, so text appears near-instantly when recording stops (0ms transcription wait when cached)
• Add pipeline metrics logging — tracks buffer conversion, transcription time, RTF (real-time factor), text insertion, and total pipeline latency
• Fix stray character on text insertion — adds a small delay before typing to let key events (Globe/Fn) fully propagate

Performance Results

Recording Length	Transcription Wait	Total Pipeline	Path
2.75s	499ms	919ms	Full (short recording)
5.70s	0ms (cached)	472ms	Chunked — used partial
9.22s	606ms	1044ms	Chunked + tail
10.24s	575ms	969ms	Chunked + tail

Test plan

• Verify app starts and model loads successfully on Apple Silicon Mac
• Test short recordings (<5s) — should use full transcription path
• Test longer recordings (>5s) — should show chunk transcription in logs and faster final result
• Verify text selection + Bedrock enhancement flow still works
• Verify no stray characters prepended to transcription output
• Check [METRICS] log lines for latency tracking

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR switches the dictation backend from faster-whisper to mlx-whisper to leverage Apple Silicon (MLX/Metal) acceleration, removes temp WAV file I/O by transcribing directly from a NumPy buffer, and introduces periodic chunked transcription plus detailed pipeline metrics logging.

Changes:

• Replace faster-whisper file-based transcription with mlx-whisper NumPy-array transcription (incl. model warm-up).
• Add a chunked transcription loop during recording to cache partial results and reduce final wait time.
• Add [METRICS] logging for model load, buffer conversion, transcription, enhancement, insertion, and total pipeline latency.

Reviewed changes

Copilot reviewed 1 out of 2 changed files in this pull request and generated 5 comments.

File	Description
src/main.py	Switches transcription to mlx_whisper, adds chunked transcription + metrics, and adjusts insertion timing.
requirements.txt	Replaces faster-whisper dependency with mlx-whisper.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

cleanup() stops the recording thread but does not signal/join the chunked transcription thread. If the app exits while recording, chunk_thread can keep running and prevent a clean shutdown (or force the watchdog to os._exit). Consider setting chunked_stop in cleanup() and joining chunk_thread (with a bounded timeout), similar to stop_recording()/discard_recording().

Suggested change

	# Stop chunked transcription thread if it is running
	if hasattr(self, 'chunked_stop'):
	self.chunked_stop.set()
	if hasattr(self, 'chunk_thread') and self.chunk_thread.is_alive():
	self.chunk_thread.join(timeout=1.0)

[Copilot]

discard_recording() joins recording_thread with a timeout and then immediately reassigns self.frames = []. If recording_thread is still alive, it can append to the new list after discard, leaving unexpected audio buffered and potentially confusing later transcriptions. Consider waiting until recording_thread fully exits (or guarding frame appends with a lock / local reference) before resetting frames.

[Copilot]

The code now imports/depends on mlx_whisper, which (per MLX) only works on Apple Silicon. On Intel macOS this will fail at import time and prevent the app from starting. Consider adding an explicit runtime architecture check with a clear status/error message (and/or a fallback backend) so users get an actionable explanation instead of a crash.

[Copilot]

stop_recording() joins chunk_thread with a 5s timeout but doesn’t verify that it actually stopped before starting process_recording(). If chunked_transcribe_loop is in the middle of a long mlx_whisper.transcribe call, it may keep running concurrently with the final transcription, causing concurrent model executions and races on partial_text/last_transcribed_frame_count. Consider joining without timeout, or (if you must bound it) checking is_alive() and skipping cached/partial usage until the thread has fully exited.

Suggested change

	# Stop chunked transcription and wait for it to finish
	self.chunked_stop.set()
	if hasattr(self, 'chunk_thread'):
	self.chunk_thread.join(timeout=5.0)
	# Stop chunked transcription and wait for it to fully finish
	# before starting the final transcription pass. A timed join can
	# return while the thread is still inside a long-running transcribe
	# call, which would allow concurrent model execution and races on
	# shared transcription state.
	self.chunked_stop.set()
	if hasattr(self, 'chunk_thread'):
	self.chunk_thread.join()

[Copilot]

discard_recording() joins chunk_thread with a 2s timeout and then clears frames/UI state. If chunked_transcribe_loop is still running (e.g., stuck in mlx_whisper.transcribe), it can continue consuming resources and mutate partial_text/last_transcribed_frame_count after the recording is discarded. Consider ensuring the thread has actually exited (or making the transcription call cancellable / running it in a separate process) before resetting state.