Refactor into modular package with pluggable backends - #2
Merged
Conversation
…, benchmarks) Evolve the single-file proof-of-concept (preserved at the nano tag) into a small modular package: - Pluggable STT backends as extras: parakeet (default) and whisper. MLX stays an explicit extra so the install is Mac-first and you pick one engine. - Default backend is Parakeet (parakeet-tdt-0.6b-v3) — far faster than Whisper on Apple Silicon; Whisper remains the fallback. - Cleanup upgraded to qwen3:4b-instruct-2507 (non-thinking); strip stray <think>. - Config at ~/.v2t/config.toml (honors V2T_HOME / XDG_CONFIG_HOME) + JSONL history. - SwiftBar plugin: menu-bar toggle, status, open config/history. - v2t bench: per-machine markdown grid (STT RTF + cleanup TTFT/total), self-contained audio via macOS say. - New commands: status, stop, config, bench. Cleanup streams the Ollama HTTP API. https://claude.ai/code/session_01XTW6U37PNqStHyZ9BZEM6r
- Pluggable cleanup engines: mlx (mlx-lm, in-process, default — no Ollama daemon) and ollama (optional, for those who already run it). Drops the <think> strip; use a non-thinking model. Default mlx model: Qwen3-4B-Instruct-2507-4bit. - mlx-lm bundled into each MLX extra so the default cleanup just works. - v2t setup: guided config — pick transcription model + cleanup engine, detect Ollama and offer it, else default to mlx-lm. Writes ~/.v2t/config.toml. - bench cleanup uses engine:model specs, resilient (missing model -> n/a). - --cleanup-engine flag; history logs cleanup engine + model. - Docs (README/CHANGELOG/benchmarks/GRID) updated for mlx-lm-default cleanup. https://claude.ai/code/session_01XTW6U37PNqStHyZ9BZEM6r
This was referenced Jun 16, 2026
Make 'uv tool install voice2text' work out of the box on Apple Silicon — parakeet-mlx (STT) and mlx-lm (cleanup) move into core dependencies. Whisper becomes an optional alternative STT backend (voice2text[whisper]). Drops the PEP 771 'default extras' caveat (no longer relevant). Mac-first by design. https://claude.ai/code/session_01XTW6U37PNqStHyZ9BZEM6r
- bench_stt warms each clip once before the timed runs (MLX compiles per input shape; the app warms up too), so cells reflect steady-state, not cold-compile. - Add benchmarks/results/2026-06-16-Apple-M1-Pro.md (real numbers). - Fill GRID.md M1 Pro rows + a Transcript quality section: local say-sample WER (~0-2%, sanity only) plus published Open ASR Leaderboard WER (v2 ~6.0%, v3 ~6.3%, whisper-turbo ~7.5-8%). Validates Parakeet v3 default (fastest + ~6.3%). https://claude.ai/code/session_01XTW6U37PNqStHyZ9BZEM6r
Design tenet: an ergonomic tool is communicative — every action gets immediate visible feedback. - App writes live state (starting/idle/recording/transcribing/cleaning) to status.json on each transition and pushes an instant SwiftBar repaint (swiftbar://refreshplugin). Fixes 'Start does nothing' — the icon now flips to 'starting' the moment v2t launches instead of staying dead for the ~37s warmup. - Plugin colors the menu-bar icon per state (🟠/🟢/🔴/🟡) and shows what it's doing. - Single-instance guard (no more accidental double-launch); non-tty permission guard so a headless/SwiftBar start can't hang on input(). - Consolidate liveness onto config.read_status(); drop the dead v2t.pid file. - ponytail: drop the Ollama HTTP->subprocess fallback chain (URLError now flows to the app's existing raw-text fallback); boring md_stt_table header; -unused imports. https://claude.ai/code/session_01XTW6U37PNqStHyZ9BZEM6r
From the first real record->paste run: - 'Text went nowhere' was dead-silent audio = missing Microphone permission for the launching app. Detect peak<1e-4 and say so + open the Mic pane (once), instead of failing silently. - 'Open transcription history' opened nothing because ~/.v2t/history didn't exist yet — create run/ + history/ eagerly (app startup + plugin). - Add a Permissions submenu (Microphone / Accessibility / Input Monitoring) that opens each System Settings pane — grant, then restart the launching app. - Show the active models in the menu (parakeet-v3 · clean: Qwen3-4B). v2t status now reports state + STT + cleanup + mode (settings from config, not status.json). - Move short_model to backends so 'v2t status' stays light (no audio import). https://claude.ai/code/session_01XTW6U37PNqStHyZ9BZEM6r
This reverts commit 121463f.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Transforms v2t from a single 291-line script into a small, modular Python package with pluggable speech-to-text backends, centralized configuration, and a benchmark harness. The original proof-of-concept is preserved at the
nanotag.Key Changes
Modular architecture: Split monolithic
voice2text.pyinto:v2t/app.py— core push-to-talk engine (recording, transcription, cleanup, paste)v2t/backends.py— pluggable STT (Parakeet MLX + Whisper) and Ollama cleanupv2t/config.py— unified config file (~/.v2t/config.toml) + history trackingv2t/cli.py— command dispatcher (v2t,v2t bench,v2t config,v2t status,v2t stop)v2t/__init__.pyandv2t/__main__.py— package entry pointsParakeet as default backend: Replaces Whisper with
mlx-community/parakeet-tdt-0.6b-v3(~10× faster on Apple Silicon, multilingual). Whisper remains available as an alternative backend via extras.Pluggable backends:
STTdict allows easy addition of new speech-to-text engines; both backends share a common interface (transcribe(wav_path) -> str).Configuration system:
$V2T_HOME,$XDG_CONFIG_HOME/v2t, or~/.v2t--casual,--no-cleanup) merge with file configHistory tracking: Every transcription logged to
~/.v2t/history/transcriptions.jsonlwith metadata (backend, model, mode, timings, raw/clean text).Benchmark harness (
v2t bench):sayfor self-contained STT benchmarkingbenchmarks/results/<date>-<host>.mdfor cross-machine comparisonSwiftBar plugin (
swiftbar/v2t.5s.sh):Status file for plugins:
~/.v2t/run/status.json+v2t.pidfor external monitoring (SwiftBar, etc.).Lazy imports: Heavy MLX/audio dependencies only loaded in
app.py, sov2t configandv2t bench --cleanupwork without GPU models.Improved permissions flow: Clearer messaging about which terminal app needs Accessibility + Input Monitoring grants.
Notable Implementation Details
_resolve_hotkey()inapp.py; supports cmd_r/cmd_l, alt_r/alt_l, ctrl_r/ctrl_l./api/generateendpoint to measure time-to-first-token; falls back toollama runif server isn't up.<think>blocks stripped from LLM output so reasoning models don't leak internal thoughts.nowplaying-cli; gracefully disabled if not installed.[parakeet](https://claude.ai/code/session_01XTW6U37PNqStHyZ9BZEM6r