Run PocketSphinx Transcription with Sapat

authors/joel_cf.md

@@ -0,0 +1,10 @@
+Author: Joel CF
+Title: Software Engineer
+Description: Joel builds practical developer tooling and AI-assisted workflows, with a focus on reproducible environments, automation, and careful validation. He enjoys turning rough operational problems into clear, testable systems that other developers can run and improve.
+Author Image: <https://avatars.githubusercontent.com/joelcf001?v=4>
+Author LinkedIn:
+Author Twitter:
+Company Name: Independent
+Company Description: Independent software engineering and automation work.
+Company Logo Dark:
+Company Logo White:

definitions/20260529_definition_offline_speech_recognition.md

@@ -0,0 +1,23 @@
+---
+title: "Offline Speech Recognition"
+description: "Speech-to-text processing that runs locally without sending audio to an external API."
+date: 2026-05-29
+author: "Joel CF"
+---
+
+# Offline Speech Recognition
+
+## Definition
+
+Offline speech recognition is speech-to-text processing that runs on local
+compute instead of sending audio to an external API. The recognizer, language
+model, and any custom dictionaries are available inside the user's machine,
+container, or development workspace.
+
+## Context and Usage
+
+Offline speech recognition is useful when teams need privacy, repeatability, or
+low-friction testing before they connect a workflow to a hosted transcription
+service. In a Daytona workspace, it can help developers verify audio
+conversion, provider routing, transcript output, and batch behavior without API
+keys or network-dependent inference.

guides/20260529_run_pocketsphinx_transcription_with_sapat_in_daytona.md

@@ -0,0 +1,309 @@
+---
+title: "Run PocketSphinx Transcription with Sapat"
+description: "Build a local, no-key speech-to-text workflow in a Daytona workspace with Sapat and PocketSphinx."
+date: 2026-05-29
+author: "Joel CF"
+tags: ["sapat", "pocketsphinx", "daytona", "speech-to-text"]
+---
+
+# Run PocketSphinx Transcription with Sapat
+
+## Introduction
+
+Cloud speech-to-text services are powerful, but they are not always the best
+first step for a reproducible transcription workflow. An AI engineer may need
+to test a pipeline before API credentials exist, process internal recordings
+without sending audio to a third party, or debug a failing audio conversion
+step in a clean environment. In those cases, an
+[offline speech recognition](../definitions/20260529_definition_offline_speech_recognition.md)
+provider gives you a useful baseline.
+
+This guide walks through running Sapat with a PocketSphinx provider inside a
+Daytona workspace. Sapat handles the file workflow: convert media with FFmpeg,
+send the normalized WAV file to the selected provider, and write a `.txt`
+transcript next to the source file. PocketSphinx handles local recognition with
+no API key, no account setup, and no network call during transcription. The
+result is not meant to replace a high-accuracy hosted model for every task, but
+it is very useful for private drafts, smoke tests, and repeatable development
+checks.
+
+## TL;DR
+
+- Use Daytona to create a clean Sapat workspace that another engineer can
+  reproduce.
+- Install Sapat with the optional `pocketsphinx` extra.
+- Run `sapat --provider pocketsphinx` on a local audio or video file.
+- Keep custom acoustic, language, dictionary, or keyword files in environment
+  variables instead of hard-coding paths.
+- Validate the transcript and original media file before moving the workflow to
+  a production provider.
+
+## Why PocketSphinx Fits This Workflow
+
+PocketSphinx is a small, local speech recognizer from the CMU Sphinx family. It
+is useful when the main requirement is offline execution, deterministic setup,
+or a cheap development loop. The Python package exposes an `AudioFile` helper
+for simple file recognition and also supports custom model paths for teams that
+maintain their own acoustic model, language model, dictionary, or keyword list.
+
+That makes it a good Sapat provider for three common cases:
+
+- **Private first pass**: Create a rough transcript for internal recordings
+  before deciding whether a cloud pass is appropriate.
+- **Provider smoke test**: Verify Sapat's media conversion and output writing
+  without requiring OpenAI, Groq, Azure, or another hosted API.
+- **Keyword-focused review**: Use a keyword list or keyphrase settings to check
+  whether important terms appear in meeting, support, or demo recordings.
+
+The tradeoff is accuracy. PocketSphinx is lightweight and local, so it will not
+match modern large speech models on noisy, multilingual, or long-form audio.
+Treat the output as a baseline transcript that should be reviewed before it is
+shared.
+
+![PocketSphinx Sapat workflow](assets/20260529_run_pocketsphinx_transcription_with_sapat_in_daytona_workflow.svg)
+
+## Step 1: Create a Daytona Workspace
+
+Start from the Sapat repository so the commands and dependencies stay close to
+the project under test.
+
+```bash
+daytona create https://github.com/nibzard/sapat --code
+```
+
+Open the workspace in your preferred IDE. Daytona gives you a fresh shell,
+isolated filesystem, and a reproducible place to install optional dependencies
+without changing your laptop's global Python environment.
+
+Inside the workspace, confirm the repository is available:
+
+```bash
+pwd
+python --version
+```
+
+If `python` is not available but `python3` is, use `python3` in the commands
+below.
+
+## Step 2: Prepare FFmpeg and Python
+
+Sapat relies on FFmpeg to normalize media files before transcription. The
+PocketSphinx provider expects WAV input, so Sapat converts videos, MP3 files, or
+other supported audio into a 16 kHz mono WAV before recognition.
+
+Check whether FFmpeg is already installed:
+
+```bash
+ffmpeg -version
+```
+
+If the command is missing in a Debian or Ubuntu based workspace, install it:
+
+```bash
+sudo apt-get update
+sudo apt-get install -y ffmpeg
+```
+
+Then create a local virtual environment for the Sapat install:
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+python -m pip install --upgrade pip
+```
+
+Install Sapat with the PocketSphinx optional dependency:
+
+```bash
+python -m pip install -e '.[pocketsphinx]'
+```
+
+This installs the Sapat CLI plus the PocketSphinx Python bindings. It does not
+require an API token or cloud account.
+
+## Step 3: Check Provider Discovery
+
+Sapat discovers providers dynamically. If the optional dependency installed
+correctly, the `pocketsphinx` provider can register without any environment
+variables.
+
+Run:
+
+```bash
+sapat --help
+```
+
+The provider flag is dynamic, so the help output describes `--provider` as a
+text option rather than a hard-coded list. A practical way to verify the
+provider is to run a tiny transcription job, which you will do in the next
+step.
+
+If your team uses custom PocketSphinx assets, keep them in local environment
+variables:
+
+```bash
+export POCKETSPHINX_HMM=/workspace/models/en-us
+export POCKETSPHINX_LM=/workspace/models/domain.lm
+export POCKETSPHINX_DICT=/workspace/models/domain.dict
+```
+
+For keyword spotting, use:
+
+```bash
+export POCKETSPHINX_KWS=/workspace/models/keywords.list
+export POCKETSPHINX_KWS_THRESHOLD=1e-20
+```
+
+These variables keep model locations out of source control and make the setup
+easy to reproduce in another Daytona workspace.
+
+## Step 4: Add a Test Recording
+
+Use a short file first. A ten to thirty second clip is enough to prove the
+workflow before you run a long meeting or lecture recording.
+
+Create a workspace folder for local input files:
+
+```bash
+mkdir -p samples
+```
+
+Then add a video or audio file to `samples/`. If you already have a file named
+`demo.mp4`, place it there and run:
+
+```bash
+sapat samples/demo.mp4 --provider pocketsphinx --language en --quality L
+```
+
+Sapat will create `samples/demo.wav` as a temporary normalized file, pass it to
+PocketSphinx, write `samples/demo.txt`, and remove only the temporary WAV that
+it created.
+
+If your source file is already a WAV file, run:
+
+```bash
+sapat samples/demo.wav --provider pocketsphinx --language en
+```
+
+In this path, Sapat should preserve the original WAV file and write
+`samples/demo.txt`. That behavior matters for offline workflows because WAV is
+often the source format used by speech teams.
+
+## Step 5: Review the Output
+
+Open the transcript:
+
+```bash
+sed -n '1,120p' samples/demo.txt
+```
+
+For a local recognizer, the first review should answer four questions:
+
+- Did Sapat produce a transcript file next to the source recording?
+- Did the original media file remain in place?
+- Is the transcript good enough for indexing, triage, or smoke testing?
+- Do domain-specific words need a custom language model, dictionary, or keyword
+  list?
+
+PocketSphinx is strongest when the audio is clean, speech is close to the
+default model, and the vocabulary is expected. If the transcript misses product
+names, acronyms, or commands, try a custom dictionary or keyword list before
+assuming the workflow is broken.
+
+## Step 6: Batch a Small Folder
+
+After a single file works, try a small directory. Sapat processes `.mp4` files
+in a directory, so a simple batch test looks like this:
+
+```bash
+mkdir -p samples/batch
+cp samples/demo.mp4 samples/batch/demo-1.mp4
+cp samples/demo.mp4 samples/batch/demo-2.mp4
+sapat samples/batch --provider pocketsphinx --language en --quality L
+```
+
+You should see one `.txt` file per input video. This makes PocketSphinx useful
+as a quick check in a larger media workflow: if batch conversion, provider
+routing, and transcript writing all work locally, you can move the same folder
+through a hosted provider later with fewer unknowns.
+
+## Step 7: Decide Whether to Promote the Workflow
+
+PocketSphinx is best treated as the first gate in a transcription pipeline. It
+answers a simple operational question: can this workspace accept media,
+normalize it, route it through a provider, and produce reviewable text without
+external credentials? Once that is true, decide whether the transcript quality
+is enough for the job.
+
+For internal indexing, search, or rough meeting triage, a local transcript may
+be sufficient. Store the `.txt` file with the recording, add a short human
+summary, and keep the original media unchanged for later review. For public
+documentation, customer-facing summaries, subtitles, or compliance records,
+route the same source file through a higher-accuracy provider after the local
+smoke test passes.
+
+Use this promotion checklist before switching providers:
+
+- **Input quality**: The sample is audible, not clipped, and close to the
+  language you expect.
+- **File handling**: Sapat writes the `.txt` file and preserves original media
+  when the source already matches the provider's preferred format.
+- **Vocabulary**: Product names, commands, and acronyms are handled by a custom
+  dictionary, keyword list, or later cloud pass.
+- **Repeatability**: Another engineer can recreate the Daytona workspace and
+  rerun the same command without hidden local state.
+- **Privacy**: The team has explicitly decided when audio can leave the
+  workspace.
+
+This split keeps the low-risk local loop useful even when the final transcript
+needs a stronger recognizer.
+
+## Troubleshooting
+
+**Problem:** `No providers available.`
+
+**Solution:** Activate the virtual environment and reinstall the optional
+dependency:
+
+```bash
+source .venv/bin/activate
+python -m pip install -e '.[pocketsphinx]'
+```
+
+**Problem:** FFmpeg conversion fails.
+
+**Solution:** Run `ffmpeg -version` in the Daytona workspace. If it is missing,
+install it with the package manager for your workspace image.
+
+**Problem:** The transcript is empty or too rough to use.
+
+**Solution:** Start with a shorter, cleaner English recording. Then add a
+custom language model, dictionary, keyphrase, or keyword list through
+`POCKETSPHINX_*` environment variables.
+
+**Problem:** A custom model path fails.
+
+**Solution:** Use absolute paths inside the workspace and confirm each file or
+directory exists:
+
+```bash
+ls -la "$POCKETSPHINX_HMM"
+ls -la "$POCKETSPHINX_LM"
+ls -la "$POCKETSPHINX_DICT"
+```
+
+## Conclusion
+
+You now have a local Sapat transcription path that runs inside a Daytona
+workspace without API keys. PocketSphinx is a pragmatic provider for private
+drafts, repeatable smoke tests, keyword checks, and early development loops. It
+also gives AI engineers a safe baseline: verify conversion and file handling
+locally, then switch to a hosted provider only when higher accuracy is worth the
+extra setup.
+
+## References
+
+- [Sapat repository](https://github.com/nibzard/sapat)
+- [PocketSphinx Python documentation](https://pocketsphinx.readthedocs.io/en/stable/pocketsphinx.html)
+- [PocketSphinx package on PyPI](https://pypi.org/project/pocketsphinx/)
+- [Daytona documentation](https://www.daytona.io/docs/)