diff --git a/.github/ISSUE_TEMPLATE/voice_pipeline.yml b/.github/ISSUE_TEMPLATE/voice_pipeline.yml index 752a5626..097acabb 100644 --- a/.github/ISSUE_TEMPLATE/voice_pipeline.yml +++ b/.github/ISSUE_TEMPLATE/voice_pipeline.yml @@ -85,7 +85,7 @@ body: id: llm_backend attributes: label: LLM backend + model - placeholder: "genie-ai-runtime / phi-4-mini-instruct-q4_k_m.gguf" + placeholder: "genie-ai-runtime / Qwen3-4B-Q4_K_M.gguf" validations: required: false diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 01768a53..35c39816 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -8,7 +8,7 @@ This repository should be understood as the Rust agent runtime that sits above: - GenieOS, the custom L4T and system image layer - `genie-voice-runtime`, the external voice runtime for wake/VAD/STT/TTS/audio - `genie-home-runtime`, the future AI-native home automation runtime -- `genie-ai-runtime`, the future Jetson-only LLM inference runtime +- `genie-ai-runtime`, the external Jetson-first LLM inference runtime It sits below: @@ -89,9 +89,9 @@ This repo can keep transitional implementations while those layers are still for The current repo still contains pragmatic adapters used to ship on Jetson now. -| Current adapter | Long-term replacement | Notes | +| Current adapter | Target boundary | Notes | | --- | --- | --- | -| `genie-ai-runtime` OpenAI-compatible client (default on Jetson) | `llama.cpp` client (selectable fallback) | Both backends ship behind the `LlmClient` facade; per-deployment selection via `[services.llm].backend` in `geniepod.toml`. Backend identity surfaces in `/api/health`, startup logs, and `genie-ctl status`. | +| `genie-ai-runtime` OpenAI-compatible client (default on Jetson) | external `genie-ai-runtime` service | `llama.cpp` remains a selectable fallback/development backend. Both backends ship behind the `LlmClient` facade; per-deployment selection is via `[services.llm].backend` in `geniepod.toml`. Backend identity surfaces in `/api/health`, startup logs, and `genie-ctl status`. | | In-repo voice pipeline under `crates/genie-core/src/voice/` and `voice_loop.rs` | [`genie-voice-runtime`](https://github.com/GeniePod/genie-voice-runtime) | Keep current code as a transitional Jetson bring-up path. New wake/VAD/STT/TTS/audio ownership should move to the external runtime. GenieClaw should consume transcripts and issue speak commands. | | Home Assistant provider | `genie-home-runtime` MCP/API client | Keep HA-specific behavior behind `ha/` and tools/home boundaries. | | Actuation safety in `genie-core` | final safety in `genie-home-runtime` | Keep current safety as an agent-side guard and confirmation layer. | @@ -281,7 +281,7 @@ The clean architecture path is incremental: 1. Make boundary language consistent in docs and config. 2. Keep Home Assistant and LLM backends behind narrow adapter traits (LLM side resolved via the `LlmClient` facade in `crates/genie-core/src/llm/`). 3. Move physical actuation authority downward into `genie-home-runtime` when it exists. -4. Move Jetson model-server specialization downward into `genie-ai-runtime`. +4. Keep Jetson model-server specialization in `genie-ai-runtime`. 5. Move voice/audio pipeline ownership downward into `genie-voice-runtime`. 6. Keep GenieClaw focused on agent policy, memory, skills, tools, channels, and household interaction. diff --git a/CODEBASE.md b/CODEBASE.md index b02fde34..9b508420 100644 --- a/CODEBASE.md +++ b/CODEBASE.md @@ -43,7 +43,9 @@ The most important runtime path is: 1. A request enters through the web UI, CLI, REPL, voice loop, or Telegram adapter. 2. `genie-core` builds prompt context from conversation history and household memory. -3. The LLM client talks to the local `llama.cpp`-compatible server. +3. The LLM facade talks to the configured OpenAI-compatible backend. Jetson + deploys default to `genie-ai-runtime`; development configs can still use a + local `llama.cpp` server. 4. If the model emits a tool call, the tool parser and dispatcher execute the tool. 5. Tool results may be returned directly or summarized, depending on the tool. 6. Conversation state and extracted memory are persisted to SQLite. @@ -62,7 +64,7 @@ Around that path: | `Cargo.lock` | Locked dependency graph. | | `Makefile` | Main developer and Jetson deploy entry points: build, test, release, cross-compile, and deploy. | | `Dockerfile` | Multi-stage container build for the local dev/runtime image. | -| `docker-compose.dev.yml` | Dev stack for `genie-core`, `genie-api`, and a local `llama.cpp` server. | +| `docker-compose.dev.yml` | Dev stack for `genie-core`, `genie-api`, and a local OpenAI-compatible model server. | | `README.md` | Product-level overview and repo orientation. | | `GETTING_STARTED.md` | Local dev, Docker, and Jetson bring-up guide. | | `ARCHITECTURE.md` | Higher-level system architecture narrative. | @@ -110,8 +112,11 @@ The primary runtime. This is where most product behavior lives. | Path | Purpose | | --- | --- | | `crates/genie-core/src/llm/mod.rs` | LLM module exports. | -| `crates/genie-core/src/llm/client.rs` | Raw TCP/OpenAI-compatible chat client for the local inference server, including streaming and compatibility fallbacks. | -| `crates/genie-core/src/llm/retry.rs` | Retry and graceful fallback layer around the base client. | +| `crates/genie-core/src/llm/openai_compat.rs` | Raw bounded OpenAI-compatible HTTP client used by local and optional provider backends. | +| `crates/genie-core/src/llm/genie_ai_runtime.rs` | Adapter for the default Jetson `genie-ai-runtime` backend and its request hints. | +| `crates/genie-core/src/llm/llama_cpp.rs` | Adapter for the legacy/development `llama.cpp` backend. | +| `crates/genie-core/src/llm/openai_compatible.rs` | Generic OpenAI-compatible provider adapter with bearer-token support. | +| `crates/genie-core/src/llm/provider.rs` | Optional provider planning and limited-context readiness checks. | #### Home Assistant Integration diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index 2aebe981..1cad6811 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -1,238 +1,148 @@ -# Getting Started with genie-core +# Getting Started With GenieClaw -Step-by-step guide from zero to a working local GeniePod Home runtime. +This guide covers the current development and Jetson bring-up paths. The +production Jetson default is `genie-ai-runtime`; the development config still +uses any local OpenAI-compatible `llama.cpp` server on `:8080`. ---- +## Option A: Development Machine -## Option A: Quick demo on your dev machine (no Jetson needed) +Prerequisites: -### Prerequisites +- Rust toolchain +- 4 GB or more free RAM for a small local test model +- an OpenAI-compatible local model server on `http://127.0.0.1:8080` -- Rust 1.75+ (`curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`) -- 4 GB free RAM (for the LLM model) - -### 1. Clone and build +Build and test: ```bash git clone https://github.com/GeniePod/genie-claw.git cd genie-claw -make test # 45 tests should pass -make release # builds optimized binaries +make test +make release ``` -### 2. Download a model - -Any GGUF model works. TinyLlama is small enough for testing: +Start a local OpenAI-compatible backend. For example, with `llama.cpp`: ```bash mkdir -p models -wget -O models/tinyllama.gguf \ - "https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf" -``` - -For better quality (needs ~3 GB RAM): - -```bash -wget -O models/nemotron-4b.gguf \ - "https://huggingface.co/nvidia/Nemotron-Mini-4B-Instruct-GGUF/resolve/main/Nemotron-Mini-4B-Instruct-Q4_K_M.gguf" -``` - -### 3. Start llama.cpp server +# Put any small GGUF test model under ./models. -Download llama.cpp or use Docker: - -```bash -# Option A: Docker (easiest) -docker run -p 8080:8080 -v $(pwd)/models:/models \ +docker run --rm -p 8080:8080 -v "$(pwd)/models:/models" \ ghcr.io/ggml-org/llama.cpp:server \ - --model /models/tinyllama.gguf --host 0.0.0.0 --port 8080 --ctx-size 2048 - -# Option B: Build from source -git clone https://github.com/ggml-org/llama.cpp.git -cd llama.cpp && cmake -B build && cmake --build build -j -./build/bin/llama-server --model ../models/tinyllama.gguf --host 127.0.0.1 --port 8080 + --model /models/your-model.gguf \ + --host 0.0.0.0 \ + --port 8080 \ + --ctx-size 4096 ``` -Verify it's running: +Verify the backend: ```bash -curl http://127.0.0.1:8080/health -# Should return: {"status":"ok"} +curl -sf http://127.0.0.1:8080/health && echo ``` -### 4. Start genie-core +Run the core and dashboard with the dev config: ```bash -cd genie-claw GENIEPOD_CONFIG=deploy/config/geniepod.dev.toml cargo run --release --bin genie-core +GENIEPOD_CONFIG=deploy/config/geniepod.dev.toml cargo run --release --bin genie-api ``` -You should see: - -``` -INFO GeniePod core starting -INFO memory loaded, memories=0 -INFO conversation store loaded, conversations=0 -INFO genie-core HTTP server listening, addr=127.0.0.1:3000 -``` - -### 5. Chat! +Open: -**Browser:** Open http://localhost:3000 +- Chat UI: `http://127.0.0.1:3000` +- Dashboard: `http://127.0.0.1:3080` -**CLI:** +CLI examples: ```bash -# In another terminal: +cargo run --release --bin genie-ctl -- status cargo run --release --bin genie-ctl -- chat "what time is it" -cargo run --release --bin genie-ctl -- chat "what is the weather in Denver" -cargo run --release --bin genie-ctl -- chat "set a timer for 5 minutes" cargo run --release --bin genie-ctl -- search --limit 3 "ESP32-C6 Thread support" cargo run --release --bin genie-ctl -- tools cargo run --release --bin genie-ctl -- health ``` -**curl:** +## Option B: Docker Compose -```bash -curl -X POST http://127.0.0.1:3000/api/chat \ - -H "Content-Type: application/json" \ - -d '{"message": "hello, what can you do?"}' -``` - -### 6. (Optional) Start the system dashboard +Use this path for a quick local service bring-up without installing every Rust +binary manually: ```bash -GENIEPOD_CONFIG=deploy/config/geniepod.dev.toml cargo run --release --bin genie-api -# Open http://localhost:3080 -``` - ---- - -## Option B: Docker compose (no Rust needed) - -```bash -cd genie-claw - -# Download a model mkdir -p models -wget -O models/tinyllama.gguf \ - "https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf" +# Put a small GGUF test model under ./models and adjust docker-compose.dev.yml +# if the filename differs from the compose default. -# Start everything (builds from Dockerfile, ~5 min first time) docker compose -f docker-compose.dev.yml up --build - -# Open http://localhost:3000 (chat UI) -# Open http://localhost:3080 (system dashboard) ``` ---- - -## Option C: Deploy to Jetson Orin Nano - -### Prerequisites - -- Jetson Orin Nano Super devkit ($249) with JetPack 6.x flashed -- SSH access to the Jetson (`ssh geniepod@`) -- Cross-compiler installed on your dev machine: `sudo apt install gcc-aarch64-linux-gnu` +Open: -### 1. Cross-compile - -```bash -cd genie-claw -make jetson -``` +- Chat UI: `http://127.0.0.1:3000` +- Dashboard: `http://127.0.0.1:3080` -This builds all 5 binaries for aarch64. Output in `target/aarch64-unknown-linux-gnu/release/`. +## Option C: Jetson Orin Nano -### 2. Download models on the Jetson +Prerequisites on the development machine: ```bash -ssh geniepod@ -sudo mkdir -p /opt/geniepod/models -cd /opt/geniepod/models - -# Nemotron 4B — the primary model (2.8 GB, ~18 tok/s on Orin Nano) -wget "https://huggingface.co/nvidia/Nemotron-Mini-4B-Instruct-GGUF/resolve/main/Nemotron-Mini-4B-Instruct-Q4_K_M.gguf" \ - -O nemotron-4b-q4_k_m.gguf - -# Whisper small model for STT (future use) -# wget "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin" -O whisper-small.bin -``` - -If you already have the Nemotron model under an older OpenClaw path, move it instead of downloading again: - -```bash -sudo mkdir -p /opt/geniepod/models -sudo mv /opt/orinclaw/models/nemotron-4b-q4_k_m.gguf /opt/geniepod/models/ -ls -lh /opt/geniepod/models/nemotron-4b-q4_k_m.gguf +sudo apt install gcc-aarch64-linux-gnu +rustup target add aarch64-unknown-linux-gnu ``` -### 3. Install llama.cpp on the Jetson +Deploy current binaries, config, systemd units, and helper scripts: ```bash -ssh geniepod@ -git clone https://github.com/ggml-org/llama.cpp.git -cd llama.cpp -cmake -B build -DGGML_CUDA=ON -cmake --build build -j$(nproc) -sudo cp build/bin/llama-server /opt/geniepod/bin/ +make deploy \ + JETSON_HOST= \ + JETSON_USER= ``` -### 4. Deploy GeniePod binaries + config +Run first-boot setup on the Jetson. The current default backend is +`genie-ai-runtime` with the Qwen3 4B model path from `deploy/config/geniepod.toml`. ```bash -# From your dev machine: -cd genie-claw -make deploy JETSON_HOST= JETSON_USER=geniepod +ssh @ 'bash /opt/geniepod/setup-jetson.sh --runtime genie-ai-runtime' ``` -This copies: -- Binaries to `/opt/geniepod/bin/` -- Config to `/etc/geniepod/` (won't overwrite existing) -- systemd units to `/etc/systemd/system/` -- Docker compose config to `/opt/geniepod/docker/` - -Run the first-boot setup on the Jetson before starting `genie-core`. This fixes directory ownership for `/opt/geniepod/data`, secures config permissions, verifies the model path, and enables the systemd units. +Start or restart the appliance stack: ```bash -ssh geniepod@ 'bash /opt/geniepod/setup-jetson.sh' +ssh @ '/opt/geniepod/bin/genie-restart-all.sh --hard' ``` -If you already tried starting `genie-core` and saw `unable to open database file: /opt/geniepod/data/memory.db`, fix the deployed permissions and retry: +Verify services: ```bash -ssh geniepod@ -sudo chown -R $(whoami):$(whoami) /opt/geniepod /run/geniepod -sudo chmod 600 /etc/geniepod/geniepod.toml +ssh @ ' + curl -sf http://127.0.0.1:8080/health && echo + curl -sf http://127.0.0.1:3000/api/health && echo + curl -sf http://127.0.0.1:3080/api/status && echo + /opt/geniepod/bin/genie-ctl status +' ``` -### 5. Install and start Home Assistant on the Jetson +Open: -For Ubuntu-based Jetson installs, use Home Assistant Container. Install Docker Engine and the Docker Compose plugin using Docker's official Ubuntu instructions: +- Chat UI: `http://:3000` +- Dashboard: `http://:3080` -- https://docs.docker.com/engine/install/ubuntu/ -- https://docs.docker.com/compose/install/linux/ +## Home Assistant -Once Docker is installed, start the managed Home Assistant service: +Home Assistant is optional and remains a transitional provider until +`genie-home-runtime` exists. If Home Assistant runs on the Jetson, enable the +managed service and complete onboarding: ```bash -ssh geniepod@ +ssh @ sudo systemctl enable --now homeassistant -sudo systemctl status homeassistant --no-pager curl http://127.0.0.1:8123/ ``` -Complete the Home Assistant onboarding flow in your browser: - -```bash -http://:8123 -``` - -Then create a long-lived access token in Home Assistant and wire it into `genie-core` with a systemd drop-in: +Set the Home Assistant token as a systemd environment value rather than storing +the token in TOML: ```bash -ssh geniepod@ sudo mkdir -p /etc/systemd/system/genie-core.service.d sudo tee /etc/systemd/system/genie-core.service.d/homeassistant.conf > /dev/null <<'EOF' [Service] @@ -242,7 +152,7 @@ sudo systemctl daemon-reload sudo systemctl restart genie-core genie-health genie-governor ``` -If Home Assistant is running on another box in the home, point GeniePod at that URL instead: +If Home Assistant runs elsewhere, update the service URL: ```toml [services.homeassistant] @@ -250,119 +160,65 @@ url = "http://:8123/" systemd_unit = "homeassistant.service" ``` -### 6. Start services - -```bash -ssh geniepod@ - -# Start llama.cpp manually first to verify: -/opt/geniepod/bin/llama-server \ - --model /opt/geniepod/models/nemotron-4b-q4_k_m.gguf \ - --host 127.0.0.1 \ - --port 8080 \ - --ctx-size 2048 \ - --n-gpu-layers 999 \ - --flash-attn on \ - --cache-type-k q8_0 \ - --cache-type-v q8_0 \ - --threads 4 +## Configuration -# In another SSH session, start genie-core: -/opt/geniepod/bin/genie-core +Main Jetson config: -# Open http://:3000 in your browser +```text +/etc/geniepod/geniepod.toml ``` -### 7. Enable systemd services (persistent) +Source templates: -```bash -ssh geniepod@ -sudo systemctl daemon-reload -sudo systemctl enable --now genie-llm.service -sudo systemctl enable --now genie-core.service -sudo systemctl enable --now genie-governor.service -sudo systemctl enable --now genie-health.service - -# Check status: -sudo systemctl status genie-core -genie-ctl status -genie-ctl health -``` +- `deploy/config/geniepod.toml`: Jetson/default appliance config +- `deploy/config/geniepod.dev.toml`: dev-machine config -### 7. Measure RAM +The key LLM section is: -```bash -# After all services are running: -genie-ctl status -free -h -tegrastats --interval 5000 # watch GPU + RAM in real time +```toml +[services.llm] +url = "http://127.0.0.1:8080/health" +systemd_unit = "genie-ai-runtime.service" +backend = "genie_ai_runtime" ``` -Expected day mode: ~4.8-5.8 GB used, 2.2-3.2 GB free. - ---- - -## Configuration - -Main config: `/etc/geniepod/geniepod.toml` +To use the legacy `llama.cpp` fallback instead, set: ```toml -[core] -port = 3000 # Chat API port -bind_host = "127.0.0.1" # Use "0.0.0.0" only behind a trusted gateway/firewall -ha_token = "" # Home Assistant token (or set HA_TOKEN env) -max_history_turns = 20 # Conversation context window - -[governor] -poll_interval_ms = 5000 -night_start_hour = 23 -day_start_hour = 6 -night_model_swap = false # Set true to use 9B model at night - -[governor.pressure] -stop_optins_mb = 500 # Stop Nextcloud/Jellyfin below this +[services.llm] +url = "http://127.0.0.1:8080/health" +systemd_unit = "genie-llm.service" +backend = "llama_cpp" ``` -See `deploy/config/geniepod.toml` for all options. - ---- - ## Troubleshooting -### genie-core can't connect to LLM +LLM backend offline: ```bash -curl http://127.0.0.1:8080/health -# If this fails, llama.cpp isn't running. +curl -v http://127.0.0.1:8080/health +systemctl status genie-ai-runtime genie-llm --no-pager +journalctl -u genie-ai-runtime -n 120 --no-pager ``` -### Chat responses are empty +Core API offline: -The LLM model may be too small. Try a larger model (Nemotron 4B instead of TinyLlama). +```bash +curl -v http://127.0.0.1:3000/api/health +systemctl status genie-core --no-pager +journalctl -u genie-core -n 120 --no-pager +``` -### Governor offline +Dashboard offline: ```bash -genie-ctl status -# If governor is offline, start it: -sudo systemctl start genie-governor +curl -v http://127.0.0.1:3080/api/status +systemctl status genie-api --no-pager ``` -### Cross-compile fails +Cross-compile fails: ```bash -# Install the cross-compiler: sudo apt install gcc-aarch64-linux-gnu rustup target add aarch64-unknown-linux-gnu ``` - ---- - -## What's next - -After you have the basic demo running: - -1. **Connect Home Assistant** — set `HA_TOKEN` in config, try "turn on the lights" -2. **Try voice mode** — enable `voice_enabled` or launch `genie-core --voice` -3. **Add household context** — place profile files under `/opt/geniepod/data/profile/` -4. **Test governor modes** — `genie-ctl mode media`, `genie-ctl mode night_b` diff --git a/LAUNCH_PREP.md b/LAUNCH_PREP.md deleted file mode 100644 index bf803045..00000000 --- a/LAUNCH_PREP.md +++ /dev/null @@ -1,108 +0,0 @@ -# Launch Prep Checklist - -Internal-facing. Not a public marketing doc. The questions here are the -ones I'd want answered before posting GenieClaw anywhere external — Reddit -`r/homeassistant`, HN Show, Twitter, GitTensor, anywhere. - -The goal is **honest readiness**, not maximum hype. Launching a real -private-AI-for-home project to an audience that distrusts AI marketing -means the project has to look like it works, not like it's pretending. - ---- - -## 1. Demo (gate) - -The single biggest lever in the marketing plan, and the only one I cannot -fake in code. Without it, none of the rest matters. - -- [ ] 20-30 second screen + audio capture: speak → STT → LLM → Piper TTS reply → Home Assistant action. -- [ ] Show real wall-clock latency on a stop-watch overlay (TTFT, total turn). -- [ ] Run on the actual Jetson Orin Nano Super, 25 W mode. No desktop GPU fakery. -- [ ] One real device action — turning a light on, asking room temperature, etc. — not a canned response. -- [ ] Captions on the speech turn for accessibility + skim-readers. -- [ ] Upload to YouTube unlisted first; review for any leaked household details before flipping public. -- [ ] Replace the `` placeholder in `README.md` with the embed. - -If the demo isn't representative of what a stranger would experience on -their own hardware, the launch will produce more pain than stars. - -## 2. Install audit - -The marketing plan calls for `./install.sh && ./run.sh`. We do not have -that today. Before posting externally, either deliver it or be very -explicit it isn't there. - -- [ ] Run `GETTING_STARTED.md` Option A (dev-machine path) end-to-end on a clean Ubuntu container. Time it. Note every step that took manual fiddling. -- [ ] Same for the Jetson path. Note every place a non-author would get stuck (`nvpmodel`, audio device picking, whisper-server / Piper / llama-server port collisions, HA token entry). -- [ ] If total Jetson time > 30 min, write a `scripts/jetson-bringup.sh` that compresses what it can. Don't over-promise; leave manual steps that genuinely need a human (HA token, model download). -- [ ] Add a known-bad-and-good versions table to `GETTING_STARTED.md`: L4T R36.x, JetPack 6.x, CUDA 12.x, llama.cpp commit, whisper.cpp commit. Drift here costs hours. - -## 3. Repo hygiene - -The README hook will pull people into the repo. Anything broken there is -visible immediately. - -- [ ] All links in `README.md` resolve (the new ARCHITECTURE.md, CHANGELOG.md, GETTING_STARTED.md, genie-ai-runtime links). -- [ ] CI is green on main. `make test` passes locally and in CI. -- [ ] `make release` builds cleanly on a fresh Jetson clone. -- [ ] No secrets, tokens, or local paths in committed config files. `deploy/config/geniepod.dev.toml` reviewed. -- [ ] `LICENSE` is the intended one (AGPL-3.0 today; integrating projects need to know upfront). -- [ ] `cargo audit` clean, or known issues documented. -- [ ] Issues template + PR template present, or explicitly removed if not wanted. -- [ ] At least one `good-first-issue` open so contributors have a foothold. - -## 4. Roadmap visibility - -Stars come from "I might use this later" as much as from "I use this now." -The roadmap has to be on the repo, not in someone's head. - -- [ ] One-page roadmap visible from README. Quarter granularity is fine; "v1.0 = 24-hour soak + packaging" is fine. -- [ ] At least one near-term milestone labeled with the model (Qwen3-4B today, whatever target next). -- [ ] Hardware roadmap referenced (GeniePod custom carrier). If it's still aspirational, say so — don't imply it exists. - -## 5. Performance numbers — honest - -Anyone in the local-AI scene will compare against llama.cpp directly. The -README should pre-empt that with real measurements, not vibes. - -- [ ] Run llama.cpp baseline on the same Jetson, same model, same prompt. Record prefill tok/s, decode tok/s, TTFT. (This closes [genie-ai-runtime#2](https://github.com/GeniePod/genie-ai-runtime/issues/2) as a bonus.) -- [ ] Add a small benchmarks table to `README.md` (or link to a `BENCHMARKS.md`): GenieClaw end-to-end voice-turn latency, broken down (mic → STT → first LLM token → first TTS audio → speaker). Honest about variance. -- [ ] Flag what isn't tuned yet (decode tok/s vs llama.cpp's reference, currently behind — Path C is open). - -## 6. Positioning anchors - -These are the lines that will get pasted into Reddit/HN/Twitter copy. -Decide them now so they're consistent everywhere. - -- One-liner: "A private, always-on AI for your home. Runs entirely on a Jetson Orin Nano. Voice in, voice out, controls Home Assistant, no cloud." -- 30-second pitch (for HN intro paragraph): TBD — draft + critique before posting. -- The "this is not" list (already in README under `What It Is Not`): keep it short, keep it true. - -## 7. Distribution sequencing - -Once everything above is green, post in this order. Each step gives feedback -that should shape the next. - -1. **GitHub repo polish** — README, demo, install. Sit on it for a day, re-read with fresh eyes, fix the things that bug you. -2. **Soft launch on r/homeassistant** — Mid-week, mid-day pacific. Title: "I built a local AI assistant that actually controls Home Assistant — fully on-device on a Jetson". Read every comment in the first 2 hours, reply to all of them honestly. Expect "this already exists" — have the answer ready (it'll be in `What It Is Not`). -3. **Home Assistant community forum** — Same week. Different audience tone (more deployers, less reddit-snark). Link the reddit post; don't repost the same body. -4. **GitTensor + Twitter/X** — Same week. Use the same demo clip. Twitter thread can do the architecture deep-dive that doesn't fit in Reddit. -5. **HN Show** — Wait at least 1 week after Reddit. Use the feedback to tighten the README. Title: "Show HN: GenieClaw — local AI agent for your home, on a Jetson". Be online for the first 4 hours after post. - -Do **not** queue Reddit + HN + Twitter same-day. The fastest way to lose a -viral moment is to scatter feedback across channels you can't keep up with. - -## 8. What we are NOT promising - -Easy to over-promise on a marketing pass. The following are deliberately -absent from the README and should stay absent until they're real. - -- "Plug-and-play": setup is 30-60 min, not five. -- "Faster than X": we don't have the comparison numbers yet (item 5). -- "Multi-language": Whisper supports it; the agent doesn't tune for it. -- "Mobile app": planned, not built. -- "Custom hardware": GeniePod carrier exists as direction, not product. -- "Production-ready": this is alpha.4. Say alpha. Always. - -If we keep the README honest about all of these, the demo + the architecture -do the selling. If we don't, the first Reddit comment will catch it. diff --git a/crates/genie-common/src/config.rs b/crates/genie-common/src/config.rs index b88063a5..1b10450d 100644 --- a/crates/genie-common/src/config.rs +++ b/crates/genie-common/src/config.rs @@ -1685,6 +1685,21 @@ mod tests { ); } + #[test] + fn core_defaults_match_current_jetson_runtime() { + let config = test_config(); + assert_eq!(config.core.llm_model_name, "qwen"); + assert_eq!( + config.core.llm_model_path, + PathBuf::from("/opt/geniepod/models/Qwen3-4B-Q4_K_M.gguf") + ); + assert_eq!( + config.core.whisper_model, + PathBuf::from("/opt/geniepod/models/ggml-small.bin") + ); + assert!(config.core.wakeword_script.as_os_str().is_empty()); + } + #[test] fn portable_agent_profile_parses() { let config: AgentConfig = toml::from_str( @@ -2622,10 +2637,10 @@ mod defaults { "GENIEPOD_AI_PROVIDER_OAUTH_TOKEN".into() } pub fn llm_model_name() -> String { - "phi".into() + "qwen".into() } pub fn whisper_model() -> PathBuf { - PathBuf::from("/opt/geniepod/models/whisper-small.bin") + PathBuf::from("/opt/geniepod/models/ggml-small.bin") } pub fn piper_model() -> PathBuf { PathBuf::from("/opt/geniepod/voices/en_US-amy-medium.onnx") @@ -2661,9 +2676,9 @@ mod defaults { "auto".into() } pub fn audio_denoiser() -> String { - // alpha.7 default: try the neural denoiser first. Runtime falls back to - // sox then none if the binary is absent, so this is safe even on hosts - // that have not run the alpha.7 setup-jetson.sh yet. + // Try the neural denoiser first. Runtime falls back to sox then none + // if the binary is absent, so this is safe on hosts that have not run + // the full Jetson setup script yet. "deepfilternet".into() } pub fn deep_filter_path() -> PathBuf { @@ -2689,10 +2704,10 @@ mod defaults { 3 } pub fn llm_model_path() -> PathBuf { - PathBuf::from("/opt/geniepod/models/phi-4-mini-instruct-q4_k_m.gguf") + PathBuf::from("/opt/geniepod/models/Qwen3-4B-Q4_K_M.gguf") } pub fn wakeword_script() -> PathBuf { - PathBuf::from("/opt/geniepod/bin/genie-wake-listen.py") + PathBuf::new() } pub fn speaker_identity_confidence() -> String { "high".into() diff --git a/crates/genie-core/src/lib.rs b/crates/genie-core/src/lib.rs index 74c45bbc..dfa7c7fc 100644 --- a/crates/genie-core/src/lib.rs +++ b/crates/genie-core/src/lib.rs @@ -17,7 +17,7 @@ //! | Module | What it does | //! |--------|-------------| //! | [`agent_harness`] | Limited-context prompt/tool/memory/provider contract checks | -//! | [`llm`] | OpenAI-compatible local LLM client (llama.cpp, Ollama, any API) | +//! | [`llm`] | OpenAI-compatible LLM facade (`genie-ai-runtime`, llama.cpp, optional providers) | //! | [`ha`] | Home Assistant provider boundary, structure cache, and REST client | //! | [`tools`] | Compiled tool dispatch + parser for LLM JSON output | //! | [`memory`] | SQLite + FTS5 persistent memory with confidence decay | @@ -33,7 +33,7 @@ //! ## Design principles //! //! - **No HTTP framework** — raw tokio TcpListener (keeps binary small) -//! - **No AI framework** — direct OpenAI API over TCP (no langchain, no autogen) +//! - **No AI framework** — direct OpenAI-compatible HTTP clients (no langchain, no autogen) //! - **Bundled SQLite** — no external database dependency //! - **Single-threaded** — `tokio::main(flavor = "current_thread")` //! - **AGPL-3.0-only** — network-facing modifications must stay available to users diff --git a/crates/genie-core/src/tools/dispatch.rs b/crates/genie-core/src/tools/dispatch.rs index ff2ffc98..0f38e3c1 100644 --- a/crates/genie-core/src/tools/dispatch.rs +++ b/crates/genie-core/src/tools/dispatch.rs @@ -23,8 +23,8 @@ const ACTUATION_RATE_WINDOW_MS: u64 = 60_000; /// Tool definition for LLM function calling. /// -/// These are sent to llama.cpp as part of the system prompt or -/// via the `tools` parameter (OpenAI function-calling format). +/// These are sent to the configured LLM backend as part of the system prompt or +/// via the `tools` parameter when a backend supports OpenAI function-calling. #[derive(Debug, Clone, Serialize)] pub struct ToolDef { pub name: String, diff --git a/crates/genie-core/src/voice/stt.rs b/crates/genie-core/src/voice/stt.rs index 66878ebe..7ab6747d 100644 --- a/crates/genie-core/src/voice/stt.rs +++ b/crates/genie-core/src/voice/stt.rs @@ -575,10 +575,9 @@ pub async fn flush_mic_buffer(device: &str, sample_rate: u32) { /// Variants in increasing strength / latency: /// - `None` — bandpass + peak-normalize only (debug / no-denoise A/B) /// - `Sox` — sox `noisered` spectral subtraction against a per-host -/// noise profile (alpha.6 baseline, see PR #11) +/// noise profile /// - `DeepFilterNet` — neural denoiser via the `deep-filter` subprocess -/// (alpha.7, see issue #12). Handles non-stationary noise -/// without a noise profile. +/// Handles non-stationary noise without a noise profile. #[derive(Debug, Clone)] pub enum Denoiser { None, @@ -699,9 +698,9 @@ pub async fn record_audio( // a hard compand gate. // - Sox: spectral subtraction with a per-host noise profile // captured by setup-jetson.sh, plus a compand gate + - // quiet-speech lift (alpha.6 baseline). - // - None: bandpass + compand + normalize only (alpha.6 fallback - // when no noise profile is available). + // quiet-speech lift. + // - None: bandpass + compand + normalize only when no noise + // profile is available. let normalized_path = preprocess_capture(&wav_path, &denoiser).await?; let _ = tokio::fs::copy(&normalized_path, "/tmp/geniepod-last-rec.wav").await; if normalized_path != wav_path { @@ -734,8 +733,8 @@ async fn preprocess_capture(wav_path: &str, denoiser: &Denoiser) -> Result /// sox(gain -n -3) → normalized_path diff --git a/deploy/config/geniepod.toml b/deploy/config/geniepod.toml index c0a6178b..dae9cd59 100644 --- a/deploy/config/geniepod.toml +++ b/deploy/config/geniepod.toml @@ -28,7 +28,7 @@ port = 3000 bind_host = "127.0.0.1" # Safer default. Use "0.0.0.0" only behind a trusted gateway/firewall. ha_token = "" # Only needed when Home Assistant service is enabled below. Or set HA_TOKEN env var. llm_model_name = "qwen" # For prompt optimization. Options: nemotron-4b, tinyllama, llama, qwen, phi - # alpha.9 default: Qwen3-4B Q4_K_M + genie-ai-runtime. + # Jetson default: Qwen3-4B Q4_K_M + genie-ai-runtime. # Phi-4-mini fallback (issue #44): # 1) deploy/setup-jetson.sh --model phi-4-mini # 2) flip the two lines below: @@ -40,7 +40,7 @@ llm_model_name = "qwen" # For prompt optimization. Options: nemotron-4 # fallback) # 4) sudo systemctl restart genie-core # (see ARCHITECTURE.md "Current Transitional Adapters".) -whisper_model = "/opt/geniepod/models/ggml-small.bin" # alpha.5 default — with the +whisper_model = "/opt/geniepod/models/ggml-small.bin" # With the # long-running whisper-server (genie-whisper.service) # this model stays resident in iGPU memory, so per-call # STT is ~500 ms on Orin Nano (good for conversational @@ -48,7 +48,7 @@ whisper_model = "/opt/geniepod/models/ggml-small.bin" # alpha.5 default — wi # accuracy) for max responsiveness, or ggml-base.bin # (~200 ms) for a middle ground. Keep the genie-whisper # systemd unit's GENIEPOD_WHISPER_MODEL in sync. -whisper_port = 8178 # alpha.5: long-running whisper-server via +whisper_port = 8178 # Long-running whisper-server via # genie-whisper.service. Model stays in GPU memory # so per-utterance latency is ~50 ms instead of ~1.5 s # cold-start. Set to 0 to fall back to spawning @@ -94,13 +94,13 @@ audio_sample_rate = 24000 # Match the AHUB's I2S2 Sample Rate (24 kHz on # ~1 s wall-clock). whisper-server resamples to # 16 kHz internally so STT is unaffected. audio_denoiser = "deepfilternet" # Capture-side noise reduction backend. Options: - # "deepfilternet" — neural denoiser (alpha.7, issue #12). + # "deepfilternet" — neural denoiser. # Handles non-stationary noise # (fan cycling, typing, background # voices) without a noise profile. # Run setup-jetson.sh to fetch the # binary into deep_filter_path. - # "sox" — alpha.6 baseline; spectral + # "sox" — spectral # subtraction against a profile # captured in setup-jetson.sh, # plus compand quiet-speech lift. @@ -121,7 +121,7 @@ post_tts_silence_ms = 1500 # Half-duplex gate (issue #15): ms to wait aft # transcribes the assistant's voice as the next "user" # utterance. Set to 0 on installs with full physical # isolation (headphones / headset). -voice_enabled = true # Voice mode on by default in alpha.5 (push-to-talk). +voice_enabled = true # Voice mode on by default (push-to-talk unless wakeword_script is set). # Set false on hosts without whisper/piper installed. voice_record_secs = 3 # Seconds to record per voice interaction. Most # household commands ("what time is it", "turn on @@ -132,7 +132,7 @@ voice_continuous_secs = 3 # Shorter recording for follow-up conversation llm_model_path = "/opt/geniepod/models/Qwen3-4B-Q4_K_M.gguf" # For GPU time-sharing in voice mode. # Phi-4-mini fallback (issue #44): # llm_model_path = "/opt/geniepod/models/phi-4-mini-instruct-q4_k_m.gguf" -wakeword_script = "" # Empty = push-to-talk (alpha.5 default). +wakeword_script = "" # Empty = push-to-talk. # Set to "/opt/geniepod/bin/genie-wake-listen.py" once # wake-word detection is validated for your install. diff --git a/deploy/setup-jetson.sh b/deploy/setup-jetson.sh index cd88f7d5..cd61f5b3 100755 --- a/deploy/setup-jetson.sh +++ b/deploy/setup-jetson.sh @@ -4,11 +4,9 @@ # ssh geniepod@ 'bash /opt/geniepod/setup-jetson.sh' # # Flags: -# --model phi-4-mini Explicit form of today's default -# (Phi-4-mini Q4_K_M). -# --model qwen3-4b Download Qwen3-4B Q4_K_M instead -# (issue #44). Recommended pairing with -# genie-ai-runtime once both are installed. +# --model qwen3-4b Explicit form of the current Jetson default +# (Qwen3-4B Q4_K_M). +# --model phi-4-mini Download the Phi-4-mini Q4_K_M fallback. # The flag only changes the download target; # it does NOT rewrite llm_model_path in # /etc/geniepod/geniepod.toml — flip that @@ -30,17 +28,15 @@ CONFIG_DIR="/etc/geniepod" MODEL_DIR="$GENIEPOD_DIR/models" DATA_DIR="$GENIEPOD_DIR/data" -# Phi-4-mini Q4_K_M — the current default. Pinned to lmstudio-community's -# GGUF mirror because that conversion has been verified end-to-end on this -# repo's Tegra/aarch64 + llama.cpp + flash-attn stack. +# Phi-4-mini Q4_K_M — legacy fallback. Pinned to lmstudio-community's GGUF +# mirror because that conversion has been verified end-to-end on this repo's +# Tegra/aarch64 + llama.cpp + flash-attn stack. PHI_MODEL_FILENAME="phi-4-mini-instruct-q4_k_m.gguf" PHI_MODEL_URL="https://huggingface.co/lmstudio-community/Phi-4-mini-instruct-GGUF/resolve/main/Phi-4-mini-instruct-Q4_K_M.gguf" PHI_MODEL_LABEL="Phi-4-mini Q4_K_M (~2.4 GB)" -# Qwen3-4B Q4_K_M — opt-in alternative (issue #44). Sourced from upstream -# Qwen GGUF release. Stronger reasoning / multilingual / JSON tool-call -# behavior than Phi-4-mini; per-token decode is slower, which is what -# genie-ai-runtime is meant to address downstream. +# Qwen3-4B Q4_K_M — current Jetson default. Sourced from upstream Qwen GGUF +# release and paired with genie-ai-runtime. QWEN3_MODEL_FILENAME="Qwen3-4B-Q4_K_M.gguf" QWEN3_MODEL_URL="https://huggingface.co/Qwen/Qwen3-4B-GGUF/resolve/main/Qwen3-4B-Q4_K_M.gguf" QWEN3_MODEL_LABEL="Qwen3-4B Q4_K_M (~2.5 GB)" @@ -305,8 +301,8 @@ fi # 4. Ensure the configured LLM model exists. # Selection rules (issue #44): # - Without --model: honor llm_model_path in geniepod.toml if set, else -# fall back to the Phi-4-mini default path. Auto-download only when -# the resolved path matches the default for the active model choice. +# fall back to the Qwen3 default path. Auto-download only when the resolved +# path matches the default for the active model choice. # - With --model : download 's canonical artifact to # $MODEL_DIR/. Does NOT rewrite llm_model_path — operator # flips that line by hand to switch the running LLM. @@ -358,7 +354,7 @@ if [ -n "$MODEL_CHOICE" ] && [ "$MODEL_CHOICE" != "qwen3-4b" ]; then echo " llm_model_path = \"$GGUF\"" echo " llm_model_name = \"phi\" # selects the Phi prompt template" echo " update GENIEPOD_LLM_MODEL in the active LLM systemd unit" - echo " (genie-ai-runtime.service for the alpha.9 default, or" + echo " (genie-ai-runtime.service for the Jetson default, or" echo " genie-llm.service for the llama.cpp fallback), then:" echo " sudo systemctl restart genie-core" fi @@ -571,7 +567,7 @@ else VOICE_MISSING=1 fi -# alpha.5: record_audio peak-normalizes captures with `sox gain -n` so +# record_audio peak-normalizes captures with `sox gain -n` so # weak mic signals reach whisper at nominal level. Not strictly required # (genie-core falls back to raw recording with a warning), but strongly # recommended for accuracy. @@ -582,7 +578,7 @@ else echo " (genie-core falls back to raw audio, but STT accuracy suffers on quiet captures)" fi -# alpha.5: whisper-server is preferred for STT (long-running, model stays in +# whisper-server is preferred for STT (long-running, model stays in # GPU memory). Optional in dev hosts where whisper_port = 0 forces CLI mode. WHISPER_SERVER="$GENIEPOD_DIR/bin/whisper-server" WHISPER_PORT="$(read_toml_string whisper_port)" @@ -636,7 +632,7 @@ if [ "$VOICE_MISSING" -eq 1 ]; then echo " Until installed, keep voice_enabled = false in $CONFIG_DIR/geniepod.toml." fi -# 5f. Install DeepFilterNet `deep-filter` binary (alpha.7, issue #12). +# 5f. Install DeepFilterNet `deep-filter` binary. # Used by record_audio when audio_denoiser = "deepfilternet". The binary is # self-contained — DFN3 model is statically linked via tract. License: MIT/ # Apache-2.0 dual (the project explicitly clarifies AGPL compatibility). diff --git a/deploy/systemd/genie-core.service b/deploy/systemd/genie-core.service index 2aabaf93..04052514 100644 --- a/deploy/systemd/genie-core.service +++ b/deploy/systemd/genie-core.service @@ -2,7 +2,7 @@ Description=GeniePod Core — Local Home AI Engine Documentation=https://github.com/GeniePod/genie-claw # Order after both LLM units so genie-core never starts before whichever -# backend is configured. genie-ai-runtime.service is the alpha.9 default; +# backend is configured. genie-ai-runtime.service is the Jetson default; # genie-llm.service is the llama.cpp fallback. They Conflict= each other, # so only one will ever be active at a time and the unused After= entry # is a no-op. diff --git a/doc/README.md b/doc/README.md index 1ebe769e..4a7bdc33 100644 --- a/doc/README.md +++ b/doc/README.md @@ -28,7 +28,6 @@ Where current code is transitional, the docs call that out explicitly. - [milestone-1-portable-home-agent.md](milestone-1-portable-home-agent.md): M1 architecture movement for portable validation without weakening the limited-context home-agent goal - [repo-map.md](repo-map.md): top-level files, directories, and module map - [research-agentic-ai.md](research-agentic-ai.md): research notes from current agentic AI application patterns and what GenieClaw adopts -- [alpha5-reflection.md](alpha5-reflection.md): critique-first next-alpha preparation notes - [../CHANGELOG.md](../CHANGELOG.md): alpha release notes ## Runtime At A Glance @@ -42,7 +41,9 @@ GenieClaw is a local-first home AI runtime centered on `genie-core`. - `genie-governor` manages mode changes, memory-pressure reactions, and service lifecycle decisions. - `genie-health` polls service endpoints and stores health history. - `genie-ctl` is the local operator CLI. -- `llama-server` is external to this Rust workspace, but it is the default LLM backend expected by the deploy assets. +- `genie-ai-runtime` is external to this Rust workspace and is the default + Jetson LLM backend expected by the deploy assets; `llama.cpp` remains a + selectable development/fallback backend. ## Canonical Deep Dives Still Kept At Repo Root @@ -67,5 +68,9 @@ implemented code from roadmap work. For the canonical status matrix, read There are a few intentional limits: - Hardware behavior that depends on a specific Jetson image, kernel, or manual systemd override is documented as operational guidance, not as a stable code contract. -- `llama.cpp`, Home Assistant, Piper, Whisper, and Telegram Bot API internals are external dependencies. This repo documents how GenieClaw integrates with them, not their full upstream behavior. -- `genie-os`, `genie-voice-runtime`, `genie-home-runtime`, and `genie-ai-runtime` are documented as architectural boundaries unless code in this repo already implements a transitional adapter. +- `genie-ai-runtime`, `llama.cpp`, Home Assistant, Piper, Whisper, and Telegram + Bot API internals are external dependencies. This repo documents how + GenieClaw integrates with them, not their full upstream behavior. +- `genie-os`, `genie-voice-runtime`, `genie-home-runtime`, and + `genie-ai-runtime` are documented as architectural boundaries unless code in + this repo already implements a client or transitional adapter. diff --git a/doc/alpha5-reflection.md b/doc/alpha5-reflection.md deleted file mode 100644 index 48991bd6..00000000 --- a/doc/alpha5-reflection.md +++ /dev/null @@ -1,105 +0,0 @@ -# Alpha 5 Reflection Notes - -This is a critique-first preparation note for the next alpha. It focuses on -what is weak or risky in the current implementation, not on expanding the -feature list. - -## Current Judgment - -GenieClaw is moving in the right direction for the Genie ecosystem: it is -local-first, Jetson-conscious, memory-aware, and increasingly policy-driven. -The strongest recent work is the runtime contract, actuation policy, skill -manifest audit, tool audit, and support bundle path. - -The main risk is not missing features. The main risk is allowing the agent -layer to become a large, trusted monolith before lower runtime boundaries are -ready. - -## Architecture Critique - -1. The HTTP API was too exposed for a physical agent. - - A local assistant that can touch memory, tools, and home actuation should - not bind to the LAN by default. - - Next-alpha change: default `[core].bind_host` is now `127.0.0.1`; operators - must explicitly opt into `0.0.0.0`. - -2. `server.rs`, `tools/dispatch.rs`, and `memory/mod.rs` are too large. - - They are still understandable, but they combine routing, policy, execution, - persistence, and response formatting. - - Alpha 5 should split by stable seams only: request origin/auth, tool audit, - memory management API, and home runtime handoff. - -3. Request handling is still sequential. - - This keeps SQLite ownership simple, but a long LLM turn can delay health, - dashboard, confirmation, and memory API responses. - - Do not blindly spawn per connection until shared state is wrapped behind - safe concurrency boundaries. - - Preferred next step: isolate read-only health/runtime endpoints first, then - decide whether conversation writes need a serialized actor. - -4. Transitional Home Assistant support is still useful but strategically - dangerous. - - The code mostly keeps HA behind provider boundaries, but product language - and tests must keep reinforcing that HA is not the final home runtime. - - Alpha 5 should define the `genie-home-runtime` client trait shape before - adding more HA-specific behavior. - -5. Voice speaker identity was scaffolded but cleanup order was wrong. - - The captured WAV was deleted before the future biometric recognizer could - inspect it. - - Next-alpha change: identity now receives the WAV path before cleanup. - -6. Skill policy is still audit-heavy, not sandbox-heavy. - - Manifest and signature-material presence are useful, but they are not - security isolation. - - Alpha 5 should keep native skills disabled or tightly allowlisted in - untrusted deployments until process isolation exists. - -## Alpha 5 Priorities - -The next alpha should be a reliability and boundary release. - -1. Secure local surface by default. - - Localhost bind by default. - - Clear warning when binding wildcard. - - Origin headers from first-party clients. - - Document reverse-proxy or gateway expectations. - -2. Split control-plane code at real seams. - - Move runtime contract and health response composition out of `server.rs`. - - Move tool audit and policy helpers out of `tools/dispatch.rs`. - - Keep behavior unchanged while reducing file size and review risk. - -3. Make voice pipeline measurable. - - Persist per-turn timing: record, STT, quick route, LLM, TTS, total. - - Surface recent voice failures in support bundles. - - Keep `/no_think` default for voice unless explicit deep reasoning is needed. - -4. Prepare home-runtime boundary. - - Define a narrow trait/API for device graph, status, proposed actuation, and - final actuation result. - - Keep Home Assistant as one adapter behind that boundary. - -5. Finish operational hardening. - - Add an operator command that prints runtime contract and policy status in a - human-readable form. - - Add release checks for bind host, runtime drift, tool policy, skill policy, - and actuation audit availability. - -## Non-Goals For Alpha 5 - -- Do not add broad marketplace behavior. -- Do not add cloud account features. -- Do not implement emotion detection. -- Do not expand Home Assistant-specific product assumptions. -- Do not turn `genie-core` into `genie-home-runtime` or `genie-ai-runtime`. - -## Exit Criteria - -Alpha 5 is ready when: - -- default deployment does not expose `genie-core` beyond localhost -- support bundle captures enough runtime state to diagnose a failed field box -- voice identity and multilingual routing have no obvious lifecycle bugs -- unsafe physical action paths are blocked without relying on prompt obedience -- the next extraction boundary for `genie-home-runtime` is documented and tested diff --git a/doc/configuration.md b/doc/configuration.md index 49afc826..5f3847b9 100644 --- a/doc/configuration.md +++ b/doc/configuration.md @@ -343,7 +343,7 @@ Operational variables used by systemd/deploy surfaces outside the Rust config: | Variable | Purpose | | --- | --- | -| `GENIEPOD_LLM_MODEL` | Model path used by `genie-llm.service` / `llama-server` | +| `GENIEPOD_LLM_MODEL` | Model path used by the active local LLM unit (`genie-ai-runtime.service` by default, or `genie-llm.service` for llama.cpp fallback) | ## Config Resolution Rules diff --git a/doc/core-subsystems.md b/doc/core-subsystems.md index 4203e9d2..d86d385b 100644 --- a/doc/core-subsystems.md +++ b/doc/core-subsystems.md @@ -7,16 +7,22 @@ their runtime role. Source: -- `crates/genie-core/src/llm/client.rs` -- `crates/genie-core/src/llm/retry.rs` - `crates/genie-core/src/llm/mod.rs` +- `crates/genie-core/src/llm/openai_compat.rs` +- `crates/genie-core/src/llm/genie_ai_runtime.rs` +- `crates/genie-core/src/llm/llama_cpp.rs` +- `crates/genie-core/src/llm/openai_compatible.rs` +- `crates/genie-core/src/llm/provider.rs` Responsibilities: -- OpenAI-compatible HTTP calls to the configured local model server +- OpenAI-compatible HTTP calls to the configured model server +- Jetson-default `genie-ai-runtime` request shaping and hint metadata +- legacy/development `llama.cpp` fallback support +- optional OpenAI-compatible provider auth/readiness planning - health checking - request serialization and response parsing -- retry and fallback behavior for selected request classes +- bounded connect/read/request timeouts for blocking and streaming calls ## Prompt Builder And Reasoning Mode diff --git a/doc/deployment-and-ops.md b/doc/deployment-and-ops.md index 0f836a46..6cff1af8 100644 --- a/doc/deployment-and-ops.md +++ b/doc/deployment-and-ops.md @@ -25,10 +25,15 @@ The practical reason is simple: - `deploy/systemd/genie-api.service` - `deploy/systemd/genie-governor.service` - `deploy/systemd/genie-health.service` +- `deploy/systemd/genie-ai-runtime.service` +- `deploy/systemd/genie-ai-runtime-warmup.service` - `deploy/systemd/genie-llm.service` +- `deploy/systemd/genie-llm-warmup.service` - `deploy/systemd/genie-mqtt.service` - `deploy/systemd/genie-audio.service` - `deploy/systemd/genie-wakeword.service` +- `deploy/systemd/genie-whisper.service` +- `deploy/systemd/genie-whisper-warmup.service` - `deploy/systemd/homeassistant.service` - `deploy/systemd/geniepod.target` - `deploy/systemd/geniepod-late.target` @@ -51,8 +56,9 @@ The practical reason is simple: ### Dev Machine -Use `deploy/config/geniepod.dev.toml` and point `genie-core` at a local -`llama-server`. +Use `deploy/config/geniepod.dev.toml` and point `genie-core` at any local +OpenAI-compatible model server. The checked-in dev config uses `llama.cpp` +on `:8080`. Main references: @@ -75,7 +81,8 @@ Use `deploy/setup-jetson.sh` and the systemd units under `deploy/systemd/`. Typical production expectations: -- `genie-llm.service` provides the local model server +- `genie-ai-runtime.service` provides the default local model server +- `genie-llm.service` is available as the legacy `llama.cpp` fallback - `genie-core.service` exposes the main runtime on `127.0.0.1:3000` by default - `genie-governor.service` and `genie-health.service` are active - `genie-api.service` serves dashboard/status @@ -90,9 +97,9 @@ the safe default. Common commands: ```bash -systemctl status genie-core genie-governor genie-health genie-api genie-llm +systemctl status genie-core genie-governor genie-health genie-api genie-ai-runtime journalctl -u genie-core -n 200 --no-pager -journalctl -u genie-llm -n 200 --no-pager +journalctl -u genie-ai-runtime -n 200 --no-pager curl -s http://127.0.0.1:3000/api/health curl -s http://127.0.0.1:3000/api/tools genie-ctl status @@ -197,7 +204,7 @@ These are current system realities, not bugs in the docs: Minimum checks after deployment: -1. Verify `llama-server` health. +1. Verify the configured LLM backend health, normally `genie-ai-runtime`. 2. Verify `genie-core` health and tool list. 3. Verify `genie-governor` socket and status. 4. Verify `genie-api` dashboard/status responses. @@ -210,7 +217,7 @@ Minimum checks after deployment: | Symptom | First Place To Check | | --- | --- | | Chat UI loads but no answers | `genie-core` logs and `/api/health` | -| `llm: offline` | `genie-llm.service` and `llama-server` flags | +| `llm: offline` | configured LLM unit (`genie-ai-runtime.service` by default, `genie-llm.service` for llama.cpp fallback) and `/api/health` backend details | | Wrong or missing Home Assistant behavior | Home Assistant service config, token resolution, `ha/` boundary logs | | Voice hears but does not answer | STT path, language selection, Piper path, audio device | | Governor appears offline | `/run/geniepod/governor.sock` and `genie-governor.service` | diff --git a/doc/implementation-status.md b/doc/implementation-status.md index be12663a..621e3903 100644 --- a/doc/implementation-status.md +++ b/doc/implementation-status.md @@ -1,6 +1,6 @@ # Implementation Status -Last reconciled against the repository code: 2026-05-07. +Last reconciled against the repository code: 2026-05-28. This page is the source of truth for what is implemented in this repository versus what is architecture, transitional integration, or future ecosystem work. @@ -42,7 +42,7 @@ Status labels: | Area | Status | What Exists | What Is Still Missing | | --- | --- | --- | --- | | Home Assistant integration | Partial / transitional | Provider boundary, status/control/history/undo, local action safety, HA token/config path | Home Assistant is not reimplemented in Rust here. Final device graph, automations, and deterministic physical safety belong in `genie-home-runtime`. | -| LLM runtime | External / transitional | OpenAI-compatible client points at local `llama-server`; deploy units configure `llama.cpp` | Jetson-only optimized C++ runtime, CUDA kernels, memory planner, and model-serving replacement belong in `genie-ai-runtime`. | +| LLM runtime | External / integrated | `crates/genie-core/src/llm/*`, `deploy/systemd/genie-ai-runtime.service`, `[services.llm].backend = "genie_ai_runtime"` | Jetson deploys default to the external `genie-ai-runtime` on `:8080`; `llama.cpp` remains a selectable fallback/development backend. | | Voice multilingual support | Partial | STT language hint/auto mode, language detection, and optional per-language Piper model selection | Full quality for Chinese, Spanish, German, etc. depends on installed Whisper/Piper models and device testing. It is not a certified full-language product yet. | | Speaker recognition | Partial | Local acoustic fingerprints from WAV profiles and runtime matching | Not robust biometric authentication, anti-spoofing, enrollment UX, or security-grade identity. | | ESP32-C6 connectivity | Partial boundary | Config, status endpoint, capability model, UART path validation, Thread/Matter capability intent | No real UART protocol controller, no Thread/Matter stack, no ESP-Hosted-NG implementation in this repo. ESP-Hosted-NG belongs in `genie-os`; protocol ownership belongs in `genie-home-runtime`/connectivity services. | @@ -56,7 +56,7 @@ Status labels: | Area | Status | Correct Owner | | --- | --- | --- | | `genie-home-runtime` | Planned / separate repo | Rust AI-native home automation engine, device graph, automations, MCP server, deterministic final physical safety layer. | -| `genie-ai-runtime` | Planned / separate repo | Jetson-only C++ inference runtime customized from `llama.cpp`, optimized CUDA kernels, and memory planner. | +| `genie-ai-runtime` | External / separate repo | Jetson-only inference runtime, CUDA kernels, memory planner, and OpenAI-compatible serving surface. GenieClaw owns the client contract, not the runtime implementation. | | `genie-voice-runtime` | Initial / separate repo | External voice runtime for wake, VAD, STT, TTS, audio streaming, and voice session protocol. | | `genie-os` | Planned / separate repo | Custom L4T image, board bring-up, drivers, OTA base image, service supervision, ESP-Hosted-NG OS integration. | | Full Matter/Thread/Zigbee/BLE production stack | Planned outside this repo | Lower connectivity/home-runtime layers. | @@ -67,12 +67,11 @@ Status labels: ## Current Alpha Truth -The workspace version is currently `1.0.0-alpha.4`. +The workspace version is currently `1.0.0-alpha.9`. -The `Unreleased` changelog contains next-alpha work already implemented in the -codebase, including safer localhost defaults, origin-aware first-party requests, -and local speaker-profile management. Those features are present in source, but -the crate version has not yet been bumped to a new alpha release. +The current alpha line defaults Jetson deployments to `genie-ai-runtime`, +preserves the 4096-token agent harness, and keeps optional remote/API providers +behind explicit config, credential-env, and context-budget checks. ## How To Keep This Page Honest diff --git a/doc/lyrat-jetson-audio.md b/doc/lyrat-jetson-audio.md index d508848b..a05fb40b 100644 --- a/doc/lyrat-jetson-audio.md +++ b/doc/lyrat-jetson-audio.md @@ -202,13 +202,13 @@ Rate` to 48 kHz produces 2× chipmunk playback; 24 kHz produces natural pitch). Reason unknown — likely an APLL/MCLK divider constraint or slot-width fallback inside the ESP32 I2S clock generator. -For alpha.5 the workaround is: tell the Jetson AHUB to expect 24 kHz. +The current workaround is: tell the Jetson AHUB to expect 24 kHz. `genie-audio-init` writes `I2S2 Sample Rate = 24000` for this reason. Capture-side parameters in `[core]` (e.g. `audio_sample_rate = 16000`) work fine — ALSA `plughw:APE,0` downsamples 24 kHz → 16 kHz cleanly. Investigating the ESP-IDF clock setup so the LyraT actually emits 48 kHz -LRCK as configured is tracked as alpha.6 work. +LRCK as configured remains future hardware/firmware work. ## Limitations / known gaps diff --git a/doc/overview.md b/doc/overview.md index 56a2c005..8d554e58 100644 --- a/doc/overview.md +++ b/doc/overview.md @@ -30,9 +30,10 @@ The long-term Genie stack is split by responsibility: - `genie-claw` for agent policy, memory, tools, skills, smart-home intent, and interaction - web/mobile apps for setup, control, memory management, and confirmations -This repository is `genie-claw`. Some current integrations still live here as -transitional adapters, especially `llama.cpp` and Home Assistant, but the code -should keep those behind narrow boundaries. +This repository is `genie-claw`. It integrates with external lower runtimes +through narrow clients: `genie-ai-runtime` is the Jetson default LLM backend, +`llama.cpp` remains a selectable development/fallback backend, and Home +Assistant is the current transitional home provider. For the exact implemented/partial/planned breakdown, use [implementation-status.md](implementation-status.md). In short, this repo @@ -64,7 +65,7 @@ In daemon mode, Telegram can also be enabled as a side-channel adapter. Typical Jetson deployment: ```text -llama-server (:8080) +genie-ai-runtime (:8080) ^ | genie-core (:3000) <---- genie-ctl @@ -74,6 +75,9 @@ genie-core (:3000) <---- genie-ctl +---- optional Home Assistant provider +---- optional ESP32-C6 connectivity controller boundary +Selectable fallback: llama.cpp `llama-server` can also serve the same +OpenAI-compatible LLM endpoint on `:8080`. + genie-governor ---- controls service modes and pressure response genie-health ---- polls health endpoints and stores health history genie-api ---- serves dashboard/status data diff --git a/doc/repo-map.md b/doc/repo-map.md index 7c575d20..6a30cbc5 100644 --- a/doc/repo-map.md +++ b/doc/repo-map.md @@ -47,11 +47,16 @@ ### LLM - `llm/mod.rs` -- `llm/client.rs` -- `llm/retry.rs` +- `llm/openai_compat.rs` +- `llm/genie_ai_runtime.rs` +- `llm/llama_cpp.rs` +- `llm/openai_compatible.rs` +- `llm/provider.rs` -This is the current AI-runtime adapter. It points at `llama.cpp` today and -should point at `genie-ai-runtime` later. +This is the LLM backend facade. Jetson deploys default to the external +`genie-ai-runtime`; `llama.cpp` remains selectable as a legacy fallback and +development backend. Optional OpenAI-compatible providers are disabled by +default and must pass the same limited-context harness before use. ### Prompt And Reasoning @@ -135,6 +140,10 @@ Current integration-style tests outside `src/`: - `crates/genie-core/tests/tool_dispatch_test.rs` - `crates/genie-core/tests/tools_test.rs` +- `crates/genie-core/tests/memory_recall.rs` +- `crates/genie-core/tests/prompt_sha_test.rs` +- `crates/genie-core/tests/tool_gate_integration_test.rs` +- `crates/genie-core/tests/voice_loop_integration.rs` Most other tests are colocated unit tests inside the module files. diff --git a/doc/services-and-crates.md b/doc/services-and-crates.md index ca709ae2..f4b1aafb 100644 --- a/doc/services-and-crates.md +++ b/doc/services-and-crates.md @@ -10,8 +10,10 @@ Jetson appliance deployment, but the long-term boundary is clear: - `genie-core` is the agent runtime. - `genie-api` is a lightweight local dashboard/status service, not the final product app. - `genie-governor` and `genie-health` are appliance support services. -- Home Assistant and `llama.cpp` are transitional lower-runtime adapters. -- Future `genie-home-runtime` and `genie-ai-runtime` should replace those lower-runtime adapters. +- Home Assistant is the current transitional home-runtime adapter. +- `genie-ai-runtime` is the default external Jetson LLM runtime; `llama.cpp` + remains a selectable fallback and development backend. +- Future `genie-home-runtime` should replace the Home Assistant lower-runtime adapter. - `genie-voice-runtime` is the new external owner for wake/VAD/STT/TTS/audio behavior. For the current truth matrix, see @@ -137,7 +139,8 @@ Key files: ### Network Endpoints - `genie-core`: `:3000` -- `llama-server`: `:8080` today; future replacement is `genie-ai-runtime` +- LLM backend: `:8080`; Jetson default is `genie-ai-runtime`, fallback/dev is + `llama.cpp` `llama-server` - Home Assistant: commonly `:8123` today; future replacement is `genie-home-runtime` - `genie-voice-runtime`: external voice runtime; protocol and port are still stabilizing - `genie-api`: separate dashboard service port, depending on deploy setup @@ -161,10 +164,15 @@ Defined under `deploy/systemd/`: - `genie-api.service` - `genie-governor.service` - `genie-health.service` +- `genie-ai-runtime.service` +- `genie-ai-runtime-warmup.service` - `genie-llm.service` +- `genie-llm-warmup.service` - `genie-mqtt.service` - `genie-audio.service` - `genie-wakeword.service` +- `genie-whisper.service` +- `genie-whisper-warmup.service` - `homeassistant.service` - `geniepod.target` - `geniepod-late.target` diff --git a/doc/workflow/prompt.md b/doc/workflow/prompt.md deleted file mode 100644 index b0ca9a2a..00000000 --- a/doc/workflow/prompt.md +++ /dev/null @@ -1,458 +0,0 @@ -# GenieClaw — Workflow Diagram Prompts - -Prompts for generating 8 architectural workflow diagrams that document how -GenieClaw works end-to-end. Each prompt is standalone — paste it into any -text-to-image diagram generator (or a Mermaid / PlantUML LLM) preceded by -the shared style preamble below. - -Component names, ports, file paths, and config field names are written -verbatim from the codebase so the renderer should not paraphrase them. - -## Shared style preamble (prepend to every prompt below) - -``` -Generate an architectural workflow diagram for "GenieClaw", a local home AI -assistant running on a Jetson Orin Nano with an ESP32-LyraT V4.3 I2S microphone -frontend. Style: clean, modern, isometric-or-flat architecture diagram with -rounded rectangular nodes, labeled arrows for data/control flow, distinct -colors for hardware (dark slate), Rust services (orange), C++ inference -binaries (blue), config/systemd (purple), external user-facing surfaces (green). -Use mono-spaced font for code/path labels. Include a title bar at top. -Background: light neutral. Resolution: 1920x1080. Avoid emojis. Component names -should appear EXACTLY as written below — these are real binaries, services, -files, and ports from the codebase. -``` - -If your renderer prefers Mermaid/PlantUML over a pixel image, swap -"architectural workflow diagram" for "Mermaid flowchart" (or -"PlantUML component diagram") and the rest carries over. - ---- - -## 1. Entire workflow (one big picture) - -``` -Title: "GenieClaw — End-to-End System Workflow" - -Show three horizontal swim lanes from top to bottom: - (a) HARDWARE: ESP32-LyraT V4.3 (I2S mic) → Jetson 40-pin header (I2S2) → - Jetson Orin Nano Super Devkit (7.6 GB iGPU) → speaker/headphone out - → optional ESP32-C6 sidecar (Thread/Matter via UART /dev/ttyTHS1). - (b) SERVICES on Jetson, grouped under "geniepod.target": homeassistant, - genie-audio (one-shot AHUB route setup), genie-whisper + - genie-whisper-warmup, genie-llm + genie-llm-warmup, genie-core, - genie-governor, genie-health, genie-api, genie-mqtt. - (c) USER SURFACES: voice push-to-talk (LyraT mic + speaker), chat UI - (http://jetson:3000), dashboard (http://jetson:3080), Home Assistant - (8123), Telegram bot (optional). - -Draw clear arrows: audio in from LyraT to genie-core; HTTP between -genie-core and genie-llm (8080) and genie-whisper (8178); Piper TTS out via -aplay; Home Assistant calls; MQTT pub/sub on 1883. Mark systemd-managed -boundaries. Label the iGPU memory budget under genie-llm (~2.4 GB Phi-4-mini) -and genie-whisper (~487 MB whisper-small). -``` - ---- - -## 2. Voice pipeline workflow in detail - -``` -Title: "GenieClaw — Voice Pipeline (Push-to-Talk Cycle)" - -Linear left-to-right flowchart, one cycle of voice_loop::voice_cycle(), -with timing markers shown above each transition: - - [User presses Enter] - → flush_mic_buffer (1 s throwaway arecord) — drains stale samples - → arecord -D plughw:APE,0 -c 2 -r 24000 -d 3 [≈3.02 s wall] - → [AUDIO_CAPTURED marker stamped] (speech end) - → preprocess_capture branches on audio_denoiser config: - ├─ deepfilternet: sox(channels 1, highpass 100, lowpass 7000) - │ → deep-filter --atten-lim-db 100 - │ → sox(gain -n -3) [≈820 ms total] - ├─ sox: sox(channels 1, highpass, lowpass, noisered, compand, gain) - └─ none: bandpass + compand + normalize only - → aec::process_aec (skips stale references, NLMS otherwise) - → stt::transcribe_via_server → HTTP POST → whisper-server :8178 - ─ whisper.cpp ggml-small, CUDA, flash-attn, model resident in iGPU - ─ multipart/form-data: language=en, temperature=0.0, - response_format=json - [≈285 ms warm] - → [STT_DONE marker] - → intent::assess_transcript (reject hallucinations / ambient narration) - → conversation store append (user turn) - → handle_quick_tool_for_voice (try memory_recall, get_time, etc.) - → streaming::stream_and_speak: - ─ build_memory_context_with_read_context - ─ apply_reasoning_mode - ─ HTTP POST → llama-server :8080 (Phi-4-mini Q4_K_M, --ctx-size 2048, - --flash-attn on, GPU layers 999) - ─ stream tokens → split into sentences - ─ per sentence: tts::TtsEngine::speak → spawn Piper → spawn aplay - ├─ [FIRST_SPEAK marker] - └─ [FIRST_AUDIO marker — before aplay.stdin.write_all] - → aplay finishes - → half-duplex gate: tokio::sleep(post_tts_silence_ms = 1500 ms) - ─ ALSA HW buffer drains, room reverb decays below no-speech-thold - → extract::extract_and_store (background memory write) - → loop back - -In a side box, show the FIRST-REPLY LATENCY BANNER fields it produces once -per process: - preprocess (DFN+sox) - STT - LLM until first sentence - TTS first synth - speech end → first audio -``` - ---- - -## 3. AI agent & system prompt workflow - -``` -Title: "GenieClaw — AI Agent Reasoning & System Prompt Composition" - -Diagram showing how a user utterance becomes an LLM call: - - Inputs (left side, parallel): - ─ User transcript (from STT, with detected_language) - ─ Speaker identity (speaker_identity::identify → name + confidence) - ─ Memory read context (build_memory_read_context) - ─ Conversation history (recent N turns, max_history_turns = 20) - ─ System prompt template (prompt::build for model="phi", family=Phi) - ─ Memory injection (inject::build_memory_context_with_read_context → - per-query namespace tree lookup + shared-room redaction filter) - ─ Reasoning mode (reasoning::apply_reasoning_mode → - InteractionKind::Voice) - - → Composed messages: [system, ...history, user] - → LLM streaming call (genie-llm at :8080, streaming response) - → Token stream → response text - - → Tool detection: tools::try_tool_call_with_context → - ToolExecutionContext { memory_read_context, request_origin=Voice, - confirmed=false } - → ToolDispatcher - → On hit: tool_result.tool + tool_result.output - - → If tool hit: build summary_msgs (with summary system prompt), - apply_reasoning_mode (InteractionKind::ToolSummary), second LLM - call, speak the summary. - - → Auto-fact capture: extract::extract_and_store (after TTS, non-blocking) - -Show distinct lanes for: - ─ "system prompt" (purple) - ─ "memory" (green, with sub-boxes: durable MEMORY.md, namespaces/INDEX.md, - person/private/restricted notes — redaction-aware projection) - ─ "tools" (orange, with sub-boxes: get_time, memory_recall, web_search, - home_status, home_control, plus skill-loaded tools) - ─ "actuation safety gate" (red) intercepts home_control before execution. -``` - ---- - -## 4. ESP32 + Thread/WiFi/BLE/Home Assistant integration - -``` -Title: "GenieClaw — ESP32 Sidecar & Home/Network Integration" - -Show TWO ESP32 boards distinctly: - - (1) ESP32-LyraT V4.3 (capture-only mic frontend): - ─ Custom IDF firmware "lyrat_jp4_passthrough" (in - espressif/esp-adf fork) - ─ ES8388 codec → MCLK/SCLK/LRCK/SDOUT pins → JP4 connector - ─ Wires to Jetson 40-pin header: GPIO5 SCLK, GPIO25 LRCK, - GPIO35 ASDOUT, GPIO0 MCLK - ─ Pure I2S slave, no networking - ─ Firmware flashed via Windows ESP32 flash download tool over - single USB-micro-B - - (2) ESP32-C6 (optional connectivity sidecar): - ─ Thread/Matter via UART /dev/ttyTHS1 @ 115200 (configurable - device_path /dev/ttyACM0 or /dev/ttyUSB0 on dev boards) - ─ MTU 1024, response_timeout_ms 250 - ─ Reset GPIO 24, no hardware flow control - ─ Connects to Thread border router / Matter fabric for home devices - ─ Configured via [connectivity] block in geniepod.toml - - Jetson runtime networking: - ─ WiFi LAN → Home Assistant container (homeassistant.service, - :8123) — HA_TOKEN auth - ─ MQTT broker (mosquitto.conf, port 1883) — local subscription - for genie-mqtt - ─ Optional Telegram long-poll (TELEGRAM_BOT_TOKEN, allowlist by - chat_id) - ─ Optional ESP32-C6 commands relayed from voice/HA/Telegram via - genie-core's connectivity subsystem (currently state=Disabled - by default) - - Show how a "turn on kitchen light" voice command flows: - voice → STT → LLM → tool_call "home_control" - → actuation_safety::evaluate (confidence ≥ 0.78, allowed_origins, - rate_limit max_actions_per_minute_by_origin) - → HA REST API → device state changes - → genie-mqtt picks up state change event - → genie-core speaks confirmation via Piper -``` - ---- - -## 5. OS / Bring-up workflow - -``` -Title: "GenieClaw — OS, First-Boot & Service Bring-Up" - -Vertical flow from "Jetson boot" to "voice loop ready": - - POWER ON → JetPack 6.x → systemd starts geniepod.target - geniepod.target pulls in (in dependency order, parallel where possible): - - ─ [5b/6] nvpmodel -m 1 (25 W max), jetson_clocks (max) - ─ /etc/sysctl.d/99-geniepod.conf applied (vm.min_free_kbytes etc.) - ─ Optional: cma=256M boot-arg already set on extlinux.conf - - Service tree: - ┌── homeassistant.service (docker compose, :8123) - ├── genie-audio.service (one-shot) - │ /opt/geniepod/bin/genie-audio-init - │ amixer cset I2S2 routes: - │ ADMAIF1 Mux = I2S2, codec master cbm-cfm, i2s framing, - │ I2S2 Sample Rate = 24000, channels=2, bits=16 - ├── genie-whisper.service (whisper-server :8178, ggml-small, CUDA) - │ │ - │ └─ After: genie-whisper-warmup.service (one-shot) - │ ─ nc -z :8178 (poll readiness ≤ 90 s) - │ ─ sox -n -r 16000 -c 1 trim 0 1 → silent WAV - │ ─ curl -F file=@ -F language=en :8178/inference - │ ─ forces ggml-small + CUDA kernels into iGPU - ├── genie-llm.service (llama-server :8080, Phi-4-mini Q4_K_M, - │ --ctx-size 2048, --n-gpu-layers 999, --flash-attn on) - │ │ - │ └─ After: genie-llm-warmup.service (one-shot) - │ ─ curl /health poll ≤ 90 s - │ ─ curl POST /completion {"prompt":"hi","n_predict":1} - │ ─ forces Phi-4-mini into iGPU - ├── genie-core.service (main runtime, :3000) - ├── genie-governor.service (memory pressure / model swap) - ├── genie-health.service (alert webhook, 30 s polls) - ├── genie-api.service (dashboard :3080) - └── genie-mqtt.service (mosquitto bridge) - - Side box: setup-jetson.sh phases (one-time deploy audit): - [1/6] mkdir; clean stale drop-ins - [2/6] verify 6 Rust binaries - [3/6] /etc/geniepod/geniepod.toml chmod 600 - [4/6] Phi-4-mini-instruct-Q4_K_M.gguf present (auto-download - from HuggingFace if default path) - [5/6] llama-server present - [5b] nvpmodel, jetson_clocks - [5c] sysctl + CMA hints - [5e] voice prereqs audit (whisper-cli, whisper-server, sox, - ggml-small.bin, piper, voice .onnx + .onnx.json) - [5f] deep-filter binary auto-download - (deep-filter-0.5.6-aarch64-unknown-linux-gnu, ~39 MB) - [6/6] systemctl enable each genie-* unit + geniepod.target -``` - ---- - -## 6. Security workflow - -``` -Title: "GenieClaw — Security Boundaries, Gates & Audit" - -Cross-sectional view showing layered enforcement: - - STARTUP AUDIT (genie-core::security::run_audit): - ─ Critical: data dir world-readable → flag for chmod 700 - ─ Warn: data dir group-readable - ─ Warn: ha_token plaintext in config (suggest env) - ─ Warn: process running as root (suggest dedicated geniepod uid) - ─ Info: HTTP API bound to 127.0.0.1 only (not exposed) - ─ Info: sensitive env vars (HA_TOKEN, TELEGRAM_BOT_TOKEN, *_KEY, - *_SECRET, *_TOKEN) excluded from tool execution - - REQUEST ORIGIN AT ENTRY: - voice → ToolExecutionContext { request_origin: Voice } - api → ToolExecutionContext { request_origin: Api } - telegram → ToolExecutionContext { request_origin: Telegram } - repl → ToolExecutionContext { request_origin: Repl } - - TOOL GATE (CoreConfig.tool_policy — partially enforced today, - fully enforced under issue #22): - allowed_tools_by_origin / denied_tools_by_origin / wildcards - rate-limit window: max_actions_per_minute / per origin - confirmation flow: confirmed=false → require second call - - ACTUATION SAFETY (CoreConfig.actuation_safety): - min_target_confidence ≥ 0.78 - min_sensitive_confidence ≥ 0.90 - deny_multi_target_sensitive = true - require_available_state = true - allowed_origins list - sliding-window rate caps per origin - - SKILL LOADER (CoreConfig.skill_policy): - require_manifest / require_signature - denied_permissions list (network.raw, filesystem.write, …) - each loaded skill's manifest hash logged - - AUDIT TRAIL (today): - /opt/geniepod/data/runtime/contracts.jsonl ← prompt + tool + policy - contract hashes recorded each boot - - AUDIT TRAIL (beta-track #24): - /opt/geniepod/data/audit/events.jsonl ← append-only, hash-chained - events: voice_cycle, tool_call_decision, tool_call_executed, - actuation, memory_write, skill_load, config_change - daily rotation + sealed footer signature - genie-ctl audit { tail | verify | export } - - BETA-TRACK GAPS (call out as TODO in red): - #22 single chokepoint enforcement for tool calls - #23 drop root, landlock/bubblewrap, subprocess + network allowlists - #24 tamper-evident hash-chained audit log -``` - ---- - -## 7. LLM runtime workflow (genie-ai-runtime) - -``` -Title: "GenieClaw — LLM Runtime (llama.cpp on iGPU)" - -Show the LLM serving subsystem in detail: - - STATIC CONFIG: - /opt/geniepod/models/phi-4-mini-instruct-q4_k_m.gguf (~2.4 GB) - llama-server invocation flags from genie-llm.service: - --model - --host 0.0.0.0 --port 8080 - --ctx-size 2048 ← halved from 4096 in #2 - --n-gpu-layers 999 - --threads 4 - --parallel 1 - --flash-attn on - --no-warmup ← warmup handled by genie-llm-warmup - - Inline NOTE in service unit: --cache-type-k/v q4_0 quant disabled — - crashes Phi-3/Phi-4 attn graph on aarch64 CUDA via ggml_reshape_2d. - - iGPU MEMORY BUDGET (7.6 GB Orin Nano total): - Phi-4-mini Q4_K_M weights ~2.4 GB - KV cache (ctx=2048, fp16) ~570 MB - whisper-small + CUDA kernels ~487 MB + activations - DeepFilterNet (subprocess) tract-loaded, ~50 MB peak - Piper (subprocess per call) short-lived, ~100 MB - System / other ~3 GB headroom - - BOOT SEQUENCE: - systemd starts genie-llm.service: - ExecStartPre: sync && drop_caches (3) ← Jetson NvMap wants - contiguous blocks - llama-server boots, loads weights, opens :8080 - genie-llm-warmup.service: - poll /health ≤ 90 s - POST /completion {"prompt":"hi","n_predict":1} - → first inference loads kernels + materializes attention - scratchpads → "warm" - - REQUEST PATH (per voice cycle): - genie-core → HTTP /completion (streaming) on :8080 - body: { messages: [system, ...history, user], - max_tokens: 256, temperature varies by reasoning mode, - stream: true } - → token stream → genie-core buffers tokens - → streaming::stream_and_speak (current: waits for full stream, - then splits sentences; #26 will stream sentences as they - complete) - - GOVERNOR INTERACTION (genie-governor.service): - poll_interval_ms = 5000 - night_start_hour 23, day_start_hour 6 - night_model_swap toggle (off by default; would swap to 9B at night) - pressure thresholds: - stop_optins_mb (Nextcloud/Jellyfin) = 500 - reduce_context_mb = 300 - swap_stt_mb = 200 (downgrade whisper to tiny) - zram_mb = 100 (enable 2 GB zram as last resort) - - TUNING LEVERS (color callouts): - ─ #25 future CPU pinning (whisper cores 2-3, llama core 4, - genie-core core 5) - ─ #26 future real streaming TTS - ─ #5 future whisper-medium fallback with context-aware - confidence escalation -``` - ---- - -## 8. Memory & conversation data flow - -``` -Title: "GenieClaw — Memory, Conversation & Speaker Identity Data Flow" - -Show the per-utterance data flow through the memory subsystem: - - USER UTTERANCE (after STT) → - ┌────────────────────────────────────────────────────────────────┐ - │ speaker_identity::identify │ - │ provider: none | fixed | local_biometric │ - │ inputs: wav_path, transcript, detected_language │ - │ output: SpeakerIdentity { name, confidence } │ - │ storage: /opt/geniepod/data/speakers/ │ - └────────────────────────────────────────────────────────────────┘ - ↓ - ┌────────────────────────────────────────────────────────────────┐ - │ identity::build_memory_read_context │ - │ filters: shared-room safety (default), redaction rules │ - │ sensitivity: shared | person | private | restricted │ - │ spoken_policy: allowed | redacted │ - └────────────────────────────────────────────────────────────────┘ - ↓ - ┌────────────────────────────────────────────────────────────────┐ - │ inject::build_memory_context_with_read_context │ - │ ranks: durable MEMORY.md + namespaces/ + INDEX.md │ - │ redaction: non-shared-safe entries projected as [redacted] │ - │ output: "Relevant household context:" prelude appended to │ - │ the system prompt │ - └────────────────────────────────────────────────────────────────┘ - ↓ - LLM → response - - POST-RESPONSE BACKGROUND TASK: - extract::extract_and_store - ─ heuristic + LLM-assisted extraction of durable facts - ─ tags: scope, sensitivity, spoken_policy - ─ writes to memory store (sqlite + markdown projection) - - PERSISTENT STATE: - sqlite (conversation_store): - conv_id, role, content, tool_name (optional), - created_at; bounded by max_history_turns - durable memory: - memory/MEMORY.md ← shared-room safe entries only - memory/INDEX.md ← generated tree entry point - memory/namespaces/ ← person / private / restricted - (markdown projection of structured - rows, redacted by default) - data/runtime/contracts.jsonl ← prompt/tool/policy hashes - at each boot - - TOOL `memory_recall`: - voice → "Who is Christine?" - → memory_recall(query) - → fuzzy + namespace-scoped search bounded by read_context - → returns short fact ("Your name is Jared") - → LLM speaks summary via Piper - - CONFIG KNOBS: - [core] max_history_turns = 20 - [core.speaker_identity] enabled, provider, fixed_name, - fixed_confidence, local_profile_dir, - local_min_score -```