Skip to content

feat(goal): /config skeptic-local — auto-managed local model for the skeptic#111

Merged
davidrhodus merged 1 commit into
mainfrom
goal-local-skeptic-auto
Jul 17, 2026
Merged

feat(goal): /config skeptic-local — auto-managed local model for the skeptic#111
davidrhodus merged 1 commit into
mainfrom
goal-local-skeptic-auto

Conversation

@davidrhodus

Copy link
Copy Markdown
Contributor

Stacked on #110 (opt-in skeptic_endpoint). Makes the local /goal skeptic fully automated behind one toggle: /config skeptic-local on.

What it does

/config skeptic-local on runs the whole setup, each step failing soft:

  1. Detect backend — Apple-Silicon MLX or NVIDIA CUDA (nvidia-smi probe). No backend → message, skeptic stays on the main model.
  2. Resolve model — a small default review model (~3B, 4-bit) per backend, overridable via HI_SKEPTIC_LOCAL_REPO / HI_SKEPTIC_LOCAL_MODEL_ID / HI_SKEPTIC_LOCAL_GGUF.
  3. Download if missing — reuses the same .hi/models cache layout as /hf run --mlx, so a model fetched either way is shared.
  4. Launch + health-poll — spawns hi-local serve and waits for /health (verbatim of the proven /hf path).
  5. Wire it — sets skeptic_endpoint/skeptic_model and rebuilds the skeptic provider. The driver and planner stay on the main model.

/config skeptic-local off stops the server and restores the prior skeptic settings.

Safety / graceful degradation

  • Config is only mutated after the server reports healthy — any failure (no backend, missing hi-local, download fail, unhealthy server) returns an error and leaves the skeptic on the main provider. No hangs, no half-wired state.
  • Server is torn down on session exit: CLI via kill_background_processes, TUI via a drop guard covering every exit path (private registry, stop_all_local_servers).
  • The blocking, progress-to-terminal download runs only on the plain CLI; the TUI returns NeedsDownload instead of corrupting its alternate screen.

Structure

  • hi-tools/local_server.rs (new) — process/HTTP mechanics: spawn into a private background registry, /health wait, stop-by-id / stop-all, skeptic_model_dir cache path.
  • hi-agent/local_skeptic.rs (new) — policy + orchestration: backend detection, default-model registry, presence checks, serve arg builder, and Agent::enable_local_skeptic / disable_local_skeptic.
  • /config skeptic-local <on|off> parsing (ConfigArg::SkepticLocal) wired into both frontends; the CLI routes it through an async arm (it may download/spawn).

Tests

  • 8 unit tests over the deterministic core (backend selection, model registry, presence checks, arg builder, config parsing/routing). cargo test -p hi-agent -p hi-tools green (564 + 229).
  • fmt / clippy (-D warnings) / file-size ratchet all pass.
  • Not exercised in CI: the live multi-GB download + real MLX/CUDA server spawn — validated on local hardware.

🤖 Generated with Claude Code

@davidrhodus
davidrhodus changed the base branch from goal-local-skeptic to main July 17, 2026 03:59
…skeptic

Turn the /goal skeptic review over to a small local model with one command,
building on the opt-in skeptic_endpoint from #110. `/config skeptic-local on`
now does the whole setup automatically:

  - detect the machine's local-inference backend (Apple-Silicon MLX or NVIDIA
    CUDA); no backend → clear message, skeptic stays on the main model
  - resolve a small default review model (~3B, 4-bit), overridable via
    HI_SKEPTIC_LOCAL_REPO / _MODEL_ID / _GGUF
  - download it if it isn't cached, reusing the same .hi/models layout as
    `/hf run --mlx` so a model fetched either way is shared
  - launch `hi-local serve` and wait for /health, then point
    skeptic_endpoint/skeptic_model at it and rebuild the skeptic provider
  - `/config skeptic-local off` stops the server and restores prior settings

Every step degrades gracefully: a missing backend, missing binary, failed
download, or unhealthy server returns an error and leaves the skeptic on the
main provider — config is only mutated after the server is healthy. The server
is torn down on session exit (CLI via kill_background_processes, TUI via a drop
guard). The blocking, progress-to-terminal download runs only on the plain CLI;
the TUI reports NeedsDownload rather than corrupting its alternate screen.

New: hi-tools `local_server` (spawn + health, private registry) and hi-agent
`local_skeptic` (backend/model policy + Agent enable/disable). Deterministic
core is unit-tested; the live download+spawn is hardware-validated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@davidrhodus
davidrhodus force-pushed the goal-local-skeptic-auto branch from 289d6be to d85f709 Compare July 17, 2026 03:59
@davidrhodus
davidrhodus merged commit 39d13f2 into main Jul 17, 2026
1 check passed
@davidrhodus
davidrhodus deleted the goal-local-skeptic-auto branch July 17, 2026 03:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant