feat(goal): opt-in local skeptic — route the review to a separate endpoint#110
Merged
Conversation
…point The /goal skeptic reviews every advancing turn and is fail-open — the ideal role to offload to a local model (cost/rate-limit free, private) while the coding driver and planner stay on the strong session model. Add an optional skeptic endpoint: when config.skeptic_endpoint is set, the agent builds a separate OpenAI-compatible provider (e.g. a local hi-local MLX/CUDA server) and the per-turn skeptic review runs against it instead of the session provider. Off by default; requires skeptic_model to name a model the endpoint serves. Set via HI_SKEPTIC_ENDPOINT (+ optional HI_SKEPTIC_ENDPOINT_KEY); the agent stays backend-agnostic — MLX vs CUDA is entirely hi-local's concern. Everything else (driver, planner, audit, milestone split) is unchanged and stays on the main provider. Test: with a side provider set, the skeptic review routes there (its objection takes effect) while the main provider scripts only the drive turn — proving the review no longer hits the session provider. Unset falls back exactly as before (covered by the existing skeptic suite). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
davidrhodus
pushed a commit
that referenced
this pull request
Jul 17, 2026
…skeptic Turn the /goal skeptic review over to a small local model with one command, building on the opt-in skeptic_endpoint from #110. `/config skeptic-local on` now does the whole setup automatically: - detect the machine's local-inference backend (Apple-Silicon MLX or NVIDIA CUDA); no backend → clear message, skeptic stays on the main model - resolve a small default review model (~3B, 4-bit), overridable via HI_SKEPTIC_LOCAL_REPO / _MODEL_ID / _GGUF - download it if it isn't cached, reusing the same .hi/models layout as `/hf run --mlx` so a model fetched either way is shared - launch `hi-local serve` and wait for /health, then point skeptic_endpoint/skeptic_model at it and rebuild the skeptic provider - `/config skeptic-local off` stops the server and restores prior settings Every step degrades gracefully: a missing backend, missing binary, failed download, or unhealthy server returns an error and leaves the skeptic on the main provider — config is only mutated after the server is healthy. The server is torn down on session exit (CLI via kill_background_processes, TUI via a drop guard). The blocking, progress-to-terminal download runs only on the plain CLI; the TUI reports NeedsDownload rather than corrupting its alternate screen. New: hi-tools `local_server` (spawn + health, private registry) and hi-agent `local_skeptic` (backend/model policy + Agent enable/disable). Deterministic core is unit-tested; the live download+spawn is hardware-validated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
davidrhodus
pushed a commit
that referenced
this pull request
Jul 17, 2026
…skeptic Turn the /goal skeptic review over to a small local model with one command, building on the opt-in skeptic_endpoint from #110. `/config skeptic-local on` now does the whole setup automatically: - detect the machine's local-inference backend (Apple-Silicon MLX or NVIDIA CUDA); no backend → clear message, skeptic stays on the main model - resolve a small default review model (~3B, 4-bit), overridable via HI_SKEPTIC_LOCAL_REPO / _MODEL_ID / _GGUF - download it if it isn't cached, reusing the same .hi/models layout as `/hf run --mlx` so a model fetched either way is shared - launch `hi-local serve` and wait for /health, then point skeptic_endpoint/skeptic_model at it and rebuild the skeptic provider - `/config skeptic-local off` stops the server and restores prior settings Every step degrades gracefully: a missing backend, missing binary, failed download, or unhealthy server returns an error and leaves the skeptic on the main provider — config is only mutated after the server is healthy. The server is torn down on session exit (CLI via kill_background_processes, TUI via a drop guard). The blocking, progress-to-terminal download runs only on the plain CLI; the TUI reports NeedsDownload rather than corrupting its alternate screen. New: hi-tools `local_server` (spawn + health, private registry) and hi-agent `local_skeptic` (backend/model policy + Agent enable/disable). Deterministic core is unit-tested; the live download+spawn is hardware-validated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Optional local model for the
/goalskepticFollowing the discussion: the
/goalskeptic reviews every advancing turn and is fail-open — the ideal role to run on a local model. Offloading it is cost/rate-limit free and private, and loses almost nothing (a weak local skeptic just approves), while the coding driver and planner stay on the strong session model where quality matters.What it does
AgentConfig::skeptic_endpoint(+skeptic_endpoint_key). When set, the agent builds a separate OpenAI-compatible provider and the per-turn skeptic review runs against it instead of the session provider. Unset → the skeptic uses the main provider exactly as before.hi-local's concern — point it at a runninghi-localserver.How to use it
Then
/goaldrives on your session model and reviews on the local one.Scope
Only the skeptic review is routed. The driver, planner, completion audit, and milestone split all stay on the main provider (the planner is the one role where a weak model measurably hurts, so it's intentionally left alone).
(Fully-local
/goal— driver included — already works today by pointing the whole session provider at a local endpoint; no code needed. This PR adds the more valuable hybrid.)Tests & gates
-D warnings), and the ratchet pass (verified against the CI toolchain).