feat: add managed local AI with llama-server - #1178
Conversation
|
🦞👀 Pull request received. I will update this pull request when review starts. |
|
Codex review: needs real behavior proof before merge. Reviewed August 20, 2026, 9:42 PM ET / August 21, 2026, 01:42 UTC. ClawSweeper reviewWhat this changesThe PR adds opt-in Windows Local AI onboarding, verified llama-server and model installation, WSL gateway provider routing, and tray lifecycle controls. Merge readiness⛔ Blocked until stronger real behavior proof is added - 8 items remain Keep open. The Local AI direction remains unique, but runtime download redirects and GPU capacity qualification have two concrete P2 blockers, and the available live proof predates the current head. Priority: P2 Review scores
Verification
How this fits togetherWindows onboarding qualifies local hardware, downloads and verifies native inference components, then publishes a loopback provider to the app-managed WSL gateway. The tray observes and controls the managed runtime after setup. flowchart LR
A[Windows onboarding] --> B[Hardware qualification]
B --> C[Verified runtime and model]
C --> D[Native local inference]
D --> E[WSL gateway provider]
E --> F[Tray status and controls]
Before merge
Findings
Agent review detailsSecurityNeeds attention: The pinned runtime hashes are useful, but unchecked automatic redirects leave a concrete trusted-download-boundary gap. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Manually restrict runtime redirects to approved HTTPS release hosts, qualify capacity against the full llama-server recipe or reduce its context target, then attach a redacted current-head WSL recovery proof. Do we have a high-confidence way to reproduce the issue? Yes, source-reproducible: runtime archive requests follow redirects without final-target validation, and qualification counts weights plus 2 GiB while the router configures a 262K F16 KV cache. Is this the best way to solve the issue? No. The current approach needs explicit runtime redirect policy and full recipe-based capacity accounting before it is safe to ship. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against d3ed6c78bf14. LabelsLabel justifications:
EvidenceSecurity concerns:
What I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (14 earlier review cycles; latest 8 shown)
|
|
This is an impressive, thoughtfully structured feature, especially the immutable artifact pins, SHA-256 checks, path containment, redirect allowlist, process ownership, and rollback compare-and-swap behavior. I validated the exact head on Windows: the full build and Shared, Tray, and SetupEngine unit suites pass. I also verified the llama.cpp and Hugging Face revisions, filenames, sizes, and hashes against upstream. It is not merge-ready yet. Two independent reviews agree on these blockers:
The proof gap is also material: this adds 9,089 changed production lines and 127 runtime types with zero new focused Before merge, please fix the five blockers, add focused regression coverage, and provide redacted current-head proof of a verified download, first inference, WSL reachability, restart, rollback, and interrupted-install recovery. The direction is excellent. The remaining work is about making this large native execution surface supportable and safe to ship. |
6e8fd0b to
411493c
Compare
|
Current-head E2E behavior proof is now recorded in the PR body for
The PR body also states the remaining proof boundary honestly: this run reused already hash-verified artifacts, and destructive interruption/rollback were covered by focused tests rather than repeated as live destructive demonstrations. |
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Sanitized current-head onboarding and Local AI behavior proof for The recording shows the full maximized setup flow, successful native inference before WSL provisioning, the connected dashboard, and a live Local AI restart returning to |
411493c to
db8fd59
Compare
a102f79 to
d241b7c
Compare
Move physical-memory discovery into one shared implementation. Give setup and inference code a consistent RAM capacity value. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
@karkarl @jacobtomlinson Updated PR with new detection scheme (not depending on specific GPU but just GPU memory capacity and runtime components). |
c6c96f8 to
1e4c2f7
Compare
Keep a compact Local AI review visible when hardware, WSL, or networking blocks setup. Hide unavailable controls and show every detected dependency reason through See why. Co-authored-by: Karen Lai <7976322+karkarl@users.noreply.github.com> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Accept full CUDA offload when either NVML memory movement or the parsed CUDA model buffer proves placement. Reject startup when neither signal accounts for the selected model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
The consent InfoBar content sat flush against the bottom edge of the warning, leaving the message and checkbox visually cramped. Add the same bottom content margin the sandbox InfoBar already uses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L8LRNTZ5fBgjGwmtBCrSp3 Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
1e4c2f7 to
2bcf986
Compare
Wait for the direct child process independently of redirected-pipe EOF, then bound the final output drain. Existing-distro inspection now fails closed when WSL times out or returns an unknown error. Show progress during the Welcome-page inspection so a cold WSL service does not make setup look frozen. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
After an elevated WSL install fails, run a best-effort GitHub quota diagnostic and distinguish likely quota exhaustion from other download failures. Diagnostic timeouts never replace the original failure. Offer Store, winget, and elevated PowerShell recovery routes, and allow the platform step to be retried. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
What Problem This Solves
Resolves a problem where Windows users who wanted local inference had no guided,
hardware-aware setup path and could be left with stale runtime, model, process,
port, or gateway state after an interruption. Users whose machines were not
ready for Local AI also lacked a concise explanation of the missing dependency.
The setup flow now qualifies compatible NVIDIA hardware before WSL installation,
offers Local AI only as an explicit opt-in, proves native inference first, and
then continues with the managed WSL gateway.
Why This Change Was Made
Local AI qualification is based on runtime capability rather than a hard-coded
GPU SKU or CPU pairing. A trusted NVML probe records NVIDIA UUID, driver, CUDA,
total memory, and free memory facts; unambiguous DXGI association can add usable
shared memory. Automatic selection chooses the largest pinned model whose
weights plus a 2 GiB reserve fit both total and currently free capacity.
The native llama.cpp runtime and GGUF model are downloaded from pinned catalog
entries, verified, installed under app-owned paths, started, and checked for a
real inference response and GPU placement before WSL setup proceeds. The
companion then resolves the single setup-managed WSL distribution from registry
state and owns the router and provider lifecycle for that distribution.
The implementation also adds exact-manifest reuse, strict partial GGUF resume,
catalog-owned orphan reconciliation, reparse-safe cleanup, fresh-process
uninstall, cancellation rollback, listener ownership checks, wildcard rejection,
and explicit 127.0.0.1 provider publication with drift-safe compensation.
Boundaries: this change qualifies NVIDIA devices exposed by NVML, not other GPU
vendors; recommends only fitting entries from the pinned model catalog; binds
the managed inference endpoint to local IPv4 loopback; and leaves Windows
session-container inference and additional model catalogs for later work. This
code-focused branch does not add documentation, screenshots, reports, or video.
User Impact
Users with a qualifying NVIDIA GPU can opt into Local AI during onboarding,
choose from models that fit their detected capacity, and see native inference
working before committing to WSL setup. Setup can safely recover from an
interrupted download or install and can uninstall owned state from a fresh
process without removing user-modified configuration.
Users who cannot run Local AI see a small unavailable notice and can open See why
for hardware, driver, CUDA, WSL, and networking diagnostics. The tray exposes
Local AI status and lifecycle controls after setup.
Evidence
All automated results below were collected from exact local head
c6c96f8:
explicitly recreated only the isolated Dev distribution; Ubuntu and the
production OpenClawGateway remained untouched.
The first E2E attempt stopped before product validation because the required x64
tray artifact was absent. After an x64 build completed with 0 errors and one
generated-code warning, the corrected run passed all 17 tests. This was an
infrastructure-only first-attempt failure.
No screenshot, recording, PID, port, inference, GPU, WSL, or restart value from
an older head is used as proof for this revision.
Fresh live proof from exact c6c96f8:
layers, and 20,816,330,752 bytes of observed GPU load growth.
2026-08-20T23:15:01.6521200Z, listening only on 127.0.0.1:54935.
{"status":"ok"}from the current endpoint.model matched the installed Qwen alias.
to 30380:54921 and then 36768:54935 while the gateway remained Connected.
verified at 3270x2180/30 fps. Its SHA-256 is
FD756C5BB6BD96D18EBDAC81B0462673087A781A80EC1DCCDF2DD6E05CEC302D. It opens
Local AI and shows Stop, Start, Restart, Running, and Connected. The local
artifact remains uncommitted and is not part of this code diff.
Post-push status: PR #1178 now resolves to exact head c6c96f8 with 25 commits
on base d3ed6c7, and Allow edits by maintainers remains enabled. CodeQL and
repository hygiene are green in Build and Test run
32428513668;
the remaining build/test and E2E jobs are still running.
Change Type
Scope
Validation
Commands and results at c6c96f8:
Focused bundled-config validation:
Setup-and-connect E2E:
Repository check:
Real Behavior Proof
and an NVIDIA GPU qualified by capability rather than SKU; plus the automated
Windows setup-and-connect E2E environment.
c6c96f8.
Validation, followed by a fresh isolated Dev onboarding run, Windows and WSL
health probes, manifest/receipt verification, and Stop/Start/Restart controls.
setup journal; exact manifest receipt; Windows and WSL health output; verified
local video metadata and boundary frames.
inference and full GPU offload were proven before WSL creation; WSL reached
the managed endpoint; lifecycle controls recovered on new owned ports.
remain uncommitted and outside the code diff.
locally and uncommitted by design, so there is no remote media URL in this
code-only update. No older-head runtime values are substituted.
Security Impact
New permissions or capabilities? Yes
Secrets or tokens handling changed? No
New or changed network calls? Yes
Command or tool execution surface changed? Yes
Data access scope changed? Yes
If any answer is Yes, explain the risk and mitigation:
Setup can download pinned runtime/model artifacts, invoke the native inference
process and scoped WSL commands, read NVIDIA telemetry, and write app-owned
runtime, model, manifest, provider, and consented WSL configuration state.
Downloads require expected size and digest verification before promotion.
NVML loads only from trusted locations. Cleanup validates app-owned paths and
rejects reparse traversal. The router rejects wildcard binds, validates
listener ownership by PID and process start time, and publishes an explicit
127.0.0.1 endpoint. Configuration rollback and uninstall preserve concurrent
or user-drifted state.
Compatibility and Migration
Backward compatible? Yes
Config or environment changes? Yes
Migration needed? No
If yes, list the exact upgrade steps:
No manual upgrade step is required. Legacy Local AI manifest/profile data is
read compatibly. New installations create app-owned manifests and provider
state. Mirrored WSL networking is changed only after explicit user consent,
with byte-preserving restore and rollback behavior.
Review Conversations
These boxes should be completed only after the rewritten head is pushed, CI
finishes, and the current-head proof comment receives re-review.