feat: add managed local AI with llama-server - #1178
Conversation
|
🦞👀 Pull request received. I will update this pull request when review starts. |
|
Codex review: found issues before merge. Reviewed August 20, 2026, 7:53 PM ET / 23:53 UTC. ClawSweeper reviewWhat this changesThis PR adds opt-in, hardware-qualified Windows Local AI setup that installs and manages pinned llama-server and model artifacts, connects them to the WSL gateway, and exposes tray controls. Merge readinessKeep open. The current-head proof is meaningful, but two P2 blockers remain: the runtime downloader follows unvalidated redirects, and hardware qualification ignores the configured 262K F16 context cache. Priority: P2 Review scores
Verification
How this fits togetherThe setup wizard qualifies NVIDIA hardware, installs and verifies native inference artifacts, then publishes a loopback endpoint to the managed WSL gateway. The tray owns later runtime lifecycle controls and displays their state. flowchart LR
A[Setup choices] --> B[Hardware qualification]
B --> C[Verified runtime and model]
C --> D[Native local inference]
D --> E[WSL gateway provider]
E --> F[Tray status and controls]
Decision needed
Why: The current 2 GiB margin cannot express the actual cache requirement, so choosing the supported-memory contract changes the user-visible product promise. Before merge
Findings
Agent review detailsSecurityNeeds attention: The new runtime downloader follows automatic redirects without the explicit destination validation used for model downloads. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Use one bounded, tested redirect policy for both runtime and model artifacts, then make eligibility calculate the actual recipe footprint or deliberately choose a smaller context profile for lower-memory GPUs. Do we have a high-confidence way to reproduce the issue? Yes for the source-level failures: a redirecting runtime asset request reaches the default automatic-redirect handler, and the added 16 GiB qualification case selects a recipe whose cache is not included in admission sizing. Is this the best way to solve the issue? No. Pinned hashes protect artifact integrity after transfer, but redirect destinations still need validation, and model weights plus 2 GiB does not model the configured inference footprint. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against d3ed6c78bf14. LabelsLabel changes:
Label justifications:
EvidenceSecurity concerns:
What I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (11 earlier review cycles; latest 8 shown)
|
|
This is an impressive, thoughtfully structured feature, especially the immutable artifact pins, SHA-256 checks, path containment, redirect allowlist, process ownership, and rollback compare-and-swap behavior. I validated the exact head on Windows: the full build and Shared, Tray, and SetupEngine unit suites pass. I also verified the llama.cpp and Hugging Face revisions, filenames, sizes, and hashes against upstream. It is not merge-ready yet. Two independent reviews agree on these blockers:
The proof gap is also material: this adds 9,089 changed production lines and 127 runtime types with zero new focused Before merge, please fix the five blockers, add focused regression coverage, and provide redacted current-head proof of a verified download, first inference, WSL reachability, restart, rollback, and interrupted-install recovery. The direction is excellent. The remaining work is about making this large native execution surface supportable and safe to ship. |
6e8fd0b to
411493c
Compare
|
Current-head E2E behavior proof is now recorded in the PR body for
The PR body also states the remaining proof boundary honestly: this run reused already hash-verified artifacts, and destructive interruption/rollback were covered by focused tests rather than repeated as live destructive demonstrations. |
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Sanitized current-head onboarding and Local AI behavior proof for The recording shows the full maximized setup flow, successful native inference before WSL provisioning, the connected dashboard, and a live Local AI restart returning to |
411493c to
db8fd59
Compare
a102f79 to
d241b7c
Compare
Move physical-memory discovery into one shared implementation. Give setup and inference code a consistent RAM capacity value. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Load NVML from trusted locations and collect NVIDIA GPU capabilities. Avoid unsafe library resolution while providing selector-grade hardware data. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Correlate NVML adapters with unambiguous DXGI memory observations and count shared memory for any NVIDIA GPU. Fail closed on duplicate or ambiguous adapter names so model selection cannot borrow another device's budget. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Resume interrupted GGUF downloads with strict Range and Content-Range handling. Verify size and hash before atomic promotion, restarting safely when resume is unsupported. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Install and inspect the pinned llama-server and CUDA runtime components. Safely reconcile exact orphan runtime paths so interrupted promotion can retry. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Launch llama-server on demand with an OS-assigned or validated fixed port. Prove listener ownership by child PID and start time before health checks or persistence. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Make Local AI opt-in and inspect hardware plus WSL viability without mutation. Acquire verified native inference before WSL provisioning and resolve bundled defaults reliably from RID-specific output. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Reconcile and reuse only exact manifest-owned runtime and model artifacts after interruption. Persist a proven healthy endpoint and clean durable app-owned state on fresh uninstall. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Probe health, execute real inference, and verify GPU activity on the owned endpoint. Fail setup cleanly when the installed Local AI stack is not operational. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Use explicit 127.0.0.1 for Windows-to-WSL gateway connections and persisted setup state. Remove only exact managed Local AI provider state during uninstall, preserving drift. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Bind Local AI to the singleton setup-managed distro resolved from the loaded gateway registry. Quiesce exact provider state before transitions, then publish only healthy owned endpoints and compensate failures. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Surface Local AI eligibility, explicit consent, review, and setup progress. Show native inference before WSL setup and describe dynamic multi-gigabyte disk use accurately. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Add Local AI navigation, status, controls, logs, and localized resources. Wire the page through application services and retain focused UI contracts. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Keep a compact Local AI review visible when hardware, WSL, or networking blocks setup. Hide unavailable controls and show every detected dependency reason through See why. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Accept full CUDA offload when either NVML memory movement or the parsed CUDA model buffer proves placement. Reject startup when neither signal accounts for the selected model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
The consent InfoBar content sat flush against the bottom edge of the warning, leaving the message and checkbox visually cramped. Add the same bottom content margin the sandbox InfoBar already uses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L8LRNTZ5fBgjGwmtBCrSp3 Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
d241b7c to
c6c96f8
Compare
Exact-head automated proofValidated commit:
E2E artifact: The E2E run removed its disposable OpenClawE2E-739379a1 distribution. The later The first E2E attempt was an infrastructure-only failure before product No runtime values or media from d241b7c, 411493c, or another stale head are Fresh redacted live proofThe isolated Dev onboarding pipeline completed all 36 steps in 450.0 seconds. The installed model is 5,868,826,976 bytes and a fresh SHA-256 calculation The Local AI page exercised Stop, Start, and Restart. The owned runtime changed The exact-head onboarding video is a real-motion 120.033-second H.264 recording Not repeated destructively in this visible run: forced interrupted acquisition @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
Current-head onboarding and Local AI behavior video for c6c96f8. This is the real full-screen setup flow of the connected dashboard and Local AI Stop, Start, and Restart controls. Video SHA-256: FD756C5BB6BD96D18EBDAC81B0462673087A781A80EC1DCCDF2DD6E05CEC302D openclaw-onboarding-timelapse.mp4 |
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
@karkarl @jacobtomlinson Updated PR with new detection scheme (not depending on specific GPU but just GPU memory capacity and runtime components). |
What Problem This Solves
Resolves a problem where Windows users who wanted local inference had no guided,
hardware-aware setup path and could be left with stale runtime, model, process,
port, or gateway state after an interruption. Users whose machines were not
ready for Local AI also lacked a concise explanation of the missing dependency.
The setup flow now qualifies compatible NVIDIA hardware before WSL installation,
offers Local AI only as an explicit opt-in, proves native inference first, and
then continues with the managed WSL gateway.
Why This Change Was Made
Local AI qualification is based on runtime capability rather than a hard-coded
GPU SKU or CPU pairing. A trusted NVML probe records NVIDIA UUID, driver, CUDA,
total memory, and free memory facts; unambiguous DXGI association can add usable
shared memory. Automatic selection chooses the largest pinned model whose
weights plus a 2 GiB reserve fit both total and currently free capacity.
The native llama.cpp runtime and GGUF model are downloaded from pinned catalog
entries, verified, installed under app-owned paths, started, and checked for a
real inference response and GPU placement before WSL setup proceeds. The
companion then resolves the single setup-managed WSL distribution from registry
state and owns the router and provider lifecycle for that distribution.
The implementation also adds exact-manifest reuse, strict partial GGUF resume,
catalog-owned orphan reconciliation, reparse-safe cleanup, fresh-process
uninstall, cancellation rollback, listener ownership checks, wildcard rejection,
and explicit 127.0.0.1 provider publication with drift-safe compensation.
Boundaries: this change qualifies NVIDIA devices exposed by NVML, not other GPU
vendors; recommends only fitting entries from the pinned model catalog; binds
the managed inference endpoint to local IPv4 loopback; and leaves Windows
session-container inference and additional model catalogs for later work. This
code-focused branch does not add documentation, screenshots, reports, or video.
User Impact
Users with a qualifying NVIDIA GPU can opt into Local AI during onboarding,
choose from models that fit their detected capacity, and see native inference
working before committing to WSL setup. Setup can safely recover from an
interrupted download or install and can uninstall owned state from a fresh
process without removing user-modified configuration.
Users who cannot run Local AI see a small unavailable notice and can open See why
for hardware, driver, CUDA, WSL, and networking diagnostics. The tray exposes
Local AI status and lifecycle controls after setup.
Evidence
All automated results below were collected from exact local head
c6c96f8:
explicitly recreated only the isolated Dev distribution; Ubuntu and the
production OpenClawGateway remained untouched.
The first E2E attempt stopped before product validation because the required x64
tray artifact was absent. After an x64 build completed with 0 errors and one
generated-code warning, the corrected run passed all 17 tests. This was an
infrastructure-only first-attempt failure.
No screenshot, recording, PID, port, inference, GPU, WSL, or restart value from
an older head is used as proof for this revision.
Fresh live proof from exact c6c96f8:
layers, and 20,816,330,752 bytes of observed GPU load growth.
2026-08-20T23:15:01.6521200Z, listening only on 127.0.0.1:54935.
{"status":"ok"}from the current endpoint.model matched the installed Qwen alias.
to 30380:54921 and then 36768:54935 while the gateway remained Connected.
verified at 3270x2180/30 fps. Its SHA-256 is
FD756C5BB6BD96D18EBDAC81B0462673087A781A80EC1DCCDF2DD6E05CEC302D. It opens
Local AI and shows Stop, Start, Restart, Running, and Connected. The local
artifact remains uncommitted and is not part of this code diff.
Post-push status: PR #1178 now resolves to exact head c6c96f8 with 25 commits
on base d3ed6c7, and Allow edits by maintainers remains enabled. CodeQL and
repository hygiene are green in Build and Test run
32428513668;
the remaining build/test and E2E jobs are still running.
Change Type
Scope
Validation
Commands and results at c6c96f8:
Focused bundled-config validation:
Setup-and-connect E2E:
Repository check:
Real Behavior Proof
and an NVIDIA GPU qualified by capability rather than SKU; plus the automated
Windows setup-and-connect E2E environment.
c6c96f8.
Validation, followed by a fresh isolated Dev onboarding run, Windows and WSL
health probes, manifest/receipt verification, and Stop/Start/Restart controls.
setup journal; exact manifest receipt; Windows and WSL health output; verified
local video metadata and boundary frames.
inference and full GPU offload were proven before WSL creation; WSL reached
the managed endpoint; lifecycle controls recovered on new owned ports.
remain uncommitted and outside the code diff.
locally and uncommitted by design, so there is no remote media URL in this
code-only update. No older-head runtime values are substituted.
Security Impact
New permissions or capabilities? Yes
Secrets or tokens handling changed? No
New or changed network calls? Yes
Command or tool execution surface changed? Yes
Data access scope changed? Yes
If any answer is Yes, explain the risk and mitigation:
Setup can download pinned runtime/model artifacts, invoke the native inference
process and scoped WSL commands, read NVIDIA telemetry, and write app-owned
runtime, model, manifest, provider, and consented WSL configuration state.
Downloads require expected size and digest verification before promotion.
NVML loads only from trusted locations. Cleanup validates app-owned paths and
rejects reparse traversal. The router rejects wildcard binds, validates
listener ownership by PID and process start time, and publishes an explicit
127.0.0.1 endpoint. Configuration rollback and uninstall preserve concurrent
or user-drifted state.
Compatibility and Migration
Backward compatible? Yes
Config or environment changes? Yes
Migration needed? No
If yes, list the exact upgrade steps:
No manual upgrade step is required. Legacy Local AI manifest/profile data is
read compatibly. New installations create app-owned manifests and provider
state. Mirrored WSL networking is changed only after explicit user consent,
with byte-preserving restore and rollback behavior.
Review Conversations
These boxes should be completed only after the rewritten head is pushed, CI
finishes, and the current-head proof comment receives re-review.