Skip to content

feat: add managed local AI with llama-server - #1178

Open
joelagnel wants to merge 27 commits into
openclaw:mainfrom
joelagnel:feature/local-ai-llama-server-code-only
Open

feat: add managed local AI with llama-server#1178
joelagnel wants to merge 27 commits into
openclaw:mainfrom
joelagnel:feature/local-ai-llama-server-code-only

Conversation

@joelagnel

@joelagnel joelagnel commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Resolves a problem where Windows users who wanted local inference had no guided,
hardware-aware setup path and could be left with stale runtime, model, process,
port, or gateway state after an interruption. Users whose machines were not
ready for Local AI also lacked a concise explanation of the missing dependency.

The setup flow now qualifies compatible NVIDIA hardware before WSL installation,
offers Local AI only as an explicit opt-in, proves native inference first, and
then continues with the managed WSL gateway.

Why This Change Was Made

Local AI qualification is based on runtime capability rather than a hard-coded
GPU SKU or CPU pairing. A trusted NVML probe records NVIDIA UUID, driver, CUDA,
total memory, and free memory facts; unambiguous DXGI association can add usable
shared memory. Automatic selection chooses the largest pinned model whose
weights plus a 2 GiB reserve fit both total and currently free capacity.

The native llama.cpp runtime and GGUF model are downloaded from pinned catalog
entries, verified, installed under app-owned paths, started, and checked for a
real inference response and GPU placement before WSL setup proceeds. The
companion then resolves the single setup-managed WSL distribution from registry
state and owns the router and provider lifecycle for that distribution.

The implementation also adds exact-manifest reuse, strict partial GGUF resume,
catalog-owned orphan reconciliation, reparse-safe cleanup, fresh-process
uninstall, cancellation rollback, listener ownership checks, wildcard rejection,
and explicit 127.0.0.1 provider publication with drift-safe compensation.

Boundaries: this change qualifies NVIDIA devices exposed by NVML, not other GPU
vendors; recommends only fitting entries from the pinned model catalog; binds
the managed inference endpoint to local IPv4 loopback; and leaves Windows
session-container inference and additional model catalogs for later work. This
code-focused branch does not add documentation, screenshots, reports, or video.

User Impact

Users with a qualifying NVIDIA GPU can opt into Local AI during onboarding,
choose from models that fit their detected capacity, and see native inference
working before committing to WSL setup. Setup can safely recover from an
interrupted download or install and can uninstall owned state from a fresh
process without removing user-modified configuration.

Users who cannot run Local AI see a small unavailable notice and can open See why
for hardware, driver, CUDA, WSL, and networking diagnostics. The tray exposes
Local AI status and lifecycle controls after setup.

Evidence

All automated results below were collected from exact local head
c6c96f8:

  • Build: PASS for all five ARM64 projects.
  • OpenClaw.Shared.Tests: 3,808 passed, 32 skipped, 0 failed.
  • OpenClaw.Tray.Tests: 2,702 passed, 0 failed.
  • OpenClaw.SetupEngine.Tests: 933 passed, 0 failed.
  • OpenClaw.Connection.Tests: 680 passed, 0 failed.
  • x64 bundled-config regression with OPENCLAW_REPO_ROOT unset: 1/1 passed.
  • git diff --check: clean.
  • Setup-and-connect E2E: 17/17 passed in 294.5 seconds.
  • TRX: TestResults/E2E/SetupAndConnect-c6c96f85.trx.
  • Cleanup: disposable OpenClawE2E-739379a1 was removed. The later video run
    explicitly recreated only the isolated Dev distribution; Ubuntu and the
    production OpenClawGateway remained untouched.

The first E2E attempt stopped before product validation because the required x64
tray artifact was absent. After an x64 build completed with 0 errors and one
generated-code warning, the corrected run passed all 17 tests. This was an
infrastructure-only first-attempt failure.

No screenshot, recording, PID, port, inference, GPU, WSL, or restart value from
an older head is used as proof for this revision.

Fresh live proof from exact c6c96f8:

  • Full setup completed 36 steps in 450.0 seconds.
  • The 5,868,826,976-byte Qwen3.5 9B artifact matched its SHA-256 receipt.
  • First inference completed before WSL mutation: 32 generated tokens, 34/34 GPU
    layers, and 20,816,330,752 bytes of observed GPU load growth.
  • The final managed process was PID 36768, started
    2026-08-20T23:15:01.6521200Z, listening only on 127.0.0.1:54935.
  • Windows and OpenClawGateway-Dev both received HTTP 200
    {"status":"ok"} from the current endpoint.
  • The WSL provider base URL matched http://127.0.0.1:54935/v1 and its primary
    model matched the installed Qwen alias.
  • Stop, Start, and Restart were exercised. The runtime moved from 28472:54883
    to 30380:54921 and then 36768:54935 while the gateway remained Connected.
  • A real-motion 120.033-second onboarding-to-dashboard recording was visually
    verified at 3270x2180/30 fps. Its SHA-256 is
    FD756C5BB6BD96D18EBDAC81B0462673087A781A80EC1DCCDF2DD6E05CEC302D. It opens
    Local AI and shows Stop, Start, Restart, Running, and Connected. The local
    artifact remains uncommitted and is not part of this code diff.

Post-push status: PR #1178 now resolves to exact head c6c96f8 with 25 commits
on base d3ed6c7, and Allow edits by maintainers remains enabled. CodeQL and
repository hygiene are green in Build and Test run
32428513668
;
the remaining build/test and E2E jobs are still running.

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs or instructions
  • Tests or validation
  • Security hardening
  • Chore or infrastructure

Scope

  • Tray or WinUI UX
  • Windows node capability
  • Local MCP or winnode
  • Gateway, connection, or pairing
  • Setup or onboarding
  • Permissions, privacy, or security
  • Tests, CI, or docs

Validation

Commands and results at c6c96f8:

powershell -NoProfile -ExecutionPolicy Bypass -File .\build.ps1
PASS: all five ARM64 projects

dotnet test .\tests\OpenClaw.Shared.Tests\OpenClaw.Shared.Tests.csproj --no-restore
PASS: 3,808 passed; 32 skipped; 0 failed

dotnet test .\tests\OpenClaw.Tray.Tests\OpenClaw.Tray.Tests.csproj --no-restore
PASS: 2,702 passed; 0 failed

dotnet test .\tests\OpenClaw.SetupEngine.Tests\OpenClaw.SetupEngine.Tests.csproj --no-restore
PASS: 933 passed; 0 failed

dotnet test .\tests\OpenClaw.Connection.Tests\OpenClaw.Connection.Tests.csproj --no-restore
PASS: 680 passed; 0 failed

Focused bundled-config validation:

Remove-Item Env:OPENCLAW_REPO_ROOT -ErrorAction SilentlyContinue
dotnet test .\tests\OpenClaw.SetupEngine.Tests\OpenClaw.SetupEngine.Tests.csproj --no-restore --filter "FullyQualifiedName~SetupConfigTests.BundledConfig_RequiresExplicitLocalAiOptIn"
PASS: 1/1 in the x64 validation environment

Setup-and-connect E2E:

dotnet test .\tests\OpenClaw.E2ETests\OpenClaw.E2ETests.csproj --no-restore --filter "FullyQualifiedName~SetupAndConnectTests" --logger "trx;LogFileName=E2E/SetupAndConnect-c6c96f85.trx"
PASS: 17/17 in 294.5 seconds

Repository check:

git diff --check
PASS: no whitespace errors

Real Behavior Proof

  • Environment tested: Windows ARM64 with NVIDIA driver 616.00, CUDA UMD 13.4,
    and an NVIDIA GPU qualified by capability rather than SKU; plus the automated
    Windows setup-and-connect E2E environment.
  • PR head or commit tested:
    c6c96f8.
  • Exact steps or command run: SetupAndConnectTests invocation shown in
    Validation, followed by a fresh isolated Dev onboarding run, Windows and WSL
    health probes, manifest/receipt verification, and Stop/Start/Restart controls.
  • Evidence after fix: TestResults/E2E/SetupAndConnect-c6c96f85.trx; redacted
    setup journal; exact manifest receipt; Windows and WSL health output; verified
    local video metadata and boundary frames.
  • Observed result: 17/17 E2E passed; fresh setup completed; first native
    inference and full GPU offload were proven before WSL creation; WSL reached
    the managed endpoint; lifecycle controls recovered on new owned ports.
  • Screenshot or artifact links verified? Yes locally. The video and reports
    remain uncommitted and outside the code diff.
  • Not verified or blocked: Post-push CI is still running. The video is retained
    locally and uncommitted by design, so there is no remote media URL in this
    code-only update. No older-head runtime values are substituted.

Security Impact

  • New permissions or capabilities? Yes

  • Secrets or tokens handling changed? No

  • New or changed network calls? Yes

  • Command or tool execution surface changed? Yes

  • Data access scope changed? Yes

  • If any answer is Yes, explain the risk and mitigation:

    Setup can download pinned runtime/model artifacts, invoke the native inference
    process and scoped WSL commands, read NVIDIA telemetry, and write app-owned
    runtime, model, manifest, provider, and consented WSL configuration state.
    Downloads require expected size and digest verification before promotion.
    NVML loads only from trusted locations. Cleanup validates app-owned paths and
    rejects reparse traversal. The router rejects wildcard binds, validates
    listener ownership by PID and process start time, and publishes an explicit
    127.0.0.1 endpoint. Configuration rollback and uninstall preserve concurrent
    or user-drifted state.

Compatibility and Migration

  • Backward compatible? Yes

  • Config or environment changes? Yes

  • Migration needed? No

  • If yes, list the exact upgrade steps:

    No manual upgrade step is required. Legacy Local AI manifest/profile data is
    read compatibly. New installations create app-owned manifests and provider
    state. Mirrored WSL networking is changed only after explicit user consent,
    with byte-preserving restore and rollback behavior.

Review Conversations

  • I replied to or resolved every bot review conversation addressed by this PR.
  • I left unresolved only conversations that still need maintainer judgment.

These boxes should be completed only after the rewritten head is pushed, CI
finishes, and the current-head proof comment receives re-review.

@clawsweeper

clawsweeper Bot commented Aug 19, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 19, 2026
@clawsweeper

clawsweeper Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed August 20, 2026, 9:42 PM ET / August 21, 2026, 01:42 UTC.

ClawSweeper review

What this changes

The PR adds opt-in Windows Local AI onboarding, verified llama-server and model installation, WSL gateway provider routing, and tray lifecycle controls.

Merge readiness

Blocked until stronger real behavior proof is added - 8 items remain

Keep open. The Local AI direction remains unique, but runtime download redirects and GPU capacity qualification have two concrete P2 blockers, and the available live proof predates the current head.

Priority: P2
Reviewed head: 145885614903b9c622197d542d7910eec5904d7c

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The branch has substantial validation work, but exact-head proof and two concrete correctness fixes remain before it is quality-ready.
Proof confidence 🦪 silver shellfish (2/6) Needs stronger real behavior proof before merge: The recording and live output are strong for c6c96f8, but the current head adds later WSL recovery behavior without current-head after-fix evidence. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality 🦐 gold shrimp (3/6) Security review found an item that needs attention.

Verification

Check Result Evidence
Real behavior Needs proof Needs stronger real behavior proof before merge: The recording and live output are strong for c6c96f8, but the current head adds later WSL recovery behavior without current-head after-fix evidence. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed 5 items Current main does not contain this feature: The current main setup pipeline has no Local AI steps, while the PR adds the Local AI installer and qualification surfaces.
Runtime redirects are unchecked: The runtime download handler leaves automatic redirects enabled, while the archive installer sends the request without validating the final redirect target.
Capacity calculation omits recipe memory: Eligibility uses model weight size plus a fixed reserve, although the selected recipe configures a 262,144-token F16 key/value cache.
Findings 2 actionable findings [P2] Restrict runtime archive redirect targets
[P2] Include the configured KV cache in GPU qualification
Security Needs attention Runtime archive redirects are not validated: The runtime client follows redirects automatically without checking the final scheme or host, unlike the model downloader's explicit redirect policy.

How this fits together

Windows onboarding qualifies local hardware, downloads and verifies native inference components, then publishes a loopback provider to the app-managed WSL gateway. The tray observes and controls the managed runtime after setup.

flowchart LR
A[Windows onboarding] --> B[Hardware qualification]
B --> C[Verified runtime and model]
C --> D[Native local inference]
D --> E[WSL gateway provider]
E --> F[Tray status and controls]
Loading

Before merge

  • Add real behavior proof - Needs stronger real behavior proof before merge: The recording and live output are strong for c6c96f8, but the current head adds later WSL recovery behavior without current-head after-fix evidence. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Restrict runtime archive redirect targets (P2) - The runtime handler leaves automatic redirects enabled, and this installer never validates the final request URI. A compromised or misconfigured release redirect can send setup to an arbitrary host before the size and hash checks run. Disable implicit redirects and apply the same bounded HTTPS host policy used by the model downloader.
  • Include the configured KV cache in GPU qualification (P2) - Qualification accepts a GPU using only GGUF weight size plus 2 GiB, but the selected recipe always starts llama-server with a 262,144-token F16 key/value cache. That can approve a machine whose router cannot allocate the configured cache, causing setup to fail after the multi-gigabyte download. Budget the recipe cache, or select a tested lower context size, before reporting eligibility.
  • Resolve security concern: Runtime archive redirects are not validated - The runtime client follows redirects automatically without checking the final scheme or host, unlike the model downloader's explicit redirect policy.
  • Resolve merge risk (P1) - The runtime archive path can follow an unchecked redirect before hash verification, weakening the declared trusted-download boundary.
  • Resolve merge risk (P1) - Hardware that passes weight-plus-reserve qualification can still fail when llama-server allocates the configured 262K F16 KV cache.
  • Resolve merge risk (P1) - The current head lacks after-fix evidence for its WSL failure-recovery behavior.
  • Complete next step (P2) - The fixes are concrete, but choosing the supported context-size and GPU-capacity contract needs maintainer review before an automated repair lane should alter this feature.

Findings

  • [P2] Restrict runtime archive redirect targets — src/OpenClaw.SetupEngine/LocalAiSetupSteps.cs:291-294
  • [P2] Include the configured KV cache in GPU qualification — src/OpenClaw.Shared/Inference/Catalog/LocalInferenceSelector.cs:126-130
  • [medium] Runtime archive redirects are not validated — src/OpenClaw.SetupEngine/LocalAiSetupSteps.cs:291
Agent review details

Security

Needs attention: The pinned runtime hashes are useful, but unchecked automatic redirects leave a concrete trusted-download-boundary gap.

Review metrics

Metric Value Why it matters
Changed surface 93 files, +14,421 -208 The feature spans setup, native runtime management, gateway routing, tray UX, and tests.
Production versus tests production +11,448 -182, tests +2,971 -26 The large production expansion needs the remaining boundary and capacity guards before landing.

Merge-risk options

Maintainer options:

  1. Repair the download and capacity guards (recommended)
    Disable implicit runtime redirects, validate each approved redirect explicitly, and include recipe-driven cache memory in qualification before collecting current-head recovery proof.
  2. Pause the Local AI landing
    Pause this branch if maintainers do not want to define and test the supported context-size and GPU-capacity contract now.

Technical review

Best possible solution:

Manually restrict runtime redirects to approved HTTPS release hosts, qualify capacity against the full llama-server recipe or reduce its context target, then attach a redacted current-head WSL recovery proof.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: runtime archive requests follow redirects without final-target validation, and qualification counts weights plus 2 GiB while the router configures a 262K F16 KV cache.

Is this the best way to solve the issue?

No. The current approach needs explicit runtime redirect policy and full recipe-based capacity accounting before it is safe to ship.

Full review comments:

  • [P2] Restrict runtime archive redirect targets — src/OpenClaw.SetupEngine/LocalAiSetupSteps.cs:291-294
    The runtime handler leaves automatic redirects enabled, and this installer never validates the final request URI. A compromised or misconfigured release redirect can send setup to an arbitrary host before the size and hash checks run. Disable implicit redirects and apply the same bounded HTTPS host policy used by the model downloader.
    Confidence: 0.98
  • [P2] Include the configured KV cache in GPU qualification — src/OpenClaw.Shared/Inference/Catalog/LocalInferenceSelector.cs:126-130
    Qualification accepts a GPU using only GGUF weight size plus 2 GiB, but the selected recipe always starts llama-server with a 262,144-token F16 key/value cache. That can approve a machine whose router cannot allocate the configured cache, causing setup to fail after the multi-gigabyte download. Budget the recipe cache, or select a tested lower context size, before reporting eligibility.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against d3ed6c78bf14.

Labels

Label justifications:

  • P2: The optional Local AI workflow has concrete setup-failure and supply-chain-boundary defects, but does not affect the existing default path.
  • merge-risk: 🚨 security-boundary: Implicit runtime archive redirects can contact an unvalidated destination before artifact verification.
  • merge-risk: 🚨 availability: The stated GPU qualification can approve systems that cannot allocate the configured inference cache.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The recording and live output are strong for c6c96f8, but the current head adds later WSL recovery behavior without current-head after-fix evidence. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • proof: 🎥 video: Contributor real behavior proof includes video or recording evidence. The recording and live output are strong for c6c96f8, but the current head adds later WSL recovery behavior without current-head after-fix evidence.

Evidence

Security concerns:

  • [medium] Runtime archive redirects are not validated — src/OpenClaw.SetupEngine/LocalAiSetupSteps.cs:291
    The runtime client follows redirects automatically without checking the final scheme or host, unlike the model downloader's explicit redirect policy.
    Confidence: 0.98

What I checked:

Likely related people:

  • shanselman: Provided the detailed maintainer-facing review identifying the initial Local AI safety and recovery expectations. (role: reviewer; confidence: high; files: src/OpenClaw.SetupEngine/LocalAiSetupSteps.cs, src/OpenClaw.SetupEngine/LocalAiArtifactInstaller.cs)
  • jacobtomlinson: Authored the shared-NVIDIA-memory qualification change and is explicitly engaged in the Local AI review discussion. (role: recent area contributor; confidence: medium; commits: 1489369c4ec4, 0b027e68258c; files: src/OpenClaw.Shared/Inference/Catalog/LocalInferenceSelector.cs, src/OpenClaw.Shared/Inference/NvmlHostHardwareProbe.cs)
  • RomneyDa: Authored the reviewed head's WSL platform failure-diagnostics change. (role: current-head contributor; confidence: high; commits: 145885614903; files: src/OpenClaw.SetupEngine/WslPlatformInstallDiagnostics.cs)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Repair both P2 findings with focused regression tests.
  • Add redacted current-head proof of the WSL install-failure recovery path.
  • Update the PR body with the new head SHA and proof artifact.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (14 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-20T18:34:03.114Z sha a102f79 :: needs real behavior proof before merge. :: [P1] Resolve the managed WSL distro from the gateway record
  • reviewed 2026-08-20T19:12:02.934Z sha d241b7c :: needs real behavior proof before merge. :: none
  • reviewed 2026-08-20T23:29:02.646Z sha c6c96f8 :: needs real behavior proof before merge. :: none
  • reviewed 2026-08-20T23:32:51.126Z sha c6c96f8 :: needs real behavior proof before merge. :: none
  • reviewed 2026-08-20T23:41:50.257Z sha c6c96f8 :: needs real behavior proof before merge. :: [P2] Validate runtime archive redirect targets
  • reviewed 2026-08-20T23:53:06.783Z sha c6c96f8 :: found issues before merge. :: [P2] Validate runtime archive redirect targets | [P2] Size qualification for the configured context cache
  • reviewed 2026-08-21T00:28:01.967Z sha 2bcf986 :: needs real behavior proof before merge. :: [P2] Validate runtime archive redirect targets | [P2] Include the configured context cache in GPU qualification
  • reviewed 2026-08-21T01:20:08.492Z sha 1458856 :: needs real behavior proof before merge. :: [P2] Validate runtime archive redirect targets | [P2] Include the configured KV cache in GPU qualification

@jacobtomlinson jacobtomlinson added status: 🚢 actively landing A maintainer or agent is actively driving this item through implementation, validation, or merge. and removed status: 🚢 actively landing A maintainer or agent is actively driving this item through implementation, validation, or merge. labels Aug 19, 2026
@shanselman

Copy link
Copy Markdown
Collaborator

This is an impressive, thoughtfully structured feature, especially the immutable artifact pins, SHA-256 checks, path containment, redirect allowlist, process ownership, and rollback compare-and-swap behavior. I validated the exact head on Windows: the full build and Shared, Tray, and SetupEngine unit suites pass. I also verified the llama.cpp and Hugging Face revisions, filenames, sizes, and hashes against upstream.

It is not merge-ready yet. Two independent reviews agree on these blockers:

  1. Local AI defaults on. default-config.json sets LocalAi.Enabled=true, contradicting the stated opt-in contract. Bundled/headless setup can fail on unsupported hardware or missing mirrored-networking consent, and qualified UI setup starts selected before an explicit user choice.
  2. .wslconfig corruption. Inserting networkingMode=mirrored into an existing [wsl2] section corrupts a common file with no trailing newline. [wsl2]\nmemory=8GB becomes memory=8GBnetworkingMode=mirrored.
  3. Interrupted install and uninstall recovery. A partial runtime directory blocks every later setup attempt, and uninstall only removes artifacts created during the current run. The multi-GB managed install needs manifest/path-policy-gated reuse or cleanup.
  4. Port lifecycle. The ephemeral port is selected and released before downloads that can take hours, then persisted with no conflict repair. A later collision leaves the UI in a dead end while the Gateway still targets that listener. Port 80 also passes preflight but is rejected later by manifest validation.
  5. Red setup E2E. The branch changes the persisted URL from localhost to 127.0.0.1, but the setup-connect contract still expects localhost.

The proof gap is also material: this adds 9,089 changed production lines and 127 runtime types with zero new focused [Fact] or [Theory] tests for the downloader, extractor, WSL config writer, manifest validation, managed process host, router, NVML qualification, inference verification, rollback, or recovery. The PR body explicitly leaves download, inference, mirrored WSL, and recovery unverified.

Before merge, please fix the five blockers, add focused regression coverage, and provide redacted current-head proof of a verified download, first inference, WSL reachability, restart, rollback, and interrupted-install recovery. The direction is excellent. The remaining work is about making this large native execution surface supportable and safe to ship.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Aug 20, 2026
@joelagnel
joelagnel force-pushed the feature/local-ai-llama-server-code-only branch from 6e8fd0b to 411493c Compare August 20, 2026 14:03
@joelagnel

Copy link
Copy Markdown
Contributor Author

Current-head E2E behavior proof is now recorded in the PR body for 411493c79848e6ff9db25f00ddd42a79ba6a6e1d.

  • Full ARM64 build passed all 5 projects; Shared 3,704/32 skipped, Tray 2,631, SetupEngine 925, and Connection 644 passed.
  • Setup/connect E2E passed 17/17.
  • A fresh isolated 36-step onboarding run completed in 144.7s: 32 generated tokens, 34/34 GPU layers with 20,816,330,752 bytes of load growth, then fresh WSL creation and verified WSL-to-llama-server reachability.
  • Gateway provider configuration, service start, operator/node pairing, and final gateway E2E verification all succeeded.
  • Companion Restart changed PID 39640 / port 61000 to PID 32384 / port 61038, then returned to Running and Connected.

The PR body also states the remaining proof boundary honestly: this run reused already hash-verified artifacts, and destructive interruption/rollback were covered by focused tests rather than repeated as live destructive demonstrations.

@joelagnel

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 20, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Aug 20, 2026
@joelagnel

joelagnel commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Sanitized current-head onboarding and Local AI behavior proof for 411493c79848e6ff9db25f00ddd42a79ba6a6e1d.

The recording shows the full maximized setup flow, successful native inference before WSL provisioning, the connected dashboard, and a live Local AI restart returning to Running and Connected.
https://github.com/user-attachments/assets/33658ce4-89d2-4b8c-8322-b2c2d1e828cf

@jacobtomlinson
jacobtomlinson force-pushed the feature/local-ai-llama-server-code-only branch from 411493c to db8fd59 Compare August 20, 2026 14:30
@clawsweeper clawsweeper Bot added proof: 🎥 video Contributor real behavior proof includes video or recording evidence. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels Aug 20, 2026
@joelagnel
joelagnel force-pushed the feature/local-ai-llama-server-code-only branch from a102f79 to d241b7c Compare August 20, 2026 19:07
@clawsweeper clawsweeper Bot added feature: ✨ showcase ClawSweeper spotlight: unusually compelling feature idea for maintainer attention. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels Aug 20, 2026
Move physical-memory discovery into one shared implementation.
Give setup and inference code a consistent RAM capacity value.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 20, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. proof: 🎥 video Contributor real behavior proof includes video or recording evidence. proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. labels Aug 20, 2026
@joelagnel

Copy link
Copy Markdown
Contributor Author

@karkarl @jacobtomlinson Updated PR with new detection scheme (not depending on specific GPU but just GPU memory capacity and runtime components).

joelagnel and others added 3 commits August 20, 2026 17:22
Keep a compact Local AI review visible when hardware, WSL, or networking blocks setup.
Hide unavailable controls and show every detected dependency reason through See why.

Co-authored-by: Karen Lai <7976322+karkarl@users.noreply.github.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Accept full CUDA offload when either NVML memory movement or the parsed CUDA model buffer proves placement.
Reject startup when neither signal accounts for the selected model.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
The consent InfoBar content sat flush against the bottom edge of the
warning, leaving the message and checkbox visually cramped. Add the
same bottom content margin the sandbox InfoBar already uses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8LRNTZ5fBgjGwmtBCrSp3
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
joelagnel force-pushed the feature/local-ai-llama-server-code-only branch from 1e4c2f7 to 2bcf986 Compare August 21, 2026 00:23
@clawsweeper clawsweeper Bot added status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Aug 21, 2026
Wait for the direct child process independently of redirected-pipe EOF, then bound the final output drain. Existing-distro inspection now fails closed when WSL times out or returns an unknown error.

Show progress during the Welcome-page inspection so a cold WSL service does not make setup look frozen.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
After an elevated WSL install fails, run a best-effort GitHub quota diagnostic and distinguish likely quota exhaustion from other download failures. Diagnostic timeouts never replace the original failure.

Offer Store, winget, and elevated PowerShell recovery routes, and allow the platform step to be retried.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@RomneyDa RomneyDa added the status: 🚢 actively landing A maintainer or agent is actively driving this item through implementation, validation, or merge. label Aug 21, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature: ✨ showcase ClawSweeper spotlight: unusually compelling feature idea for maintainer attention. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal priority bug or improvement with limited blast radius. proof: 🎥 video Contributor real behavior proof includes video or recording evidence. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 🚢 actively landing A maintainer or agent is actively driving this item through implementation, validation, or merge. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants