Skip to content

feat: add managed local AI with llama-server - #1178

Open
joelagnel wants to merge 25 commits into
openclaw:mainfrom
joelagnel:feature/local-ai-llama-server-code-only
Open

feat: add managed local AI with llama-server#1178
joelagnel wants to merge 25 commits into
openclaw:mainfrom
joelagnel:feature/local-ai-llama-server-code-only

Conversation

@joelagnel

@joelagnel joelagnel commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Resolves a problem where Windows users who wanted local inference had no guided,
hardware-aware setup path and could be left with stale runtime, model, process,
port, or gateway state after an interruption. Users whose machines were not
ready for Local AI also lacked a concise explanation of the missing dependency.

The setup flow now qualifies compatible NVIDIA hardware before WSL installation,
offers Local AI only as an explicit opt-in, proves native inference first, and
then continues with the managed WSL gateway.

Why This Change Was Made

Local AI qualification is based on runtime capability rather than a hard-coded
GPU SKU or CPU pairing. A trusted NVML probe records NVIDIA UUID, driver, CUDA,
total memory, and free memory facts; unambiguous DXGI association can add usable
shared memory. Automatic selection chooses the largest pinned model whose
weights plus a 2 GiB reserve fit both total and currently free capacity.

The native llama.cpp runtime and GGUF model are downloaded from pinned catalog
entries, verified, installed under app-owned paths, started, and checked for a
real inference response and GPU placement before WSL setup proceeds. The
companion then resolves the single setup-managed WSL distribution from registry
state and owns the router and provider lifecycle for that distribution.

The implementation also adds exact-manifest reuse, strict partial GGUF resume,
catalog-owned orphan reconciliation, reparse-safe cleanup, fresh-process
uninstall, cancellation rollback, listener ownership checks, wildcard rejection,
and explicit 127.0.0.1 provider publication with drift-safe compensation.

Boundaries: this change qualifies NVIDIA devices exposed by NVML, not other GPU
vendors; recommends only fitting entries from the pinned model catalog; binds
the managed inference endpoint to local IPv4 loopback; and leaves Windows
session-container inference and additional model catalogs for later work. This
code-focused branch does not add documentation, screenshots, reports, or video.

User Impact

Users with a qualifying NVIDIA GPU can opt into Local AI during onboarding,
choose from models that fit their detected capacity, and see native inference
working before committing to WSL setup. Setup can safely recover from an
interrupted download or install and can uninstall owned state from a fresh
process without removing user-modified configuration.

Users who cannot run Local AI see a small unavailable notice and can open See why
for hardware, driver, CUDA, WSL, and networking diagnostics. The tray exposes
Local AI status and lifecycle controls after setup.

Evidence

All automated results below were collected from exact local head
c6c96f8:

  • Build: PASS for all five ARM64 projects.
  • OpenClaw.Shared.Tests: 3,808 passed, 32 skipped, 0 failed.
  • OpenClaw.Tray.Tests: 2,702 passed, 0 failed.
  • OpenClaw.SetupEngine.Tests: 933 passed, 0 failed.
  • OpenClaw.Connection.Tests: 680 passed, 0 failed.
  • x64 bundled-config regression with OPENCLAW_REPO_ROOT unset: 1/1 passed.
  • git diff --check: clean.
  • Setup-and-connect E2E: 17/17 passed in 294.5 seconds.
  • TRX: TestResults/E2E/SetupAndConnect-c6c96f85.trx.
  • Cleanup: disposable OpenClawE2E-739379a1 was removed. The later video run
    explicitly recreated only the isolated Dev distribution; Ubuntu and the
    production OpenClawGateway remained untouched.

The first E2E attempt stopped before product validation because the required x64
tray artifact was absent. After an x64 build completed with 0 errors and one
generated-code warning, the corrected run passed all 17 tests. This was an
infrastructure-only first-attempt failure.

No screenshot, recording, PID, port, inference, GPU, WSL, or restart value from
an older head is used as proof for this revision.

Fresh live proof from exact c6c96f8:

  • Full setup completed 36 steps in 450.0 seconds.
  • The 5,868,826,976-byte Qwen3.5 9B artifact matched its SHA-256 receipt.
  • First inference completed before WSL mutation: 32 generated tokens, 34/34 GPU
    layers, and 20,816,330,752 bytes of observed GPU load growth.
  • The final managed process was PID 36768, started
    2026-08-20T23:15:01.6521200Z, listening only on 127.0.0.1:54935.
  • Windows and OpenClawGateway-Dev both received HTTP 200
    {"status":"ok"} from the current endpoint.
  • The WSL provider base URL matched http://127.0.0.1:54935/v1 and its primary
    model matched the installed Qwen alias.
  • Stop, Start, and Restart were exercised. The runtime moved from 28472:54883
    to 30380:54921 and then 36768:54935 while the gateway remained Connected.
  • A real-motion 120.033-second onboarding-to-dashboard recording was visually
    verified at 3270x2180/30 fps. Its SHA-256 is
    FD756C5BB6BD96D18EBDAC81B0462673087A781A80EC1DCCDF2DD6E05CEC302D. It opens
    Local AI and shows Stop, Start, Restart, Running, and Connected. The local
    artifact remains uncommitted and is not part of this code diff.

Post-push status: PR #1178 now resolves to exact head c6c96f8 with 25 commits
on base d3ed6c7, and Allow edits by maintainers remains enabled. CodeQL and
repository hygiene are green in Build and Test run
32428513668
;
the remaining build/test and E2E jobs are still running.

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs or instructions
  • Tests or validation
  • Security hardening
  • Chore or infrastructure

Scope

  • Tray or WinUI UX
  • Windows node capability
  • Local MCP or winnode
  • Gateway, connection, or pairing
  • Setup or onboarding
  • Permissions, privacy, or security
  • Tests, CI, or docs

Validation

Commands and results at c6c96f8:

powershell -NoProfile -ExecutionPolicy Bypass -File .\build.ps1
PASS: all five ARM64 projects

dotnet test .\tests\OpenClaw.Shared.Tests\OpenClaw.Shared.Tests.csproj --no-restore
PASS: 3,808 passed; 32 skipped; 0 failed

dotnet test .\tests\OpenClaw.Tray.Tests\OpenClaw.Tray.Tests.csproj --no-restore
PASS: 2,702 passed; 0 failed

dotnet test .\tests\OpenClaw.SetupEngine.Tests\OpenClaw.SetupEngine.Tests.csproj --no-restore
PASS: 933 passed; 0 failed

dotnet test .\tests\OpenClaw.Connection.Tests\OpenClaw.Connection.Tests.csproj --no-restore
PASS: 680 passed; 0 failed

Focused bundled-config validation:

Remove-Item Env:OPENCLAW_REPO_ROOT -ErrorAction SilentlyContinue
dotnet test .\tests\OpenClaw.SetupEngine.Tests\OpenClaw.SetupEngine.Tests.csproj --no-restore --filter "FullyQualifiedName~SetupConfigTests.BundledConfig_RequiresExplicitLocalAiOptIn"
PASS: 1/1 in the x64 validation environment

Setup-and-connect E2E:

dotnet test .\tests\OpenClaw.E2ETests\OpenClaw.E2ETests.csproj --no-restore --filter "FullyQualifiedName~SetupAndConnectTests" --logger "trx;LogFileName=E2E/SetupAndConnect-c6c96f85.trx"
PASS: 17/17 in 294.5 seconds

Repository check:

git diff --check
PASS: no whitespace errors

Real Behavior Proof

  • Environment tested: Windows ARM64 with NVIDIA driver 616.00, CUDA UMD 13.4,
    and an NVIDIA GPU qualified by capability rather than SKU; plus the automated
    Windows setup-and-connect E2E environment.
  • PR head or commit tested:
    c6c96f8.
  • Exact steps or command run: SetupAndConnectTests invocation shown in
    Validation, followed by a fresh isolated Dev onboarding run, Windows and WSL
    health probes, manifest/receipt verification, and Stop/Start/Restart controls.
  • Evidence after fix: TestResults/E2E/SetupAndConnect-c6c96f85.trx; redacted
    setup journal; exact manifest receipt; Windows and WSL health output; verified
    local video metadata and boundary frames.
  • Observed result: 17/17 E2E passed; fresh setup completed; first native
    inference and full GPU offload were proven before WSL creation; WSL reached
    the managed endpoint; lifecycle controls recovered on new owned ports.
  • Screenshot or artifact links verified? Yes locally. The video and reports
    remain uncommitted and outside the code diff.
  • Not verified or blocked: Post-push CI is still running. The video is retained
    locally and uncommitted by design, so there is no remote media URL in this
    code-only update. No older-head runtime values are substituted.

Security Impact

  • New permissions or capabilities? Yes

  • Secrets or tokens handling changed? No

  • New or changed network calls? Yes

  • Command or tool execution surface changed? Yes

  • Data access scope changed? Yes

  • If any answer is Yes, explain the risk and mitigation:

    Setup can download pinned runtime/model artifacts, invoke the native inference
    process and scoped WSL commands, read NVIDIA telemetry, and write app-owned
    runtime, model, manifest, provider, and consented WSL configuration state.
    Downloads require expected size and digest verification before promotion.
    NVML loads only from trusted locations. Cleanup validates app-owned paths and
    rejects reparse traversal. The router rejects wildcard binds, validates
    listener ownership by PID and process start time, and publishes an explicit
    127.0.0.1 endpoint. Configuration rollback and uninstall preserve concurrent
    or user-drifted state.

Compatibility and Migration

  • Backward compatible? Yes

  • Config or environment changes? Yes

  • Migration needed? No

  • If yes, list the exact upgrade steps:

    No manual upgrade step is required. Legacy Local AI manifest/profile data is
    read compatibly. New installations create app-owned manifests and provider
    state. Mirrored WSL networking is changed only after explicit user consent,
    with byte-preserving restore and rollback behavior.

Review Conversations

  • I replied to or resolved every bot review conversation addressed by this PR.
  • I left unresolved only conversations that still need maintainer judgment.

These boxes should be completed only after the rewritten head is pushed, CI
finishes, and the current-head proof comment receives re-review.

@clawsweeper

clawsweeper Bot commented Aug 19, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 19, 2026
@clawsweeper

clawsweeper Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codex review: found issues before merge. Reviewed August 20, 2026, 7:53 PM ET / 23:53 UTC.

ClawSweeper review

What this changes

This PR adds opt-in, hardware-qualified Windows Local AI setup that installs and manages pinned llama-server and model artifacts, connects them to the WSL gateway, and exposes tray controls.

Merge readiness

⚠️ Needs maintainer review before merge - 7 items remain

Keep open. The current-head proof is meaningful, but two P2 blockers remain: the runtime downloader follows unvalidated redirects, and hardware qualification ignores the configured 262K F16 context cache.

Priority: P2
Reviewed head: c6c96f853a1c1f7c8c2120155a8cfa654460e318
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) Current-head proof is strong, but two concrete safety and capacity defects must be resolved before merge.
Proof confidence 🦞 diamond lobster (5/6) ✨ media proof bonus Sufficient (recording): A public recording linked for the exact head visibly demonstrates onboarding, dashboard status, and lifecycle controls, with matching current-head live output for inference and WSL reachability.
Patch quality 🦐 gold shrimp (3/6) Security review found an item that needs attention.

Verification

Check Result Evidence
Real behavior Verified Sufficient (recording): A public recording linked for the exact head visibly demonstrates onboarding, dashboard status, and lifecycle controls, with matching current-head live output for inference and WSL reachability.
Evidence reviewed 6 items Runtime redirect path: The runtime acquisition handler enables default automatic redirects, while the archive installer streams the final response without validating its final URL or redirect chain.
Contrasting model protection: Model downloads disable automatic redirects and explicitly validate each redirect host, demonstrating the missing equivalent runtime boundary.
Capacity mismatch: Qualification treats weights plus a fixed 2 GiB margin as sufficient, while the router always configures a 262,144-token F16 K/V cache with all layers offloaded and fit disabled.
Findings 2 actionable findings [P2] Validate runtime archive redirect targets
[P2] Size qualification for the configured context cache
Security Needs attention Runtime archive redirects bypass host validation: The runtime handler follows redirects by default and the installer validates only HTTPS on the original URL; a redirect can contact an unapproved host before the digest check rejects the bytes.

How this fits together

The setup wizard qualifies NVIDIA hardware, installs and verifies native inference artifacts, then publishes a loopback endpoint to the managed WSL gateway. The tray owns later runtime lifecycle controls and displays their state.

flowchart LR
A[Setup choices] --> B[Hardware qualification]
B --> C[Verified runtime and model]
C --> D[Native local inference]
D --> E[WSL gateway provider]
E --> F[Tray status and controls]
Loading

Decision needed

Question Recommendation
Should Local AI guarantee the advertised 262K-token F16 full-offload recipe on supported hardware, or offer a deliberately smaller-context recipe for lower-memory GPUs? Guarantee the advertised recipe: Add recipe-aware capacity accounting and hide or reject configurations that cannot run the configured 262K F16 full-offload profile.

Why: The current 2 GiB margin cannot express the actual cache requirement, so choosing the supported-memory contract changes the user-visible product promise.

Before merge

  • Validate runtime archive redirect targets (P2) - The runtime handler leaves AllowAutoRedirect at its default, and LocalAiArtifactInstaller streams the resulting response without checking the final URI. A compromised or redirected release URL can therefore make setup contact an arbitrary host before the hash check fails. Disable automatic redirects and validate a bounded GitHub asset redirect chain, as the model downloader already does.
  • Size qualification for the configured context cache (P2) - This admits a 16 GiB GPU based only on GGUF size plus 2 GiB, but the launched preset unconditionally requests 262,144 tokens with F16 K/V cache, full GPU offload, and fit = off. Such a machine can be shown as eligible, download the model, and only fail at first inference. Account for the selected recipe's cache footprint or select a lower-context recipe. This concern was visible at the prior current-head review; the affected files are unchanged.
  • Resolve security concern: Runtime archive redirects bypass host validation - The runtime handler follows redirects by default and the installer validates only HTTPS on the original URL; a redirect can contact an unapproved host before the digest check rejects the bytes.
  • Resolve merge risk (P1) - The runtime installer can follow a release redirect to an unapproved host before hash verification rejects an unexpected payload.
  • Resolve merge risk (P1) - A 16 GiB GPU can be presented as eligible for a 262K-token, full-offload F16-cache recipe that may fail only after downloading and beginning setup.
  • Complete next step (P2) - A maintainer must choose the supported memory-profile contract, then the contributor can make the two bounded repairs on this branch.

Findings

  • [P2] Validate runtime archive redirect targets — src/OpenClaw.SetupEngine/LocalAiSetupSteps.cs:291-294
  • [P2] Size qualification for the configured context cache — src/OpenClaw.Shared/Inference/Catalog/LocalInferenceSelector.cs:126-130
  • [medium] Runtime archive redirects bypass host validation — src/OpenClaw.SetupEngine/LocalAiSetupSteps.cs:291
Agent review details

Security

Needs attention: The new runtime downloader follows automatic redirects without the explicit destination validation used for model downloads.

Review metrics

Metric Value Why it matters
Change surface 88 files affected The feature crosses setup, native runtime, gateway, tray, and tests, so focused boundary review matters.
Production versus test delta production +11,280/-150, tests +2,848/-26 The substantial production expansion has meaningful accompanying test coverage, but the two uncovered boundary assumptions still block merge.

Merge-risk options

Maintainer options:

  1. Bound runtime archive redirects (recommended)
    Disable automatic redirects and validate every GitHub release redirect against a narrow HTTPS host policy before streaming an archive.
  2. Set the memory contract before merge
    Choose and test either recipe-aware admission for the current 262K profile or a separately qualified lower-context profile.

Technical review

Best possible solution:

Use one bounded, tested redirect policy for both runtime and model artifacts, then make eligibility calculate the actual recipe footprint or deliberately choose a smaller context profile for lower-memory GPUs.

Do we have a high-confidence way to reproduce the issue?

Yes for the source-level failures: a redirecting runtime asset request reaches the default automatic-redirect handler, and the added 16 GiB qualification case selects a recipe whose cache is not included in admission sizing.

Is this the best way to solve the issue?

No. Pinned hashes protect artifact integrity after transfer, but redirect destinations still need validation, and model weights plus 2 GiB does not model the configured inference footprint.

Full review comments:

  • [P2] Validate runtime archive redirect targets — src/OpenClaw.SetupEngine/LocalAiSetupSteps.cs:291-294
    The runtime handler leaves AllowAutoRedirect at its default, and LocalAiArtifactInstaller streams the resulting response without checking the final URI. A compromised or redirected release URL can therefore make setup contact an arbitrary host before the hash check fails. Disable automatic redirects and validate a bounded GitHub asset redirect chain, as the model downloader already does.
    Confidence: 0.99
  • [P2] Size qualification for the configured context cache — src/OpenClaw.Shared/Inference/Catalog/LocalInferenceSelector.cs:126-130
    This admits a 16 GiB GPU based only on GGUF size plus 2 GiB, but the launched preset unconditionally requests 262,144 tokens with F16 K/V cache, full GPU offload, and fit = off. Such a machine can be shown as eligible, download the model, and only fail at first inference. Account for the selected recipe's cache footprint or select a lower-context recipe. This concern was visible at the prior current-head review; the affected files are unchanged.
    Confidence: 0.93
    Late finding: first raised on code an earlier review cycle already covered.

Overall correctness: patch is incorrect
Overall confidence: 0.96

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against d3ed6c78bf14.

Labels

Label changes:

  • add merge-risk: 🚨 availability: Qualification can accept hardware that cannot satisfy the fixed high-context inference configuration.
  • add proof: sufficient: Contributor real behavior proof is sufficient. A public recording linked for the exact head visibly demonstrates onboarding, dashboard status, and lifecycle controls, with matching current-head live output for inference and WSL reachability.
  • add proof: 🎥 video: Contributor real behavior proof includes video or recording evidence. A public recording linked for the exact head visibly demonstrates onboarding, dashboard status, and lifecycle controls, with matching current-head live output for inference and WSL reachability.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (recording): A public recording linked for the exact head visibly demonstrates onboarding, dashboard status, and lifecycle controls, with matching current-head live output for inference and WSL reachability.
  • remove status: 📣 needs proof: Current PR status label is status: ⏳ waiting on author.
  • remove merge-risk: 🚨 compatibility: Current PR review merge-risk labels are merge-risk: 🚨 security-boundary, merge-risk: 🚨 availability.
  • remove merge-risk: 🚨 auth-provider: Current PR review merge-risk labels are merge-risk: 🚨 security-boundary, merge-risk: 🚨 availability.

Label justifications:

  • P2: This optional but substantial setup feature has concrete merge blockers without evidence of an existing production outage.
  • merge-risk: 🚨 security-boundary: The new native-runtime downloader follows redirects without validating the destination host.
  • merge-risk: 🚨 availability: Qualification can accept hardware that cannot satisfy the fixed high-context inference configuration.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (recording): A public recording linked for the exact head visibly demonstrates onboarding, dashboard status, and lifecycle controls, with matching current-head live output for inference and WSL reachability.
  • proof: sufficient: Contributor real behavior proof is sufficient. A public recording linked for the exact head visibly demonstrates onboarding, dashboard status, and lifecycle controls, with matching current-head live output for inference and WSL reachability.
  • proof: 🎥 video: Contributor real behavior proof includes video or recording evidence. A public recording linked for the exact head visibly demonstrates onboarding, dashboard status, and lifecycle controls, with matching current-head live output for inference and WSL reachability.

Evidence

Security concerns:

  • [medium] Runtime archive redirects bypass host validation — src/OpenClaw.SetupEngine/LocalAiSetupSteps.cs:291
    The runtime handler follows redirects by default and the installer validates only HTTPS on the original URL; a redirect can contact an unapproved host before the digest check rejects the bytes.
    Confidence: 0.99

What I checked:

Likely related people:

  • shanselman: A collaborator supplied the detailed prior review of this Local AI setup and security surface; current history also shows recent work on the tray/setup paths. (role: reviewer and likely product-direction owner; confidence: high; commits: a51029e; files: src/OpenClaw.Tray.WinUI/App.xaml.cs, src/OpenClaw.SetupEngine/SetupPipeline.cs)
  • jacobtomlinson: The PR history credits this contributor with the shared-GPU-memory and GPU-placement changes central to qualification and first-inference verification. (role: recent inference-area contributor; confidence: high; commits: 1489369c4ec4, c966870cb97a; files: src/OpenClaw.Shared/Inference/Catalog/LocalInferenceSelector.cs, src/OpenClaw.SetupEngine/LocalAiGpuVerification.cs)
  • RomneyDa: The current reviewed head is authored by this contributor and touches the Local AI onboarding consent presentation. (role: current-head UI contributor; confidence: medium; commits: c6c96f853a1c; files: src/OpenClaw.SetupEngine.UI/Pages/CapabilitiesPage.xaml)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Add a bounded GitHub release redirect policy with focused redirect tests.
  • Set and test the GPU-memory contract for the configured 262K F16 recipe or an explicit lower-context alternative.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (11 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-20T14:10:39.900Z sha 411493c :: needs real behavior proof before merge. :: [P1] Restore a usable primary model when removing the provider
  • reviewed 2026-08-20T14:35:14.650Z sha db8fd59 :: needs real behavior proof before merge. :: [P1] Keep a usable primary model while the provider is quiesced
  • reviewed 2026-08-20T18:01:54.540Z sha b81d026 :: needs real behavior proof before merge. :: [P1] Keep a valid primary model while quiescing the provider
  • reviewed 2026-08-20T18:34:03.114Z sha a102f79 :: needs real behavior proof before merge. :: [P1] Resolve the managed WSL distro from the gateway record
  • reviewed 2026-08-20T19:12:02.934Z sha d241b7c :: needs real behavior proof before merge. :: none
  • reviewed 2026-08-20T23:29:02.646Z sha c6c96f8 :: needs real behavior proof before merge. :: none
  • reviewed 2026-08-20T23:32:51.126Z sha c6c96f8 :: needs real behavior proof before merge. :: none
  • reviewed 2026-08-20T23:41:50.257Z sha c6c96f8 :: needs real behavior proof before merge. :: [P2] Validate runtime archive redirect targets

@jacobtomlinson jacobtomlinson added status: 🚢 actively landing A maintainer or agent is actively driving this item through implementation, validation, or merge. and removed status: 🚢 actively landing A maintainer or agent is actively driving this item through implementation, validation, or merge. labels Aug 19, 2026
@shanselman

Copy link
Copy Markdown
Collaborator

This is an impressive, thoughtfully structured feature, especially the immutable artifact pins, SHA-256 checks, path containment, redirect allowlist, process ownership, and rollback compare-and-swap behavior. I validated the exact head on Windows: the full build and Shared, Tray, and SetupEngine unit suites pass. I also verified the llama.cpp and Hugging Face revisions, filenames, sizes, and hashes against upstream.

It is not merge-ready yet. Two independent reviews agree on these blockers:

  1. Local AI defaults on. default-config.json sets LocalAi.Enabled=true, contradicting the stated opt-in contract. Bundled/headless setup can fail on unsupported hardware or missing mirrored-networking consent, and qualified UI setup starts selected before an explicit user choice.
  2. .wslconfig corruption. Inserting networkingMode=mirrored into an existing [wsl2] section corrupts a common file with no trailing newline. [wsl2]\nmemory=8GB becomes memory=8GBnetworkingMode=mirrored.
  3. Interrupted install and uninstall recovery. A partial runtime directory blocks every later setup attempt, and uninstall only removes artifacts created during the current run. The multi-GB managed install needs manifest/path-policy-gated reuse or cleanup.
  4. Port lifecycle. The ephemeral port is selected and released before downloads that can take hours, then persisted with no conflict repair. A later collision leaves the UI in a dead end while the Gateway still targets that listener. Port 80 also passes preflight but is rejected later by manifest validation.
  5. Red setup E2E. The branch changes the persisted URL from localhost to 127.0.0.1, but the setup-connect contract still expects localhost.

The proof gap is also material: this adds 9,089 changed production lines and 127 runtime types with zero new focused [Fact] or [Theory] tests for the downloader, extractor, WSL config writer, manifest validation, managed process host, router, NVML qualification, inference verification, rollback, or recovery. The PR body explicitly leaves download, inference, mirrored WSL, and recovery unverified.

Before merge, please fix the five blockers, add focused regression coverage, and provide redacted current-head proof of a verified download, first inference, WSL reachability, restart, rollback, and interrupted-install recovery. The direction is excellent. The remaining work is about making this large native execution surface supportable and safe to ship.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Aug 20, 2026
@joelagnel
joelagnel force-pushed the feature/local-ai-llama-server-code-only branch from 6e8fd0b to 411493c Compare August 20, 2026 14:03
@joelagnel

Copy link
Copy Markdown
Contributor Author

Current-head E2E behavior proof is now recorded in the PR body for 411493c79848e6ff9db25f00ddd42a79ba6a6e1d.

  • Full ARM64 build passed all 5 projects; Shared 3,704/32 skipped, Tray 2,631, SetupEngine 925, and Connection 644 passed.
  • Setup/connect E2E passed 17/17.
  • A fresh isolated 36-step onboarding run completed in 144.7s: 32 generated tokens, 34/34 GPU layers with 20,816,330,752 bytes of load growth, then fresh WSL creation and verified WSL-to-llama-server reachability.
  • Gateway provider configuration, service start, operator/node pairing, and final gateway E2E verification all succeeded.
  • Companion Restart changed PID 39640 / port 61000 to PID 32384 / port 61038, then returned to Running and Connected.

The PR body also states the remaining proof boundary honestly: this run reused already hash-verified artifacts, and destructive interruption/rollback were covered by focused tests rather than repeated as live destructive demonstrations.

@joelagnel

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 20, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Aug 20, 2026
@joelagnel

joelagnel commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Sanitized current-head onboarding and Local AI behavior proof for 411493c79848e6ff9db25f00ddd42a79ba6a6e1d.

The recording shows the full maximized setup flow, successful native inference before WSL provisioning, the connected dashboard, and a live Local AI restart returning to Running and Connected.
https://github.com/user-attachments/assets/33658ce4-89d2-4b8c-8322-b2c2d1e828cf

@jacobtomlinson
jacobtomlinson force-pushed the feature/local-ai-llama-server-code-only branch from 411493c to db8fd59 Compare August 20, 2026 14:30
@clawsweeper clawsweeper Bot added proof: 🎥 video Contributor real behavior proof includes video or recording evidence. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels Aug 20, 2026
@joelagnel
joelagnel force-pushed the feature/local-ai-llama-server-code-only branch from a102f79 to d241b7c Compare August 20, 2026 19:07
@clawsweeper clawsweeper Bot added feature: ✨ showcase ClawSweeper spotlight: unusually compelling feature idea for maintainer attention. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels Aug 20, 2026
Move physical-memory discovery into one shared implementation.
Give setup and inference code a consistent RAM capacity value.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
joelagnel and others added 15 commits August 20, 2026 14:56
Load NVML from trusted locations and collect NVIDIA GPU capabilities.
Avoid unsafe library resolution while providing selector-grade hardware data.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Correlate NVML adapters with unambiguous DXGI memory observations and count shared memory for any NVIDIA GPU.

Fail closed on duplicate or ambiguous adapter names so model selection cannot borrow another device's budget.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Resume interrupted GGUF downloads with strict Range and Content-Range handling.
Verify size and hash before atomic promotion, restarting safely when resume is unsupported.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Install and inspect the pinned llama-server and CUDA runtime components.
Safely reconcile exact orphan runtime paths so interrupted promotion can retry.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Launch llama-server on demand with an OS-assigned or validated fixed port.
Prove listener ownership by child PID and start time before health checks or persistence.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Make Local AI opt-in and inspect hardware plus WSL viability without mutation.
Acquire verified native inference before WSL provisioning and resolve bundled defaults reliably from RID-specific output.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Reconcile and reuse only exact manifest-owned runtime and model artifacts after interruption.
Persist a proven healthy endpoint and clean durable app-owned state on fresh uninstall.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Probe health, execute real inference, and verify GPU activity on the owned endpoint.
Fail setup cleanly when the installed Local AI stack is not operational.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Use explicit 127.0.0.1 for Windows-to-WSL gateway connections and persisted setup state.
Remove only exact managed Local AI provider state during uninstall, preserving drift.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Bind Local AI to the singleton setup-managed distro resolved from the loaded gateway registry.
Quiesce exact provider state before transitions, then publish only healthy owned endpoints and compensate failures.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Surface Local AI eligibility, explicit consent, review, and setup progress.
Show native inference before WSL setup and describe dynamic multi-gigabyte disk use accurately.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Add Local AI navigation, status, controls, logs, and localized resources.
Wire the page through application services and retain focused UI contracts.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Keep a compact Local AI review visible when hardware, WSL, or networking blocks setup.
Hide unavailable controls and show every detected dependency reason through See why.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Accept full CUDA offload when either NVML memory movement or the parsed CUDA model buffer proves placement.
Reject startup when neither signal accounts for the selected model.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
The consent InfoBar content sat flush against the bottom edge of the
warning, leaving the message and checkbox visually cramped. Add the
same bottom content margin the sandbox InfoBar already uses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8LRNTZ5fBgjGwmtBCrSp3
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
joelagnel force-pushed the feature/local-ai-llama-server-code-only branch from d241b7c to c6c96f8 Compare August 20, 2026 23:24
@joelagnel

Copy link
Copy Markdown
Contributor Author

Exact-head automated proof

Validated commit:
c6c96f8

Check Exact result
ARM64 build PASS: all five projects
OpenClaw.Shared.Tests PASS: 3,808 passed, 32 skipped, 0 failed
OpenClaw.Tray.Tests PASS: 2,702 passed, 0 failed
OpenClaw.SetupEngine.Tests PASS: 933 passed, 0 failed
OpenClaw.Connection.Tests PASS: 680 passed, 0 failed
x64 bundled-config regression with OPENCLAW_REPO_ROOT unset PASS: 1/1
git diff --check PASS: clean
Setup-and-connect E2E PASS: 17/17 in 294.5 seconds

E2E artifact:
TestResults/E2E/SetupAndConnect-c6c96f85.trx

The E2E run removed its disposable OpenClawE2E-739379a1 distribution. The later
video run explicitly recreated only OpenClawGateway-Dev after validating its
isolated Lxss base path. Ubuntu-24.04 and production OpenClawGateway were not
removed or shut down.

The first E2E attempt was an infrastructure-only failure before product
validation because the x64 tray artifact was missing. After the x64 build
completed with 0 errors and one generated-code warning, the corrected run passed
all 17 tests.

No runtime values or media from d241b7c, 411493c, or another stale head are
included below.

Fresh redacted live proof

The isolated Dev onboarding pipeline completed all 36 steps in 450.0 seconds.
The native Local AI sequence completed before the first WSL mutation:

acquire-local-ai-runtime       completed  Installed verified llama-server
acquire-local-ai-model         completed  Downloaded Qwen3.5 9B from the pinned revision
start-local-ai-runtime         completed  Managed router healthy
verify-local-ai-inference      completed  32 generated tokens [response redacted]
verify-local-ai-gpu-load       completed  34/34 GPU layers; 20,816,330,752 bytes load growth
ensure-wsl-platform            started    after native inference proof
wsl-create                     completed  Created OpenClawGateway-Dev
verify-local-ai-wsl            completed  App-owned WSL distro reached llama-server
configure-local-ai-gateway     completed  Managed provider published
_pipeline                      completed  elapsed=450.0s

The installed model is 5,868,826,976 bytes and a fresh SHA-256 calculation
matched its manifest receipt. The final runtime state and independent health
checks were:

process:       llama-server PID 36768
process start: 2026-08-20T23:15:01.6521200Z
listener:      127.0.0.1:54935 owned by PID 36768
wildcard:      none
Windows:       HTTP 200 {"status":"ok"}
WSL:           HTTP 200 {"status":"ok"} from OpenClawGateway-Dev
manifest:      requestedPort=0 endpoint=http://127.0.0.1:54935/v1
provider:      baseUrl matched the manifest endpoint
primary:       llamacpp/qwen3.5-9b-mtp-q4-k-m

The Local AI page exercised Stop, Start, and Restart. The owned runtime changed
from PID/port 28472:54883 to 30380:54921 and then 36768:54935; after each start
the provider followed the new healthy endpoint and the gateway remained
Connected. Unrelated Ubuntu-24.04 and production OpenClawGateway distributions
remained registered.

The exact-head onboarding video is a real-motion 120.033-second H.264 recording
at 3270x2180/30 fps, SHA-256
FD756C5BB6BD96D18EBDAC81B0462673087A781A80EC1DCCDF2DD6E05CEC302D. It cuts
only waiting and transition spans, contains animated pointer movement, opens
the Local AI dashboard page, and shows Stop, Start, Restart, Running, and
Connected. Boundary frames were checked: no console/private window appears and
the setup controls are not clipped. The local media remains outside Git.

Not repeated destructively in this visible run: forced interrupted acquisition
and fresh-process uninstall of the completed Local AI tree. Those failure paths
are covered by the focused recovery, rollback, safe-path, and uninstall tests in
the passing SetupEngine suite.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 20, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot removed the proof: 🎥 video Contributor real behavior proof includes video or recording evidence. label Aug 20, 2026
@joelagnel

Copy link
Copy Markdown
Contributor Author

Current-head onboarding and Local AI behavior video for c6c96f8. This is the real full-screen setup flow of the connected dashboard and Local AI Stop, Start, and Restart controls.

Video SHA-256: FD756C5BB6BD96D18EBDAC81B0462673087A781A80EC1DCCDF2DD6E05CEC302D

openclaw-onboarding-timelapse.mp4

@joelagnel

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 20, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. proof: 🎥 video Contributor real behavior proof includes video or recording evidence. proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. labels Aug 20, 2026
@joelagnel

Copy link
Copy Markdown
Contributor Author

@karkarl @jacobtomlinson Updated PR with new detection scheme (not depending on specific GPU but just GPU memory capacity and runtime components).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature: ✨ showcase ClawSweeper spotlight: unusually compelling feature idea for maintainer attention. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. proof: 🎥 video Contributor real behavior proof includes video or recording evidence. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants