Skip to content

fix: support memory-qualified NVIDIA GPUs for local AI - #1185

Closed
karkarl wants to merge 22 commits into
feature/local-ai-llama-server-code-onlyfrom
karkarl-pr-1178-hardware-spacing
Closed

fix: support memory-qualified NVIDIA GPUs for local AI#1185
karkarl wants to merge 22 commits into
feature/local-ai-llama-server-code-onlyfrom
karkarl-pr-1178-hardware-spacing

Conversation

@karkarl

@karkarl karkarl commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator
Additional instructions

MUST: Keep Allow edits from maintainers enabled for this PR so maintainers
can help update the branch when needed.

What Problem This Solves

Fixes an issue where users with an otherwise unknown NVIDIA GPU could not enable managed Local AI even when the GPU met the supported architecture, memory, driver, and CUDA requirements. It also corrects excess spacing in the Local AI unavailable InfoBar.

Why This Change Was Made

Named x64 and ARM64 hardware profiles remain the preferred selection path. Unknown NVIDIA GPUs can now use an architecture-specific pinned runtime when GPU-visible memory is at least 24,000 MiB, while the existing NVIDIA driver 615.0 and CUDA 13 requirements remain mandatory. Shared memory is intentionally excluded from the generic fallback.

User Impact

Users with supported x64 or ARM64 systems and NVIDIA GPUs with at least 24,000 MiB of visible memory can install Local AI without requiring an allowlisted GPU name. The unavailable-state InfoBar content is also spaced correctly.

Evidence

  • Added coverage for unknown x64 and ARM64 NVIDIA GPUs exactly at the 24,000 MiB threshold.
  • Added below-threshold rejection coverage.
  • Added coverage proving a 24,512 MiB device proceeds to the mandatory driver-version check.
  • Added a scoped XAML contract assertion for the first content StackPanel inside LocalAiUnavailablePanel.
  • Direct ARM64 tray build completed with 0 errors.

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs or instructions
  • Tests or validation
  • Security hardening
  • Chore or infrastructure

Scope

  • Tray or WinUI UX
  • Windows node capability
  • Local MCP or winnode
  • Gateway, connection, or pairing
  • Setup or onboarding
  • Permissions, privacy, or security
  • Tests, CI, or docs

Validation

  • powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\build.ps1: blocked by the host's known Windows 10 SDK detection prerequisite.
  • dotnet restore .\src\OpenClaw.Tray.WinUI\OpenClaw.Tray.WinUI.csproj -r win-arm64 -p:NuGetAudit=false followed by dotnet build .\src\OpenClaw.Tray.WinUI\OpenClaw.Tray.WinUI.csproj -c Debug -r win-arm64 --no-restore -p:NuGetAudit=false: passed with 0 errors and 1 existing obsolete Icon warning.
  • dotnet test .\tests\OpenClaw.Shared.Tests\OpenClaw.Shared.Tests.csproj --no-restore: 3,801 passed, 32 skipped, 0 failed.
  • dotnet test .\tests\OpenClaw.Tray.Tests\OpenClaw.Tray.Tests.csproj --no-restore: 2,699 passed, 0 failed.

Real Behavior Proof

  • Environment tested: Windows ARM64, .NET SDK 10.0.400
  • PR head or commit tested: 9d6ab525ed6d1962f7b81d07cbeca4aaa35902d7 (tree content matches the validated pre-push commit exactly)
  • Exact steps or command run: direct ARM64 tray build plus the shared and tray test commands listed above
  • Evidence after fix: threshold, architecture-specific fallback, driver guard, and scoped InfoBar XAML assertions all passed in their respective suites
  • Observed result: supported unknown NVIDIA GPUs select the architecture-specific fallback; below-threshold GPUs remain unsupported; driver and CUDA checks remain enforced
  • Screenshot or artifact links verified? (Yes/No/N/A): N/A
  • Not verified or blocked: Live UI screenshot was not captured. Full build.ps1 remained blocked by Windows SDK prerequisite detection, with the direct ARM64 tray build passing instead.

Security Impact

  • New permissions or capabilities? (Yes/No): No
  • Secrets or tokens handling changed? (Yes/No): No
  • New or changed network calls? (Yes/No): No
  • Command or tool execution surface changed? (Yes/No): No
  • Data access scope changed? (Yes/No): No
  • If any answer is Yes, explain the risk and mitigation: N/A

Compatibility and Migration

  • Backward compatible? (Yes/No): Yes
  • Config or environment changes? (Yes/No): No
  • Migration needed? (Yes/No): No
  • If yes, list the exact upgrade steps: N/A

Review Conversations

  • I replied to or resolved every bot review conversation addressed by this PR.
  • I left unresolved only conversations that still need maintainer judgment.

joelagnel and others added 20 commits August 20, 2026 11:28
@joelagnel
refactor(local-ai): add durable runtime manifests
b159030 
Persist installed runtime and model metadata with atomic file updates.
Give setup, startup, and cleanup one durable source of installation truth.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
refactor(local-ai): add native Windows managed-process host
dd3f253 
Host native Local AI processes with bounded startup and shutdown behavior.
Capture output and terminate owned process trees reliably on Windows.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(inference): add pinned runtime and model catalogs
9851ed1 
Define immutable llama.cpp, CUDA, and GGUF catalog entries.
Pin versions, URLs, hashes, and hardware requirements for reproducible installs.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(inference): select qualified hardware profiles
6982b9c 
Select the best qualified Local AI recipe from the host snapshot.
Return deterministic eligibility decisions and actionable rejection reasons.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(inference): probe NVIDIA hardware through trusted NVML
c1c1bbc 
Load NVML from trusted locations and collect NVIDIA GPU capabilities.
Avoid unsafe library resolution while providing selector-grade hardware data.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(huggingface): download verified GGUF models
63cc278 
Resume interrupted GGUF downloads with strict Range and Content-Range handling.
Verify size and hash before atomic promotion, restarting safely when resume is unsupported.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(llama-server): install verified native CUDA runtime
f7c5aae 
Install and inspect the pinned llama-server and CUDA runtime components.
Safely reconcile exact orphan runtime paths so interrupted promotion can retry.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(llama-server): add request-driven managed router
327db16 
Launch llama-server on demand with an OS-assigned or validated fixed port.
Prove listener ownership by child PID and start time before health checks or persistence.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(setup): select and acquire qualified Local AI plan
16fcfac 
Make Local AI opt-in and inspect hardware plus WSL viability without mutation.
Acquire and verify native inference before later WSL and OpenClaw provisioning.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(setup): persist and start managed Local AI
ed930e1 
Reconcile and reuse only exact manifest-owned runtime and model artifacts after interruption.
Persist a proven healthy endpoint and clean durable app-owned state on fresh uninstall.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(setup): verify local inference readiness
2066c6c 
Probe health, execute real inference, and verify GPU activity on the owned endpoint.
Fail setup cleanly when the installed Local AI stack is not operational.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(setup): configure IPv4 llama-server gateway provider
5b50653 
Use explicit 127.0.0.1 for Windows-to-WSL gateway connections and persisted setup state.
Remove only exact managed Local AI provider state during uninstall, preserving drift.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(local-ai): own llama router in companion lifecycle
6cdc66f 
Quiesce the exact managed WSL provider before every runtime transition or failure path.
Reject non-loopback binds, publish only healthy owned endpoints, and compensate failed publication.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(setup-ui): add Local AI onboarding
ca407ff 
Surface Local AI eligibility, explicit consent, review, and setup progress.
Show native inference before WSL setup and describe dynamic multi-gigabyte disk use accurately.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
feat(tray): add Local AI status and controls
ef12f77 
Add Local AI navigation, status, controls, logs, and localized resources.
Wire the page through application services and retain focused UI contracts.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@jacobtomlinson @joelagnel
fix(local-ai): support Spark N1X core variants
64e98aa 
Recognize shipping Spark processor-name variants during hardware qualification.
Keep detection compatible when N1X systems report alternate SKU text.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@jacobtomlinson @joelagnel
fix(local-ai): account for Spark shared GPU memory
014ca4e 
Include shared GPU memory when qualifying Spark unified-memory systems.
Avoid hiding Local AI when dedicated VRAM alone understates usable capacity.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@joelagnel
improve(setup): explain unavailable Local AI
43cd664 
Keep a compact Local AI review visible when hardware, WSL, or networking blocks setup.
Hide unavailable controls and show every detected dependency reason through See why.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@jacobtomlinson @joelagnel
fix(local-ai): verify shared GPU model placement
f6b2ad4 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@RomneyDa @claude @joelagnel
style(setup-ui): pad WSL networking consent content
d241b7c 
The consent InfoBar content sat flush against the bottom edge of the
warning, leaving the message and checkbox visually cramped. Add the
same bottom content margin the sandbox InfoBar already uses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8LRNTZ5fBgjGwmtBCrSp3
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
@clawsweeper

clawsweeper Bot commented Aug 20, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@karkarl karkarl added the status: 🚢 actively landing A maintainer or agent is actively driving this item through implementation, validation, or merge. label Aug 20, 2026
@clawsweeper

clawsweeper Bot commented Aug 20, 2026

Copy link
Copy Markdown

ClawSweeper status: review started.

I am starting a fresh review of this pull request: fix: support memory-qualified NVIDIA GPUs for local AI This is item 1/1 in the current shard. Shard 0/1.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

karkarl and others added 2 commits August 20, 2026 14:41
@karkarl
fix: allow memory-qualified local AI GPUs
db3bb78 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@karkarl
fix: correct local AI InfoBar spacing
c6fe7c1 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@karkarl
karkarl force-pushed the karkarl-pr-1178-hardware-spacing branch from 9d6ab52 to c6fe7c1 Compare August 20, 2026 21:44
@karkarl

karkarl commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by the cross-fork follow-up targeting joelagnel:feature/local-ai-llama-server-code-only.

@karkarl karkarl closed this Aug 20, 2026
@karkarl karkarl removed the status: 🚢 actively landing A maintainer or agent is actively driving this item through implementation, validation, or merge. label Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@karkarl @joelagnel @jacobtomlinson @RomneyDa