Skip to content

feat: add clean Windows validation infrastructure - #1116

Draft
bkudiess wants to merge 58 commits into
mainfrom
bkudiess-hyper-v-unattended-setup
Draft

feat: add clean Windows validation infrastructure#1116
bkudiess wants to merge 58 commits into
mainfrom
bkudiess-hyper-v-unattended-setup

Conversation

@bkudiess

@bkudiess bkudiess commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds a Windows-native clean-machine lifecycle for owned Hyper-V Windows 11 guests, including unattended installation, preparation, verification, installed smoke, rollback, and integrity-checked artifact retrieval.
  • Adds installed, upgrade, active-desktop, live model/Discord, and Crabbox test routing, skills, and documentation without claiming lanes that were not real-run in this closeout.
  • Preserves strict VM, VHD, owner-marker, checkpoint, credential, rollback, and no-delete safety contracts.
  • Allocates a unique artifact directory per run and verifies guest archive hashes, host manifests, file counts, phase gates, and recovery state.

Validation

  • ./build.ps1: 5/5 targets passed.
  • dotnet test ./tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj --no-restore: 3,170 passed, 31 skipped.
  • dotnet test ./tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj --no-restore: 2,278 passed.
  • dotnet test ./tests/OpenClaw.SetupEngine.Tests/OpenClaw.SetupEngine.Tests.csproj --no-restore: 744 passed.
  • WindowsProofSkillContractTests: 20 passed.
  • Focused CreateWslInstance tests: 23 passed.
  • Final scoped read-only Claude Opus 4.8 max-effort review: clean, with no remaining actionable findings.

Real behavior proof

  • Environment tested: owned local Windows 11 Hyper-V guest OpenClaw-Clean-Windows.
  • PR head tested: 86c929002ba095d88d147884a48b6023da6726e3.
  • Current-head artifact path: TestResults\CleanWindows\HyperV\Installed\20260806-201406-469-49d515b7.
  • Guest archive SHA-256: 5FDF2D0DAC58CBEF58E254CF67FD1F99A7A9FD2BE02CE5DD89951FEEBCC573E8.
  • Host manifest SHA-256: 999D5D5032AA6212313427C941FA8208F13434319F01CAB302745FBD96A07AC7.
  • Artifact count: 27 manifest artifacts, 28 files including host-smoke-manifest.json.
  • Passed phases: preflight, build, install, installed-payload, and cleanup.
  • Failed phase: roundtrip.
  • Recovery proof: one validation-session recovery and one artifact-session recovery succeeded. The exact generated distro was installed once, then terminated and unregistered safely. No second install occurred.
  • Final ownership proof: VM ID and owner were unchanged; the VM was restored Off at the owned openclaw-prerequisites checkpoint.
  • Cleanup proof: no scheduled-task, launcher, or generated-distro residue remained.
  • Screenshots: none collected because no active unlocked desktop proof was run.

Known blocker

The pristine nested WSL first launch terminated with Wsl/Service/E_UNEXPECTED after two bounded 30-second readiness timeouts. The controller correctly failed closed and cleaned up only the exact generated distro. Full installed roundtrip is not verified, so this PR remains draft.

Upgrade, active-desktop screenshot proof, Crabbox Azure, and live model/Discord are implemented or configured lanes but were not real-run in this closeout.

Review

The final scoped Claude Opus 4.8 max-effort review completed cleanly. An earlier review identified that timed-out readiness probes containing incidental partial output could be misclassified as terminal. The implementation was corrected so explicit timeouts remain retryable under the exact-distro and absolute-deadline guards, while genuine terminal WSL diagnostics remain fail-closed. Focused tests cover the corrected behavior.

Security and compatibility

  • Credentials remain current-user DPAPI-protected and are never passed as plaintext command-line arguments or written to logs.
  • Temporary unattended media is owned, bounded, detached, and removed after readiness; the setup credential is rotated.
  • Existing manual Create behavior remains available, while the automated path requires explicit generated-credential consent.
  • The controller never deletes unowned VMs, VHDs, or checkpoints and never force-recovers ambiguous ownership state.

bkudiess and others added 30 commits July 23, 2026 22:37
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Automate verified Windows 11 Enterprise Evaluation installation with owned answer media, DPAPI credentials, bounded readiness, cleanup, and focused lifecycle coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Set the signed-in user as owner when UAC-created setup material defaults to Administrators, and document safe pre-VM recovery.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use the canonical MicrosoftWindows template and safely resume an owned Off VM whose security setup failed before first boot.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Establish and preserve the VM key protector before applying secure-boot firmware, then conditionally enable vTPM and reverify before start. Pin the shared fresh and resume ordering contract and document safe partial-state recovery.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reject every four-byte Hyper-V key-protector sentinel, re-read newly created protectors, and fail before firmware or vTPM unless the host reports a substantive blob. Add executable sentinel and ordering regressions and update resume guidance.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Poll Hyper-V until owned installation media detach state converges and permit already-detached continuation only for the exact owned powershell-direct-ready recovery state. Preserve strict first-attempt media requirements and complete credential rotation and guest verification on resume.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Persist owned pending checkpoint intent before creation, poll for eventual snapshot visibility, and finalize identity only after observation. Add explicit confirmed recovery for exact completed unattended clean-windows snapshots while keeping pending markers unauthorized for destructive operations.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Treat an active Hyper-V differencing disk as owned only when a bounded, cycle-safe Get-VHD parent chain terminates at the exact owner-marked base VHD. Preserve strict single-disk and VM marker identity checks with focused checkpoint ancestry coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Enable WSL platform features before reboot, install the package through bounded native process capture, reboot again after installation, and verify WSL before developer prerequisites. Preserve PowerShell Direct failure reasons with bounded sanitized diagnostics.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Route status-ready version-failing clean guests through one fixed noninteractive WSL web update, require the second owned reboot after update, and retain final zero-exit status and version verification before developer prerequisites.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Treat WSL status as authoritative: install immediately when absent without invoking the interactive version bootstrap, and probe version only for status-ready guests before deciding whether an update is required.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Force the exact machine-scope PowerShell 7.6.4 Wix package in the Hyper-V guest and verify the trusted executable path and engine version after installation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Require a clean committed HEAD and move one validated Git archive into the guest with hash, extraction, provenance, and cleanup guards.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
bkudiess and others added 21 commits July 29, 2026 18:12
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Allocate a canonical, unique child beneath the configured artifact base for every smoke invocation so retries preserve prior evidence while retaining CreateNew extraction safety.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Install only the pinned VC Redist component through the clean-runner package recovery path and verify its exact vswhere and x64 CRT payload before checkpointing.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Require the individual x64/x86 VC tools component alongside the redistributable update so clean-runner publish verification matches the loose CRT files consumed by MSBuild.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Wait for the existing guest validation process to close its artifact writers after an exact PowerShell Direct transport loss, without launching validation twice. Record recovery proof separately and cover bounded success, failure, timeout, and classifier behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Scale the aggregate WSL gateway configuration deadline by emitted command count using real clean-distro timing, and retain sanitized bounded failure diagnostics.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Poll guest completion in short bounded calls so an existing smoke process can survive multiple PowerShell Direct target recycles without duplicate validation or premature artifact packaging.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Treat a timed-out gateway start as success only when the active systemd service MainPID owns the configured listener and the normal HTTP health check passes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Poll the exact generated distro under one bounded monotonic deadline after successful installation, with fail-closed identity checks, sanitized diagnostics, and exact-distro cleanup.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the bounded diagnostic regression while avoiding a dummy value that review bundle safety correctly classifies as secret-like.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep terminal WSL diagnostics fail-closed, but treat the command runner's explicit timeout signal as retryable under the existing exact-distro and absolute-deadline guards.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P1 Urgent regression or broken agent/channel workflow affecting real users now. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. labels Aug 7, 2026
@clawsweeper

clawsweeper Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codex review: found issues before merge. Reviewed August 9, 2026, 4:44 AM ET / 08:44 UTC.

ClawSweeper review

What this changes

This branch adds clean-Windows VM, installed and upgrade smoke, desktop-proof, and opt-in live-parity validation infrastructure, alongside setup recovery and test coverage changes.

Merge readiness

Blocked by patch quality or review findings - 9 items remain

Keep open. This collaborator-authored draft adds validation capabilities not on current main, but it has an unfixed P1 self-hosted workflow security defect, incomplete central roundtrip proof, and a dirty merge state. Likely related people: Barbara Kudiess for recent setup recovery work, Dallin Romney for recent release-validation work, and Karen for fresh-WSL retry behavior.

Priority: P1
Reviewed head: 86c929002ba095d88d147884a48b6023da6726e3
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🧂 unranked krab (1/6) PR readiness rating was derived from proof quality, review findings, security review, and reviewer confidence.
Proof confidence 🌊 off-meta tidepool Not applicable: Real behavior proof is not required for maintainer- or bot-authored pull requests.
Patch quality 🧂 unranked krab (1/6) Security review found an item that needs attention.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: Real behavior proof is not required for maintainer- or bot-authored pull requests.
Evidence reviewed 5 items Unverified executable execution: The new self-hosted desktop-proof workflow downloads WindowsAppRuntimeInstall.exe from an aka.ms redirect and immediately executes it, without validating its Authenticode signature, Microsoft publisher, or expected version.
Prior blocker remains at current head: The prior ClawSweeper review requested installer identity validation, and its reviewed SHA is the same current PR head; the workflow still has the direct download-and-execute sequence.
Incomplete real behavior proof: The PR body reports that the clean guest passed preflight, build, install, installed-payload, and cleanup, but its required roundtrip phase failed with Wsl/Service/E_UNEXPECTED and no active desktop proof was collected.
Findings 1 actionable finding [P1] Verify the downloaded Windows App Runtime before execution
Security Needs attention Runtime installer lacks identity verification: A redirected download is executed with runner privileges without Authenticode signer, publisher, or expected-version validation.

How this fits together

The Windows validation subsystem builds and installs the tray app, drives gateway and desktop checks on Windows runners or owned VMs, and publishes proof artifacts for maintainers. Its workflows download prerequisites, start test environments, then collect smoke and UI evidence.

flowchart LR
A[PR validation change] --> B[Manual validation workflows]
B --> C[Runtime installer download]
C --> D[Trust verification]
D --> E[Desktop proof runner]
A --> F[Clean Windows guest]
F --> G[Installed gateway smoke]
E --> H[Proof artifacts]
G --> H
Loading

Decision needed

Question Recommendation
Should clean-VM lifecycle, desktop-proof, and live-parity lanes be accepted as one supported core validation program, or sponsored and landed as smaller independently proven surfaces? Split into supported lanes: Land only independently rebased, security-verified, and real-run validation lanes with a clear owner and proof contract.

Why: The 76-file feature spans multiple long-lived validation lanes and repository policy calls for one cohesive seam per PR; only maintainers can decide the permanent supported scope.

Before merge

  • Verify the downloaded Windows App Runtime before execution (P1) - This still downloads an executable through an aka.ms redirect and immediately runs it on the self-hosted desktop runner. Validate its Authenticode signature, Microsoft publisher, and the expected version before invoking it; the prior P1 remains unfixed because this is still the reviewed head.
  • Resolve security concern: Runtime installer lacks identity verification - A redirected download is executed with runner privileges without Authenticode signer, publisher, or expected-version validation.
  • Resolve merge risk (P1) - The manual self-hosted workflow executes a redirected downloaded installer without cryptographic identity checks, which can run an unintended binary with runner privileges.
  • Resolve merge risk (P1) - The central installed roundtrip remains blocked, so the clean-machine lifecycle is not proven end to end on the current PR head.
  • Resolve merge risk (P1) - GitHub reports a dirty merge state while current main changed overlapping setup and CI paths; rebase and proof refresh are required before merge.
  • Complete next step (P2) - A maintainer must decide the supported scope of this large validation program after the contributor resolves the concrete workflow security flaw and supplies complete proof.
  • Improve patch quality - Resolve the security review concern or explain why the changed path is safe.
  • Improve patch quality - Address the highest-priority review finding and re-run the changed-surface validation.

Findings

  • [P1] Verify the downloaded Windows App Runtime before execution — .github/workflows/windows-desktop-proof.yml:64-65
  • [high] Runtime installer lacks identity verification — .github/workflows/windows-desktop-proof.yml:64
Agent review details

Security

Needs attention: The new self-hosted workflow executes a downloaded Runtime installer without verifying its identity.

Review metrics

Metric Value Why it matters
Review surface 76 files affected, +36,779 / -287 lines The scope combines workflows, PowerShell, setup behavior, UI capture, documentation, and multiple test suites, making a single merge hard to validate.

Merge-risk options

Maintainer options:

  1. Rebase, verify, and prove the lifecycle (recommended)
    Verify the downloaded Runtime identity before execution, rebase the branch, and attach a successful current-head clean-machine roundtrip before merge.
  2. Split the validation program
    Pause the combined branch and propose narrower proof lanes with their own reviewed trust and runtime evidence.

Technical review

Best possible solution:

Split or rebase the work into maintainable validation seams, verify the Runtime installer signer, Microsoft publisher, and expected version before execution, then publish a successful current-head clean-machine roundtrip artifact.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: the workflow's current head downloads a redirected executable and invokes it without a signature, publisher, or version check.

Is this the best way to solve the issue?

No. The implementation needs a verified installer trust boundary, a rebase, and a successful end-to-end proof before this broad validation program is a safe landing path.

Full review comments:

  • [P1] Verify the downloaded Windows App Runtime before execution — .github/workflows/windows-desktop-proof.yml:64-65
    This still downloads an executable through an aka.ms redirect and immediately runs it on the self-hosted desktop runner. Validate its Authenticode signature, Microsoft publisher, and the expected version before invoking it; the prior P1 remains unfixed because this is still the reviewed head.
    Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.99

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 582a44713860.

Labels

Label justifications:

  • P1: An unverified executable is downloaded and run on a self-hosted Windows desktop runner.
  • merge-risk: 🚨 security-boundary: The new workflow creates a runner code-execution trust boundary without validating the downloaded installer identity.
  • merge-risk: 🚨 availability: Setup recovery and clean-machine smoke behavior remain unproven end to end and overlap current-main setup changes.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🌊 off-meta tidepool and patch quality is 🧂 unranked krab.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: Real behavior proof is not required for maintainer- or bot-authored pull requests.

Evidence

Security concerns:

  • [high] Runtime installer lacks identity verification — .github/workflows/windows-desktop-proof.yml:64
    A redirected download is executed with runner privileges without Authenticode signer, publisher, or expected-version validation.
    Confidence: 0.99

What I checked:

  • Unverified executable execution: The new self-hosted desktop-proof workflow downloads WindowsAppRuntimeInstall.exe from an aka.ms redirect and immediately executes it, without validating its Authenticode signature, Microsoft publisher, or expected version. (.github/workflows/windows-desktop-proof.yml:64, 86c929002ba0)
  • Prior blocker remains at current head: The prior ClawSweeper review requested installer identity validation, and its reviewed SHA is the same current PR head; the workflow still has the direct download-and-execute sequence. (.github/workflows/windows-desktop-proof.yml:65, 86c929002ba0)
  • Incomplete real behavior proof: The PR body reports that the clean guest passed preflight, build, install, installed-payload, and cleanup, but its required roundtrip phase failed with Wsl/Service/E_UNEXPECTED and no active desktop proof was collected. (86c929002ba0)
  • Current-main drift: The PR diverged from merge base e157ceb before current main added setup and CI changes, including fresh-WSL retry work; GitHub reports the PR as dirty, so it needs a rebase and renewed validation rather than assuming the old head is merge-safe. (src/OpenClaw.SetupEngine/SetupSteps.cs:745, 2eab151b6e13)
  • Applicable repository proof policy: Repository policy requires current-head real behavior proof for setup and UI work, explicitly recording blockers when focused proof is unavailable. (AGENTS.md:75, 582a44713860)

Likely related people:

  • Barbara Kudiess: Recent current-main commits cover gateway and WSL recovery adjacent to this branch's setup changes. (role: recent setup recovery contributor; confidence: high; commits: 603886a81d34, e9d571a9b395; files: src/OpenClaw.SetupEngine/SetupSteps.cs)
  • Dallin Romney: Recent current-main commits own release and gateway candidate validation workflows adjacent to the proposed validation lanes. (role: recent validation pipeline contributor; confidence: high; commits: 520f8051c19a, 9f9a8eda6340; files: .github/workflows/ci.yml)
  • Karen: Introduced the current-main fresh-distro root-probe retry behavior that overlaps the branch's readiness changes. (role: fresh-WSL retry contributor; confidence: medium; commits: 2eab151b6e13; files: src/OpenClaw.SetupEngine/SetupSteps.cs, tests/OpenClaw.SetupEngine.Tests/SetupStepsTests.cs)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)
  • reviewed 2026-08-07T01:06:29.773Z sha 86c9290 :: needs changes before merge. :: [P1] Verify the Windows App Runtime installer before executing it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant