Skip to content

feat: add profile auto-heal watchdog example#40

Open
khang-dogo wants to merge 7 commits into
CloakHQ:mainfrom
khang-dogo:main
Open

feat: add profile auto-heal watchdog example#40
khang-dogo wants to merge 7 commits into
CloakHQ:mainfrom
khang-dogo:main

Conversation

@khang-dogo

@khang-dogo khang-dogo commented Jun 7, 2026

Copy link
Copy Markdown

Summary

  • adds a host-side watchdog to automatically relaunch CloakBrowser profiles when the browser/profile crashes while the manager process is still alive
  • closes the current operational gap: upstream auto_launch=true only applies at manager startup and does not continuously self-heal afterward
  • keeps the PR low-risk and easy to review: this adds ops/watchdog/ examples and runbook guidance without changing the upstream core runtime

Root cause → fix

  • Root cause: built-in auto-launch only handles the initial manager startup path
    • Fix: add a watchdog that polls /api/profiles and calls POST /api/profiles/<id>/launch for profiles with auto_launch=true but status != running
  • Root cause: there is no standard host-side runbook for enabling auto-heal behavior
    • Fix: add a sample systemd unit plus install/verification/smoke-test documentation

Flow

flowchart TD
    A[cloakbrowser-manager healthy] --> B[profile or browser crashes]
    B --> C[profile status becomes stopped]
    C --> D[host watchdog polls /api/profiles every 15s]
    D --> E{auto_launch = true?}
    E -- no --> F[leave profile stopped]
    E -- yes --> G[POST /api/profiles/<id>/launch]
    G --> H[profile returns to running]
Loading

Before / After

  • Before:
    • the manager container can remain healthy
    • a crashed profile stays in stopped
    • an operator must relaunch it manually
  • After:
    • the manager container can remain healthy
    • the watchdog detects the stopped profile
    • any profile with auto_launch=true is relaunched automatically within about 15 seconds

Files changed

  • README.md
    • adds a section describing the custom fork behavior and operating caveat
  • ops/watchdog/cloakbrowser_watchdog.py
    • adds a host-side Python watchdog that polls the manager API and relaunches eligible profiles
  • ops/watchdog/cloakbrowser-watchdog.service
    • adds a sample systemd service for durable host-side execution
  • ops/watchdog/README.md
    • adds install, verification, smoke-test, and manual-stop caveat documentation

Scope

  • ✅ adds auto-heal behavior at the host/operator layer
  • ✅ preserves upstream behavior for users who do not enable the watchdog
  • ✅ documents the auto_launch=false caveat for intentionally keeping a profile down
  • ❌ does not yet make the watchdog a built-in core manager feature
  • ❌ does not yet add a dedicated UI override for intentional manual-stop persistence

Evidence

  • verified against the real runtime on Dogo it-workstation-2 with a smoke test:
    • created a test profile
    • set auto_launch=true
    • waited for the watchdog launch -> running
    • stopped it manually
    • waited for the watchdog relaunch -> running
  • real logs confirmed the expected pattern:
    • relaunching profile ... status=stopped auto_launch=True
    • relaunched profile ... status=running

Notes

  • if an operator wants a profile to stay down, they should set auto_launch=false first
  • otherwise the watchdog treats a manual stop the same way it treats a crash and will bring the profile back up

* feat: add fingerprint rotation option with auto-restart

- Add fingerprint_rotation_interval field (minutes, 0 = disabled)
- Auto-generate new random seed and restart browser at interval
- Background rotation tasks managed per-profile, cancelled on stop
- DB migration: ALTER TABLE profiles ADD COLUMN fingerprint_rotation_interval

* feat: add fingerprint rotation UI to frontend form

- Add fingerprint_rotation_interval to Profile & ProfileCreateData types
- Add rotation interval input (minutes) in Behavior section of form
- Show "Rotate every X min" as subtitle on running profiles
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant