RFC 0013 addendum: application-consistent recovery for scale-to-zero

[giodl73-repo]

Summary

Scale-to-zero means OpenClaw can safely hibernate when idle and wake when work
arrives, so hosts do not pay for idle containers. Vincent's snapshot work gives
OpenClaw durable state capture and restore, while Peter's Gateway suspension
and Cron work provides much of the lifecycle foundation.

The remaining need is a small handoff with the host: remember that work is
waiting, wake a replacement, restore its accepted state, and wait until the
Gateway is ready before sending the queued work. This PR defines that
application-consistency contract without moving the host's scheduler,
transport, placement, or retained payloads into OpenClaw.

The detailed proposal adds three optional, host-neutral lifecycle
specifications to the completed RFC 0013 SQLite snapshot contract:

• recovery-point component composition;
• portable final handoff; and
• restored admission.

Together they define the application-consistency proof a host needs before it
can safely replace a Gateway generation or remove idle compute. They do not
define a new snapshot implementation, pause API, storage provider, or wake
system.

This PR is stacked on RFC PR #20.
RFC 0013 and
openclaw/openclaw#105718
remain authoritative for one-database SQLite snapshot creation, verification,
local publication, and fresh-target restore.

Why This Is Needed

Per-user and event-driven hosts can scale resident compute to zero only if they
can prove all of the following:

1. the retiring Gateway generation closed the complete owner-selected state;
2. the host durably accepted the exact required byte set;
3. source-compute retirement was authorized for that exact generation; and
4. replacement admission remained closed until the same recovery point and
   required owner state were restored.

A verified SQLite snapshot proves none of those aggregate lifecycle facts by
itself. Without a shared contract, hosts must either keep idle Gateways warm or
maintain private shutdown, copying, restore-ordering, and readiness-inference
paths.

The motivating managed-host deployment already has durable ingress and cron
wake machinery. Its production issue cluster includes accepted work outliving
the compute that should process it, cold-start head-of-line blocking, cron wake
racing idle retirement, scheduler continuity failing after replacement, and
cold runtimes appearing ready before the required owner state is restored.
OpenClaw's broader state-ownership risk is also tracked in
openclaw/openclaw#101290.
The complete user/operator case, including missed recurring work, competitor
scale-to-zero primitives, and Microsoft Scout's host-side composition, is
tracked in
openclaw/openclaw#114145.

Industry Baseline And OpenClaw's Missing Layer

Infrastructure and sandbox platforms already expose the compute primitives:

Platform	Primitive
Fly Machines	Traffic-driven stop-to-zero and autostart
E2B	Persistent pause and automatic resume
Daytona	Persistent stop/start, archive, and VM pause/resume
Azure Container Apps	Event-driven scale-to-zero
Cloudflare Durable Objects	Hibernation with wakeable WebSocket delivery

Those systems cannot infer OpenClaw's complete owner inventory, SQLite
invariants, scheduler state, or restored-readiness boundary. This addendum
defines that application-consistent layer. Retained ingress, wake registration,
compute placement, and external durability remain host responsibilities.

What Changed From The Earlier Continuity Direction

The sidecars compose contracts already shipped on OpenClaw main:

• openclaw backup sqlite from
  #105718; and
• gateway.suspend.prepare|status|resume from
  #103618, including
  Vincent's validation repair in
  #103925.

They replace the earlier broad continuity direction rather than adding another
SQLite capture path, Gateway pause API, provider framework, Lobster transport,
CAPE configuration surface, or Elastic wake mechanism.

The suspension contract remains a cooperative tracked-work fence, not full
process quiescence. The host still owns external ingress fencing, third-party
Channel transports, unregistered work, process supervision, encryption,
retention, placement, and wake.

Closed Boundaries

• The runtime state owner supplies a canonical required-component inventory;
  missing and extra components fail closed, and the inventory is covered by
  the recovery-point identity.
• Ordinary RFC 0013 snapshots remain host-protected. They cannot be called
  credential-free portable without exact owner-authored portability receipts.
• Obligations use a closed kind/owner/treatment vocabulary and contain no
  secret values.
• Host acceptance binds canonical aggregate-manifest bytes plus every exact
  owner-manifest and artifact byte sequence. Packaging and paths remain
  host-owned.
• Restored admission binds the same acceptance-set identity through hold,
  restored-start, and ready evidence. When RFC 0018 canonical readiness is
  available and enabled, it projects that same evidence without adding a
  second evaluator.
• Unknown outcomes hold or quarantine. Quarantine has no automatic exit.

The lifecycle facts are deliberately separate:

GatewaySuspensionReady
  -> SourceWritersClosed
  -> RecoveryPointCaptured
  -> RecoveryPointAccepted
  -> SourceComputeRetirementAuthorized
  -> RecoveryPointRestored
  -> AdmissionReady

There is no global /synced state. Each fact has a different owner, subject,
lifetime, and failure boundary. The diagram is the success path, not an
unconditional linear workflow: retained work may race retirement, atomically
revoking it while source compute exists or joining the destination wake after
retirement.

Observable Outcome Of The Complete Composition

• Idle resident compute may reach zero without treating raw live files as a
  recovery point.
• Accepted ingress remains retained until restored admission succeeds.
• Autonomous scheduled work can recover from absent compute without an
  unrelated user message.
• Source compute is never retired before exact durable acceptance.
• Replacement readiness identifies the accepted recovery point it restored.

These are system-level acceptance criteria. The three current OpenClaw PRs
prove only the owner-side recovery-point, final-capture, and restored-admission
slices; they do not claim to complete host acceptance, retained ingress, wake,
or source-compute retirement.

One Contract, Two Directions

The lifecycle can be presented as one bidirectional host/runtime contract
without putting host lifecycle authority in OpenClaw. OpenClaw supplies the
running-source suspension methods, offline final-capture and accepted-restore
operations, and the restored-readiness fact. The host supplies durable
acceptance, compute generations, retained wake, placement, and source-compute
retirement.

Quiesced
  -> RecoveryPointCaptured
  -> RecoveryPointAccepted
  -> generation-scoped SourceComputeRetirementAuthorized
  -> retained wake cause
  -> RecoveryPointRestored
  -> AdmissionReady
  -> owner delivery and acknowledgement

This is a protocol boundary, not a generic base class or new callback
framework. A host may compose it behind prepareHibernate,
ensureRuntimeReady, and inspectLifecycle, but those remain host operations.
The existing gateway.suspend.*, scheduler hooks, recovery operations, and
admission fence are reused.

The RFC now recommends one concrete missing seam:

gateway.restore.status({ restoreOperationId })
  -> { status: "not-restored" }
   | { status: "held", reason, retryAfterMs, ...restoreIdentities }
   | { status: "ready", ...restoreIdentities, schedulerIdentity,
       ownerReadinessIdentity, readinessIdentity }

This follows Peter's gateway.suspend.status pattern: closed Gateway Protocol
schemas, operator.read, stable operation identity, no mutation, exact replay,
and typed conflict behavior. It projects PR 112896's existing durable ready
record rather than adding another journal or state machine. Offline restore
and quarantine do not become invented live phases. The method remains callable
only through an authenticated pre-admission control path while ordinary work
is held. That path is a tracked, bounded, non-enrolling host probe: restart
still wins, and nodes, pairing, remote clients, presence-bearing sessions, and
ordinary RPCs remain fenced. There is no gateway.restore.admit; OpenClaw
computes readiness and opens its own admission.

When RFC 0018 is available and enabled, the same record also publishes a
required RecoveryPointRestored condition through /readyz, openclaw ready,
Status, and Gateway readiness RPC. Stable subject refs identify the destination
Gateway, restore operation, accepted recovery point, runtime generation,
scheduler reconciliation, and required state-owner roles; current IDs and
generations stay in RFC 0018's identity package. The generic readiness
projection and operation-fenced status method must report the same readiness
generation. Neither may infer restore success from process health or
database-open success. gateway.restore.status remains independently useful
when canonical readiness is disabled or has not landed.

Planned handoff may create a new recovery point only after Gateway suspension,
host-proven source-writer closure, final capture, and durable acceptance.
Forced source loss creates no clean-retirement fact and recovers from the last
previously accepted point under its declared RPO.

Implementation Evidence

• openclaw/openclaw#112385
  composes verified RFC 0013 snapshots into one deterministic host-protected
  recovery point and exact acceptance-byte inventory, using one canonical
  generation-bound state-owner inventory receipt across the stack.
• openclaw/openclaw#112865
  is the final-capture PR stacked on #112385. It binds
  host-supplied closure evidence, creates final owner snapshots, and durably
  replays one committed recovery point.
• openclaw/openclaw#112896
  is the restored-admission PR stacked on #112865. It restores an exact
  accepted recovery point to fresh paths and reuses Gateway work admission to
  remain closed until scheduler reconciliation and required-owner readiness
  produce one durable ready record. It now also implements the proposed
  read-only gateway.restore.status projection and its tracked local probe
  handshake as representative proof. The
  public Gateway Protocol addition remains an explicit owner-review decision
  before merge or ship.
• Fork-only Lobster drafts
  #39,
  #41,
  #42,
  #43, and
  #45,
  #46,
  #47, and
  #51 (Microsoft access
  required) demonstrate permanent cron projection, cancellation arbitration,
  Peter's shipped hook/proxy-pipe composition, and a default-off private
  Blob/per-user ECS/already-due sleep-veto boundary, followed by one permanent
  payload-free retained-wake authority that coalesces concurrent owner causes
  into an exact destination generation, production-backed metadata-only
  Teams/cron owner validation, and exact-readiness-gated per-cause delivery
  with durable crash replay.

The retained-wake drafts deliberately do not activate a production route or
cohort. Lobster still needs an owner-approved accepted-final-handoff lookup and
canonical mapping from opaque logical runtime identity to the Teams/cron
semantic owners before those call sites can be wired safely.

These PRs and fork drafts are non-normative feasibility and owner-review
evidence. None moves external ingress fencing, clean shutdown, host acceptance,
publication, wake, compute placement, or source-compute retirement into
OpenClaw.

If Vincent, Peter, Josh Lehman, or other maintainers prefer a smaller or
different implementation that meets the RFC and issue #114145 outcomes, please
use it. The fork stack is evidence for review, not a requirement to preserve
this exact decomposition.

Current Main Update

Recent owner work narrows the remaining integration:

• Peter's merged #118393
  keeps each unresolved Cron run visible to Gateway suspension through bounded
  shutdown.
• Peter's merged #117705
  removes most local startup tax from Gateway-backed agent turns.
• Vincent's #113306
  campaign has landed durable parent publication and pending-snapshot recovery;
  generic staging-cleanup ownership remains open.
• Vincent's open #117258
  adds per-owner isolation for post-commit auth snapshot publication.

These foundations improve shutdown accounting, cold-start cost, snapshot
durability, and owner-local reconciliation. They do not yet provide the
accepted-final-handoff lookup, retained Teams/API/Cron wake authority,
generation placement, or exact-readiness-gated delivery replay required for
scale-to-zero.

The OpenClaw implementation stack was rebuilt on current main and is
mergeable at 32809a14fc6 (#112385), 77e09a8f5c9 (#112865), and
247457d3544 (#112896). Cross-fork PRs cannot target fork-only parent branches,
so GitHub displays cumulative child diffs even though branch ancestry is
linear.

Repository review also raises one placement decision: keep these optional
lifecycle contracts under completed RFC 0013, or move them to a standalone
draft RFC. The branch now states that RFC 0013 remains authoritative only for
the completed SQLite artifact contract until maintainers settle that boundary.

Review Request

Please review the ownership split and public status method before the
implementation stack is promoted:

1. RFC 0013 remains the completed SQLite owner primitive.
2. Current-main gateway.suspend.* remains the cooperative pause primitive.
3. The state owner, not aggregate composition, supplies the complete selected
   agent inventory from the activation-pinned runtime and binds its revision to
   the source generation.
4. Ordinary snapshots remain host-protected; portability requires owner proof.
5. Acceptance binds a closed logical byte set without standardizing a storage
   container.
6. The host owns ingress fencing, suspension orchestration, clean shutdown,
   wake, and compute lifecycle; final capture only binds supplied closure
   evidence.
7. OpenClaw core owns recovery-journal schema, migration, corruption, repair,
   retention, and cleanup semantics; unknown journal state never falls back to
   ordinary startup.
8. When available and enabled, RFC 0018 owns the canonical current-readiness
   projection, while gateway.restore.status remains the independent exact
   operation-fenced observation seam.

[giodl73-repo]

Hey @vincentkoc — this is the small follow-on referenced from #20.

The review question is only the ownership split:

1. RFC 0013 and backup sqlite remain authoritative for each SQLite artifact.
2. gateway.suspend.* remains the cooperative tracked-work fence.
3. These sidecars add only aggregate recovery-point identity, durable handoff acceptance, and restored admission/replay above those shipped contracts.

We are holding implementation until that boundary is accepted. The earlier RFC 0021 / twelve-PR design is now retained only as evidence, not proposed as the path forward.

[giodl73-repo]

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

• State: Review in progress
• Detail: The exact-review queue leased this run; Codex is reviewing the item.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/30010475441
• Updated: 2026-07-23T13:16:35.194Z

[clawsweeper]

Codex review: needs real behavior proof before merge. Reviewed August 15, 2026, 9:57 AM ET / 13:57 UTC.

ClawSweeper review

What this changes

This PR adds scale-to-zero recovery, handoff, and restore-admission specifications to the RFC 0013 SQLite snapshot proposal.

Merge readiness

⚠️ Ready for maintainer review - 5 items remain

Keep open: the prior P2 boundary finding remains unresolved on the unchanged head. This broad host/runtime lifecycle and proposed Gateway API should be reviewed as a standalone draft RFC, not embedded in the unmerged SQLite snapshot RFC.

Priority: P2
Reviewed head: 2a70d5ce706b99b9b0ebf07f686ab5c9b6f85e89
Owner decision: Required. See Decision needed.

Review scores

Measure	Result	What it means
Overall readiness	🦐 gold shrimp (3/6)	The proposal is substantial, but its RFC and protocol boundary needs maintainer resolution before it is ready.
Proof confidence	🌊 off-meta tidepool	Not applicable: All changed files are RFC documentation, so runtime behavior proof is not applicable.
Patch quality	🦐 gold shrimp (3/6)	1 actionable review finding remain.

Verification

Check	Result	Evidence
Real behavior	Not applicable	Not applicable: All changed files are RFC documentation, so runtime behavior proof is not applicable.
Evidence reviewed	6 items	Current-main boundary: Current main has no RFC 0013 or RFC 0018 document, so the proposed addendum is not an update to a completed main-branch RFC. Expanded contract: The branch calls RFC 0013 completed while adding recovery, handoff, and admission contracts plus a proposed Gateway Protocol method. Protocol scope: The restored-admission sidecar specifies the new read-only gateway.restore.status API and an authenticated pre-admission path.
Findings	1 actionable finding	[P2] Move the lifecycle contract into a standalone draft RFC
Security	None	None.

How this fits together

The RFC repository records technical contracts for OpenClaw. This proposal connects SQLite recovery artifacts to host-controlled compute retirement, replacement startup, and Gateway work admission.

flowchart LR
A[Gateway state owners] --> B[Recovery point]
B --> C[Host acceptance]
C --> D[Compute retirement]
D --> E[Replacement startup]
E --> F[Restore admission]
F --> G[Queued work delivery]

Loading

Decision needed

Question	Recommendation
Should scale-to-zero recovery become a standalone RFC rather than an addendum to the unmerged RFC 0013 proposal?	Split into a standalone RFC: Move the lifecycle contract and its sidecars to a new draft RFC that names RFC 0013 as a dependency.

Why: This choice defines the permanent review and protocol boundary; it cannot be resolved mechanically from the documentation.

Before merge

• Add real behavior proof - Not applicable: All changed files are RFC documentation, so runtime behavior proof is not applicable.
• Move the lifecycle contract into a standalone draft RFC (P2) - This preserves the prior finding on the unchanged head. RFC 0013 is framed as a completed one-database artifact contract, but this patch adds three lifecycle protocols and a proposed Gateway API on its unmerged stacked branch. Move the scale-to-zero contract into its own draft RFC that references RFC 0013.
• Resolve merge risk (P1) - Merging this as an RFC 0013 addendum would make a broader host lifecycle and Gateway Protocol direction appear coupled to an unmerged SQLite artifact contract.
• Complete next step (P2) - Maintainers need to choose the RFC and Gateway-protocol boundary before the contributor can make a focused documentation revision.

Findings

• [P2] Move the lifecycle contract into a standalone draft RFC — rfcs/0013-cloud-serializable-sqlite-state.md:176-181

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
PR documentation delta	4 files, +1,411/-1	The branch substantially expands the stacked RFC with three normative lifecycle specifications.

Merge-risk options

Maintainer options:

1. Split the lifecycle RFC (recommended)
   Move the recovery, handoff, and admission contract into a standalone draft before further implementation review.
2. Approve the combined boundary
   Explicitly accept RFC 0013 as the owner of the added lifecycle and authenticated Gateway status contract.

Technical review

Best possible solution:

Publish the lifecycle contract as a standalone draft RFC that references RFC 0013 only as a dependency, then seek focused host and Gateway protocol approval.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is an RFC documentation proposal, not a report of a failing current behavior.

Is this the best way to solve the issue?

No: a standalone lifecycle RFC is the narrower, more maintainable way to review these host and protocol decisions.

Full review comments:

• [P2] Move the lifecycle contract into a standalone draft RFC — rfcs/0013-cloud-serializable-sqlite-state.md:176-181
  This preserves the prior finding on the unchanged head. RFC 0013 is framed as a completed one-database artifact contract, but this patch adds three lifecycle protocols and a proposed Gateway API on its unmerged stacked branch. Move the scale-to-zero contract into its own draft RFC that references RFC 0013.
  Confidence: 0.94

Overall correctness: patch is incorrect
Overall confidence: 0.94

AGENTS.md: not found in the target repository.

Codex review notes: model internal, reasoning high; reviewed against af708f0ddb6f.

Labels

Label justifications:

• P2: This is a substantial architecture proposal requiring focused owner review without changing running behavior directly.
• merge-risk: 🚨 compatibility: The proposal specifies recovery and Gateway behavior that future hosts and integrations may depend on.
• merge-risk: 🚨 security-boundary: The proposed Gateway status method includes an authenticated pre-admission control path that needs protocol-owner approval.
• rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
• status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: All changed files are RFC documentation, so runtime behavior proof is not applicable.

Evidence

What I checked:

• Current-main boundary: Current main has no RFC 0013 or RFC 0018 document, so the proposed addendum is not an update to a completed main-branch RFC. (rfcs/0013-cloud-serializable-sqlite-state.md:1, af708f0ddb6f)
• Expanded contract: The branch calls RFC 0013 completed while adding recovery, handoff, and admission contracts plus a proposed Gateway Protocol method. (rfcs/0013-cloud-serializable-sqlite-state.md:176, 2a70d5ce706b)
• Protocol scope: The restored-admission sidecar specifies the new read-only gateway.restore.status API and an authenticated pre-admission path. (rfcs/0013/restored-admission-v1-spec.md:243, 2a70d5ce706b)
• Repository RFC shape: Repository guidance defines one Markdown file per RFC, with sibling material supporting that RFC; a standalone lifecycle RFC is the reviewable shape for this proposal. (README.md:19, af708f0ddb6f)
• Proposal provenance: The behavior appears to originate with the branch’s initial lifecycle-sidecar commit and was most recently clarified at the current head. (rfcs/0013-cloud-serializable-sqlite-state.md:176, 2a70d5ce706b)
• Re-review continuity: The current head is identical to the prior reviewed head, so the existing P2 finding remains rather than being newly raised. (rfcs/0013-cloud-serializable-sqlite-state.md:176, 2a70d5ce706b)

Likely related people:

• Gio Della-Libera: Authored the initial lifecycle-sidecar commit and the current-head clarification; no merged current-main history exists for this RFC path. (role: proposal-path author; confidence: medium; commits: 17fcd5d894d7, 2a70d5ce706b; files: rfcs/0013-cloud-serializable-sqlite-state.md, rfcs/0013/portable-handoff-v1-spec.md, rfcs/0013/recovery-point-components-v1-spec.md)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Split the lifecycle material into a standalone draft RFC or obtain explicit owner approval for the combined scope.

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (74 earlier review cycles; latest 8 shown)

• reviewed 2026-08-09T18:52:06.271Z sha 2f290e9 :: needs real behavior proof before merge. :: [P2] Move the lifecycle contract into a standalone draft RFC
• reviewed 2026-08-09T19:46:48.394Z sha 2f290e9 :: needs real behavior proof before merge. :: [P2] Move the lifecycle contract into a standalone draft RFC
• reviewed 2026-08-09T22:01:16.756Z sha 2f290e9 :: needs real behavior proof before merge. :: [P2] Move the lifecycle contract into a standalone draft RFC
• reviewed 2026-08-09T23:10:26.171Z sha 2f290e9 :: needs real behavior proof before merge. :: [P2] Move the lifecycle contract into a standalone draft RFC
• reviewed 2026-08-12T01:54:31.313Z sha 2f290e9 :: needs real behavior proof before merge. :: [P2] Separate the lifecycle contract into a draft RFC
• reviewed 2026-08-13T02:43:44.174Z sha 540fb7c :: needs real behavior proof before merge. :: [P2] Move the lifecycle contract into a standalone draft RFC
• reviewed 2026-08-14T01:30:24.697Z sha 2a70d5c :: needs real behavior proof before merge. :: [P2] Move the lifecycle contract into a standalone draft RFC
• reviewed 2026-08-15T11:06:29.372Z sha 2a70d5c :: needs real behavior proof before merge. :: [P2] Move the lifecycle contract into a standalone draft RFC

[giodl73-repo]

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Review in progress
• Detail: The exact-review queue leased this run; Codex is reviewing the item.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/30525032804
• Updated: 2026-07-30T08:02:30.209Z