fix(lease) task lease generation ABA

[wchwawa]

Summary

• Preserve one inactive terminal record per todo so release and committed terminal writeback cannot reset the next lease to version 1.
• Add the LoopX-owned lease_epoch, require the current version for lease mutations and terminal writeback, and reject immediate reuse of a retired execution key.
• Keep NoKV provider generations separate from LoopX authority generations, with regression coverage for local and cross-repository execution paths.

Issue Or Task

• Related: [Feature]: Integrate shared-goal authority through the native NoKV CLI and Python SDK #3390 and RFC: shared-goal coordination authority and pluggable state provider (v0) #2787
• Contributor task ID: N/A

Validation

• python3 -m py_compile for every changed Python file
• Public-boundary scan for all 19 changed files
• Other: 157 focused lease, todo, handoff, projection, and CLI tests passed
• Other: python3 examples/control_plane/task-lease-runtime-smoke.py
• Other: loopx canary premerge --from-git-diff selected and passed 18 checks with no warning or manual hold
• Other: live cross-repository source E2E with NoKV 9d609d10a1feda3512bfc4b49f889b94ea2f65d1, its Python SDK 0.11.0 and native CLI, isolated etcd, and digest-pinned RustFS

The live E2E first reproduced release followed by same-key reacquire and stale completion. With this branch, same-key reacquire is rejected, a new key receives the next version, and stale key/version completion is rejected. NoKV still showed exact receipt replay and independent artifact generations 1 -> 2 -> 3. Its create-only event replay, payload-mismatch rejection, historical commit replay, and per-commit restore also stayed intact for frozen-oracle use.

Type of Change

• Bug fix
• New feature
• Breaking change
• Refactoring (no functional changes)
• Documentation update
• Test update

LoopX Area

• Control plane (goals, todos, quota, scheduler, registry, runtime)
• Benchmark boundary (adapters, runners, verifiers, scoring, evidence)
• Capability or extension (providers, adapters, skills)
• Public docs or presentation surface (README, protocols, dashboard)
• Build, packaging, installer, or CI
• Host or runtime integration

Technical Direction

• Core control-plane hardening

• Long-horizon benchmark evidence

• Operator surface and IM integration

• Shared Goal Authority and cross-host coordination

• Architecture and research incubator

• Target base branch: main

• Direction tracker or promotion unit: RFC: shared-goal coordination authority and pluggable state provider (v0) #2787

Compatibility And Tradeoffs

• Hard-lease callers must now carry the version returned by acquire or renew into renew, transfer, release, complete, and supersede. This closes the partial-fence path that made a reused key unsafe.
• The local authority retains one small terminal JSON record per todo. Active projections ignore it, while exact release retries can recover the terminal result and the next acquire advances monotonically.
• This PR does not add the production NoKV provider. [Feature]: Integrate shared-goal authority through the native NoKV CLI and Python SDK #3390 tracks the native CLI-first, Python SDK-second integration; NoKV storage generation remains an opaque provider CAS token rather than a LoopX lease epoch.

Boundary Checklist

• I did not commit .loopx/, .codex/goals/, live ACTIVE_GOAL_STATE.md, credentials, private benchmark traces, verifier output, raw agent sessions, internal document links, or local machine paths.
• I did not duplicate maintainer-owned benchmark work unless a maintainer split out a public issue for it.
• I kept the change scoped to the linked issue/task.
• Every commit includes a DCO Signed-off-by trailer (git commit -s).

[wchwawa]

Validation update for exact head fa9baff0116f6e5be89f955928009c2ea319f946:

• Changed surfaces: task-lease state transitions and persistence, lease mutation/terminal-writeback CLI fencing, active todo/goal projections, the durable task-lease runtime smoke, and the corresponding protocol/RFC text.
• Full repository suite: PATH=.venv/bin:$PATH .venv/bin/python -m pytest -q — 3648 passed, 10 skipped, 1 warning, 0 failed in 413.25 seconds. The warning is the existing Pydantic incomplete-forward-reference warning emitted by test_native_controller_blocks_model_visible_mcp_mutations; this branch did not change that surface.
• Focused regression set: 157 lease, todo, handoff, projection, and CLI tests passed.
• Durable CLI/runtime smoke: python3 examples/control_plane/task-lease-runtime-smoke.py passed, including release tombstone, next-generation acquire, stale fence rejection, and exact release replay.
• Risk selection: loopx canary premerge --from-git-diff passed all 18 selected checks; public/private boundary scan passed for all 19 changed files; no warning or manual hold.
• Real cross-repository E2E used NoKV 9d609d10a1feda3512bfc4b49f889b94ea2f65d1, its source Python SDK 0.11.0 and native CLI, isolated etcd, and digest-pinned RustFS. It reproduced the old release/reacquire ABA before the fix, then proved same-key reuse rejection, monotonic versioning, stale key/version rejection, exact NoKV receipt replay, and preserved frozen-oracle commit/restore behavior after the fix.

This coverage is sufficient for the changed boundary because the focused tests cover the legal/illegal transition matrix, the durable smoke crosses the shipped CLI and persisted local record, and the live E2E crosses the real LoopX-to-NoKV receipt/generation path without treating NoKV's storage generation as a LoopX lease epoch.

[huangruiteng]

动机

这个 PR 修复的是本地 task_lease_v0 的真实 ABA 窗口：旧实现会在 release 或已验证的 terminal writeback 后删除 per-todo lease JSON，下一次 acquire 因而把 version 重建为 1。若执行 key 也被复用，旧 worker 持有的 key/version 就可能再次与新一代 lease 相同，无法可靠拒绝 stale renew、release 或 Todo 终态写回。该缺陷不仅影响本地 hard-lease 调用方，也会让后续 shared-goal provider 难以把 LoopX 自身的 authority generation 与 NoKV 等底层 provider generation 清晰分离。最近的更小修复若只禁止同 key 的一次复用，仍不能跨多代阻止旧 version 回绕，因此保留一个 per-todo 终态记录并让 CAS version 单调前进是合适的最小持久化边界。

改动思路

实现继续使用现有 per-goal file lock 和原子 replace，不引入新 provider 或第二套 authority。每个 lease mutation 都推进 version；lease_epoch 只在 acquire 或 transfer 形成新的所有权/执行代际时推进，renew 保持 epoch 不变。release 不再 unlink，而是原子写入 status=released 的 inactive tombstone；活动投影仍只消费 status=active 且未过期的记录。

正向路径是 acquire (v1,e1) → renew (v2,e1) → transfer (v3,e2) → release tombstone → 新 key reacquire (v4,e3) → 当前 key/version 完成 Todo 并再次留下 tombstone。负向路径在 canonical Todo 写入前 fail closed：退役 key 返回 idempotency_key_reuse 或 lease_cas_mismatch，旧 version 返回 version_mismatch，renew/transfer/release/terminal writeback 缺少 version 返回 version_required。NoKV/storage generation 仍是 provider 的 opaque CAS token，没有被误用为 LoopX 的 lease epoch。

具体改动

本 PR 共改 19 个文件（608 additions / 107 deletions），可分为三个完整 surface：约 129/20 行产品运行时代码、432/65 行测试与 durable smoke、47/22 行协议和架构文档。

关键代码讲解

• lease_epoch() 为已有无字段记录提供明确迁移语义：缺记录为 0，legacy 记录为 1，非法或非正整数按 corrupt_lease 拒绝，避免靠 prose 猜代际。
• released_lease_payload() / persist_released_lease() 在原有 goal lock 内保留最后一代 key、version、epoch 与 release 时间；terminal Todo 写回只有在已验证 fence 且 canonical write committed 后才调用 release hook。
• acquire_task_lease() 对 active retry 保持原幂等语义，对 expired/released 当前 key 的立即复用 fail closed，并从 tombstone 的 version/epoch 单调产生下一代。
• require_expected_version() 被接入 renew、transfer、release 及非 auto-acquired lifecycle writeback，使 machine-enforced obligation 与文档一致；transfer 还要求新 execution key，并推进 epoch。
• hold_task_lease_mutation_fence() 将 lease_epoch 投到成功与错误 payload，验证 owner、key、version 后才允许 complete/supersede；goal_channel_projection、visible governance 和 CLI Markdown 同步暴露代际信息。

测试覆盖了 legacy epoch 迁移/损坏、完整 acquire-renew-transfer-release-reacquire 链、release replay、退役 key、旧 key、旧 version、缺 version、Todo complete/supersede、event projection、dry-run、release 持久化失败、hard/soft/legacy handoff 及可见投影。durable CLI smoke 也真实跨过 acquire、release tombstone、新代 acquire 和 stale completion rejection。文档同步更新了 architecture、provider RFC、host integration、Todo contract、课程、roadmap 与 self-repair pattern，没有把 storage generation 或 provider-specific 术语写进通用控制面义务。

对主干的风险

主要兼容性风险是有意的 breaking change：已有 hard-lease host 若在 renew、transfer、release 或 terminal writeback 时只传 key、不传当前 version，会从成功或可选 CAS 变为 version_required。PR body、CLI help、host protocol 和 Todo contract 都明确披露了受影响命令及迁移方式，仓库内生产调用点和测试调用点也已同步。另一个成本是每个使用过 hard lease 的 Todo 保留一个小型 terminal JSON；这是阻止 version 回绕所需的有界 lineage，活动冲突和 dashboard 投影会过滤 inactive tombstone。

失败路径保持保守：Todo canonical write 成功而 tombstone 写入失败时，完成结果不会被回滚，但返回的 fence 标记 released=false，原 active record 留给显式 repair/release；这延续了既有 post-commit cleanup 失败语义，没有静默宣称 release 成功。typed state rule、domain neutrality、behavior-change disclosure、guidance-vs-obligation 四个 review lens 均通过；未发现 substring/prose-only 分类、领域专用控制面文案或把 machine obligation 称为 guidance 的问题。剩余风险主要是第三方 host 是否按 breaking contract 升级，仓库与公开协议已给出足够的 fail-fast/readback 路径。

我的整体评价

我批准 exact head fa9baff0116f6e5be89f955928009c2ea319f946。修复在现有 lease owner 内完成，没有提前引入 NoKV provider 或泛化 storage abstraction；状态机、调用方、投影、协议和回归证据是一个可独立 review/revert 的 cohesive batch。独立验证通过 240 个相关 pytest、task-lease-runtime-smoke.py、Ruff、changed-file py_compile、git diff --check，以及风险选择的 18 个 pre-merge checks（含 public/private boundary，0 failure、0 warning、0 manual hold）；GitHub 的 pytest、Windows、build、DCO 和 dependency-review 也全部为绿色。未发现阻塞合并的问题。

English verdict: APPROVE exact head fa9baff0116f6e5be89f955928009c2ea319f946; the retained terminal lease generation plus mandatory key/version fencing closes the local ABA path without conflating LoopX authority epochs with provider CAS generations, and focused tests, the durable CLI smoke, 18 risk-selected canaries, boundary checks, and GitHub CI are green.

[wchwawa]

Got it

…

________________________________ From: huangruiteng ***@***.***> Sent: Friday, 21 August 2026 15:10:33 To: huangruiteng/loopx ***@***.***> Cc: Jason ***@***.***>; Author ***@***.***> Subject: Re: [huangruiteng/loopx] fix(lease) task lease generation ABA (PR #3393) Merged #3393<#3393> into main. — Reply to this email directly, view it on GitHub<#3393?email_source=notifications&email_token=A44NEPTQSWD3F3QGAGHARF35K7KUTA5CNFSNUABQM5UWIORPF5TWS5BNNB2WEL2JONZXKZKFOZSW45CON52GSZTJMNQXI2LPNYXTEOJXHA2DEMZVHA4TPJTSMVQXG33OUZQXK5DIN5ZKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#event-29784235897>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A44NEPXFIQFHBPNHTTZK7NL5K7KUTAVCNFSNUABGKJSXA33TNF2G64TZHMYTENJVGIYTOOJTHA5US43TOVSTWNJSGA2DCOJXG42DTILWAI>. You are receiving this because you authored the thread.Message ID: ***@***.***>