Skip to content

ops/decision: canary-enable the self-learning loops on self-host (review memory, self-tune, knob autotunes) #8762

Description

@JSONbored

Context

Parent: #8757. The audit confirmed the self-improvement machinery is fully built and rigorously guarded (shadow-soak 24h + strictly-tightening-only promotion in src/review/auto-apply.ts:399-448 with re-verification :155-183; held-out Pareto-floor validation in loosening-knobs; review-memory advisory-only by construction) — and ALL of it ships flag-OFF: LOOPOVER_REVIEW_SELFTUNE, LOOPOVER_REVIEW_MEMORY, SATISFACTION_FLOOR_AUTOTUNE_ENABLED, AI_REVIEW_CLOSE_CONFIDENCE_AUTOTUNE_ENABLED, AI_REVIEW_CLOSE_CONFIDENCE_TIGHTEN_ENABLED (wrangler.jsonc:43-45,160,173). The only live loop today is the merge/close precision circuit breaker (auto-tune.ts, always on).

Requirements

  • Canary on the self-host deployment first (not live-hosted for customers yet — natural canary surface).
  • Order: (1) LOOPOVER_REVIEW_MEMORY (advisory-rendering only, lowest risk); (2) LOOPOVER_REVIEW_SELFTUNE in its shadow-soak observe window; (3) knob autotune flags last, only after the corpus fixes land (see blocked-by) so decisions are not made from the biased block-mode-only sample.
  • Define the watch criteria BEFORE enabling: what soak output/metrics get reviewed after one week, and the explicit rollback (flags back to false; overrides table cleared) path.
  • Record the decision + observations as issue comments, not just config edits.

Deliverables

  • Written canary plan (order, watch window, rollback) as the issue's first comment.
  • Phase 1 + 2 enablement with observations.
  • Phase 3 go/no-go decision recorded with evidence.

Boundaries

Blocked by the corpus fixes (advisory-mode recording + implicit confirmations): knob decisions from the current biased corpus are exactly the failure the audit warned about. Self-host config changes only — no hosted/wrangler.jsonc flag flips in this issue.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions