Skip to content

feat(guardrail): permanent, fail-closed domain gate for subnet-critical paths #2384

Description

@matedev01

Problem (current state, with file refs)

Two guardrail mechanisms exist today; neither classifies an arbitrary external PR by the domain of code it changes.

  1. scripts/benchmark_pr_policy.py is path-based but scoped to this repo's own agent surface. touches_guardrail(paths) is literally not is_agent_submission(paths) (lines 63-65); is_agent_submission (lines 50-60) whitelists only agent.py / agent/** (+ companion tests/**). It protects the agent/benchmark code from contributor edits — a whitelist of our own files. It says nothing about which subnet domain (emission/weights, validator-consensus, scoring, payment, wallet/keys) a PR touches, and does not transfer to an external product repo.

  2. openvang/factory.py gates by action TYPE, not by the domain of a merged diff. _OWNER_ACTIONS (lines 90-99) names GITHUB_WRITE, ONCHAIN_TRANSACTION, WALLET_ACCESS, EMISSION_CHANGE, GOVERNANCE_VOTE, PUBLICATION; RoleContract.__post_init__ rejects any role granted an owner action (lines 127-128); can_auto_execute() returns False "ever" (lines 434-438). But action-type gating cannot classify a merge by the domain of the diff it merges. A merge is one GITHUB_WRITE; nothing marks "this particular PR edits validator-consensus code, so it is forever-human."

  3. The live recommendation has zero path awareness. agent/review.py:37ACTIONS = ["merge", "request-changes", "reject", "comment"]. review_pr recommends regardless of which files a PR touches.

No module anywhere classifies subnet-critical code paths: grepping weights|emission|validator|consensus|wallet|payment|incentive|hotkey|coldkey across agent/, scripts/, openvang/, vanguarstew_runtime/ returns only the benchmark's blend weights and the factory's action-type enums — never a path classifier.

Why it blocks autonomous external maintenance

On a live incentive-subnet product repo the highest-consequence changes — emission/weight-setting, validator consensus and scoring, payment, wallet/key handling, the incentive mechanism itself — are exactly the ones with no safe automatic verdict. As the sibling #2385 establishes, an external PR produces no benchmark-delta artifact, so the decision falls back on the judge — which per #2379 saturates and cannot discriminate (the real fix, challenger-vs-king #2382, is unbuilt; documented as M2+ scope in benchmark/runner.py:7). A merge there is irreversible and can move real emissions or funds. Autonomy must never reach these paths — and "never" has to be a structural, fail-closed classifier, not a documented intention nor a property a future well-behaved judge is trusted to preserve.

Proposed direction (concrete, staged)

Add a permanent, path-pattern, fail-closed domain-guardrail classifier that every decision and (future) execution surface must consult.

  • Stage 1 — pure classifier. New module (e.g. openvang/domain_guardrail.py) mirroring benchmark_pr_policy.evaluate_policy / _normalized_paths (reuse the path hardening at lines 26-47). Input = repo-relative changed paths; output = {critical: bool, matched: [...], reason}. Match a versioned pattern set for: emission/weight-setting, validator consensus, scoring, payment, wallet/keys (incl. hotkey/coldkey), incentive mechanism, on-chain/governance. Fail closed: malformed/unparseable paths, empty input, or any PR that cannot be fully classified resolve to critical=True. A per-repo override may only add patterns, never remove a built-in critical class.
  • Stage 2 — wire into the recommendation. In agent/review.py, a critical match forces the action away from any write verdict (never merge; downgrade to request-changes/comment) and surfaces the matched domain. vanguarstew_runtime records the flag as a hard property on the persisted result. This is the absolute veto the mergeability-verdict sibling consumes.
  • Stage 3 — bind to the never-auto-execute contract. A critical match maps onto the factory's owner-action path so it resolves to an ActionIntent whose can_auto_execute is False permanently. Publish the forever-gated domain set through public_contract() (factory.py:373-393) as a versioned commitment, extending owner_actions_require_external_approval (line 390) with a domain_critical_paths entry, so any operator or external repo owner can audit exactly which classes are permanently human-only.
  • Stage 4 — fail-closed precondition on any executor. When the owner-action execution gateway (ROADMAP §4) is built, the classifier is a mandatory precondition it calls before any write; a critical match is a hard stop no score, band, judge verdict, or approval can override. On this repo, add a CI check mirroring benchmark-change-policy / pr-target-check.yml.

Acceptance criteria

  • Pure, dependency-free classifier module with unit tests; returns critical/matched/reason deterministically.
  • Fail-closed proven: malformed paths, empty/None input, and paths that do not fully classify all yield critical=True.
  • Pattern set covers ≥ weight/emission setting, validator consensus, scoring, payment, wallet/keys (incl. hotkey/coldkey), incentive mechanism, on-chain/governance — each with a regression test.
  • Per-repo config can add but a test proves it cannot remove or weaken any built-in critical class.
  • agent/review.py never returns action: "merge" for a domain-critical PR; matched domain appears in the JSON (test).
  • vanguarstew_runtime records the domain-critical flag on the persisted result.
  • A domain-critical change maps to a factory ActionIntent with can_auto_execute False; a test asserts no configuration flips this.
  • public_contract() publishes the versioned domain-critical path set.
  • A CI workflow enforces the classifier on this repo, failing closed.
  • AGENTS.md / docs state the domain set is PERMANENTLY human-gated (a forever set, not a temporary hold).

Dependencies

Out of scope / stays human-gated

  • Setting or changing emissions/weights, validator consensus and scoring parameters, payment flows, wallet/key material, and the incentive mechanism itself remain PERMANENTLY human-gated. Autonomy is earned only on the low-risk surface; it never extends to these paths, by design, regardless of benchmark score, judge verdict, or operator convenience.
  • This issue does not build the write/execution gateway, define per-repo autonomy levels, or grant any automatic GitHub write — can_auto_execute stays False and the runtime stays advisory-only.
  • No change to how value is measured on non-critical PRs; that is the mergeability-verdict sibling.

Metadata

Metadata

Assignees

No one assigned

    Labels

    autonomy-roadmapRoadmap toward autonomous external-repo maintenance

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions