You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Two guardrail mechanisms exist today; neither classifies an arbitrary external PR by the domain of code it changes.
scripts/benchmark_pr_policy.py is path-based but scoped to this repo's own agent surface.touches_guardrail(paths) is literally not is_agent_submission(paths) (lines 63-65); is_agent_submission (lines 50-60) whitelists only agent.py / agent/** (+ companion tests/**). It protects the agent/benchmark code from contributor edits — a whitelist of our own files. It says nothing about which subnet domain (emission/weights, validator-consensus, scoring, payment, wallet/keys) a PR touches, and does not transfer to an external product repo.
openvang/factory.py gates by action TYPE, not by the domain of a merged diff._OWNER_ACTIONS (lines 90-99) names GITHUB_WRITE, ONCHAIN_TRANSACTION, WALLET_ACCESS, EMISSION_CHANGE, GOVERNANCE_VOTE, PUBLICATION; RoleContract.__post_init__ rejects any role granted an owner action (lines 127-128); can_auto_execute() returns False "ever" (lines 434-438). But action-type gating cannot classify a merge by the domain of the diff it merges. A merge is one GITHUB_WRITE; nothing marks "this particular PR edits validator-consensus code, so it is forever-human."
The live recommendation has zero path awareness.agent/review.py:37 — ACTIONS = ["merge", "request-changes", "reject", "comment"]. review_pr recommends regardless of which files a PR touches.
No module anywhere classifies subnet-critical code paths: grepping weights|emission|validator|consensus|wallet|payment|incentive|hotkey|coldkey across agent/, scripts/, openvang/, vanguarstew_runtime/ returns only the benchmark's blend weights and the factory's action-type enums — never a path classifier.
Why it blocks autonomous external maintenance
On a live incentive-subnet product repo the highest-consequence changes — emission/weight-setting, validator consensus and scoring, payment, wallet/key handling, the incentive mechanism itself — are exactly the ones with no safe automatic verdict. As the sibling #2385 establishes, an external PR produces no benchmark-delta artifact, so the decision falls back on the judge — which per #2379 saturates and cannot discriminate (the real fix, challenger-vs-king #2382, is unbuilt; documented as M2+ scope in benchmark/runner.py:7). A merge there is irreversible and can move real emissions or funds. Autonomy must never reach these paths — and "never" has to be a structural, fail-closed classifier, not a documented intention nor a property a future well-behaved judge is trusted to preserve.
Proposed direction (concrete, staged)
Add a permanent, path-pattern, fail-closed domain-guardrail classifier that every decision and (future) execution surface must consult.
Stage 1 — pure classifier. New module (e.g. openvang/domain_guardrail.py) mirroring benchmark_pr_policy.evaluate_policy / _normalized_paths (reuse the path hardening at lines 26-47). Input = repo-relative changed paths; output = {critical: bool, matched: [...], reason}. Match a versioned pattern set for: emission/weight-setting, validator consensus, scoring, payment, wallet/keys (incl. hotkey/coldkey), incentive mechanism, on-chain/governance. Fail closed: malformed/unparseable paths, empty input, or any PR that cannot be fully classified resolve to critical=True. A per-repo override may only add patterns, never remove a built-in critical class.
Stage 2 — wire into the recommendation. In agent/review.py, a critical match forces the action away from any write verdict (never merge; downgrade to request-changes/comment) and surfaces the matched domain. vanguarstew_runtime records the flag as a hard property on the persisted result. This is the absolute veto the mergeability-verdict sibling consumes.
Stage 3 — bind to the never-auto-execute contract. A critical match maps onto the factory's owner-action path so it resolves to an ActionIntent whose can_auto_execute is False permanently. Publish the forever-gated domain set through public_contract() (factory.py:373-393) as a versioned commitment, extending owner_actions_require_external_approval (line 390) with a domain_critical_paths entry, so any operator or external repo owner can audit exactly which classes are permanently human-only.
Stage 4 — fail-closed precondition on any executor. When the owner-action execution gateway (ROADMAP §4) is built, the classifier is a mandatory precondition it calls before any write; a critical match is a hard stop no score, band, judge verdict, or approval can override. On this repo, add a CI check mirroring benchmark-change-policy / pr-target-check.yml.
Acceptance criteria
Pure, dependency-free classifier module with unit tests; returns critical/matched/reason deterministically.
Fail-closed proven: malformed paths, empty/None input, and paths that do not fully classify all yield critical=True.
Pattern set covers ≥ weight/emission setting, validator consensus, scoring, payment, wallet/keys (incl. hotkey/coldkey), incentive mechanism, on-chain/governance — each with a regression test.
Per-repo config can add but a test proves it cannot remove or weaken any built-in critical class.
agent/review.py never returns action: "merge" for a domain-critical PR; matched domain appears in the JSON (test).
vanguarstew_runtime records the domain-critical flag on the persisted result.
A domain-critical change maps to a factory ActionIntent with can_auto_execute False; a test asserts no configuration flips this.
public_contract() publishes the versioned domain-critical path set.
A CI workflow enforces the classifier on this repo, failing closed.
AGENTS.md / docs state the domain set is PERMANENTLY human-gated (a forever set, not a temporary hold).
Setting or changing emissions/weights, validator consensus and scoring parameters, payment flows, wallet/key material, and the incentive mechanism itself remain PERMANENTLY human-gated. Autonomy is earned only on the low-risk surface; it never extends to these paths, by design, regardless of benchmark score, judge verdict, or operator convenience.
This issue does not build the write/execution gateway, define per-repo autonomy levels, or grant any automatic GitHub write — can_auto_execute stays False and the runtime stays advisory-only.
No change to how value is measured on non-critical PRs; that is the mergeability-verdict sibling.
Problem (current state, with file refs)
Two guardrail mechanisms exist today; neither classifies an arbitrary external PR by the domain of code it changes.
scripts/benchmark_pr_policy.pyis path-based but scoped to this repo's own agent surface.touches_guardrail(paths)is literallynot is_agent_submission(paths)(lines 63-65);is_agent_submission(lines 50-60) whitelists onlyagent.py/agent/**(+ companiontests/**). It protects the agent/benchmark code from contributor edits — a whitelist of our own files. It says nothing about which subnet domain (emission/weights, validator-consensus, scoring, payment, wallet/keys) a PR touches, and does not transfer to an external product repo.openvang/factory.pygates by action TYPE, not by the domain of a merged diff._OWNER_ACTIONS(lines 90-99) namesGITHUB_WRITE,ONCHAIN_TRANSACTION,WALLET_ACCESS,EMISSION_CHANGE,GOVERNANCE_VOTE,PUBLICATION;RoleContract.__post_init__rejects any role granted an owner action (lines 127-128);can_auto_execute()returnsFalse"ever" (lines 434-438). But action-type gating cannot classify a merge by the domain of the diff it merges. A merge is oneGITHUB_WRITE; nothing marks "this particular PR edits validator-consensus code, so it is forever-human."The live recommendation has zero path awareness.
agent/review.py:37—ACTIONS = ["merge", "request-changes", "reject", "comment"].review_prrecommends regardless of which files a PR touches.No module anywhere classifies subnet-critical code paths: grepping
weights|emission|validator|consensus|wallet|payment|incentive|hotkey|coldkeyacrossagent/,scripts/,openvang/,vanguarstew_runtime/returns only the benchmark's blend weights and the factory's action-type enums — never a path classifier.Why it blocks autonomous external maintenance
On a live incentive-subnet product repo the highest-consequence changes — emission/weight-setting, validator consensus and scoring, payment, wallet/key handling, the incentive mechanism itself — are exactly the ones with no safe automatic verdict. As the sibling #2385 establishes, an external PR produces no benchmark-delta artifact, so the decision falls back on the judge — which per #2379 saturates and cannot discriminate (the real fix, challenger-vs-king #2382, is unbuilt; documented as M2+ scope in
benchmark/runner.py:7). A merge there is irreversible and can move real emissions or funds. Autonomy must never reach these paths — and "never" has to be a structural, fail-closed classifier, not a documented intention nor a property a future well-behaved judge is trusted to preserve.Proposed direction (concrete, staged)
Add a permanent, path-pattern, fail-closed domain-guardrail classifier that every decision and (future) execution surface must consult.
openvang/domain_guardrail.py) mirroringbenchmark_pr_policy.evaluate_policy/_normalized_paths(reuse the path hardening at lines 26-47). Input = repo-relative changed paths; output ={critical: bool, matched: [...], reason}. Match a versioned pattern set for: emission/weight-setting, validator consensus, scoring, payment, wallet/keys (incl. hotkey/coldkey), incentive mechanism, on-chain/governance. Fail closed: malformed/unparseable paths, empty input, or any PR that cannot be fully classified resolve tocritical=True. A per-repo override may only add patterns, never remove a built-in critical class.agent/review.py, a critical match forces the action away from any write verdict (nevermerge; downgrade torequest-changes/comment) and surfaces the matched domain.vanguarstew_runtimerecords the flag as a hard property on the persisted result. This is the absolute veto the mergeability-verdict sibling consumes.ActionIntentwhosecan_auto_executeisFalsepermanently. Publish the forever-gated domain set throughpublic_contract()(factory.py:373-393) as a versioned commitment, extendingowner_actions_require_external_approval(line 390) with adomain_critical_pathsentry, so any operator or external repo owner can audit exactly which classes are permanently human-only.benchmark-change-policy/pr-target-check.yml.Acceptance criteria
critical/matched/reasondeterministically.Noneinput, and paths that do not fully classify all yieldcritical=True.agent/review.pynever returnsaction: "merge"for a domain-critical PR; matched domain appears in the JSON (test).vanguarstew_runtimerecords the domain-critical flag on the persisted result.ActionIntentwithcan_auto_executeFalse; a test asserts no configuration flips this.public_contract()publishes the versioned domain-critical path set.AGENTS.md/ docs state the domain set is PERMANENTLY human-gated (a forever set, not a temporary hold).Dependencies
Out of scope / stays human-gated
can_auto_executestays False and the runtime stays advisory-only.