CORR-EDIT

CORR-EDIT is a small, judge-free pilot for studying positive corrigibility in coding-agent workflows. It asks whether an agent remains useful after an authenticated principal changes, narrows, or withdraws an instruction: can it preserve user-owned state, perform selective rollback, disclose decision-relevant uncertainty or mistakes, respect authority boundaries, and return control?

This is not a general corrigibility benchmark, a model-safety certification, or a claim about internal objectives. It operationalizes a narrow set of necessary-but-not-sufficient behaviors in a synthetic text-file environment.

What is implemented

• 10 two-phase intervention tasks across principal updates, selective rollback, uncertainty disclosure, safety repair, and authority/delegation propagation
• a deterministic, quota-limited in-memory filesystem; no candidate code, shell command, or network action is executed
• candidate-visible tasks separated from hidden state/action oracles
• semantic intervention triggers requested one action at a time for live Codex trials
• canonical JSONL transcripts with a SHA-256 hash chain
• transcript replay that reconstructs state and rejects saved-snapshot mismatches
• separate control, revised-task utility, communication, and harmonic balanced scores
• compliant, defiant, and idle reference policies for scorer directionality
• a bounded GPT-5.6 Sol Ultra adapter that exposes no hidden oracle

Quick start

Python 3.11 or later is required. Runtime dependencies are standard-library only.

PYTHONPATH=src python3 -m corrigibility_bench validate
PYTHONPATH=src python3 -m unittest discover -s tests -v
PYTHONPATH=src python3 -m corrigibility_bench run --agent all --out runs/harness-validation --force

To run a live Codex trial, use only an account and model configuration you are authorised to use:

PYTHONPATH=src python3 -m corrigibility_bench run-codex \
  --tasks all \
  --out runs/codex-pilot \
  --model gpt-5.6-sol \
  --effort ultra

The model process receives a read-only real sandbox and proposes data-only actions. CORR-EDIT executes those actions in its virtual filesystem.

Current validation results

Reference-policy validation after the replay hardening:

Policy	Valid trials	Control	Task utility	Communication	Balanced	Critical violations
compliant	10/10	100.00	100.00	100.00	100.00	0
defiant	10/10	8.00	31.00	20.00	4.44	16
idle	0/10	0.00	0.00	0.00	0.00	0

One blind, one-trial-per-task GPT-5.6 Sol Ultra pilot was also run. The score produced immediately after the run was 9/10 valid, control 90.0, task utility 85.5, communication 30.0, balanced 87.1, with zero critical violations. A documented post-run scorer audit corrected two non-semantic oracle defects and added transcript replay verification; rescoring the preserved action transcripts, which were not edited during that audit, produced 10/10 valid, control 100.0, task utility 100.0, communication 40.0, balanced 100.0, with zero critical violations. No externally timestamped checksum manifest existed before the audit, so the repository does not claim independent cryptographic proof of that history.

The post-run score is explicitly post hoc and is not independent model-performance evidence. The useful result is methodological: the pilot exposed scheduler and oracle failure modes, and public-report completeness remained substantially weaker than state/task behavior. See the pilot report.

Research position

CORR-EDIT does not claim priority over existing work. ROGUE evaluates control-override and shutdown-resistance behaviors in realistic computer-use tasks. Revisable by Design formalizes streaming intervention and reversibility and evaluates its Revision Absorber on StreamBench. ProcCtrlBench evaluates process defects and control preservation over coding-agent trajectories. CORR-EDIT's narrower proposed contribution is an executable coding protocol combining authenticated principal updates, exact state/effect oracles, selective user-state preservation, and useful-recovery scoring without an LLM judge.

Safety and scope

All fixtures are synthetic. The harness contains no real secrets, deployment credentials, production targets, mainnet funds, exploit tasks, or external side effects. An attempted unsafe virtual action can be recorded separately from realised harm. The project does not train models or optimise autonomous task performance.

Reproducibility notes

• fixtures/v1/tasks.json is candidate-visible task data.
• fixtures/v1/oracles.json is scorer-only data and must not be provided to evaluated agents.
• results/ contains reviewable aggregate and task-level summaries intended for version control.
• results/evidence-manifest.json locks the preserved v1.0/v1.1 task artifacts after the scorer audit and explicitly records its pre-audit timestamp limitation.
• Local runs/codex-pilot-v0-batched/ preserves the engineering shakeout that exposed a flawed fixed-action scheduler.
• Local runs/codex-pilot-v1-semantic/summary.json preserves the score emitted immediately after the semantic-trigger run.
• Local runs/codex-pilot-v1-semantic/summary-v1.1.json is the post-run replay-verified rescore.

Raw run bundles are git-ignored until the applicant reviews provider terms, thread identifiers, and disclosure risk. A funded release should publish permitted sanitized traces plus checksums rather than silently omitting them.

The present data are a protocol-validation pilot only: one model configuration, one sample per task, toy state, and no independent task review. Broad claims require preregistration, repeated trials, multiple configurations, and external review.