-
Notifications
You must be signed in to change notification settings - Fork 1
fix: create configs/safety.yaml with sensible defaults #100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| # EVOSEAL Safety Configuration (Immutable) | ||
| # | ||
| # This file defines safety thresholds and constraints for the evolution pipeline. | ||
| # It is enforced as IMMUTABLE by the edit-scope allowlist — the evolution loop | ||
| # must never modify this file. See threat model §3 and ADR 0001. | ||
| # | ||
| # NOTE: This directory is `configs/` (plural), NOT `config/` (singular). | ||
| # `config/` is a Python package containing mutable settings (budget.yaml, | ||
| # logging.yaml, settings.py). `configs/` holds standalone immutable YAML | ||
| # files referenced by path. Both are intentional — do not merge them. | ||
|
|
||
| # Edit-scope enforcement | ||
| edit_scope: | ||
| enabled: true | ||
| # Paths that the evolution loop is forbidden from modifying | ||
| forbidden_paths: | ||
| - .env | ||
| - Makefile | ||
| # Directories that are entirely off-limits | ||
| forbidden_dirs: | ||
| - .git | ||
| - .github/workflows | ||
| - .evoseal | ||
| - .pytest_cache | ||
| - __pycache__ | ||
| - .venv | ||
| - venv | ||
| - configs | ||
| # Directories that ARE mutable by the evolution loop | ||
| allowed_dirs: | ||
| - evoseal/ | ||
| - tests/ | ||
| - examples/ | ||
| - docs/ | ||
| - scripts/ | ||
| - benchmarks/ | ||
| - config/ | ||
|
|
||
| # Regression detection thresholds | ||
| regression: | ||
| # Global default regression threshold (5%) | ||
| regression_threshold: 0.05 | ||
| # Per-metric thresholds (positive = increase is regression, negative = decrease is regression) | ||
| metric_thresholds: | ||
| # Performance metrics (lower is better) | ||
| duration_sec: | ||
| regression: 0.10 | ||
| critical: 0.25 | ||
| memory_mb: | ||
| regression: 0.10 | ||
| critical: 0.30 | ||
| execution_time: | ||
| regression: 0.10 | ||
| critical: 0.25 | ||
| # Quality metrics (higher is better — negative thresholds) | ||
| success_rate: | ||
| regression: -0.05 | ||
| critical: -0.10 | ||
| pass_rate: | ||
| regression: -0.05 | ||
| critical: -0.10 | ||
| correctness: | ||
| regression: -0.01 | ||
| critical: -0.05 | ||
|
Comment on lines
+44
to
+64
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win Preserve thresholds for all monitored metrics.
🤖 Prompt for AI Agents |
||
|
|
||
| # Checkpoint and rollback policy | ||
| checkpoint: | ||
| # Maximum number of checkpoints to retain (oldest pruned first) | ||
| max_checkpoints: 10 | ||
| # Automatically rollback on critical regression | ||
| auto_rollback_on_critical: true | ||
|
|
||
| # Sandboxed test execution | ||
| sandbox: | ||
| enabled: true | ||
| # Files mounted read-only during test execution | ||
| readonly_files: | ||
| - configs/safety.yaml | ||
| - .env | ||
| # Resource limits for test subprocesses | ||
| cpu_limit_secs: 120 | ||
| memory_limit_mb: 512 | ||
| fd_limit: 256 | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Align the edit-scope schema with its consumer.
SafetyIntegrationreadsedit_scope.allowed_patternsas a mapping, while this file definesallowed_dirsas a list;enabledis also not connected to the top-level enforcement flags used bySafetyIntegration. As written, these settings are inert and the validator falls back to hard-coded defaults instead of using this immutable configuration.🤖 Prompt for AI Agents