From 0d337dc5a6c36c480c3751864f1501490ae7f6d5 Mon Sep 17 00:00:00 2001 From: Kresna Sucandra Date: Fri, 31 Jul 2026 22:16:51 +0000 Subject: [PATCH] docs: improve API reference to match actual codebase Rewrite docs/api/index.md to document the real public API surface: - CLI commands (init, config, pipeline, start, doctor, estimate-cost, etc.) - Core Python API (EvolutionPipeline, EvolutionConfig) - Safety layer (SafetyIntegration, CheckpointManager, EditScopeValidator, etc.) - Phase 3 services (ContinuousEvolutionService, TrainingManager) - Monitoring dashboard quick reference with auth/CORS info The previous version documented fabricated classes (EVOSEAL.evolve(), OpenAIModel, AnthropicModel) that don't exist in the codebase. Also fix stale CORS and auth info in docs/API_REFERENCE.md: - Auth: document optional bearer token (added in PR #109) - CORS: origins now default to host:port, not wildcard * Addresses TODO.md item: Improve API reference --- TODO.md | 2 +- docs/API_REFERENCE.md | 28 ++-- docs/api/index.md | 322 ++++++++++++++++++++++++------------------ 3 files changed, 204 insertions(+), 148 deletions(-) diff --git a/TODO.md b/TODO.md index 33d4a3ad..e7dcced8 100644 --- a/TODO.md +++ b/TODO.md @@ -282,7 +282,7 @@ - [ ] **Add a "How It Actually Works" tutorial** - Walk through a single evolution cycle step by step with real logs - Lower the barrier for new contributors -- [ ] **Improve API reference** +- [x] **Improve API reference** _(done 2026-07-31)_ - Ensure all public classes/functions have docstrings - Auto-generate API docs (MkDocs + mkdocstrings) diff --git a/docs/API_REFERENCE.md b/docs/API_REFERENCE.md index b6e823ca..458d03eb 100644 --- a/docs/API_REFERENCE.md +++ b/docs/API_REFERENCE.md @@ -307,22 +307,30 @@ WebSocket connections handle errors gracefully: ## Authentication -### Current Implementation -- No authentication required (local access only) -- Dashboard only accessible on localhost interface +### Bearer Token (Optional) + +When the dashboard is constructed with `auth_token`, all `/api/*` and `/ws` requests must +present a valid token: + +- **HTTP**: `Authorization: Bearer ` header +- **WebSocket**: `?token=` query parameter +- The dashboard HTML page (`/`) is not gated by auth. + +When `auth_token` is `None` (the default), no authentication is enforced. ### Security Considerations -- Dashboard binds only to localhost (127.0.0.1) -- No external network access +- Dashboard defaults to localhost (127.0.0.1) binding +- Binding to `0.0.0.0` logs a security warning — ensure `auth_token` is set or restrict + access via firewall when exposing externally - Runs as user service (no root privileges) ## CORS Configuration -Cross-Origin Resource Sharing (CORS) is configured to allow: -- **Origins**: All origins (`*`) -- **Methods**: All methods -- **Headers**: All headers -- **Credentials**: Allowed +Cross-Origin Resource Sharing (CORS) is configured as follows: + +- **Origins**: Defaults to the dashboard's own `host:port` (not `*`) +- Wildcard `*` origins explicitly disable `allow_credentials` per the CORS specification +- Methods and headers are unrestricted for allowed origins ## Usage Examples diff --git a/docs/api/index.md b/docs/api/index.md index 15bd0552..77d98aef 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -1,180 +1,228 @@ # API Reference -This document provides detailed information about the EVOSEAL API. +This document provides detailed information about the EVOSEAL public API — both the CLI +and the Python internals. For the Phase 3 monitoring dashboard REST/WebSocket API, see +[API_REFERENCE.md](../API_REFERENCE.md). -## Core Classes +--- -### EVOSEAL +## CLI -The main class for interacting with the EVOSEAL system. +EVOSEAL exposes a Typer-based CLI (`evoseal`) defined at `evoseal.cli.main:app`. -```python -class EVOSEAL: - def __init__( - self, - model: Optional[BaseModel] = None, - fitness_function: Optional[Callable] = None, - config: Optional[Dict] = None, - ): - """ - Initialize the EVOSEAL system. - - Args: - model: The language model to use (default: OpenAI's GPT-4) - fitness_function: Custom fitness function for solution evaluation - config: Configuration dictionary - """ - - def evolve( - self, task: str, max_iterations: int = 100, population_size: int = 20, **kwargs - ) -> EvolutionResult: - """ - Run the evolutionary algorithm. - - Args: - task: Description of the task to solve - max_iterations: Maximum number of iterations - population_size: Number of solutions in each generation - **kwargs: Additional parameters for the evolution process - - Returns: - EvolutionResult containing the best solution and metrics - """ - - def save_checkpoint(self, filepath: str) -> None: - """Save the current state to a checkpoint file.""" - - @classmethod - def load_checkpoint(cls, filepath: str) -> "EVOSEAL": - """Load a previously saved checkpoint.""" -``` +### Top-level -## Models +| Command | Description | +|---|---| +| `evoseal --version` | Print the installed version | +| `evoseal init project` | Scaffold a new EVOSEAL project | +| `evoseal doctor` | Validate environment (API keys, configs, dependencies, git state) | +| `evoseal estimate-cost` | Estimate token/cost for a given number of iterations | -### BaseModel +### Config (`evoseal config`) -Abstract base class for all language models. +| Subcommand | Description | +|---|---| +| `evoseal config show` | Display current configuration | +| `evoseal config set ` | Set a config value | +| `evoseal config unset ` | Remove a config value | -```python -class BaseModel(ABC): - @abstractmethod - def generate(self, prompt: str, **kwargs) -> str: - """Generate text from a prompt.""" - - @abstractmethod - def get_embeddings(self, text: str) -> List[float]: - """Get embeddings for the given text.""" -``` +### Pipeline (`evoseal pipeline`) -### OpenAIModel +| Subcommand | Description | +|---|---| +| `evoseal pipeline init` | Initialize pipeline state | +| `evoseal pipeline start` | Start the evolution pipeline | +| `evoseal pipeline pause` / `resume` | Pause or resume a running pipeline | +| `evoseal pipeline stop` | Stop the pipeline | +| `evoseal pipeline status` | Show current pipeline status | +| `evoseal pipeline config` | Show/modify pipeline configuration | +| `evoseal pipeline logs` | Display pipeline logs | -Wrapper for OpenAI models. +### Continuous Evolution (`evoseal start`) -```python -class OpenAIModel(BaseModel): - def __init__(self, model: str = "gpt-4", **kwargs): - """ - Initialize with a specific OpenAI model. - - Args: - model: Model name (e.g., 'gpt-4', 'gpt-3.5-turbo') - **kwargs: Additional parameters for the OpenAI API - """ -``` +| Subcommand | Description | +|---|---| +| `evoseal start evolution` | Start the `ContinuousEvolutionService` daemon (Phase 3 loop) | -### AnthropicModel +### Subsystem Commands -Wrapper for Anthropic models. +| Command | Description | +|---|---| +| `evoseal dgm improve/evaluate/compare` | Interact with the DGM subsystem | +| `evoseal openevolve run/resume/analyze` | Interact with OpenEvolve | +| `evoseal export results/variant/all` | Export evolution results | -```python -class AnthropicModel(BaseModel): - def __init__(self, model: str = "claude-3-opus", **kwargs): - """ - Initialize with a specific Anthropic model. - - Args: - model: Model name (e.g., 'claude-3-opus', 'claude-3-sonnet') - **kwargs: Additional parameters for the Anthropic API - """ -``` +--- + +## Core Python API -## Data Types +### `EvolutionPipeline` -### EvolutionResult +The central orchestrator. Integrates DGM, OpenEvolve, and SEAL to run the +code evolution workflow. + +**Module:** `evoseal.core.evolution_pipeline` ```python -@dataclass -class EvolutionResult: - best_solution: str - fitness: float - iterations: int - history: List[Dict[str, Any]] - metadata: Dict[str, Any] +from evoseal.core.evolution_pipeline import EvolutionPipeline, EvolutionConfig + +# Initialize with defaults +pipeline = EvolutionPipeline() + +# Or with explicit config +pipeline = EvolutionPipeline( + EvolutionConfig( + dgm_config={...}, + openevolve_config={...}, + seal_config={...}, + test_config={...}, + max_iterations=1000, + ) +) + +# Run one or more evolution iterations +results = await pipeline.run_evolution_cycle(iterations=1) + +# Run with safety gates (checkpoint + rollback on regression) +results = await pipeline.run_evolution_cycle_with_safety(iterations=1) ``` -## Utilities +#### `EvolutionConfig` -### Fitness Functions +Dataclass holding per-subsystem configuration and runaway-control knobs: -```python -def default_fitness(solution: str, **kwargs) -> float: - """ - Default fitness function that evaluates solution quality. +| Field | Type | Default | Description | +|---|---|---|---| +| `dgm_config` | `dict` | `{}` | DGM-specific settings | +| `openevolve_config` | `dict` | `{}` | OpenEvolve-specific settings | +| `seal_config` | `dict` | `{}` | SEAL-specific settings | +| `test_config` | `dict` | `{}` | Test runner settings | +| `metrics_config` | `dict` | `{}` | Metrics tracker settings | +| `validation_config` | `dict` | `{}` | Improvement validation settings | +| `version_control_config` | `dict` | `{}` | Version control settings | +| `max_iterations` | `int` | `1000` | Hard cap on evolution iterations | +| `max_consecutive_rejections` | `int` | `5` | Stuck-generator circuit threshold | - Args: - solution: The solution to evaluate - **kwargs: Additional parameters +#### Key Methods - Returns: - Fitness score (higher is better) - """ -``` +| Method | Signature | Description | +|---|---|---| +| `run_evolution_cycle` | `async (iterations=1) -> list[dict]` | Run N evolution iterations | +| `run_evolution_cycle_with_safety` | `async (iterations=1) -> list[dict]` | Same, with checkpoint/rollback safety | +| `pause` | `() -> bool` | Pause the pipeline | +| `resume` | `() -> bool` | Resume a paused pipeline | -### Checkpointing +--- -```python -def save_checkpoint(evoseal: EVOSEAL, filepath: str) -> None: - """Save EVOSEAL instance to a file.""" +## Safety Layer +### `SafetyIntegration` -def load_checkpoint(filepath: str) -> EVOSEAL: - """Load EVOSEAL instance from a file.""" -``` +Coordinates safety checkpoints, edit-scope validation, and regression detection. -## Examples +**Module:** `evoseal.core.safety_integration` -### Basic Usage +| Method | Description | +|---|---| +| `create_safety_checkpoint(description)` | Create a git-backed safety checkpoint | +| `validate_version_safety(version_id, ...)` | Validate a proposed version against safety rules | +| `execute_safe_evolution_step(...)` | Run one evolution step inside the safety envelope | +| `validate_edit_path(file_path)` | Check a file path against the edit-scope allowlist | +| `validate_edits_before_apply(edited_files)` | Batch pre-apply edit validation | +| `get_safety_status()` | Return current safety status dict | -```python -from evoseal import EVOSEAL +### `CheckpointManager` -# Initialize with default settings -evoseal = EVOSEAL() +Git-backed checkpointing and rollback. -# Run evolution -result = evoseal.evolve( - task="Create a Python function that implements binary search", - max_iterations=30, - population_size=15, -) +**Module:** `evoseal.core.checkpoint_manager` -# Access results -print(f"Best solution: {result.best_solution}") -print(f"Fitness: {result.fitness}") -``` +| Method | Description | +|---|---| +| `create_checkpoint(version_id, changes, ...)` | Create a named checkpoint | +| `restore_checkpoint(version_id)` | Restore to a previous checkpoint | +| `get_checkpoint_path(version_id)` | Get the filesystem path for a checkpoint | -### Custom Model and Fitness +### `EditScopeValidator` -```python -from evoseal import EVOSEAL, OpenAIModel +Validates that self-modifications stay within the allowed scope. + +**Module:** `evoseal.core.edit_scope_validator` + +### `RegressionDetector` +Detects metric regressions between versions. -def custom_fitness(solution, **kwargs): - # Your custom fitness logic here - return score +**Module:** `evoseal.core.regression_detector` +### `TestRunner` / `SandboxedTestRunner` -# Initialize with custom model and fitness -model = OpenAIModel(model="gpt-4") -evoseal = EVOSEAL(model=model, fitness_function=custom_fitness) +Executes test suites, optionally in a sandboxed environment (Tier 1 isolation). + +**Module:** `evoseal.core.testrunner` + +--- + +## Phase 3 — Continuous Evolution + +### `ContinuousEvolutionService` + +Long-running daemon that drives the autonomous evolution → training → deployment loop. + +**Module:** `evoseal.services.continuous_evolution_service` + +```python +from evoseal.services.continuous_evolution_service import ContinuousEvolutionService + +service = ContinuousEvolutionService(config={...}) +await service.start() # enters the service loop +await service.shutdown() # graceful stop ``` + +| Method | Description | +|---|---| +| `start()` | Start the daemon (signal handlers, service loop) | +| `shutdown()` | Graceful shutdown | +| `generate_service_report()` | Return a status/evolution/training report dict | + +### `TrainingManager` + +Manages fine-tuning readiness checks, data preparation, and training cycles. + +**Module:** `evoseal.fine_tuning.training_manager` + +| Method | Description | +|---|---| +| `check_training_readiness()` | Check if enough data exists and no training is in progress | +| `prepare_training_data()` | Build training dataset from evolution results | +| `run_training_cycle()` | Execute one full training cycle | +| `get_training_status()` | Return current training status | + +--- + +## Monitoring Dashboard + +The Phase 3 monitoring dashboard serves a REST API and WebSocket interface on port 9613 +(default). Full endpoint documentation is in [API_REFERENCE.md](../API_REFERENCE.md). + +### Quick Reference + +| Endpoint | Method | Description | +|---|---|---| +| `/` | GET | Dashboard HTML page | +| `/api/status` | GET | Service status + basic metrics | +| `/api/metrics` | GET | Comprehensive system metrics | +| `/api/report` | GET | Detailed evolution report | +| `/ws` | WS | Real-time metrics updates (every 30s) | + +### Authentication + +When the dashboard is constructed with `auth_token`, all `/api/*` and `/ws` requests must +include `Authorization: Bearer `. WebSocket clients may pass `?token=` as a +query parameter instead. + +### CORS + +Allowed origins default to the dashboard's own `host:port`. Binding to `0.0.0.0` logs a +security warning. Wildcard `*` origins disable `allow_credentials` per the CORS spec.