Security hardening: sandbox mode, shell injection fixes, config centralization by pookNast · Pull Request #15 · hexo-ai/sia

pookNast · 2026-05-30T13:09:17Z

Summary

Production security hardening audit with 25 atomic commits across 5 phases:

Phase 1 (Config + Errors): Centralized all hardcoded values into sia/config.py with SIA_* env var overrides. Narrowed 11 broad except Exception handlers to specific types.
Phase 2 (Sandbox): Eliminated all shell=True subprocess calls in core framework. Added optional Docker sandbox mode (--sandbox docker) with read-only dataset mount, no network, and resource limits.
Phase 3 (Decompose): Refactored the 630-line main() into 10 well-named functions with TaskFiles and RunSetup dataclasses.
Phase 4 (Size Limits): Added file size guards on all JSON loading to prevent unbounded memory usage. Centralized truncation limits in config.
Phase 5 (Tests): Added 27 new tests covering config, evaluation, generation loop, Docker sandbox, and size limits.

Key changes

Area	Before	After
Config	8 hardcoded values across 3 files	Single `Config` dataclass with env var overrides
Shell safety	3 `subprocess.run(shell=True)` calls	0 in core code; Docker sandbox option
Error handling	11 `except Exception`	Specific `OSError`, `JSONDecodeError`, `SubprocessError`
`main()` size	630 lines	~150 lines (thin orchestrator)
Test count	34 (4 files)	61 (10 files)
File safety	No size limits on JSON/file reads	Configurable size guards with graceful degradation

Files changed

sia/config.py (new) — centralized configuration
sia/__main__.py (new) — python -m sia support
SECURITY.md (new) — threat model documentation
sia/orchestrator.py — decomposed, shell=True removed, sandbox added
sia/context_manager.py — safe file reads, logging, config wiring
5 new test files in tests/

Test plan

All 61 tests pass (python -m pytest tests/ -v)
Ruff lint clean (ruff check sia/ tests/)
CLI interface preserved (sia --help exits 0)
No secrets or sensitive data in changes
Manual: run 1 generation with --task gpqa --max_gen 1
Manual: test Docker sandbox with --sandbox docker

🤖 Generated with Claude Code

Extract hardcoded model names, timeouts, limits, and defaults into a single config module with environment variable overrides. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace 8 inline hardcoded defaults with Config.* references. No behavioral change. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace hardcoded truncation limits and default models with config imports. No behavioral change. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Environment variables (SIA_META_MODEL, SIA_TASK_MODEL, etc.) now override defaults with lower priority than explicit CLI flags. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace broad except Exception with specific exception types (JSONDecodeError, OSError, SubprocessError) for better error diagnosis. Keep a safety-net handler at the generation loop level. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace broad except Exception with specific exception types (OSError, JSONDecodeError, RuntimeError) for better error handling. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace shell=True piped tee with direct subprocess.run(arg_list) and file write. Adds configurable timeout via Config.EVAL_TIMEOUT. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace shell=True piped tee with Popen streaming stdout to both console and log file simultaneously. Removes bash dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

When --sandbox docker is specified, target agents run inside a Docker container with read-only dataset mount, read-write working directory, no network access, and resource limits. Default is sandbox=none (current behavior unchanged). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Document the security model, execution modes, bypassPermissions rationale, and Docker sandbox usage. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Section 1 of main() extracted into a load_task_files() function returning a TaskFiles dataclass. Improves readability and testability. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Section 2 of main() extracted into setup_run_directory() and _create_venv() helper functions. Returns RunSetup dataclass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Meta-agent and feedback-agent prompt templates extracted into dedicated builder functions for maintainability. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The 400+ line generation loop extracted into run_generation() and helper functions. main() is now a thin orchestrator. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Smoke tests for --help, missing args, and invalid task name. Also adds __main__.py to support `python -m sia` invocation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add _safe_read_file() and _safe_load_json() helpers that enforce Config.MAX_CONTEXT_FILE_SIZE limits. Prevents unbounded memory usage from oversized execution logs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Enforce Config.MAX_EXECUTION_LOG_SIZE on trajectory files and results.json. Oversized files are skipped with a warning instead of loading into memory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace remaining magic number truncation limits with Config constants for discoverability and tuning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Test default values, env var overrides, and invalid value fallback. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Mock subprocess tests for skipped, success, failure, and timeout scenarios. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Test target agent execution and context tracking with mocked subprocess. Verifies directory structure and context.md creation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Verify feedback agent invocation, directory creation across generations, and context.md tracking. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Verify Docker command construction, mount flags, and sandbox mode selection with mocked subprocess. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Test _safe_read_file and _safe_load_json with files at, above, and below size limits. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Fix 13 ruff lint errors (unused imports, f-string, sorted imports) - Change DEFAULT_MAX_TURNS/CONTEXT_SUMMARY_MAX_TURNS from str to int - Replace sys.exit(1) with graceful return in _run_target_agent - Add truncation to single-trajectory JSON in feedback prompt - Replace print() with logger in context_manager - Mock _generate_llm_summary in test_multiple_generations_track_deltas All 61 tests passing, ruff clean. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Add HiveMind config fields (endpoint, model, task_model) to Config - Add --hivemind CLI flag to route target agents through local LLMs - Add reference_target_agent_hivemind.py template using OpenAI-compatible API - Support SIA_HIVEMIND_ENDPOINT and SIA_HIVEMIND_MODEL env var overrides Usage: sia --task gpqa --max_gen 3 --hivemind Routes target agents to qwen3.6-27b via HiveMind (:8400), meta/feedback agents still use Claude for quality reasoning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

selvamHexo · 2026-06-02T17:12:45Z

@pookNast thanks for submitting the PR. Can you address the lint failure? Also, please split out the hivemind change into a separate PR.

This reverts commit b3c7bfa.

Address the reviewer-requested security contact email and clean up Ruff lint failures so the hardening PR can pass CI. Co-Authored-By: OpenClaude (gpt-5.5) <openclaude@gitlawb.com>

pookNast · 2026-06-02T18:12:52Z

Addressed, thanks.

Fixed the lint failures on the hardening branch.
Updated the SECURITY.md contact to security@hexo.ai.
Split the HiveMind change into separate PR Add HiveMind gateway integration #16.

Local validation:

ruff check sia/ tests/
python -m pytest tests -q

pookNast and others added 26 commits May 29, 2026 18:34

feat: add sia/config.py with centralized constants

ad23890

Extract hardcoded model names, timeouts, limits, and defaults into a single config module with environment variable overrides. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: orchestrator uses centralized config from sia.config

1e70369

Replace 8 inline hardcoded defaults with Config.* references. No behavioral change. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: context_manager uses centralized config

c0dc7b8

Replace hardcoded truncation limits and default models with config imports. No behavioral change. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat: config supports SIA_* env var overrides

2d49540

Environment variables (SIA_META_MODEL, SIA_TASK_MODEL, etc.) now override defaults with lower priority than explicit CLI flags. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix: narrow exception handlers in context_manager.py

b422918

Replace broad except Exception with specific exception types (OSError, JSONDecodeError, RuntimeError) for better error handling. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix: remove shell=True from run_evaluation subprocess

6d2b001

Replace shell=True piped tee with direct subprocess.run(arg_list) and file write. Adds configurable timeout via Config.EVAL_TIMEOUT. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix: remove shell=True from target agent execution

419a0ff

Replace shell=True piped tee with Popen streaming stdout to both console and log file simultaneously. Removes bash dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

docs: add SECURITY.md with threat model and sandbox docs

90bb2b1

Document the security model, execution modes, bypassPermissions rationale, and Docker sandbox usage. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: extract load_task_files from main()

0d6f214

Section 1 of main() extracted into a load_task_files() function returning a TaskFiles dataclass. Improves readability and testability. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: extract setup_run_directory from main()

c8bfd1c

Section 2 of main() extracted into setup_run_directory() and _create_venv() helper functions. Returns RunSetup dataclass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: extract prompt builders from main()

5afd3f9

Meta-agent and feedback-agent prompt templates extracted into dedicated builder functions for maintainability. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: extract run_generation from main()

83825cb

The 400+ line generation loop extracted into run_generation() and helper functions. main() is now a thin orchestrator. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

test: verify CLI interface unchanged after decomposition

94427ff

Smoke tests for --help, missing args, and invalid task name. Also adds __main__.py to support `python -m sia` invocation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix: add file size checks in context_manager.py

8e4f163

Add _safe_read_file() and _safe_load_json() helpers that enforce Config.MAX_CONTEXT_FILE_SIZE limits. Prevents unbounded memory usage from oversized execution logs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix: add size guards on JSON loading in orchestrator

e76abbe

Enforce Config.MAX_EXECUTION_LOG_SIZE on trajectory files and results.json. Oversized files are skipped with a warning instead of loading into memory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix: centralize all truncation limits in config

bd64826

Replace remaining magic number truncation limits with Config constants for discoverability and tuning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

test: add unit tests for config module

9ba8dd9

Test default values, env var overrides, and invalid value fallback. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

test: add tests for run_evaluation helper

15d88cf

Mock subprocess tests for skipped, success, failure, and timeout scenarios. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

test: add integration test for single generation loop

fc3c9bb

Test target agent execution and context tracking with mocked subprocess. Verifies directory structure and context.md creation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

test: add multi-generation evolution test

372a708

Verify feedback agent invocation, directory creation across generations, and context.md tracking. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

test: add Docker sandbox execution tests

c125d3f

Verify Docker command construction, mount flags, and sandbox mode selection with mocked subprocess. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

test: add file size limit enforcement tests

6f03e34

Test _safe_read_file and _safe_load_json with files at, above, and below size limits. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

selvamHexo self-requested a review June 2, 2026 17:15

selvamHexo reviewed Jun 2, 2026

View reviewed changes

Comment thread SECURITY.md Outdated

Revert "feat: wire SIA to HiveMind gateway for local LLM evolution"

0ebeb89

This reverts commit b3c7bfa.

Fix PR lint and security contact

4cd0c70

Address the reviewer-requested security contact email and clean up Ruff lint failures so the hardening PR can pass CI. Co-Authored-By: OpenClaude (gpt-5.5) <openclaude@gitlawb.com>

pookNast mentioned this pull request Jun 2, 2026

Add HiveMind gateway integration #16

Closed

2 tasks

selvamHexo merged commit 446270f into hexo-ai:main Jun 5, 2026
6 of 7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Security hardening: sandbox mode, shell injection fixes, config centralization#15

Security hardening: sandbox mode, shell injection fixes, config centralization#15
selvamHexo merged 28 commits into
hexo-ai:mainfrom
pookNast:hardening

pookNast commented May 30, 2026

Uh oh!

selvamHexo commented Jun 2, 2026

Uh oh!

Uh oh!

pookNast commented Jun 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

pookNast commented May 30, 2026

Summary

Key changes

Files changed

Test plan

Uh oh!

selvamHexo commented Jun 2, 2026

Uh oh!

Uh oh!

pookNast commented Jun 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants