Skip to content

docxology/template

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

628 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🔬 Docxology Template

Production-grade scaffold for reproducible computational research Pipelines · Manuscripts · Cryptographic Provenance · AI-Agent Collaboration

CI Python 3.10+ uv Ruff License: Apache 2.0 Version DOI

📄 Published: A template/ approach to Reproducible Generative Research: Architecture and Ergonomics from Configuration through Publication — DOI: 10.5281/zenodo.19139090

⚡ Quick Start · 📐 Architecture · 🔄 Pipeline · 🤖 AI Collaboration · 🔒 Provenance · 📚 Docs


About .github/

This folder is the GitHub integration surface: Actions workflows, Dependabot, and issue/PR templates. It is not importable application code.

Document Use
This file Human overview while browsing the repo on GitHub
AGENTS.md Technical index: job names, triggers, coverage thresholds
workflows/README.md CI job graph and commands to mirror CI locally

Related (repo root): .cursor/skill_manifest.json lists agent SKILL.md descriptors. After adding or editing infrastructure/**/SKILL.md, run uv run python -m infrastructure.skills write and commit the updated manifest (see infrastructure/skills/).


What Is This?

The Docxology Template solves the structural root of research irreproducibility: fragmentation between code, tests, manuscripts, and provenance. Instead of patching tools together, it enforces integrity at the architectural level.

You get How
Reproducible builds 10-stage DAG pipeline from env setup → tests → PDF → hashed artifact
Real test enforcement Zero-Mock policy · ≥90% project coverage · ≥60% infra coverage
Cryptographic provenance SHA-256/512 hashing + steganographic watermarking on every PDF
Horizontal scaling N independent projects share one infrastructure layer — no coupling
AI-agent-ready codebase AGENTS.md + README.md per directory; SKILL.md + .cursor/skill_manifest.json for routing
Interactive orchestration run.sh TUI menu for humans · --pipeline flag for CI

⚡ Quick Start

# 1. Create your research repo from this template
gh repo create my-research --template docxology/template --private
cd my-research

# 2. Install dependencies
uv sync

# 3. Run the interactive pipeline menu
./run.sh

# 4. Or run non-interactively against the exemplar project
./run.sh --pipeline --project template_code_project

# Outputs → output/template_code_project/

Don't have uv?curl -Ls https://astral.sh/uv/install.sh | sh

See the full walkthrough in docs/RUN_GUIDE.md and docs/guides/getting-started.md.


📐 Architecture

The repository is organized into two strictly separated layers — shared infrastructure that never changes per-project, and self-contained project workspaces that know nothing about each other.

graph TD
    Root["/ (Repository Root)"] --> Infra["infrastructure/ (Layer 1 — Shared)"]
    Root --> Scripts["scripts/ (Pipeline Stage Scripts)"]
    Root --> Projects["projects/ (Layer 2 — Project Workspaces)"]
    Root --> Docs["docs/ (see documentation-index.md)"]
    Root --> Output["output/ (Final Deliverables)"]

    subgraph "Layer 1 · 15 Python packages under infrastructure/ (+ logrotate.d templates) · 16 docs areas (see docs/modules/modules-guide.md)"
        Infra --> Core["core/ — logging, config, exceptions"]
        Infra --> Rendering["rendering/ — Pandoc + XeLaTeX"]
        Infra --> Stego["steganography/ — SHA-256 + watermarking"]
        Infra --> Valid["validation/ — PDF + Markdown integrity"]
        Infra --> LLM["llm/ — Ollama review + translation"]
        Infra --> More["+ publishing, reporting, scientific, project, documentation…"]
    end

    subgraph "Layer 2 · Active Project Workspaces"
        Projects --> CP["template_code_project/ ← Master numerical exemplar"]
        Projects --> CCD["cognitive_case_diagrams/ ← Case diagrams manuscript"]
        Projects --> TP["template/ ← Meta-documentation"]
        Projects --> Dots["More under projects/ ← Auto-discovered"]
    end
Loading

Authoritative slugs: docs/_generated/active_projects.md. Archived exemplars (for example blake_bimetalism) live under projects_archive/.

Directory Reference

Path Persistence Purpose
infrastructure/ Permanent 15 top-level Python packages under infrastructure/ (plus logrotate.d/ configs and a docker/ config tree, neither a Python package); 16 documented areas in docs/modules/modules-guide.md — see also infrastructure/AGENTS.md
projects/ Permanent Active projects — discovered and executed by pipeline (docs/_generated/active_projects.md)
projects_in_progress/ Transient Staging area: scaffold here before promoting to projects/
projects_archive/ Permanent Completed/retired work — preserved, not executed
scripts/ Permanent 8 generic pipeline stage scripts (Stages 00–07)
output/ Disposable Final PDFs, dashboards, reports — cleaned on each run
docs/ Permanent Large documentation hub — inventory in docs/documentation-index.md (counts vary; do not rely on a single “N files” figure across READMEs)
tests/ Permanent Infrastructure-level test suite (≥ 60% coverage gate)

Key invariant: All domain logic lives in projects/{name}/src/. Scripts are thin orchestrators — they import and call, never implement. See docs/architecture/thin-orchestrator-summary.md.


📁 Active Projects

template_code_project — Master Numerical Exemplar

projects/template_code_project/ is the canonical example of a complete, working project. Use it as the reference when building your own.

Feature Implementation
Gradient descent optimization projects/template_code_project/src/optimizer.py
Scientific benchmarking uses infrastructure.scientific
45 tests, 100% coverage tests/ — Zero-Mock, real operations only
6 publication-quality figures generated in scripts/, registered via FigureManager
Full pipeline output PDF rendered, validated, steganographically signed
Complete documentation AGENTS.md + README.md throughout

Details: projects/template_code_project/AGENTS.md · projects/template_code_project/README.md

template — Meta-Documentation Project (rotating)

projects/template/ is a self-referential manuscript that programmatically documents this very repository.

Feature Implementation
Repository introspection src/template/introspection.py — discovers modules, projects, stages
Metrics injection src/template/metrics.py + inject_metrics.py${var} substitution
65 tests, 94.4% coverage tests/ — Zero-Mock, real filesystem
4 architecture figures scripts/generate_architecture_viz.py — generated from live data
21-chapter manuscript ~13,500 words covering Two-Layer Architecture, pipeline, provenance

Details: projects/template/AGENTS.md · projects/template/README.md

cognitive_case_diagrams — Cognitive Case Diagrams (rotating)

projects/cognitive_case_diagrams/ is an active manuscript project (categorial grammar, case diagrams, and related chapters).

Details: projects/cognitive_case_diagrams/AGENTS.md · projects/cognitive_case_diagrams/README.md

fep_lean — FEP / Lean catalogue (rotating)

projects/fep_lean/ — Free Energy Principle / Active Inference topics with Lean 4 sketches, Hermes (OpenRouter), and math-inc Open Gauss (gauss) when workflows are enabled.

Details: projects/fep_lean/AGENTS.md · projects/fep_lean/README.md

Archived exemplars

Retired or sample projects are kept under projects_archive/ (not executed by ./run.sh until moved back into projects/). Example: blake_bimetalism (digital humanities manuscript) — projects_archive/blake_bimetalism/AGENTS.md.

Project Directory Layout

flowchart LR
    P[/projects/<name>/]
    P --> SRC[/src/&lt;name&gt;<br/>All domain logic · algorithms · analysis/]
    P --> T[/tests<br/>Real tests · no mocks · ≥ 90% coverage/]
    P --> SC[/scripts<br/>Thin orchestrators calling src/]
    P --> M[/manuscript<br/>Markdown chapters · config.yaml/]
    P --> O[/output<br/>Pipeline artifacts · generated/]
    P --> AG[AGENTS.md<br/>AI-agent context for this project]

    classDef d fill:#0f172a,stroke:#0f172a,color:#fff
    classDef f fill:#0f766e,stroke:#0f172a,color:#fff
    class P,SRC,T,SC,M,O d
    class AG f
Loading

System Architecture Overview

graph TB
    subgraph Entry["🚀 Entry Points"]
        RUNSH[./run.sh<br/>Interactive menu<br/>Full pipeline control]
        RUNALL[uv run python scripts/execute_pipeline.py --core-only<br/>Programmatic<br/>Core pipeline]
        INDIVIDUAL[Individual Scripts<br/>scripts/00-07_*.py<br/>Stage-specific execution]
    end

    subgraph Orchestration["⚙️ Orchestration Layer"]
        SETUP[Environment Setup<br/>Dependencies & validation]
        TESTING[Test Execution<br/>Coverage requirements]
        ANALYSIS[Script Discovery<br/>Project analysis execution]
        RENDERING[PDF Generation<br/>Manuscript compilation]
        VALIDATION[Quality Assurance<br/>Content validation]
        DELIVERY[Output Distribution<br/>Final deliverables]
    end

    subgraph Core["🧠 Core Systems (DAG Engine)"]
        INFRASTRUCTURE[Infrastructure Modules<br/>13 specialized modules<br/>Validation, rendering, LLM]
        BUSINESS_LOGIC[Business Logic<br/>Project algorithms<br/>100% test coverage]
        CONFIGURATION[Configuration System<br/>YAML + environment<br/>Runtime flexibility]
    end

    subgraph Data["📊 Data Flow"]
        SOURCE_CODE[Source Code<br/>Python modules<br/>Algorithm implementation]
        MANUSCRIPT_CONTENT[Manuscript Content<br/>Markdown sections<br/>Research writing]
        GENERATED_OUTPUTS[Generated Outputs<br/>PDFs, figures, data<br/>Research deliverables]
    end

    subgraph Quality["✅ Quality Assurance"]
        UNIT_TESTS[Unit Tests<br/>Function validation<br/>Real data, no mocks]
        INTEGRATION_TESTS[Integration Tests<br/>System validation<br/>End-to-end workflows]
        VALIDATION_CHECKS[Content Validation<br/>Quality assurance<br/>Academic standards]
    end

    RUNSH --> Orchestration
    RUNALL --> Orchestration
    INDIVIDUAL --> Orchestration

    Orchestration --> Core
    Core --> Data
    Data --> Quality

    Quality -.->|Feedback| Orchestration

    classDef entry fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
    classDef orchestration fill:#fff3e0,stroke:#e65100,stroke-width:2px
    classDef core fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
    classDef data fill:#e8f5e8,stroke:#2e7d32,stroke-width:2px
    classDef quality fill:#fce4ec,stroke:#c2185b,stroke-width:2px

    class Entry entry
    class Orchestration orchestration
    class Core core
    class Data data
    class Quality quality
Loading

To add your own project, follow docs/guides/new-project-setup.md.

Project Lifecycle

stateDiagram-v2
    [*] --> InProgress: Create scaffold
    InProgress --> Active: Add src/ + tests/ + manuscript/config.yaml
    Active --> Archive: Complete / retire
    Archive --> Active: Reactivate

    InProgress : projects_in_progress/ — not executed
    Active : projects/ — auto-discovered, pipeline-executed
    Archive : projects_archive/ — preserved, not executed
Loading

🔄 Pipeline

run.sh executes a 10-stage declarative DAG pipeline configured via pipeline.yaml. secure_run.sh appends steganographic post-processing.

flowchart TD
    subgraph Input["📥 Input Data"]
        SOURCE_CODE["Source Code<br/>projects/{name}/src/*.py<br/>Algorithm implementations"]
        ANALYSIS_SCRIPTS["Analysis Scripts<br/>projects/{name}/scripts/*.py<br/>Workflow orchestrators"]
        MANUSCRIPT_FILES["Manuscript Files<br/>projects/{name}/manuscript/*.md<br/>Research content"]
        CONFIG_FILES[Configuration<br/>config.yaml<br/>Runtime parameters]
    end

    subgraph Processing["⚙️ 10-Stage DAG Pipeline"]
        STAGE0["Stage 0 — Clean<br/>(built-in / executor)"]
        STAGE1["Stage 1 — Setup<br/>00_setup_environment.py"]
        STAGE2["Stage 2 — Infra tests<br/>01_run_tests.py --infra-only"]
        STAGE3["Stage 3 — Project tests<br/>01_run_tests.py --project-only"]
        STAGE4["Stage 4 — Analysis<br/>02_run_analysis.py"]
        STAGE5["Stage 5 — Render PDF<br/>03_render_pdf.py"]
        STAGE6["Stage 6 — Validate<br/>04_validate_output.py"]
        STAGE7["Stage 7 — LLM reviews<br/>06_llm_review.py --reviews-only"]
        STAGE8["Stage 8 — LLM translations<br/>06_llm_review.py --translations-only"]
        STAGE9["Stage 9 — Copy outputs<br/>05_copy_outputs.py"]
    end

    subgraph Output["📤 Generated Outputs"]
        PDF_DOCS["PDF Documents<br/>output/{name}/pdf/*.pdf<br/>Professional manuscripts"]
        FIGURES["Figures<br/>output/{name}/figures/*.png<br/>Publication-quality plots"]
        DATA_FILES["Data Files<br/>output/{name}/data/*.csv<br/>Analysis results"]
        REPORTS["Reports<br/>output/{name}/reports/*.md<br/>Validation summaries"]
    end

    SOURCE_CODE --> STAGE3
    ANALYSIS_SCRIPTS --> STAGE4
    MANUSCRIPT_FILES --> STAGE5
    CONFIG_FILES --> STAGE1

    STAGE0 --> STAGE1 
    STAGE1 --> STAGE2
    STAGE1 --> STAGE3
    STAGE3 --> STAGE4
    STAGE4 --> STAGE5
    STAGE5 --> STAGE6
    STAGE6 --> STAGE7
    STAGE6 --> STAGE8
    STAGE6 --> STAGE9

    STAGE4 --> FIGURES
    STAGE4 --> DATA_FILES
    STAGE5 --> PDF_DOCS
    STAGE6 --> REPORTS

    classDef input fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
    classDef process fill:#fff3e0,stroke:#e65100,stroke-width:2px
    classDef output fill:#e8f5e8,stroke:#2e7d32,stroke-width:2px

    class Input input
    class Processing process
    class Output output
Loading

Stage indices 0–9 are pipeline positions; they do not match scripts/NN_*.py numeric prefixes (e.g. stage 2 uses 01_run_tests.py). The table below is authoritative.

Stage Script Tags Failure mode
0 Clean Output Directories built-in _run_clean_outputs core, clean soft fail
1 Environment Setup 00_setup_environment.py core hard fail
2 Infrastructure Tests 01_run_tests.py --infra-only --verbose core, tests configurable tolerance
3 Project Tests 01_run_tests.py --project-only --verbose core, tests configurable tolerance
4 Project Analysis 02_run_analysis.py core hard fail
5 PDF Rendering 03_render_pdf.py core hard fail
6 Output Validation 04_validate_output.py core warning + report
7 LLM Scientific Review 06_llm_review.py --reviews-only llm skipped if Ollama absent
8 LLM Translations 06_llm_review.py --translations-only llm skipped if Ollama absent
9 Copy Outputs 05_copy_outputs.py core soft fail

Full stage details: docs/core/workflow.md · docs/core/how-to-use.md.


🤖 AI Collaboration

Every directory at every level contains two documentation files:

  • README.md — Human-readable overview and quick-start
  • AGENTS.md — Machine-readable spec for AI coding assistants: API tables, dependency graphs, architectural constraints, naming conventions

Under infrastructure/, each subpackage also has SKILL.md (YAML frontmatter). The aggregated list for editors is .cursor/skill_manifest.json (regenerate with uv run python -m infrastructure.skills write). Cursor project skills live under .cursor/skills/.

flowchart TB
    CL[CLAUDE.md · root<br/>Global constraints: Zero-Mock ·<br/>Thin Orchestrator · naming]
    AG[AGENTS.md · per directory<br/>Local API surfaces · file inventories ·<br/>integration patterns]
    RD[README.md<br/>Human navigation · quick-start]

    CL --> AG --> RD

    classDef root fill:#0f172a,stroke:#0f172a,color:#fff
    classDef tech fill:#1e3a8a,stroke:#0f172a,color:#fff
    classDef hum fill:#0f766e,stroke:#0f172a,color:#fff
    class CL root
    class AG tech
    class RD hum
Loading

See docs/rules/ for standards and infrastructure/SKILL.md for the infrastructure skill hub.


🔒 Security & Provenance

Every rendered PDF is automatically processed by the steganographic pipeline via secure_run.sh:

Layer Mechanism Survives
PDF Metadata XMP + Info dictionary (author, DOI, ORCID, build timestamp) All viewers
Hash manifest SHA-256 + SHA-512 in *.hashes.json External verification
Alpha overlay Low-opacity text per page (build time + commit hash) Standard PDF operations, printing
QR code Repository URL injected on final page Redistribution

Full specification: docs/security/steganography.md · docs/security/hashing_and_manifests.md · docs/security/secure_execution.md.


🧪 Testing Standards

Standard Requirement
Zero-Mock policy No MagicMock, mocker.patch, or unittest.mock anywhere
Real operations Tests use real filesystem, subprocess, and HTTP calls
Infrastructure coverage ≥ 60% (currently achieving 83%+)
Project coverage ≥ 90% (template_code_project achieves 100% · template achieves 94.4%)
Optional service skipping @pytest.mark.requires_ollama for graceful degradation
# Mirror CI locally
uv run pytest tests/infra_tests/ --cov=infrastructure --cov-fail-under=60 -m "not requires_ollama"
uv run pytest projects/template_code_project/tests/ --cov-fail-under=90 -m "not requires_ollama"
python scripts/verify_no_mocks.py

See docs/development/testing/ and docs/guides/testing-and-reproducibility.md.


📚 Documentation Hub

graph TD
    subgraph Core["📂 Core — Start Here"]
        HOW[how-to-use.md]
        WORK[workflow.md]
        ARCH[architecture.md]
    end

    subgraph Build["🏗️ Architecture"]
        TLA[two-layer-architecture.md]
        THIN[thin-orchestrator-summary.md]
        DECIDE[decision-tree.md]
    end

    subgraph Learn["📖 Guides"]
        START[getting-started.md]
        NEW[new-project-setup.md]
        TEST[testing-and-reproducibility.md]
        FIG[figures-and-analysis.md]
    end

    subgraph Enforce["📏 Rules"]
        TSTD[testing_standards.md]
        CODE[code_style.md]
        DOC[documentation_standards.md]
    end

    subgraph Operate["🔧 Operational"]
        BUILD[build/]
        CONF[config/]
        TROUBLE[troubleshooting/]
    end

    HOW -->|"Deep dive"| TLA
    HOW -->|"First run"| START
    WORK -->|"Stage details"| BUILD
    ARCH -->|"Pattern"| THIN
    START -->|"Add project"| NEW
    NEW -->|"Write tests"| TEST
    TEST -->|"Standards"| TSTD
    FIG -->|"Style"| DOC

    classDef core fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
    classDef build fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
    classDef learn fill:#e8f5e8,stroke:#2e7d32,stroke-width:2px
    classDef enforce fill:#fff3e0,stroke:#e65100,stroke-width:2px
    classDef operate fill:#fce4ec,stroke:#c2185b,stroke-width:2px

    class Core core
    class Build build
    class Learn learn
    class Enforce enforce
    class Operate operate
Loading

The docs/ directory spans many files across 13 subdirectories — see docs/documentation-index.md for the live inventory. Every subdirectory has its own README.md and AGENTS.md. Start at docs/README.md.

📂 Core (docs/core/)

Essential start-here docs — read these first.

File Purpose
how-to-use.md Step-by-step usage guide for the full system
workflow.md Pipeline workflow: stages, flags, modes
architecture.md Two-Layer Architecture overview

🏗️ Architecture (docs/architecture/)

Design decisions, patterns, and migration guides.

File Purpose
two-layer-architecture.md Deep dive into the Layer 1 / Layer 2 separation
thin-orchestrator-summary.md The Thin Orchestrator pattern — why and how
testing-strategy.md Testing architecture and Zero-Mock rationale
decision-tree.md Where does new code go? Decision guide
migration-from-flat.md Migrating a flat repo to the Two-Layer model

📖 Guides (docs/guides/)

Progressive tutorials from first project to advanced automation.

File Purpose
getting-started.md First-time setup and first pipeline run
new-project-setup.md Full checklist for adding a new project
figures-and-analysis.md Generating, registering, and embedding figures
testing-and-reproducibility.md Writing real tests, coverage, markers
extending-and-automation.md Customizing the pipeline, adding CI stages

📏 Rules (docs/rules/)

Authoritative standards enforced across the codebase.

File Purpose
testing_standards.md Zero-Mock policy, coverage thresholds, markers
code_style.md Ruff config, formatting, naming conventions
documentation_standards.md AGENTS.md / README.md duality requirements
manuscript_style.md Chapter structure, figure captions, citations
api_design.md Module API conventions, dataclass patterns
error_handling.md Exception hierarchy, pipeline flow control
security.md Dependency pinning, secrets management
llm_standards.md Ollama integration, prompt templates, markers
python_logging.md Structured logging via get_logger
type_hints_standards.md mypy-compatible type annotation requirements
infrastructure_modules.md Module API contracts and extension patterns
git_workflow.md Branch strategy, commit conventions, PRs
folder_structure.md Directory layout invariants

🔧 Operational (docs/operational/)

Build, config, logging, and troubleshooting.

Directory / File Purpose
build/ Build system internals and stage details
config/ config.yaml reference, environment variables
logging/ Logging configuration, log levels, rotation
troubleshooting/ Common errors, rendering issues, coverage gaps
error-handling-guide.md Pipeline error handling patterns
reporting-guide.md Executive reports, coverage JSON, dashboards

📦 Modules (docs/modules/)

Infrastructure subpackage documentation.

File Purpose
modules-guide.md Overview of infrastructure subpackages
pdf-validation.md infrastructure.validation — PDF integrity checking
scientific-simulation-guide.md infrastructure.scientific — stability, benchmarking
guides/ Per-module usage guides

📝 Usage (docs/usage/)

Manuscript authoring, style, and content guides.

File Purpose
markdown-template-guide.md Chapter structure, frontmatter, Pandoc quirks
style-guide.md Voice, tense, academic writing conventions
manuscript-numbering-system.md Section/figure/table numbering
visualization-guide.md Figure accessibility standards (16pt floor, colorblind palettes)
image-management.md Figure registration, paths, captions
examples.md Worked example manuscript snippets
examples-showcase.md Gallery of generated figures from exemplar projects

📐 Best Practices (docs/best-practices/)

Project hygiene, version control, and multi-project management.

File Purpose
best-practices.md Consolidated best practices across all concerns
multi-project-management.md Managing N projects, discovery rules, isolation
version-control.md Git workflow, tagging, output tracking
migration-guide.md Upgrading the template across major versions
backup-recovery.md Output preservation, disaster recovery

🛠️ Development (docs/development/)

Contributing, testing internals, roadmap.

File Purpose
contributing.md How to contribute — branch, test, PR
testing/ Test writing guide, coverage analysis, patterns
coverage-gaps.md Known low-coverage modules and improvement plans
roadmap.md Feature roadmap and planned improvements
security.md Security disclosure policy
code-of-conduct.md Community standards

🤖 Prompts (docs/prompts/)

Reusable AI agent prompt templates for common tasks.

File Purpose
infrastructure_module.md Creating a new infrastructure subpackage
feature_addition.md Adding a feature to an existing module
test_creation.md Writing Zero-Mock tests
manuscript_creation.md Authoring a new manuscript chapter
refactoring.md Safe refactoring with test preservation
validation_quality.md Adding validation and quality gates
documentation_creation.md Writing AGENTS.md / README.md
code_development.md General code development patterns
comprehensive_assessment.md Full pipeline + codebase audit

📖 Reference (docs/reference/)

API reference, glossary, cheatsheets, and FAQ.

File Purpose
api-reference.md Public API reference for infrastructure modules
api-project-modules.md Project-level module patterns and conventions
glossary.md Definitions for all template-specific terms
faq.md Frequently asked questions
quick-start-cheatsheet.md One-page command reference
common-workflows.md Recipes for common research tasks
copypasta.md Copy-paste code snippets for common patterns

🔒 Security (docs/security/)

Steganography, hashing, and secure execution.

File Purpose
steganography.md Watermarking layers, alpha overlay, QR injection
hashing_and_manifests.md SHA-256/512 hash manifests and tamper detection
secure_execution.md secure_run.sh, steganography config, output files

🔍 Audit (docs/audit/)

Documentation review reports and filepath audits.

File Purpose
documentation-review-report.md Comprehensive documentation audit results
filepath-audit-report.md File path accuracy and broken link report

🚀 Top-Level Docs

File Purpose
docs/README.md Documentation hub index and navigation
docs/documentation-index.md Full inventory of documentation files (authoritative count)
docs/RUN_GUIDE.md Complete run guide: modes, flags, troubleshooting
docs/CLOUD_DEPLOY.md Cloud deployment guide (AWS, GCP, Azure, Docker)
docs/PAI.md Personal AI Infrastructure integration guide

🔧 CI/CD

Workflows

Workflow Trigger Purpose
ci.yml push · PR · weekly · manual 10 CI jobs + conditional fep-lean when projects/fep_lean/lean/lean-toolchain exists
stale.yml Daily 01:00 UTC Close inactive issues/PRs
release.yml v*.*.* tag · manual GitHub Release with changelog

CI Job Flow

flowchart TB
    L[Lint and Type Check]
    H[Unified Health Report informational]
    VNM[Verify No Mocks]
    TI[Infra Tests matrix]
    TP[Project Tests matrix]
    FL[fep_lean optional]
    VM[Validate Manuscripts]
    SS[Security Scan]
    DL[Documentation Lint]
    PC[Performance Check]
    L --> H
    L --> VNM
    L --> VM
    L --> SS
    L --> DL
    VNM --> TI
    VNM --> TP
    VNM --> FL
    TI --> PC
    TP --> PC
Loading

See workflows/AGENTS.md for step-level detail (pip-audit ignore file, bandit.yaml, 01_run_tests.py).

Quality Gates

Gate Tool Threshold
Code style Ruff zero violations
Formatting Ruff zero diffs
Type safety mypy no errors
No mocks verify_no_mocks.py zero mock usage
Exports audit infrastructure.skills check-all-exports zero violations
Infra coverage pytest-cov ≥ 60%
Project coverage pytest-cov ≥ 90%
pip-audit blocking zero unignored vulns
Security Bandit -c bandit.yaml MEDIUM+ zero findings
Docs lint scripts/lint_docs.py clean
Performance import timer ≤ 5 s

Simulate CI Locally

# Lint + format check (mirror CI)
uvx ruff check infrastructure/ projects/*/src/
uvx ruff format --check infrastructure/ projects/*/src/
uv run mypy infrastructure/ projects/*/src/

# Tests (skip Ollama-requiring tests)
uv run pytest tests/infra_tests/ --cov=infrastructure --cov-fail-under=60 -m "not requires_ollama"
uv sync --group rendering --group monitoring --group discopy
COVERAGE_FILE=.coverage.project uv run python scripts/01_run_tests.py --project-only --all-projects --non-strict --include-slow

# Security (mirror CI)
IGNORE_ARGS=()
while IFS= read -r raw || [ -n "$raw" ]; do
  [[ "$raw" =~ ^[[:space:]]*# ]] && continue
  line="${raw%%#*}"; line="$(echo "$line" | xargs)"
  [ -z "$line" ] || IGNORE_ARGS+=(--ignore-vuln "$line")
done < .github/pip-audit-ignore.txt
uv run pip-audit "${IGNORE_ARGS[@]}"
uv run bandit -c bandit.yaml -r -ll infrastructure/ scripts/ projects/

Branch Protection

Set in Settings → Branches → main:

Required status checks:
  Lint & Type Check
  Infra Tests (ubuntu-latest, Python 3.10/3.11/3.12)
  Project Tests (ubuntu-latest, Python 3.10/3.11/3.12)
  Validate Manuscripts · Security Scan · Documentation Lint · Performance Check
  # Optional informational artefact: Unified Health Report (informational)
  # Optional: fep_lean (gauss + lake) — only when that job runs (skipped if fep_lean not in projects/)

Require PR review before merging: 1 approver
Secret Required for
CODECOV_TOKEN Coverage upload to Codecov (optional)

📋 Issue & PR Templates

Issues → New Issue

Template Labels Best for
🐛 Bug Report bug · needs-triage Reproducible errors with log output and pipeline stage
✨ Feature Request enhancement · needs-triage New capabilities with priority and alternatives
📝 Documentation documentation · needs-triage Incorrect, missing, or outdated docs with file paths

💬 Questions? Use GitHub Discussions — blank issues are disabled.

PR Checklist → PULL_REQUEST_TEMPLATE.md

  • ✅ Linked issue · type-of-change label · pipeline stage(s) affected
  • ✅ Test evidence — local run confirmation with pass rates
  • Zero-Mock confirmation — no MagicMock / mocker.patch
  • ✅ Thin Orchestrator compliance — no logic in scripts

📦 Dependency Management

dependabot.yml — weekly automated PRs:

Ecosystem Group Max PRs
GitHub Actions all minor/patch batched 5
Python (uv) dev-tools (pytest, mypy, ruff…) 5
Python (uv) scientific-core (numpy, scipy…) 5

🔍 Troubleshooting

# CI status
gh run list --workflow=CI --limit=5
gh run view <run-id> --log-failed
gh run rerun <run-id> --failed

# Fix lint locally
uvx ruff check infrastructure/ projects/*/src/ --fix
uvx ruff format infrastructure/ projects/*/src/

Common issues: docs/operational/troubleshooting/ · docs/reference/faq.md.


📖 AGENTS.md · 🚀 Run Guide · 📐 Architecture · 📋 Rules · 🐛 Issues · 💬 Discussions

Reproducibility as architecture, not afterthought.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors