feat: add reusable workflow orchestration runtime

[y1zhou]

Summary

This PR is primarily a general workflow-orchestration change. It adds the shared schemas, runtime, ledger, Modal orchestrator boundary, CLI/catalog plumbing, and app-function contracts needed to compose Biomodals apps into durable workflows. PPIFlow is updated as one consumer of that infrastructure, but the reusable runtime and workflow-compatible app surface are the main scope.

Full Diff Summary

Compared with main, this branch changes 68 files: 9,271 insertions and 2,001 deletions.

Workflow Runtime and Schemas

• Adds shared workflow/app/storage schemas under src/biomodals/schema/:
  • AppConfig, AppRunResult, AppOutput, and AppRunStatus
  • InlineBytes and VolumePath storage contracts
  • workflow artifact, selector, node/run status, placement, and execution-policy models
• Adds reusable workflow core modules under src/biomodals/workflow/core/:
  • builder.py for Python-first DAG construction and dependency validation
  • nodes.py for workflow node interfaces and node run context
  • artifacts.py for materializing inline/volume app outputs into workflow artifacts
  • ledger.py for SQLite-backed durable run, node, attempt, remote-call, artifact, and input/output state
  • runtime.py for scheduling ready nodes, bounded parallel execution, remote node dispatch, run resumption, DAG hash checks, and volume sync
  • orchestrator.py for the Modal WorkflowOrchestrator class boundary and isolated remote node execution
• Adds src/biomodals/workflow/__init__.py exports for workflow construction and core types.

Workflow Applications

• Adds src/biomodals/workflow/shortmd_workflow.py, a reusable ShortMD workflow built on the new runtime:
  • prepares GROMACS inputs once per PDB
  • clones prepared inputs into replicate runs
  • runs short production replicates in parallel
  • emits a summary artifact
  • exposes submit_shortmd_workflow
• Reworks src/biomodals/workflow/ppiflow_workflow.py from the previous draft into an app-orchestrated workflow script that stages YAML inputs, coordinates deployed Biomodals apps, runs queue-based parallel per-structure tasks, filters/ranks outputs, and packages the final run directory.

CLI, Catalog, and Helper Refactor

• Adds explicit biomodals app ... and biomodals workflow ... namespaces while keeping deprecated top-level app aliases for compatibility.
• Moves app catalog discovery from biomodals.app.catalog to biomodals.helper.catalog and extends it to discover workflow scripts.
• Moves shared constants from biomodals.app.constant to biomodals.helper.constant and adds a shared workflow-orchestrator volume.
• Updates the local biomodals helper script to route app and workflow subcommands.
• Adds stricter helper behavior for local output paths, shell/archive handling, and volume-run path conversion.

Workflow-Compatible App Contracts

• Adds src/biomodals/app/score/dockq_app.py with batch DockQ scoring, CSV output, tarball packaging, and a local entrypoint.
• Updates app modules to use the moved helper constants/catalog imports and the newer Modal image .pipe(patch_image_for_helper) pattern.
• Extends app-level contracts used by workflows:
  • AlphaFold3: more robust data/inference handling, resumable multi-container inference tarballs, sanitized run names, and updated pinned source/version
  • FlowPacker: workflow-compatible AppRunResult archive output backed by a Modal volume
  • GROMACS: more robust interrupted-trajectory time detection and production resume behavior
  • Rosetta: image/env helper updates for workflow use
  • AF3Score and related apps: import/path updates for shared helper modules

Build, Tooling, Docs, and Examples

• Bumps modal to >=1.4.3, adds orjson, adds pytest to dev dependencies, and moves Ruff configuration into pyproject.toml.
• Updates pre-commit configuration for the new Ruff setup and switches Markdown formatting to rumdl.
• Updates app examples to use the biomodals helper and namespaced biomodals app r ... commands.
• Adds CONTEXT.md and docs/agents/workflow-development.md, and updates app-development agent guidance for shared schemas, helper reuse, and workflow-compatible app functions.
• Adds .gitignore entries for .zed/ and PLAN.md.

Tests Added or Updated

• Adds schema tests for app/storage/workflow contracts.
• Adds workflow core tests for artifact materialization, DAG builder validation, SQLite ledger persistence, node contracts, runtime scheduling/recovery, and Modal orchestrator boundaries.
• Adds ShortMD workflow tests covering DAG construction, GROMACS prep/clone/replicate nodes, local entrypoint submission, and force propagation.
• Adds app/CLI/catalog tests for workflow discovery, namespaced commands, DockQ, FlowPacker workflow outputs, AlphaFold3 standalone contracts, GROMACS standalone contracts, and volume-run helpers.

Validation

Previously exercised during this branch:

• compileall src/biomodals
• uv run biomodals app list
• uv run biomodals app help ppiflow
• uv run biomodals app help dockq
• uv run biomodals app help af3score
• uv run biomodals app help rosetta
• uv run biomodals workflow list
• modal run ppiflow_workflow.py --help
• prek run / prek run --files ... on changed files

Additional local spot check while reviewing the final diff:

• rtk uv run pytest tests/workflow/test_runtime.py tests/workflow/test_ppiflow_workflow_inputs.py tests/app/test_cli_workflow_catalog.py

Not Run

• Full Modal end-to-end workflow execution for PPIFlow or ShortMD

[Copilot]

Pull request overview

Adds a Modal-based PPIFlow workflow orchestrator under src/biomodals/workflow/ that coordinates multiple deployed Biomodals apps to reproduce the upstream two-stage pipeline and package results as a .tar.zst.

Changes:

• Replace the previous monolithic PPIFlow runner with a workflow orchestrator that calls deployed apps (PPIFlow/LigandMPNN/FlowPacker/AF3Score/AlphaFold3/Rosetta/DockQ).
• Add a new DockQ Biomodals app and add workflow-oriented “*_bytes” helpers to PPIFlow, AF3Score, and Rosetta for in-memory/byte-based orchestration.
• Add workflow development and PPIFlow workflow documentation.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/biomodals/workflow/ppiflow_workflow.py	New workflow orchestrator implementing stage1/stage2 execution, queue-based fanout, filtering, refold/relax, dockq, and report packaging.
src/biomodals/app/score/dockq_app.py	New DockQ scoring app with batch function and CSV-driven entrypoint.
src/biomodals/app/score/af3score_app.py	Adds af3score_run_bytes to score in-memory PDBs and return a packaged archive for workflows.
src/biomodals/app/design/ppiflow_app.py	Adds ppiflow_run_bytes to run PPIFlow from in-memory inputs and return a packaged archive for workflows.
src/biomodals/app/bioinfo/rosetta_app.py	Adds run_rosetta_batch_bytes for in-memory Rosetta batch execution and packaging.
docs/agents/workflow-development.md	Documents workflow conventions, interfaces, and operational guidelines.
docs/agents/ppiflow-workflow.md	Documents ppiflow workflow app dependencies and design tradeoffs.
AGENTS.md	Adds a pointer to workflow development guidance for workflow changes.