Validate Terra Runtime and Expand Local Model Context - #60
Merged
Conversation
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (11)
📝 WalkthroughWalkthroughThis release updates Heartwood to Sequence Diagram(s)sequenceDiagram
participant CI
participant DockerVolume
participant Heartwood
participant Readiness
CI->>DockerVolume: download managed local model
CI->>Heartwood: start managed Terra launch smoke
Heartwood->>Readiness: poll project readiness
Readiness-->>Heartwood: return ready Terra payload
Heartwood-->>CI: emit authenticated proxy path
CI->>DockerVolume: remove volume
Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This pull request prepares the 0.2.0-beta.2 release while tightening the “local model” contract across Heartwood’s runtime layers (gateway/OpenHands/CLI/notebook/web UI) and improving Terra-specific behavior (nested Leonardo proxy routing, managed launch smoke validation), alongside a GPU runtime pin intended to match Terra’s NVIDIA driver baseline.
Changes:
- Pin and preflight the NVIDIA vLLM stack (CUDA 11.8 / PyTorch) and remove now-unneeded FFmpeg/TorchCodec dependencies and checks.
- Introduce and propagate a consistent local-model context contract (notably
context_window, plus local profilemax_input_tokens/max_output_tokens) across gateway config, CLI runtime launch args, OpenHands parameters, and UI surfaces. - Preserve Terra’s full nested Jupyter proxy base path and add/extend Terra smoke coverage (including a managed
heartwood launch --webflow).
Reviewed changes
Copilot reviewed 96 out of 99 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| VERSION.toml | Bumps canonical release version to 0.2.0-beta.2. |
| uv.lock | Aligns workspace package versions with 0.2.0b2. |
| skills/verified/omop-cohort-summary/SKILL.md | Updates verified skill metadata version to beta.2. |
| skills/verified/omop-cohort-summary/metadata.json | Updates verified skill metadata version to beta.2. |
| skills/verified/baseline-model/SKILL.md | Updates verified skill metadata version to beta.2. |
| skills/verified/baseline-model/metadata.json | Updates verified skill metadata version to beta.2. |
| skills/verified/aggregate-export/SKILL.md | Updates verified skill metadata version to beta.2. |
| skills/verified/aggregate-export/metadata.json | Updates verified skill metadata version to beta.2. |
| packages/webui/src/types.ts | Extends UI contract with token limits and context_window fields. |
| packages/webui/src/e2e/app.spec.ts | Updates e2e fixtures to include context_window. |
| packages/webui/src/components/UtilitySheet.tsx | Displays local model context window in settings/setup UI. |
| packages/webui/src/client.ts | Fixes base-path inference for nested Terra proxy routes. |
| packages/webui/src/client.test.ts | Adds coverage ensuring nested Terra proxy base path is preserved. |
| packages/webui/src/App.test.tsx | Updates app test fixtures to include context_window. |
| packages/webui/scripts/smoke-jupyter-proxy.cjs | Adjusts default service prefix toward Terra-style proxy routing. |
| packages/webui/package.json | Bumps web UI package version to beta.2. |
| packages/webui/package-lock.json | Bumps lockfile version strings to beta.2. |
| packages/skills/pyproject.toml | Bumps heartwood-skills to beta.2. |
| packages/session/src/heartwood/session/init.py | Bumps heartwood-session runtime __version__. |
| packages/session/pyproject.toml | Bumps heartwood-session package version to beta.2. |
| packages/schemas/tests/test_schema_records.py | Updates tests to expect beta.2 skill metadata versions. |
| packages/schemas/src/heartwood/schemas/init.py | Bumps heartwood-schemas runtime __version__. |
| packages/schemas/pyproject.toml | Bumps heartwood-schemas package version to beta.2. |
| packages/notebook/tests/test_notebook.py | Adds tests for Terra Leonardo proxy routing in jupyter_proxy_url. |
| packages/notebook/src/heartwood/notebook/_view_model.py | Implements Terra Leonardo proxy URL building with URL-quoting. |
| packages/notebook/src/heartwood/notebook/init.py | Bumps heartwood-notebook runtime __version__. |
| packages/notebook/pyproject.toml | Bumps heartwood-notebook package version to beta.2. |
| packages/model-policy/src/heartwood/model_policy/init.py | Bumps heartwood-model-policy runtime __version__. |
| packages/model-policy/pyproject.toml | Bumps heartwood-model-policy package version to beta.2. |
| packages/gateway/tests/test_openhands_sdk.py | Adds test ensuring OpenHands char-budget aligns with local token budget. |
| packages/gateway/tests/test_model_snapshots.py | Validates snapshot catalog includes context_window and new Terra GPU rec. |
| packages/gateway/tests/test_model_settings.py | Adds tests for (optional) max_input_tokens/max_output_tokens validation. |
| packages/gateway/tests/test_model_artifacts.py | Validates artifacts carry context_window; extends manifest fixture. |
| packages/gateway/tests/test_local_models.py | Adds context-window propagation and bounding tests; adds new recommended model. |
| packages/gateway/tests/test_gateway_contract.py | Ensures persisted local profile token budgets are set on selection. |
| packages/gateway/src/heartwood/gateway/_project_config.py | Persists/validates context_window in local model selection config. |
| packages/gateway/src/heartwood/gateway/_openhands_sdk.py | Passes token limits and derived max_message_chars into OpenHands config. |
| packages/gateway/src/heartwood/gateway/_model_snapshots.py | Adds context_window to snapshot schema and validation. |
| packages/gateway/src/heartwood/gateway/_model_settings.py | Adds token-limit fields to model profile mapping + validation helper. |
| packages/gateway/src/heartwood/gateway/_model_artifacts.py | Adds context_window to artifact schema and validation. |
| packages/gateway/src/heartwood/gateway/_local_models.py | Extracts/bounds context window from HF metadata; threads it through choices. |
| packages/gateway/src/heartwood/gateway/_gateway.py | Splits context into input/output budgets and persists them into settings. |
| packages/gateway/pyproject.toml | Bumps heartwood-gateway package version to beta.2. |
| packages/fixtures/src/heartwood/fixtures/init.py | Bumps heartwood-fixtures runtime __version__. |
| packages/fixtures/pyproject.toml | Bumps heartwood-fixtures package version to beta.2. |
| packages/detector/tests/test_platform_detection.py | Adds explicit Terra platform marker test. |
| packages/detector/src/heartwood/detector/_platforms.py | Supports explicit HEARTWOOD_PLATFORM=terra detection. |
| packages/detector/src/heartwood/detector/init.py | Bumps heartwood-detector runtime __version__. |
| packages/detector/pyproject.toml | Bumps heartwood-detector package version to beta.2. |
| packages/core-adapter/src/heartwood/core_adapter/init.py | Bumps heartwood-core-adapter runtime __version__. |
| packages/core-adapter/pyproject.toml | Bumps heartwood-core-adapter package version to beta.2. |
| packages/compliance/tests/test_documentation_assets.py | Updates notebook/doc assertions for new Terra demo content. |
| packages/compliance/tests/test_container_assets.py | Updates GPU runtime expectations (CUDA 11.8 stack, no FFmpeg/TorchCodec). |
| packages/compliance/src/heartwood/compliance/init.py | Bumps heartwood-compliance runtime __version__. |
| packages/compliance/pyproject.toml | Bumps heartwood-compliance package version to beta.2. |
| packages/cli/tests/test_launch.py | Adds context propagation, resource assessment warnings, cgroup RAM limit test, and updated vLLM preflight expectations. |
| packages/cli/tests/test_carina_launch_integration.py | Updates synthetic runtime version output expectations for vLLM/PyTorch/CUDA. |
| packages/cli/src/heartwood/cli/_launch.py | Propagates context to runtime args; adds conservative RAM/VRAM preflight; updates vLLM CUDA initialization preflight. |
| packages/cli/src/heartwood/cli/init.py | Bumps heartwood-cli runtime __version__; prints context in model listings/plans. |
| packages/cli/pyproject.toml | Bumps heartwood-cli package version to beta.2. |
| packages/audit/src/heartwood/audit/init.py | Bumps heartwood-audit runtime __version__. |
| packages/audit/pyproject.toml | Bumps heartwood-audit package version to beta.2. |
| packages/adapters/tests/test_conformance.py | Updates conformance fixture to beta.2 skill version. |
| packages/adapters/src/heartwood/adapters/conformance.py | Updates default conformance skill version to beta.2. |
| packages/adapters/src/heartwood/adapters/init.py | Bumps heartwood-adapters runtime __version__. |
| packages/adapters/pyproject.toml | Bumps heartwood-adapters package version to beta.2. |
| images/platforms.toml | Updates declared Terra GPU runtime pin to vLLM 0.10.1.1+cu118. |
| images/platform/scripts/terra_managed_launch_smoke.sh | Adds Terra managed local-model launch smoke script. |
| images/platform/Dockerfile | Removes conditional FFmpeg install for GPU runtime. |
| images/gpu/vllm.in | Pins vLLM wheel URL and constrains transformers major version. |
| images/gpu/verify_runtime.sh | Verifies PyTorch CUDA build and imports (torch/vLLM). |
| images/gpu/start_vllm.sh | Raises default local-model context env var to 32768. |
| images/generic/scripts/terra_jupyter_demo_smoke.py | Updates Terra demo smoke to validate both service-prefix and Leonardo route forms. |
| images/generic/scripts/start_local_runtime.sh | Raises default llama.cpp context env var to 32768. |
| images/generic/scripts/capable_model_e2e.sh | Raises default local-model context env var to 32768. |
| images/generic/local-runtime/snapshots.toml | Adds 16GB-GPU AWQ recommendation and declares snapshot context_window. |
| images/generic/local-runtime/models/stories260k.toml | Adds artifact context_window. |
| images/generic/local-runtime/models/qwen25-coder-7b-q4_k_m.toml | Renames/standardizes context field to context_window. |
| images/generic/local-runtime/models/qwen25-7b-instruct-q4_k_m.toml | Renames/standardizes context field to context_window. |
| images/generic/Dockerfile | Removes conditional FFmpeg install for GPU runtime. |
| fixtures/synthetic/skills/omop-cohort-summary/SKILL.md | Updates synthetic fixture skill metadata version to beta.2. |
| fixtures/synthetic/skills/omop-cohort-summary/metadata.json | Updates synthetic fixture skill metadata version to beta.2. |
| docs/terra-jupyter-demo.md | Updates Terra runbook (GPU vs CPU images, proxy route requirements, launch flow). |
| docs/terra-jupyter-demo.ipynb | Updates notebook to show full authenticated proxy link and updated guidance. |
| docs/releases.md | Updates release instructions and examples for beta.2. |
| docs/platform-support.md | Updates platform-support evidence statements for beta.2 and new validations. |
| docs/getting-started-offline.md | Documents context window visibility and resource preflight warnings. |
| docs/container-images.md | Updates image tags and NVIDIA runtime description for beta.2. |
| docs/carina-cli.md | Updates Carina installer instructions and removes FFmpeg references. |
| design/03-architecture.md | Documents the unified context/resource contract and local/hosted limit responsibilities. |
| deploy/tests/native_installer_smoke.sh | Removes FFmpeg assertions; improves portable permission checks across stat variants. |
| deploy/carina/environment.yml | Removes FFmpeg dependency from Carina bootstrap environment. |
| deploy/carina/bootstrap.sh | Updates bootstrap verification to import torch/vLLM and report CUDA build. |
| ACRONYMS.md | Adds AWQ acronym definition. |
| .github/workflows/gpu-container-image.yml | Updates runtime-lock validation checks for new CUDA 11.8 pin. |
| .github/workflows/dependency-review.yml | Updates dependency-review commentary for new vLLM/PyTorch versions. |
| .github/workflows/container-smoke.yml | Adds Terra managed local-model launch smoke to container smoke workflow. |
| .github/workflows/container-image.yml | Adds staged Terra managed local-model launch smoke to container-image workflow. |
Files not reviewed (1)
- packages/webui/package-lock.json: Generated file
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/cli/src/heartwood/cli/_launch.py (1)
243-254: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winUnfiltered environment (potential secrets) is passed to the GPU-memory preflight subprocess.
_print_resource_assessment(selection, env)is called with the rawenvbeforeruntime_env(the security-filtered allowlist used everywhere else in this function, e.g. by_preflight_vllm) is computed. For vLLM selections, this raw env flows into_available_gpu_memory_bytes(env)→subprocess.run(..., env=dict(env)), spawningnvidia-smiwith the full unfiltered process environment instead of the allowlisted one — inconsistent with the deliberate secret-scrubbing done for every other subprocess call in this module.🔒 Proposed fix: compute `runtime_env` first and reuse it
- _print_resource_assessment(selection, env) runtime_executable = _resolve_runtime_executable(runtime_kind) if not runtime_executable.is_file() or not os.access(runtime_executable, os.X_OK): print(f"{_runtime_label(runtime_kind)} executable is unavailable: {runtime_executable}") return 69 runtime_env = _runtime_environment(env, project=options.project) + _print_resource_assessment(selection, runtime_env) _stage(2, 6, "Validate the local inference runtime")Consider also adding a regression test that asserts the env passed to the mocked
_available_gpu_memory_bytes/subprocess excludes credential-only variables (the current tests only stub the helper directly, so they wouldn't catch this).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/src/heartwood/cli/_launch.py` around lines 243 - 254, Compute the security-filtered runtime_env before calling _print_resource_assessment in the launch flow, then pass runtime_env to _print_resource_assessment and reuse it for _preflight_vllm. Ensure the GPU-memory assessment and all runtime subprocesses receive only the allowlisted environment, and add a regression test verifying credential-only variables are excluded.
🧹 Nitpick comments (1)
ACRONYMS.md (1)
74-74: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMove the
AWQacronym to theAI and Agent Stackgroup.
AWQ(Activation-aware Weight Quantization) is a model quantization method, so it conceptually belongs in the "AI and Agent Stack" group rather than "Cloud and Infrastructure" (which contains terms like GCP, AWS, VMs, and networking protocols).As per coding guidelines: "add it to ACRONYMS.md with its expansion and a one-line description in the appropriate group."
♻️ Proposed fix
Apply this diff to move
AWQto the appropriate group:--- ACRONYMS.md +++ ACRONYMS.md @@ -71,7 +71,6 @@ |---|---|---| | AMD64 | 64-bit Advanced Micro Devices architecture | x86-64 CPU architecture identifier used by Linux Docker images | | ARM64 | 64-bit Arm architecture | CPU architecture identifier used by Apple Silicon, AWS Graviton, and Arm Linux runners | -| AWQ | Activation-aware Weight Quantization | Model quantization method used to reduce GPU memory requirements while retaining inference quality | | GCP | Google Cloud Platform | Google's cloud | | AWS | Amazon Web Services | Amazon's cloud | | VM | Virtual Machine | A virtualized computer instance | @@ -111,6 +110,7 @@ | Term | Expansion | What it is | |---|---|---| | AI | Artificial Intelligence | Broad term for systems that perform tasks associated with human intelligence | +| AWQ | Activation-aware Weight Quantization | Model quantization method used to reduce GPU memory requirements while retaining inference quality | | LLM | Large Language Model | Model used by the OpenHands conversation runtime | | GGUF | GPT-Generated Unified Format | llama.cpp model artifact format used for local inference profiles | | ML | Machine Learning | Statistical/modeling techniques that learn patterns from data |🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ACRONYMS.md` at line 74, Move the AWQ entry from the Cloud and Infrastructure group to the AI and Agent Stack group in ACRONYMS.md, preserving its expansion and existing one-line description.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@images/gpu/vllm-requirements.txt`:
- Around line 326-327: Update the certifi dependency entry in
vllm-requirements.txt from 2022.12.7 to a patched version at or above 2023.7.22,
and replace its hash with the corresponding hash for the selected release.
- Around line 326-327: Update the certifi dependency resolution associated with
vllm requirements to require a patched version at least 2023.7.22, replacing the
vulnerable 2022.12.7 pin and its hash as needed. Ensure the resulting dependency
files resolve consistently to the secure version.
In `@images/platform/scripts/terra_managed_launch_smoke.sh`:
- Around line 37-74: Update the final log-line check after the readiness loop to
retry until “[6/6] Open the web interface on 127.0.0.1:” appears in log_file,
using a bounded wait consistent with startup_timeout. Preserve the existing
failure behavior if the line is still absent after retries, and keep the success
message unchanged.
In `@packages/detector/src/heartwood/detector/_platforms.py`:
- Around line 89-94: Update the explicit-platform handling in the detector logic
to preserve any evidence already stored in matches by using setdefault and
appending the HEARTWOOD_PLATFORM marker rather than replacing the list. Apply
the same concise evidence-list pattern to the slurm_cluster handling so existing
CARINA evidence remains intact while adding the cluster marker.
In `@packages/gateway/src/heartwood/gateway/_model_snapshots.py`:
- Around line 91-92: Update the context-window validation in the model snapshot
validation method to reject values above the shared 32,768-token maximum,
preferably by reusing the existing shared maximum constant. Preserve the current
minimum-bound check and error behavior for values below 2,048.
- Line 149: Update load_model_snapshot_catalog() to preserve backward
compatibility for heartwood.model-snapshot-catalog.v1 entries missing
context_window by applying the established ModelSnapshot default during loading
instead of requiring the field. Keep explicit context_window values unchanged
and avoid requiring a schema migration for v1 catalogs.
---
Outside diff comments:
In `@packages/cli/src/heartwood/cli/_launch.py`:
- Around line 243-254: Compute the security-filtered runtime_env before calling
_print_resource_assessment in the launch flow, then pass runtime_env to
_print_resource_assessment and reuse it for _preflight_vllm. Ensure the
GPU-memory assessment and all runtime subprocesses receive only the allowlisted
environment, and add a regression test verifying credential-only variables are
excluded.
---
Nitpick comments:
In `@ACRONYMS.md`:
- Line 74: Move the AWQ entry from the Cloud and Infrastructure group to the AI
and Agent Stack group in ACRONYMS.md, preserving its expansion and existing
one-line description.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c96c6672-a68f-409c-aa30-cf7511bf65e3
⛔ Files ignored due to path filters (2)
packages/webui/package-lock.jsonis excluded by!**/package-lock.jsonuv.lockis excluded by!**/*.lock
📒 Files selected for processing (97)
.github/workflows/container-image.yml.github/workflows/container-smoke.yml.github/workflows/dependency-review.yml.github/workflows/gpu-container-image.ymlACRONYMS.mdVERSION.tomldeploy/carina/bootstrap.shdeploy/carina/environment.ymldeploy/tests/native_installer_smoke.shdesign/03-architecture.mddocs/carina-cli.mddocs/container-images.mddocs/getting-started-offline.mddocs/platform-support.mddocs/releases.mddocs/terra-jupyter-demo.ipynbdocs/terra-jupyter-demo.mdfixtures/synthetic/skills/omop-cohort-summary/SKILL.mdfixtures/synthetic/skills/omop-cohort-summary/metadata.jsonimages/generic/Dockerfileimages/generic/local-runtime/models/qwen25-7b-instruct-q4_k_m.tomlimages/generic/local-runtime/models/qwen25-coder-7b-q4_k_m.tomlimages/generic/local-runtime/models/stories260k.tomlimages/generic/local-runtime/snapshots.tomlimages/generic/scripts/capable_model_e2e.shimages/generic/scripts/start_local_runtime.shimages/generic/scripts/terra_jupyter_demo_smoke.pyimages/gpu/start_vllm.shimages/gpu/verify_runtime.shimages/gpu/vllm-requirements.txtimages/gpu/vllm.inimages/platform/Dockerfileimages/platform/scripts/terra_managed_launch_smoke.shimages/platforms.tomlpackages/adapters/pyproject.tomlpackages/adapters/src/heartwood/adapters/__init__.pypackages/adapters/src/heartwood/adapters/conformance.pypackages/adapters/tests/test_conformance.pypackages/audit/pyproject.tomlpackages/audit/src/heartwood/audit/__init__.pypackages/cli/pyproject.tomlpackages/cli/src/heartwood/cli/__init__.pypackages/cli/src/heartwood/cli/_launch.pypackages/cli/tests/test_carina_launch_integration.pypackages/cli/tests/test_launch.pypackages/compliance/pyproject.tomlpackages/compliance/src/heartwood/compliance/__init__.pypackages/compliance/tests/test_container_assets.pypackages/compliance/tests/test_documentation_assets.pypackages/core-adapter/pyproject.tomlpackages/core-adapter/src/heartwood/core_adapter/__init__.pypackages/detector/pyproject.tomlpackages/detector/src/heartwood/detector/__init__.pypackages/detector/src/heartwood/detector/_platforms.pypackages/detector/tests/test_platform_detection.pypackages/fixtures/pyproject.tomlpackages/fixtures/src/heartwood/fixtures/__init__.pypackages/gateway/pyproject.tomlpackages/gateway/src/heartwood/gateway/_gateway.pypackages/gateway/src/heartwood/gateway/_local_models.pypackages/gateway/src/heartwood/gateway/_model_artifacts.pypackages/gateway/src/heartwood/gateway/_model_settings.pypackages/gateway/src/heartwood/gateway/_model_snapshots.pypackages/gateway/src/heartwood/gateway/_openhands_sdk.pypackages/gateway/src/heartwood/gateway/_project_config.pypackages/gateway/tests/test_gateway_contract.pypackages/gateway/tests/test_local_models.pypackages/gateway/tests/test_model_artifacts.pypackages/gateway/tests/test_model_settings.pypackages/gateway/tests/test_model_snapshots.pypackages/gateway/tests/test_openhands_sdk.pypackages/model-policy/pyproject.tomlpackages/model-policy/src/heartwood/model_policy/__init__.pypackages/notebook/pyproject.tomlpackages/notebook/src/heartwood/notebook/__init__.pypackages/notebook/src/heartwood/notebook/_view_model.pypackages/notebook/tests/test_notebook.pypackages/schemas/pyproject.tomlpackages/schemas/src/heartwood/schemas/__init__.pypackages/schemas/tests/test_schema_records.pypackages/session/pyproject.tomlpackages/session/src/heartwood/session/__init__.pypackages/skills/pyproject.tomlpackages/webui/package.jsonpackages/webui/scripts/smoke-jupyter-proxy.cjspackages/webui/src/App.test.tsxpackages/webui/src/client.test.tspackages/webui/src/client.tspackages/webui/src/components/UtilitySheet.tsxpackages/webui/src/e2e/app.spec.tspackages/webui/src/types.tsskills/verified/aggregate-export/SKILL.mdskills/verified/aggregate-export/metadata.jsonskills/verified/baseline-model/SKILL.mdskills/verified/baseline-model/metadata.jsonskills/verified/omop-cohort-summary/SKILL.mdskills/verified/omop-cohort-summary/metadata.json
💤 Files with no reviewable changes (3)
- deploy/carina/environment.yml
- images/generic/Dockerfile
- images/platform/Dockerfile
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
images/gpu/vllm-requirements.txt (1)
986-993: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winUpdate
idnato a secure version.The pinned version of
idna(3.4) contains known vulnerabilities, including a denial of service (DoS) flaw via specially crafted inputs toidna.encode()(GHSA-jjg7-2v4v-x38h and GHSA-65pc-fj4g-8rjx).As indicated by the pipeline failures, please update the dependency resolution to use a patched version. To resolve this cleanly, add a constraint (e.g.,
idna>=3.7) to your upstream manifests likeimages/gpu/vllm.inorimages/gpu/vllm-overrides.txt, and regenerate this lockfile usinguv pip compile.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@images/gpu/vllm-requirements.txt` around lines 986 - 993, Update the upstream dependency constraints for idna to require version 3.7 or newer, then regenerate images/gpu/vllm-requirements.txt with uv pip compile so the lockfile uses the patched version and corresponding hash.Source: Pipeline failures
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@images/gpu/vllm-requirements.txt`:
- Around line 986-993: Update the upstream dependency constraints for idna to
require version 3.7 or newer, then regenerate images/gpu/vllm-requirements.txt
with uv pip compile so the lockfile uses the patched version and corresponding
hash.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: e0865041-43ee-45e4-a226-9f8a1680cc5c
📒 Files selected for processing (33)
.github/workflows/gpu-container-image.ymlACRONYMS.mdREADME.mddeploy/tests/native_installer_smoke.shdesign/03-architecture.mddocs/container-images.mddocs/getting-started.mddocs/platform-support.mddocs/platforms.mddocs/terra-jupyter-demo.ipynbdocs/terra-jupyter-demo.mddocs/web-interface.mdimages/gpu/heartwood_vllm.pyimages/gpu/vllm-overrides.txtimages/gpu/vllm-requirements.txtimages/gpu/vllm.inimages/platform/scripts/terra_managed_launch_smoke.shpackages/cli/src/heartwood/cli/__init__.pypackages/cli/src/heartwood/cli/_launch.pypackages/cli/tests/test_cli.pypackages/cli/tests/test_launch.pypackages/compliance/tests/test_container_assets.pypackages/detector/src/heartwood/detector/_platforms.pypackages/detector/tests/test_platform_detection.pypackages/gateway/src/heartwood/gateway/_local_model_contract.pypackages/gateway/src/heartwood/gateway/_local_models.pypackages/gateway/src/heartwood/gateway/_model_artifacts.pypackages/gateway/src/heartwood/gateway/_model_snapshots.pypackages/gateway/src/heartwood/gateway/_project_config.pypackages/gateway/tests/test_model_artifacts.pypackages/gateway/tests/test_model_snapshots.pypackages/gateway/tests/test_project_config.pyzensical.toml
🚧 Files skipped from review as they are similar to previous changes (15)
- images/gpu/vllm.in
- packages/gateway/tests/test_model_artifacts.py
- packages/gateway/src/heartwood/gateway/_model_artifacts.py
- packages/gateway/src/heartwood/gateway/_model_snapshots.py
- packages/gateway/src/heartwood/gateway/_project_config.py
- packages/detector/src/heartwood/detector/_platforms.py
- deploy/tests/native_installer_smoke.sh
- packages/cli/src/heartwood/cli/init.py
- docs/platform-support.md
- images/platform/scripts/terra_managed_launch_smoke.sh
- images/gpu/heartwood_vllm.py
- docs/container-images.md
- packages/cli/src/heartwood/cli/_launch.py
- packages/cli/tests/test_launch.py
- packages/compliance/tests/test_container_assets.py
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
docs/container-images.md (1)
42-46: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winMake the bind-mounted project writable before the first run.
These examples mount a fresh host directory, but the image runs as
10001:10001. On a normal Linux host,.heartwood/creation and model downloads can fail because$PWDis owned by a different UID. Add a reviewed--user "$(id -u):$(id -g)"mapping or explicit host ACL/ownership preparation to every genericdocker runexample, not only the troubleshooting note.Also applies to: 60-67, 78-90, 94-104, 108-126
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/container-images.md` around lines 42 - 46, Update every generic docker run example in docs/container-images.md, including the command using heartwood serve, to make the bind-mounted project writable on Linux hosts. Add the reviewed --user "$(id -u):$(id -g)" mapping to each applicable docker run invocation, or provide explicit host ownership/ACL preparation before the first run; keep troubleshooting-only examples separate.docs/terra-jupyter-demo.md (1)
45-50: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winShow the two Terra images as alternatives.
“Paste one immutable release image” is followed by a two-line block. Users may paste both tags into a single Terra image field, producing an invalid or ambiguous configuration. Label them as alternatives and show one selected value for each CPU/GPU path.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/terra-jupyter-demo.md` around lines 45 - 50, Update the Terra image instructions around the immutable release image block to clearly present the CPU and GPU tags as alternatives, not values to paste together. Label each image by its CPU/GPU path and instruct users to select exactly one appropriate value for the configured environment.images/platforms.toml (1)
61-61: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick winUse a published vLLM release or a CUDA wheel index
vllm==0.10.1.1+cu118is not a PyPI release, so this pin will not resolve from the default index.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@images/platforms.toml` at line 61, Update the gpu_runtime dependency pin in platforms.toml to reference a published vLLM release available from the default package index, or configure the dependency to use the appropriate CUDA wheel index so the existing CUDA-specific version resolves.
🧹 Nitpick comments (2)
docs/using-heartwood.md (1)
156-159: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueVary the starting words to improve readability.
Three successive bullet points start with the word "Use". Consider varying the starting words to improve the flow of the list.
💡 Proposed refactor
-- Use [Browser and Notebooks](web-interface.md) for visual setup, notebook APIs, Jupyter routing, and shared-session examples. -- Use [Project Files and State](project-state.md) before moving, backing up, or resetting a project. -- Use [Troubleshooting](troubleshooting.md) when readiness, model startup, or an interface does not behave as expected. +- Read [Browser and Notebooks](web-interface.md) for visual setup, notebook APIs, Jupyter routing, and shared-session examples. +- Review [Project Files and State](project-state.md) before moving, backing up, or resetting a project. +- Consult [Troubleshooting](troubleshooting.md) when readiness, model startup, or an interface does not behave as expected.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/using-heartwood.md` around lines 156 - 159, Vary the opening wording of the three consecutive bullets in the documentation list around “Browser and Notebooks,” “Project Files and State,” and “Troubleshooting” so they do not all begin with “Use,” while preserving their links and meanings.Source: Linters/SAST tools
packages/gateway/src/heartwood/gateway/_readiness.py (1)
511-511: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueStrip whitespace when falling back to the default platform home.
If
HEARTWOOD_PLATFORM_HOMEhappens to be set to a string containing only whitespace (e.g." "), the string evaluates as truthy and bypasses the/home/jupyterfallback. Stripping the value before the boolean check provides the same robustness used forHEARTWOOD_GPU_RUNTIMEbelow.💡 Proposed refactor
- persistent_root = Path(env.get("HEARTWOOD_PLATFORM_HOME") or "/home/jupyter").resolve() + persistent_root = Path(env.get("HEARTWOOD_PLATFORM_HOME", "").strip() or "/home/jupyter").resolve()🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/gateway/src/heartwood/gateway/_readiness.py` at line 511, Update the platform-home initialization for persistent_root to strip surrounding whitespace from HEARTWOOD_PLATFORM_HOME before applying the fallback check, so whitespace-only values use /home/jupyter. Keep the existing Path resolution behavior unchanged and align the handling with HEARTWOOD_GPU_RUNTIME.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@docs/container-images.md`:
- Around line 42-46: Update every generic docker run example in
docs/container-images.md, including the command using heartwood serve, to make
the bind-mounted project writable on Linux hosts. Add the reviewed --user "$(id
-u):$(id -g)" mapping to each applicable docker run invocation, or provide
explicit host ownership/ACL preparation before the first run; keep
troubleshooting-only examples separate.
In `@docs/terra-jupyter-demo.md`:
- Around line 45-50: Update the Terra image instructions around the immutable
release image block to clearly present the CPU and GPU tags as alternatives, not
values to paste together. Label each image by its CPU/GPU path and instruct
users to select exactly one appropriate value for the configured environment.
In `@images/platforms.toml`:
- Line 61: Update the gpu_runtime dependency pin in platforms.toml to reference
a published vLLM release available from the default package index, or configure
the dependency to use the appropriate CUDA wheel index so the existing
CUDA-specific version resolves.
---
Nitpick comments:
In `@docs/using-heartwood.md`:
- Around line 156-159: Vary the opening wording of the three consecutive bullets
in the documentation list around “Browser and Notebooks,” “Project Files and
State,” and “Troubleshooting” so they do not all begin with “Use,” while
preserving their links and meanings.
In `@packages/gateway/src/heartwood/gateway/_readiness.py`:
- Line 511: Update the platform-home initialization for persistent_root to strip
surrounding whitespace from HEARTWOOD_PLATFORM_HOME before applying the fallback
check, so whitespace-only values use /home/jupyter. Keep the existing Path
resolution behavior unchanged and align the handling with HEARTWOOD_GPU_RUNTIME.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 80bdfc62-a512-4498-bef0-e2e2a26df406
📒 Files selected for processing (47)
.github/workflows/container-smoke.yml.github/workflows/gpu-container-image.ymlAGENTS.mdREADME.mddeploy/tests/native_installer_smoke.shdesign/01-overview.mddesign/03-architecture.mddocs/README.mddocs/carina-cli.mddocs/container-images.mddocs/deployment.mddocs/getting-started-offline.mddocs/getting-started.mddocs/model-connections.mddocs/platform-support.mddocs/platforms.mddocs/project-state.mddocs/terra-jupyter-demo.ipynbdocs/terra-jupyter-demo.mddocs/troubleshooting.mddocs/using-heartwood.mddocs/web-interface.mdimages/gpu/heartwood_vllm.pyimages/gpu/vllm-overrides.txtimages/gpu/vllm-requirements.txtimages/platform/Dockerfileimages/platform/scripts/terra_image_smoke.shimages/platform/scripts/terra_managed_launch_smoke.shimages/platforms.tomlpackages/adapters/src/heartwood/adapters/platform/terra.pypackages/adapters/tests/test_generic_adapters.pypackages/cli/src/heartwood/cli/__init__.pypackages/cli/src/heartwood/cli/_launch.pypackages/cli/tests/test_cli.pypackages/cli/tests/test_launch.pypackages/compliance/tests/test_container_assets.pypackages/compliance/tests/test_documentation_assets.pypackages/gateway/src/heartwood/gateway/__init__.pypackages/gateway/src/heartwood/gateway/_gateway.pypackages/gateway/src/heartwood/gateway/_jupyter.pypackages/gateway/src/heartwood/gateway/_readiness.pypackages/gateway/tests/test_gateway_contract.pypackages/gateway/tests/test_readiness.pypackages/notebook/src/heartwood/notebook/__init__.pypackages/notebook/src/heartwood/notebook/_view_model.pypackages/notebook/tests/test_notebook.pyzensical.toml
🚧 Files skipped from review as they are similar to previous changes (16)
- images/gpu/vllm-overrides.txt
- .github/workflows/container-smoke.yml
- zensical.toml
- packages/compliance/tests/test_documentation_assets.py
- deploy/tests/native_installer_smoke.sh
- docs/getting-started.md
- .github/workflows/gpu-container-image.yml
- images/platform/Dockerfile
- design/03-architecture.md
- README.md
- docs/platform-support.md
- packages/compliance/tests/test_container_assets.py
- packages/cli/tests/test_launch.py
- docs/terra-jupyter-demo.ipynb
- images/gpu/heartwood_vllm.py
- packages/cli/src/heartwood/cli/_launch.py
1 task
1 task
PSchmiedmayer
added a commit
that referenced
this pull request
Jul 18, 2026
### ♻️ Current Situation & Problem The published Terra GPU image uses a CUDA build that cannot initialize on Terra's current NVIDIA driver baseline, and the browser client does not preserve Terra's full nested Jupyter proxy prefix. Terra workflows also need one consistent project boundary, persistent state location, model-selection contract, and set of resource recommendations across the CLI, notebook, browser interface, and container runtime. Local runtimes currently use inconsistent hard-coded context limits, so the model server, OpenHands, and user interfaces can describe different capacities. This pull request advances #42, #51, #53, and #34. Live validation used only a dedicated synthetic Terra project containing no protected health information. ### ⚙️ Release Notes - Pin the NVIDIA images to a hash-locked vLLM and PyTorch CUDA 11.8 environment, validate CUDA initialization before model loading, and remove the unnecessary TorchCodec and FFmpeg runtime path. - Add a 32,768-token local-model context contract shared by model metadata, project configuration, llama.cpp, vLLM, OpenHands, the CLI, notebook views, and the browser interface. - Treat the current directory below `/home/jupyter` as the Terra project and `.heartwood/` as its persistent state, configuration, model, session, and audit location across every interface. - Share one authenticated Jupyter proxy contract between the gateway, notebook, CLI, and browser. Terra browser links now preserve the complete Leonardo/Jupyter prefix and fail closed when the required Terra metadata is unavailable. - Preserve non-secret Terra platform and image metadata when Heartwood launches child runtimes, while continuing to filter credentials and other sensitive environment values. - Add exact default Terra policy routes for loopback inference and the supported OpenAI and Anthropic APIs. Stanford AI API Gateway and other managed routes remain deployment-configured exact policies; arbitrary endpoints are not implicitly allowed. - Order recommended local models according to the active selection and available runtime. NVIDIA deployments with vLLM prefer Qwen2.5-Coder 7B AWQ, CPU deployments prefer the quantized GGUF profile, and an explicitly selected custom model remains first. - Report conservative RAM and GPU-memory requirements before launch, including warnings when observed resources may be insufficient. - Validate project persistence, runtime metadata, model availability, provider configuration, and proxy prerequisites before the Terra notebook modifies the project or presents an interface link. - Make every generic bind-mount example writable by the current non-root host user and validate that exact project-creation path on AMD64 and ARM64 image candidates. - Prepare version `0.2.0-beta.2` across packages, release instructions, Skills, fixtures, and lockfiles. ### 🧭 Terra Recommendations - For the fastest first demonstration, use the portable Terra image with an approved hosted provider, 8 CPUs, 30 GB RAM, a 50 GB persistent disk, and a 30-minute autopause. - For offline local inference, use the NVIDIA Terra image with 8 CPUs, 48 GB RAM, a 75 GB persistent disk, and a T4-class or newer GPU with at least 16 GB VRAM. Heartwood then prioritizes the compatible 4-bit coder model. - Use CPU-only local inference with 8 CPUs, 32 GB RAM, and a 75 GB persistent disk for portability and validation; generation will be materially slower than the GPU path. - Keep projects in dedicated folders below `/home/jupyter`. Running Heartwood from that folder gives the CLI, notebook, and browser the same workspace and preserves `.heartwood/` across application replacement when the persistent disk is retained. - Retain Terra's currently validated Python base image for this release. Evaluate the newer slim Terra base independently after its Python workflow is available and the Jupyter, Leonardo, kernel, persistence, and image-detection contracts can be revalidated together. ### 📚 Documentation The local-model, container, Terra, Carina, platform-support, release, acronym, and architecture documentation now describes the shared project and state contract, model selection, context and resource requirements, NVIDIA image behavior, provider choices, persistent-disk expectations, authenticated proxy route, and exact limits of the validation evidence. The Terra guide progressively introduces the portable hosted path, GPU-local path, CPU fallback, CLI, notebook, browser interface, audit workflow, and troubleshooting. The example notebook uses the same current-directory project, model discovery, readiness checks, and proxy behavior as the CLI and browser. ### ✅ Testing - 607 Python tests pass with 90.37% total coverage; Ruff and MyPy pass across 103 source files. - Web formatting, ESLint, TypeScript, 62 unit tests, production build, license review, two Playwright workflows, gateway smoke, Jupyter proxy smoke, and the reference-analysis system test pass. - The generic and Carina native installer smoke tests pass for `0.2.0-beta.2`. - Strict and versioned documentation builds, Markdown links, Actionlint, ShellCheck, YAML lint, whitespace validation, REUSE compliance, dependency review, CodeQL, and Gitleaks pass. - The final Terra image passes the Jupyter entrypoint, Leonardo mode, current-directory persistence, OpenHands offline workflow, grouped approval and rejection, audit export, managed web launch, model download, and network-isolated llama.cpp inference smokes. - Generic image candidates run as the current host UID and GID against a fresh bind mount, persist and validate model configuration, and leave `.heartwood/` writable by the host on AMD64 and ARM64. - Pull-request CI builds both generic and Terra NVIDIA image candidates and validates their GPU image contracts. The Terra candidate completes successfully together with the aggregate release-candidate gate. - Live synthetic Terra validation confirms the base image, Heartwood kernel, current-directory project, persistence across pause and resume, model planning and download, T4 discovery, and the authenticated nested proxy route. An actual NVIDIA model interaction remains a final live Terra validation for the immutable beta image because GitHub-hosted pull-request checks can build and inspect the GPU image but cannot reproduce Terra's attached NVIDIA hardware. This pull request does not claim that hardware execution as CI-validated. ### Code of Conduct & Contributing Guidelines By creating and submitting this pull request, you agree to follow our [Code of Conduct](https://github.com/SchmiedmayerLab/.github/blob/main/CODE_OF_CONDUCT.md) and [Contributing Guidelines](https://github.com/SchmiedmayerLab/.github/blob/main/CONTRIBUTING.md): - [x] I agree to follow the [Code of Conduct](https://github.com/SchmiedmayerLab/.github/blob/main/CODE_OF_CONDUCT.md) and [Contributing Guidelines](https://github.com/SchmiedmayerLab/.github/blob/main/CONTRIBUTING.md).
1 task
1 task
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
♻️ Current Situation & Problem
The published Terra GPU image uses a CUDA build that cannot initialize on Terra's current NVIDIA driver baseline, and the browser client does not preserve Terra's full nested Jupyter proxy prefix. Terra workflows also need one consistent project boundary, persistent state location, model-selection contract, and set of resource recommendations across the CLI, notebook, browser interface, and container runtime. Local runtimes currently use inconsistent hard-coded context limits, so the model server, OpenHands, and user interfaces can describe different capacities.
This pull request advances #42, #51, #53, and #34. Live validation used only a dedicated synthetic Terra project containing no protected health information.
⚙️ Release Notes
/home/jupyteras the Terra project and.heartwood/as its persistent state, configuration, model, session, and audit location across every interface.0.2.0-beta.2across packages, release instructions, Skills, fixtures, and lockfiles.🧭 Terra Recommendations
/home/jupyter. Running Heartwood from that folder gives the CLI, notebook, and browser the same workspace and preserves.heartwood/across application replacement when the persistent disk is retained.📚 Documentation
The local-model, container, Terra, Carina, platform-support, release, acronym, and architecture documentation now describes the shared project and state contract, model selection, context and resource requirements, NVIDIA image behavior, provider choices, persistent-disk expectations, authenticated proxy route, and exact limits of the validation evidence. The Terra guide progressively introduces the portable hosted path, GPU-local path, CPU fallback, CLI, notebook, browser interface, audit workflow, and troubleshooting. The example notebook uses the same current-directory project, model discovery, readiness checks, and proxy behavior as the CLI and browser.
✅ Testing
0.2.0-beta.2..heartwood/writable by the host on AMD64 and ARM64.An actual NVIDIA model interaction remains a final live Terra validation for the immutable beta image because GitHub-hosted pull-request checks can build and inspect the GPU image but cannot reproduce Terra's attached NVIDIA hardware. This pull request does not claim that hardware execution as CI-validated.
Code of Conduct & Contributing Guidelines
By creating and submitting this pull request, you agree to follow our Code of Conduct and Contributing Guidelines: