Skip to content

feat(agents): add environment support to ExecuteAgentJob - #1486

Merged
benmccown merged 4 commits into
mainfrom
environment-spec-job-impl
Aug 25, 2026
Merged

feat(agents): add environment support to ExecuteAgentJob#1486
benmccown merged 4 commits into
mainfrom
environment-spec-job-impl

Conversation

@benmccown

@benmccown benmccown commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Gives ExecuteAgentJob the same AgentEnvironment support that PR #1379 added to AgentDeployment. An execute job can now reference an AgentEnvironment (a "workspace/name" ref or inline); at create time its EnvironmentSpec is merged into the resolved agent config and its ComputeSpec and secret-env references are snapshotted onto the job step. Before, an execute job ran only the bare agent config with no environment dependencies, resources, or secret-backed env vars; after, the run gets the merged config, executor resources from compute, and secret-backed step env vars.

Related Issue

Changes

  • ExecuteAgentJobConfig: new environment: str | AgentEnvironmentInline | None input field.
  • ExecuteAgentStepConfig: new snapshot fields environment (raw request value), compute: ComputeSpecInline | None, and secrets: dict[str, str], mirroring AgentDeployment.
  • to_spec: resolves and merges the referenced environment (reusing the shared environment_resolution helpers), snapshots compute/secrets/raw environment, and validates the merged config so an EnvironmentSpec selecting a non-local provider is rejected. EnvironmentResolutionError is surfaced as a ValueError on the create path (matching the existing "Agent not found" pattern).
  • compile: injects each secret ref as a secret-backed step env var via EnvironmentVariable(from_secret=...), guarded against a reserved set (the NEMO_JOB_*/NMP_TASK_CONFIG job-substrate vars plus the deployment-container reserved names), and maps agents ComputeResources onto the executor ResourcesSpec (cpu/memory pass through; nvidia.com/gpu -> num_gpus; any other resource key is rejected).
  • run: unchanged — Fabric reads the merged config and inherits the substrate-populated process env (MCP secrets resolved via env-var-name indirection).
  • Added unit tests in test_execute_job.py (environment ref/inline snapshot, no-environment empty snapshot, missing-ref -> ValueError, non-local-provider rejection, MCP secret indirection, compile secret env + compute resources, no-compute omits executor resources, unsupported resource key raises, reserved-name collision raises). Also refreshed a pre-existing stale snapshot assertion in the create-route test (the Fabric environment block gained env/connection/metadata defaults in feat(agents): add AgentEnvironment / EnvironmentSpec / ComputeSpec entities #1379; that assertion failed on main independently of this change).
  • Regenerated the nemo-agents plugin OpenAPI spec (make refresh-openapi); the diff is scoped to the three new fields (AgentEnvironmentInline/ComputeSpecInline schemas already existed). The nemo-agents plugin is not part of Stainless SDK generation, so no SDK regen.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with documentation updates
  • Documentation only
  • Contributor tooling or automation
  • CI, build, or test infrastructure

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Documentation updated for user-visible behavior
  • Documentation not applicable — justification: internal job wiring; no user-facing docs surface changed (environment support is not exposed in the hand-written CLI, matching the deployment path).

Verification

  • Pull request title follows the repository's Conventional Commit format
  • Every commit includes an appropriate Signed-off-by: trailer
  • uv run pre-commit run -a passes, or any blocked checks are identified below
  • Targeted tests pass, or tests are marked not applicable above
  • No secrets, API keys, or credentials are included

Targeted validation:

  • uv run ruff check plugins/nemo-agents/src/nemo_agents_plugin/jobs/execute.py plugins/nemo-agents/tests/unit/test_execute_job.py — passed.
  • uv run ruff format --check (same files) — passed (already formatted).
  • uv run --frozen ty check on changed source + test files — passed.
  • bash tools/lint/lint-python-types.sh (full-repo ty check, mirrors CI's lint-python-types) — passed. (Initial CI run flagged step["executor"]["resources"] in a compile test: the step executor is a union of executor TypedDicts and SubprocessExecutionProviderParam has no resources key. Fixed by casting the executor to dict[str, Any] before subscripting, matching the existing step["config"] pattern.)
  • uv run --frozen pytest plugins/nemo-agents/tests/unit/test_execute_job.py --import-mode=importlib -p no:cacheprovider -q — 50 passed.
  • uv run --frozen pytest plugins/nemo-agents/tests/unit --import-mode=importlib -p no:cacheprovider -q — 1335 passed (no regressions).
  • make refresh-openapi — completed; diff scoped to plugins/nemo-agents/openapi/openapi.yaml.
  • uv run pre-commit run -a — relevant hooks passed (ruff, ruff format, ty, config-reference, copyright headers, plugins-not-import-nmp-common, merge-conflict check, uv.lock drift check). Three hooks failed only for local toolchain/environment reasons unrelated to this change and not touching changed files: helm-docs (binary not installed; no Helm changes), uv-lock (local uv 0.9.21 vs required 0.9.14; no dependency changes and the uv.lock drift check passed), and studio-lint-staged (local Node engine mismatch / corepack pnpm download; no web/ changes). CI will re-run these on pinned toolchains.

Summary by CodeRabbit

  • New Features

    • Added optional stored or inline environment configuration for agent execution jobs.
    • Environment settings can include compute resources and secret-backed variables.
    • Configured resources and secrets are applied automatically during execution.
  • Bug Fixes

    • Added validation for unsupported resources and reserved environment names.
    • Invalid reserved-secret configurations now return clear compilation errors or HTTP 422 responses.
    • Environment settings are consistently resolved and preserved for each execution step.

Give ExecuteAgentJob the same AgentEnvironment support that PR #1379 added
to AgentDeployment. An execute job can now reference an AgentEnvironment
(ref or inline); its EnvironmentSpec is merged into the agent config and its
ComputeSpec and secret-env references are snapshotted onto the job step at
creation time.

- ExecuteAgentJobConfig gains an 'environment' input field.
- ExecuteAgentStepConfig snapshots environment (raw), compute, and secrets.
- to_spec resolves + merges the environment (reusing environment_resolution)
  and validates the merged config; EnvironmentResolutionError surfaces as a
  ValueError on create.
- compile injects each secret ref as a secret-backed step env var (guarded
  against reserved job/agent env names) and maps ComputeResources onto the
  executor ResourcesSpec (cpu/memory pass through, nvidia.com/gpu -> num_gpus,
  unsupported keys rejected).

Regenerated the nemo-agents plugin OpenAPI spec.

Signed-off-by: Ben McCown <bmccown@nvidia.com>
@benmccown benmccown self-assigned this Aug 24, 2026
@github-actions github-actions Bot added the feat label Aug 24, 2026
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 35330/45091 78.3% 62.8%
Integration Tests 21051/42866 49.1% 22.4%

The jobs step 'executor' is a union of executor TypedDicts, and
SubprocessExecutionProviderParam has no 'resources' key, so ty's full-repo
check (lint-python-types) flagged step["executor"]["resources"] as an
invalid key. Cast the executor to dict[str, Any] before subscripting,
matching the existing step["config"] pattern in this file.

Signed-off-by: Ben McCown <bmccown@nvidia.com>
The reserved-name secret-env collision and unsupported-compute-resource-key
checks live in ExecuteAgentJob.compile and raised a bare ValueError. The jobs
create route's compile wrapper (_compile_platform_spec) only translates
PlatformJobCompilationError into a 422 — a bare ValueError escaped to the
global handler as an opaque 500 "An unexpected error occurred", unlike the
to_spec validations (missing env ref, non-local provider) which surface as a
descriptive 422.

Wrap the two compile-time snapshot validations and re-raise their ValueError
as PlatformJobCompilationError so both reach the client as a 422 with the
original message. Update the two compile unit tests to expect the new type and
add a route-level test asserting the reserved-name collision maps to 422 at the
HTTP boundary (the existing unit test only checked the raised exception, not
the mapped status code).

Signed-off-by: Ben McCown <bmccown@nvidia.com>
@benmccown
benmccown requested a review from mikeknep August 24, 2026 21:08
@benmccown
benmccown marked this pull request as ready for review August 24, 2026 21:09
@benmccown
benmccown requested review from a team as code owners August 24, 2026 21:09
@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: fdc14e2a-a23f-4139-928c-f2036e7c0d9a

📥 Commits

Reviewing files that changed from the base of the PR and between 69aaf74 and 15f436f.

📒 Files selected for processing (3)
  • plugins/nemo-agents/openapi/openapi.yaml
  • plugins/nemo-agents/src/nemo_agents_plugin/jobs/execute.py
  • plugins/nemo-agents/tests/unit/test_execute_job.py
💤 Files with no reviewable changes (1)
  • plugins/nemo-agents/openapi/openapi.yaml

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Walkthrough

Walkthrough

Execute jobs now accept stored or inline environments. Environment, compute, and secret settings are resolved, snapshotted, compiled into platform steps, and validated through unit and HTTP tests.

Changes

Execute job environment support

Layer / File(s) Summary
Environment contract and resolution
plugins/nemo-agents/openapi/openapi.yaml, plugins/nemo-agents/src/nemo_agents_plugin/jobs/execute.py, plugins/nemo-agents/tests/unit/test_execute_job.py
Job schemas accept stored or inline environments. Step schemas retain resolved compute and secret data. Job specification creation resolves, merges, validates, and snapshots environment settings.
Step resource and secret compilation
plugins/nemo-agents/src/nemo_agents_plugin/jobs/execute.py
Compilation maps CPU, memory, and GPU settings to executor resources and injects secret-backed environment variables. Unsupported resources, reserved names, and invalid GPU counts produce PlatformJobCompilationError.
Resolution and compilation validation
plugins/nemo-agents/tests/unit/test_execute_job.py
Tests cover inline and referenced environments, MCP secrets, resource translation, canonical snapshots, and HTTP 422 responses for reserved environment-name collisions.

Sequence Diagram(s)

sequenceDiagram
  participant ExecuteJob
  participant EntityClient
  participant Executor
  participant PlatformStep
  ExecuteJob->>EntityClient: Resolve environment and compute settings
  EntityClient-->>ExecuteJob: Return environment and secrets
  ExecuteJob->>ExecuteJob: Merge configuration and snapshot settings
  ExecuteJob->>Executor: Compile compute resources
  ExecuteJob->>PlatformStep: Pass executor and environment variables
  PlatformStep-->>ExecuteJob: Return compiled step
Loading

Suggested reviewers: mikeknep

Merge Risk: ⚪ Minimal · up to 15f43

The change adds environment support to execute jobs with validation and resource/secret propagation, and the supplied checks pass; no actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 22.22% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 27 functions across 2 files. (1 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding environment support to ExecuteAgentJob.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 22.22% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 27 functions across 2 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch environment-spec-job-impl

Comment @coderabbitai help to get the list of available commands.

Comment thread plugins/nemo-agents/src/nemo_agents_plugin/jobs/execute.py Outdated
Comment thread plugins/nemo-agents/src/nemo_agents_plugin/jobs/execute.py
@benmccown
benmccown requested a review from mikeknep August 25, 2026 17:03
…tepConfig

ExecuteAgentStepConfig stored a top-level environment field that to_spec set to
exactly request.environment. Since the step config already persists the whole
request: ExecuteAgentJobConfig, that field carried no information not already in
request.environment — the raw-request provenance is already there.

Drop the redundant field and read provenance off request.environment instead.
compute/secrets stay: those are resolved snapshots that do not exist on the raw
request. Unlike AgentDeployment (which keeps a top-level environment because the
entity does not embed the raw request), the job step config already embeds it,
so the field is genuinely redundant here.

Updates the two to_spec tests to assert provenance via request.environment, and
regenerates the nemo-agents OpenAPI spec (diff scoped to the removed field).

Signed-off-by: Ben McCown <bmccown@nvidia.com>
@benmccown
benmccown added this pull request to the merge queue Aug 25, 2026
Merged via the queue into main with commit 868e265 Aug 25, 2026
60 checks passed
@benmccown
benmccown deleted the environment-spec-job-impl branch August 25, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants