Skip to content

feat: add built-in Dynamo session lineage#348

Merged
rapids-bot[bot] merged 5 commits into
NVIDIA:mainfrom
willkill07:wkk_fix/relay-404-custom-turn-lineage
Jul 2, 2026
Merged

feat: add built-in Dynamo session lineage#348
rapids-bot[bot] merged 5 commits into
NVIDIA:mainfrom
willkill07:wkk_fix/relay-404-custom-turn-lineage

Conversation

@willkill07

@willkill07 willkill07 commented Jul 1, 2026

Copy link
Copy Markdown
Member

Overview

Implements RELAY-404 by making Dynamo session lineage propagation built into managed LLM requests and representing every CLI turn as a Custom scope.

  • I confirm this contribution is my own work, or I have the right to submit it under this project's license.
  • I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

  • Injects x-dynamo-session-id and x-dynamo-parent-session-id automatically from the active Agent scope stack.
  • Applies the behavior to managed and standalone LLM request-intercept paths, including codec-backed requests.
  • Makes every CLI turn a bounded Custom scope while preserving Agent-only lineage and classification.
  • Keeps ATIF trajectory routing aware of top-level Custom turns and updates Node, Rust, CLI, and documentation coverage.

Validation: just test-rust completed all runtime suites but encountered the repository's parallel FFI shared-global race; the full 75-test FFI suite passes serially. Also ran cargo clippy --workspace --all-targets -- -D warnings, just test-node, just test-go, just docs, focused Python tests (93 passed), and pre-commit checks.

Where should the reviewer start?

Start with crates/core/src/api/shared.rs for Agent-only Dynamo lineage injection, then crates/cli/src/session.rs for the universal Custom turn scope. The ATIF top-level Custom-turn routing is in crates/core/src/observability/plugin_component.rs.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • Closes RELAY-404

Summary by CodeRabbit

  • New Features
    • Outgoing managed LLM requests now propagate Dynamo lineage headers for the current agent session, and the parent session when available.
    • Turn-based activity is emitted with distinct “turn” scope semantics to improve observability correlation.
  • Bug Fixes
    • Updated telemetry/export and OpenInference span classification so turn scopes are treated as turns (not agents), aligning stop/contract event matching.
    • Improved Dynamo lineage header selection to correctly handle missing lineage, duplicates, and intervening scope changes.
  • Documentation
    • Added “Before You Start” guidance explaining how agent lineage is derived and which Dynamo headers are propagated.

Signed-off-by: Will Killian <wkillian@nvidia.com>
@willkill07 willkill07 requested review from a team and lvojtku as code owners July 1, 2026 22:24
@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 2b3ca00a-67d8-40e2-80aa-3e8cac7099d6

📥 Commits

Reviewing files that changed from the base of the PR and between 2ee74e0 and 4ae399b.

📒 Files selected for processing (4)
  • crates/cli/tests/coverage/server_tests.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/node/tests/scope_local_tests.mjs
  • docs/instrument-applications/instrument-llm-call.mdx
💤 Files with no reviewable changes (1)
  • crates/core/src/observability/plugin_component.rs
📜 Recent review details
⏰ Context from checks skipped due to timeout. (2)
  • GitHub Check: Check / Run
  • GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (14)
**/*

📄 CodeRabbit inference engine (AGENTS.md)

Keep SPDX headers on source, docs, scripts, and configuration files.

Files:

  • crates/node/tests/scope_local_tests.mjs
  • docs/instrument-applications/instrument-llm-call.mdx
  • crates/cli/tests/coverage/server_tests.rs
**/*.{rs,py,go,ts,js,mjs,cjs,jsx,tsx,c,h}

📄 CodeRabbit inference engine (AGENTS.md)

Follow binding naming conventions: Rust and Python use snake_case, C FFI exports use the nemo_relay_ prefix, Go public APIs use PascalCase, and Node.js uses camelCase.

Files:

  • crates/node/tests/scope_local_tests.mjs
  • crates/cli/tests/coverage/server_tests.rs
crates/node/**

📄 CodeRabbit inference engine (AGENTS.md)

Node.js public entry points include the main runtime package plus nemo-relay-node/typed, nemo-relay-node/plugin, and nemo-relay-node/adaptive.

Files:

  • crates/node/tests/scope_local_tests.mjs
**

⚙️ CodeRabbit configuration file

**:

AGENTS.md

This file provides guidance to agents, including Claude Code and OpenAI Codex, when working in this repository.

Project Overview

NeMo Relay is a multi-language agent runtime framework for execution scopes, lifecycle events, middleware, plugins, and observability around tool and LLM calls. The core runtime is Rust. Primary supported bindings are Rust, Python, and Node.js. Go and the raw C FFI are experimental and source-first.

The shared runtime model is:

  1. Scope stacks decide where work belongs and which scope-local behavior is visible.
  2. Middleware registries decide what guardrails and intercepts run around managed calls.
  3. Plugins install reusable runtime behavior from configuration.
  4. Events record runtime behavior in ATOF form.
  5. Subscribers and exporters consume events in-process or export them to ATIF, OpenTelemetry, OpenInference, or other backends.

Repository Structure

The repository layout separates the Rust runtime, language bindings,
documentation, integrations, and agent-facing skills.

crates/
  core/       # Rust core runtime crate, published as nemo-relay
  adaptive/   # Adaptive runtime primitives and plugin components
  python/     # PyO3 native extension for the Python package
  ffi/        # Raw C ABI layer used by downstream bindings such as Go
  node/       # NAPI Node.js binding and JavaScript/TypeScript entry points
python/
  nemo_relay/  # Python wrapper package: scopes, tools, LLM, middleware, typed helpers, plugins, adaptive helpers
  tests/      # Python tests
go/
  nemo_relay/  # Experimental Go CGo binding and tests
fern/         # Fern documentation site
scripts/      # Stable wrappers and helper scripts; build/test/docs entry points live in justfile
skills/       # Published Codex/agent skills for NeMo Relay usage patterns

Prerequisites

Insta...

Files:

  • crates/node/tests/scope_local_tests.mjs
  • docs/instrument-applications/instrument-llm-call.mdx
  • crates/cli/tests/coverage/server_tests.rs
crates/{python,ffi,node}/**/*

⚙️ CodeRabbit configuration file

crates/{python,ffi,node}/**/*: Treat binding changes as public API changes. Check for parity with the other language bindings, FFI ownership/lifetime safety,
callback error propagation, stable type conversion, and consistent async/stream semantics.
Flag changes that update one binding without corresponding tests or documentation for the same surface elsewhere.

Files:

  • crates/node/tests/scope_local_tests.mjs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}

⚙️ CodeRabbit configuration file

{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.

Files:

  • crates/node/tests/scope_local_tests.mjs
  • crates/cli/tests/coverage/server_tests.rs
**/*.mdx

📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)

MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)

**/*.mdx: In MDX files, top-of-file comments must use JSX comment delimiters: {/* to open and */} to close; do not use HTML comments for MDX SPDX headers.
New or regenerated MDX files must use {/* ... */} for top-of-file SPDX comments.

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
docs/**/*.{md,mdx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

docs/**/*.{md,mdx}: Keep documentation under docs/ valid for Fern tooling and external links, including Markdown/MDX link checks where applicable.
Update relevant reference docs and embedded examples when public behavior, bindings, examples, or workspace structure change.

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
**/*.{rs,go,js,ts,py,html,md,mdx,toml,yml,yaml,c,h}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Include SPDX license headers in all source files using the correct comment syntax for the file type.

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
  • crates/cli/tests/coverage/server_tests.rs
**/*.{md,mdx}

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

**/*.{md,mdx}: Prefer the documented public API rather than internal shortcuts.
Keep package names, repository references, and build commands current.
Update entry-point docs when examples or reading paths change.
Keep release-process and release-notes guidance in repository-maintainer docs such as RELEASING.md, not in user-facing docs pages or CHANGELOG.md.
Keep stable user-facing wrappers at the scripts/ root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work.
When detailed dynamic plugin guides exist, keep Rust native plugin examples, Python worker plugin examples, and grpc-v1 protocol details on separate pages.
Dynamic plugin manifests in docs and examples should use compat.relay = ">=0.5,<1.0" unless deliberately narrower.
Update relevant getting-started or reference docs when the associated behavior or examples change.
Ensure example commands still match current package names and paths.
Dynamic plugin entry pages should link to native, worker, Rust example, Python example, and protocol pages when those pages exist.
When the docs site changes, run just docs; ./scripts/build-docs.sh html remains the compatibility wrapper.

**/*.{md,mdx}: Document native and worker plugins as trusted extensions: native plugins are in-process and unsandboxed, while worker plugins provide process isolation but not a security sandbox.
When detailed dynamic plugin guides exist, keep Rust native, Python worker, and grpc-v1 protocol details on separate pages.

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
docs/**/*.mdx

📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)

Update documentation if activation or usage changes

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
**/*.rs

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

**/*.rs: Any Rust change must run just test-rust
Any Rust change must run cargo fmt --all
Any Rust change must run cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all for all FFI work since it is Rust work
Run just test-rust to validate FFI changes
Run cargo clippy --workspace --all-targets -- -D warnings to enforce strict linting on FFI work

When Rust files changed as part of Go work, also run cargo fmt --all, just test-rust, and cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all when Rust files are changed as part of Node work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files are changed as part of Node work
Run just test-rust when Rust files are changed as part of Node work

**/*.rs: Use cargo fmt (rustfmt defaults) for Rust code changed in the core runtime or Rust-facing API surface.
Run cargo clippy -- -D warnings and keep Rust code warning-free; all warnings are treated as errors.
Use Rust snake_case naming conventions.

Use snake_case for Rust identifiers.

Files:

  • crates/cli/tests/coverage/server_tests.rs
crates/**/*.{rs,py,ts,js,c,h}

📄 CodeRabbit inference engine (AGENTS.md)

crates/**/*.{rs,py,ts,js,c,h}: Use Json = serde_json::Value in Rust-facing runtime APIs where the existing code expects JSON payloads.
Use Result<T> with FlowError in core runtime paths, and keep errors explicit and binding-appropriate at wrapper layers.
Keep async behavior on the existing tokio-based model; bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.

Files:

  • crates/cli/tests/coverage/server_tests.rs
🔇 Additional comments (4)
crates/cli/tests/coverage/server_tests.rs (2)

172-204: LGTM!

Also applies to: 395-457


1835-1835: 🎯 Functional Correctness

No issue: environment_ref belongs on ActiveDynamicPluginComponent.

			> Likely an incorrect or invalid review comment.
crates/node/tests/scope_local_tests.mjs (1)

270-270: LGTM!

Also applies to: 498-501, 705-708, 719-722, 1263-1265

docs/instrument-applications/instrument-llm-call.mdx (1)

35-40: LGTM!


Walkthrough

Turn scopes now use Custom classification, observability routing and coverage expectations follow that turn role, and managed LLM requests receive Dynamo lineage headers derived from Agent scope ancestry.

Changes

Turn scope and observability updates

Layer / File(s) Summary
Turn scope observability
crates/cli/src/alignment/mod.rs, crates/cli/src/session.rs, crates/core/src/observability/plugin_component.rs, crates/core/src/observability/atif.rs, crates/cli/tests/coverage/server_tests.rs, crates/cli/tests/coverage/session_tests.rs
Custom turn scopes are documented, created, routed, and asserted in observability and coverage tests.
Dynamo lineage injection
crates/core/src/api/shared.rs, crates/core/src/api/llm.rs, crates/core/tests/unit/shared_tests.rs, crates/node/tests/scope_local_tests.mjs, docs/instrument-applications/instrument-llm-call.mdx
Dynamo session headers are derived from Agent lineage, injected into outgoing LLM requests, and covered by tests and docs.

Estimated code review effort: 3 (Moderate) | ~25 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title follows Conventional Commits and accurately summarizes the main change.
Description check ✅ Passed The description includes all required template sections with a clear overview, details, review guidance, and related issue reference.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size:M PR is medium Bug issue describes bug; PR fixes bug lang:js PR changes/introduces Javascript/Typescript code lang:rust PR changes/introduces Rust code labels Jul 1, 2026
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

@willkill07 willkill07 changed the title fix: add built-in Dynamo session lineage feat: add built-in Dynamo session lineage Jul 1, 2026
@github-actions github-actions Bot added Feature a new feature and removed Bug issue describes bug; PR fixes bug labels Jul 1, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/core/src/api/shared.rs`:
- Around line 47-84: The parent session resolution in resolve_agent_session_ids
is skipping the nearest explicit Agent ancestor when two scopes resolve to the
same id. Update the logic so the parent is taken directly from the immediate
next Agent scope in the stack iteration, instead of using a search that can skip
duplicates; keep the identifier resolution behavior in agent_scope_id unchanged.

In `@crates/core/tests/unit/shared_tests.rs`:
- Around line 233-358: Add coverage for the codec-active branch in
run_request_intercepts_with_codec, since this test currently only exercises
codec: None. Introduce a case that passes a codec, then assert the Dynamo
lineage headers are still present after the encoded.headers = request.headers
path runs and that any existing codec-populated headers are preserved. Use the
existing test helpers and symbols like run_request_intercepts_with_codec,
DYNAMO_SESSION_ID_HEADER_KEY, and DYNAMO_PARENT_SESSION_ID_HEADER_KEY to keep
the new assertion near the current lifecycle checks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 399c99fc-ef97-4f37-ba68-e69f684fbb01

📥 Commits

Reviewing files that changed from the base of the PR and between cc3e5f1 and 4b5cab9.

📒 Files selected for processing (11)
  • crates/cli/src/alignment/mod.rs
  • crates/cli/src/session.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/cli/tests/coverage/session_tests.rs
  • crates/core/src/api/llm.rs
  • crates/core/src/api/shared.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/tests/unit/shared_tests.rs
  • crates/node/tests/scope_local_tests.mjs
  • docs/instrument-applications/instrument-llm-call.mdx
📜 Review details
🧰 Additional context used
📓 Path-based instructions (31)
**/*.rs

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

Use snake_case naming convention for Rust identifiers (e.g., nemo_relay_tool_call)

**/*.rs: Any Rust change must run just test-rust
Any Rust change must run cargo fmt --all
Any Rust change must run cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all for all FFI work since it is Rust work
Run just test-rust to validate FFI changes
Run cargo clippy --workspace --all-targets -- -D warnings to enforce strict linting on FFI work

When Rust files changed as part of Go work, also run cargo fmt --all, just test-rust, and cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all when Rust files are changed as part of Node work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files are changed as part of Node work
Run just test-rust when Rust files are changed as part of Node work

**/*.rs: Run cargo fmt --all to format all Rust code
Run cargo clippy --workspace --all-targets -- -D warnings to enforce all clippy lints as errors

**/*.rs: Run cargo fmt --all when Rust files changed as part of WebAssembly work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files changed as part of WebAssembly work

**/*.rs: If any Rust code changed, always run just test-rust
If any Rust code changed, also run cargo fmt --all
If any Rust code changed, also run cargo clippy --workspace --all-targets -- -D warnings
Run Rust formatting with cargo fmt --all
Run Rust linting with cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Use cargo fmt for Rust code formatting
Run cargo clippy -- -D warnings to lint Rust code and treat all warnings as errors
Use Rust snake_case naming convention for Rust identifiers
Include SPDX license header in all Rust source files using double-slash comment syntax
Validate Rust code with uv run pre-commit run --all-files to enforce cargo fmt formatting check, cargo clippy lints, and cargo deny aud...

Files:

  • crates/cli/src/alignment/mod.rs
  • crates/cli/src/session.rs
  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
  • crates/cli/tests/coverage/session_tests.rs
**/{Cargo.toml,**/*.rs}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Maintain consistency between Rust package names in Cargo.toml and their actual usage across the codebase

Files:

  • crates/cli/src/alignment/mod.rs
  • crates/cli/src/session.rs
  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
  • crates/cli/tests/coverage/session_tests.rs
**/*.{h,hpp,c,cpp,rs}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Ensure FFI header and library naming follows consistent conventions across platform-specific builds

Files:

  • crates/cli/src/alignment/mod.rs
  • crates/cli/src/session.rs
  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
  • crates/cli/tests/coverage/session_tests.rs
**/*.{rs,toml}

📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)

Update Rust crate names and module prefixes during coordinated rename operations

Files:

  • crates/cli/src/alignment/mod.rs
  • crates/cli/src/session.rs
  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
  • crates/cli/tests/coverage/session_tests.rs
**/*.{rs,py,js,ts,tsx,jsx,go,sh,toml,yaml,yml,md}

📄 CodeRabbit inference engine (AGENTS.md)

Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.

Files:

  • crates/cli/src/alignment/mod.rs
  • crates/cli/src/session.rs
  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
  • crates/cli/tests/coverage/session_tests.rs
**/*.{rs,py,go,js,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Follow binding naming conventions: Rust and Python use snake_case, C FFI exports prefixed nemo_relay_, Go uses PascalCase for public APIs, Node.js uses camelCase.

Files:

  • crates/cli/src/alignment/mod.rs
  • crates/cli/src/session.rs
  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
  • crates/cli/tests/coverage/session_tests.rs
crates/**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

crates/**/*.rs: Keep async behavior on the existing tokio-based model. Bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Use Json = serde_json::Value in Rust-facing runtime APIs for JSON payload handling.

Files:

  • crates/cli/src/alignment/mod.rs
  • crates/cli/src/session.rs
  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
  • crates/cli/tests/coverage/session_tests.rs
**

⚙️ CodeRabbit configuration file

**:

AGENTS.md

This file provides guidance to agents, including Claude Code and OpenAI Codex, when working in this repository.

Project Overview

NeMo Relay is a multi-language agent runtime framework for execution scopes, lifecycle events, middleware, plugins, and observability around tool and LLM calls. The core runtime is Rust. Primary supported bindings are Rust, Python, and Node.js. Go and the raw C FFI are experimental and source-first.

The shared runtime model is:

  1. Scope stacks decide where work belongs and which scope-local behavior is visible.
  2. Middleware registries decide what guardrails and intercepts run around managed calls.
  3. Plugins install reusable runtime behavior from configuration.
  4. Events record runtime behavior in ATOF form.
  5. Subscribers and exporters consume events in-process or export them to ATIF, OpenTelemetry, OpenInference, or other backends.

Repository Structure

The repository layout separates the Rust runtime, language bindings,
documentation, integrations, and agent-facing skills.

crates/
  core/       # Rust core runtime crate, published as nemo-relay
  adaptive/   # Adaptive runtime primitives and plugin components
  python/     # PyO3 native extension for the Python package
  ffi/        # Raw C ABI layer used by downstream bindings such as Go
  node/       # NAPI Node.js binding and JavaScript/TypeScript entry points
python/
  nemo_relay/  # Python wrapper package: scopes, tools, LLM, middleware, typed helpers, plugins, adaptive helpers
  tests/      # Python tests
go/
  nemo_relay/  # Experimental Go CGo binding and tests
fern/         # Fern documentation site
scripts/      # Stable wrappers and helper scripts; build/test/docs entry points live in justfile
skills/       # Published Codex/agent skills for NeMo Relay usage patterns

Prerequisites

Insta...

Files:

  • crates/cli/src/alignment/mod.rs
  • docs/instrument-applications/instrument-llm-call.mdx
  • crates/cli/src/session.rs
  • crates/node/tests/scope_local_tests.mjs
  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
  • crates/cli/tests/coverage/session_tests.rs
{docs/**,README.md,CONTRIBUTING.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

{docs/**,README.md,CONTRIBUTING.md}: For docs-only changes, run targeted checks only if commands, package names, or examples changed. Use just docs for docs-site builds and just docs-linkcheck when links changed
Run docs site build with just docs

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
{docs/**,README.md,CONTRIBUTING.md,**/*.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Run docs link validation with just docs-linkcheck when links change

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
{docs/**,README.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Verify README and docs entry points still match current package names and paths for large or public-facing changes

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
{docs/**,examples/**,README.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Verify examples still run with documented commands for large or public-facing changes

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
{docs/**,README.md,**/Cargo.toml,**/package.json,**/*.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Ensure renamed public surfaces are reflected consistently in manifests and docs for large or public-facing changes

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
**/*.{md,mdx,py,sh,yaml,yml,toml,json}

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

Keep package names, repo references, and build commands current

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
**/*.mdx

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

In MDX files, top-of-file comments must use JSX comment delimiters: {/* to open and */} to close. Do not use HTML comments for MDX SPDX headers.

MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
**/*.{html,md,mdx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Include SPDX license header in HTML and Markdown files using HTML comment syntax

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
docs/**/*.{md,mdx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Update embedded documentation snippets, patch docs, and binding-support notes if examples or supported bindings changed

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
docs/**

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Run just docs or ./scripts/build-docs.sh html to regenerate ignored Fern API reference pages before validation for documentation site changes

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.

Files:

  • docs/instrument-applications/instrument-llm-call.mdx
{crates/adaptive/**,python/nemo_relay/adaptive.py,python/nemo_relay/plugin.py,go/nemo_relay/adaptive/**,go/nemo_relay/!(adaptive)/**,**/node/**,**/wasm/**}

📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)

Keep adaptive surface in sync across crates/adaptive, shared plugin behavior in core and bindings, Python adaptive/plugin wrappers in python/nemo_relay/adaptive.py and python/nemo_relay/plugin.py, Go adaptive helpers under go/nemo_relay/adaptive plus shared plugin helpers in go/nemo_relay, and Node/WebAssembly adaptive helpers and plugin wrappers

Files:

  • crates/node/tests/scope_local_tests.mjs
{crates/adaptive/**,python/nemo_relay/plugin.py,go/nemo_relay/**,**/node/**,**/wasm/**}

📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)

{crates/adaptive/**,python/nemo_relay/plugin.py,go/nemo_relay/**,**/node/**,**/wasm/**}: Maintain consistent plugin lifecycle across all language bindings (Python, Go, Node/WebAssembly, and Rust)
Keep plugin context surfaces aligned across all language implementations

Files:

  • crates/node/tests/scope_local_tests.mjs
crates/{python,ffi,node}/**/*

⚙️ CodeRabbit configuration file

crates/{python,ffi,node}/**/*: Treat binding changes as public API changes. Check for parity with the other language bindings, FFI ownership/lifetime safety,
callback error propagation, stable type conversion, and consistent async/stream semantics.
Flag changes that update one binding without corresponding tests or documentation for the same surface elsewhere.

Files:

  • crates/node/tests/scope_local_tests.mjs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}

⚙️ CodeRabbit configuration file

{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.

Files:

  • crates/node/tests/scope_local_tests.mjs
  • crates/core/tests/unit/shared_tests.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/cli/tests/coverage/session_tests.rs
{crates/adaptive/**/*.rs,**/*test*.{rs,py,go,ts,js},**/*adaptive*test*.{rs,py,go,ts,js},docs/plugins/adaptive/**}

📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)

Maintain documented and tested validation and report behavior for adaptive surfaces

Files:

  • crates/core/tests/unit/shared_tests.rs
  • crates/cli/tests/coverage/server_tests.rs
  • crates/cli/tests/coverage/session_tests.rs
{crates/core,crates/adaptive}/**/*

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

Changes to crates/core or crates/adaptive must run the full language matrix

Files:

  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
crates/core/**/*.rs

📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)

If the change touched crates/core or shared runtime semantics, also use validate-change for broader validation

crates/core/**/*.rs: Use Json = serde_json::Value in Rust-facing runtime APIs where the existing code expects JSON payloads.
Use Result<T> with FlowError in core runtime paths. Keep errors explicit and binding-appropriate at the wrapper layer.

Files:

  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
crates/{core,adaptive}/**

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If crates/core or crates/adaptive changed, run the full matrix across Rust, Python, Go, Node.js, and WebAssembly

Files:

  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
crates/{core,adaptive}/**/*.rs

⚙️ CodeRabbit configuration file

crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.

Files:

  • crates/core/tests/unit/shared_tests.rs
  • crates/core/src/api/llm.rs
  • crates/core/src/observability/plugin_component.rs
  • crates/core/src/observability/atif.rs
  • crates/core/src/api/shared.rs
crates/core/src/api/**/*.rs

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

Implement behavior first in Rust core API modules: crates/core/src/api/ and related core modules such as crates/core/src/api/runtime/, crates/core/src/codec/, or crates/core/src/json.rs

Files:

  • crates/core/src/api/llm.rs
  • crates/core/src/api/shared.rs
crates/core/src/api/{tool,llm}.rs

📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)

Wire the new middleware chain into the execute path in crates/core/src/api/tool.rs or crates/core/src/api/llm.rs at the appropriate pipeline stage

Files:

  • crates/core/src/api/llm.rs
crates/core/src/observability/{atif,otel,openinference}.rs

📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)

When changing event fields in ATIF, OpenTelemetry, or OpenInference observability surfaces, keep the core event model in crates/core/src/observability/atif.rs, crates/core/src/observability/otel.rs, and crates/core/src/observability/openinference.rs in sync

Files:

  • crates/core/src/observability/atif.rs
🔇 Additional comments (11)
crates/cli/src/alignment/mod.rs (1)

612-614: LGTM!

crates/cli/src/session.rs (1)

1164-1194: LGTM!

crates/core/src/observability/plugin_component.rs (1)

15-18: LGTM!

Also applies to: 1315-1336

crates/cli/tests/coverage/server_tests.rs (1)

654-666: LGTM!

Also applies to: 1521-1522

crates/cli/tests/coverage/session_tests.rs (1)

723-756: LGTM!

Also applies to: 1819-1834, 1916-1921, 2966-2975, 2994-3011, 3109-3118, 3138-3156

crates/core/src/observability/atif.rs (1)

3124-3127: 🎯 Functional Correctness

No issue: this helper is a pure extraction of the existing ScopeType::Agent check, and custom turns already follow the fallback path handled by the existing Codex/Hermes ATIF flow.

			> Likely an incorrect or invalid review comment.
crates/core/src/api/shared.rs (2)

13-24: LGTM!


175-185: 🗄️ Data Integrity & Integration

No issue here: LlmCodec::encode implementations currently clone original.headers and do not add codec-specific headers, so this assignment does not discard codec output.

			> Likely an incorrect or invalid review comment.
crates/core/src/api/llm.rs (1)

23-24: LGTM!

Also applies to: 906-927

crates/node/tests/scope_local_tests.mjs (1)

270-270: LGTM!

docs/instrument-applications/instrument-llm-call.mdx (1)

35-42: LGTM!

Comment thread crates/core/src/api/shared.rs
Comment thread crates/core/tests/unit/shared_tests.rs
@willkill07 willkill07 added this to the 0.5 milestone Jul 1, 2026
@willkill07 willkill07 self-assigned this Jul 1, 2026
Signed-off-by: Will Killian <wkillian@nvidia.com>

@mnajafian-nv mnajafian-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall. I left two small inline polish notes around public-contract wording and local comment consistency. Please also clear the remaining Rust Runtime coverage gate before merge.

Comment thread docs/instrument-applications/instrument-llm-call.mdx Outdated
Comment thread crates/cli/src/session.rs
Signed-off-by: Will Killian <wkillian@nvidia.com>

@mnajafian-nv mnajafian-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks

@lvojtku lvojtku left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edits to use active voice. Approved with comments

Comment thread docs/instrument-applications/instrument-llm-call.mdx
willkill07 and others added 2 commits July 2, 2026 18:40
Co-authored-by: lvojtku <lvojtku@nvidia.com>
Signed-off-by: Will Killian <2007799+willkill07@users.noreply.github.com>

@mnajafian-nv mnajafian-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

@willkill07

Copy link
Copy Markdown
Member Author

/merge

@rapids-bot rapids-bot Bot merged commit be167a0 into NVIDIA:main Jul 2, 2026
65 of 67 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature a new feature lang:js PR changes/introduces Javascript/Typescript code lang:rust PR changes/introduces Rust code size:M PR is medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants