feat(vllm): auto-prepare trusted dual DGX Stations - #7030
Conversation
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Senthil Kumar Ravichandran <senthilr@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR adds DGX Station dual-peer qualification, secure SSH and resume state, managed two-node vLLM lifecycle orchestration, onboarding integration, updated documentation, and a guarded fixture-backed simulator with extensive tests. ChangesDual-Station installer and preparation
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in commit eada676 in the TypeScript / code-coverage/cliThe overall coverage in commit eada676 in the Show a code coverage summary of the most impacted files.
Updated |
PR Review Advisor — No blocking findings reportedAdvisor assessment: No blocking advisor findings reported Model lanes
Nemotron output stays in workflow artifacts and does not change the assessment above. E2E guidanceAdvisory only. E2E / PR Gate selects and runs jobs independently. Recommended E2E: 1 optional E2E recommendation
This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
ericksoa
added
platform: dgx-station
Signed-off-by: Senthil Kumar Ravichandran <senthilr@nvidia.com>
…st-prereqs Signed-off-by: Senthil Kumar Ravichandran <senthilr@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com> (cherry picked from commit 586a60d)
|
@ericksoa @apurvvkumaria Before final exact-head qualification, please confirm that the previously accepted Deferred dual-DGX Station boundary applies unchanged to the current v0.0.96 candidate Static CI is green and the canonical PR Review Advisor reports no blockers. The exact-head PR E2E observer is waiting, but its automatic control-plane run did not start. After scope confirmation, a repository maintainer can launch E2E / PR Gate Controller with |
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Factory Station images can provide CDI-backed --gpus support without registering a named nvidia Docker runtime. Verify the actual no-pull GPU container contract instead. The qualified RoCE path uses CUDA DMA-BUF/Data Direct, so it does not require nvidia_peermem. Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Factory Station iproute2 reports the selected source under the JSON from key and omits the redundant device field when route show is already filtered by device. Accept those forms while preserving the direct-route, scope-link, neighbor identity, and jumbo-frame checks. Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Factory Station iproute2 omits the redundant device field when neighbor output is already filtered by device. Accept the omission while still rejecting any emitted device that does not match the qualified rail. Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Summary
Station Express now detects one pretrusted DGX Station GB300 peer across two direct private
/30rails and prepares the qualified pair for distributed Nemotron 3 Ultra inference. Without a qualified pair, Express keeps the existing single-Station Ultra path. The dual-Station path remains Deferred and does not change the single-Station support status. Operators still own rail configuration, network isolation, SSH trust, firewall policy, and reboots.Changes
/30, 400 Gb/s, MTU-9000 CX-8 rails. The installer does not scan subnets or enroll SSH trust.NEMOCLAW_DGX_STATION_PEER,NEMOCLAW_VLLM_MODEL, and--station-deepseekauthoritative and fail closed on conflicting intent.scripts/prepare-dgx-station-host.shon both hosts with exact-byte hashing and strict noninteractive SSH execution.nemotron-ultra. Single-Station Ultra continues to servenvidia/nemotron-3-ultra-550b-a55b.O_NOFOLLOW, and only explicit managed bindings enter managed lifecycle handling.--checkto use sudo only for read-only Docker inventory, reopening SSH after peer login-required status, and binding both controller UIDs after pair qualification.HOMEhas a trailing slash and accept physicaliproute2route JSON that omits redundant filtered fields while retaining exact direct-route, device, scope, MAC, neighbor-state, and jumbo-frame checks.Type of Change
Quality Gates
/30scope applies to the current release remain pending before approval.Documentation Writer Review
no-docs-neededc8998d6f7, the final correction records the already documented qualified dual-Stationnemotron-ultraserved alias in the sealed Station Express intent. Current docs already define that alias and route behavior. The updated behavior-oriented test titles followWRITING.md; no user-facing page or changelog change is required. The previously reviewed dual-Station documentation remains unchanged.DGX Station Hardware Evidence
546e67b9ea0277b755095dfc14151cb0d5a73dfb/30rails. Exact-head preparation, relogin continuation, reciprocal pair qualification, controller binding, worker preparation, CDI validation, and exact-source/OpenShell installation passed.546e67b9through exact-head preparation and continuation. That revision incorrectly classified the public Nemotron Ultra repository as gated and stopped before model or vLLM image download. Current candidate950cb3a59restores the base branch's public-model contract; live Hugging Face metadata reportsprivate=false,gated=false, and the same pinned revision. Exact-head model staging, two-node Ray/vLLM readiness, authenticated routing, agent smoke, restart/reboot recovery, and rollback remain pending.Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailableupstream/main. The preceding exact-head feature evidence includes 542 CLI tests, 139 installer-integration tests, 105 integration and documentation-contract tests, exact Vitest membership across 1,876 files, and docs validation with 0 errors and 2 existing Fern warnings.npm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes —npm test: 1,177 passed and 2 intentional skips before the non-overlappingmainrefresh.npm run docsbuilds without warnings (doc changes only) — build passed with 0 errors and 2 existing Fern warnings.Signed-off-by: Aaron Erickson aerickson@nvidia.com
Signed-off-by: Senthil Ravichandran senthilr@nvidia.com