chore(dynamo-platform): coordinate NATS removal with Dynamo 1.4+ bump #1983
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,40 +1,54 @@ | ||
| # EKS Dynamo Networking Prerequisites | ||
|
|
||
| For `*-eks-ubuntu-inference-dynamo` recipes, AICR configures | ||
| `dynamo-platform` with Kubernetes-native discovery. As of the Dynamo 1.4+ | ||
| bump, AICR no longer installs bundled NATS by default: the request plane | ||
| defaults to TCP and the KV event plane defaults to ZMQ | ||
| (`ai-dynamo/dynamo#11951`). This removes the old `4222` NATS requirement, | ||
| but it does **not** remove the underlying cross-nodegroup networking | ||
| requirement — the request plane and KV events are now **direct | ||
| frontend↔worker pod-to-pod connections** instead of both sides talking to a | ||
| `dynamo-platform-nats` StatefulSet on the system nodegroup, and Frontend | ||
| pods still run on the system nodegroup while workers run on the GPU | ||
| nodegroup, so traffic still crosses the same GPU↔system nodegroup SG | ||
| boundary as before. | ||
|
|
||
| > **TODO before merging (tracked in NVIDIA/aicr#1836):** the port(s) below | ||
| > are not yet confirmed against a real Dynamo 1.4+ EKS deployment. What's | ||
| > known from the AICR recipes: the ZMQ KV-event endpoint is set explicitly | ||
| > per worker via `--kv-events-config`, e.g. | ||
| > `{"enable_kv_cache_events":true,"publisher":"zmq","endpoint":"tcp://*:5557"}` | ||
| > (see `tests/manifests/dynamo-vllm-smoke-test.yaml`), offset by `+dp_rank` | ||
| > for dp_rank > 0. The TCP request plane does not have one fixed, | ||
| > documented port the way NATS had `4222` — confirm the actual listening | ||
| > port(s) on a live cluster before finalizing the SG rule below: | ||
| > ```shell | ||
| > kubectl exec -n dynamo-system <frontend-pod> -- ss -tlnp | ||
| > kubectl exec -n dynamo-system <worker-pod> -- ss -tlnp | ||
| > ``` | ||
|
|
||
| If the GPU and system node groups sit in different security groups, these | ||
| ports may be blocked from GPU nodes to the frontend's node (and vice versa). | ||
| Typical symptoms: | ||
| - Dynamo frontend and vLLM worker pods stuck in `CrashLoopBackOff`, or a | ||
| frontend that starts cleanly but never successfully routes a request | ||
| through to a worker | ||
| - Worker startup probes failing with `connection refused` because the | ||
| process exits before serving | ||
| - The `inference-perf` performance validator failing after its | ||
| workload-readiness (10 min) and health (5 min) gates lapse — roughly | ||
| 15 min — while `deployment` and `conformance` pass; the workload never | ||
| reaches a ready state | ||
|
|
||
| You can confirm reachability directly from a GPU node before re-running. The | ||
| toleration is required because the GPU node groups on these clusters are | ||
| tainted (`NoSchedule`/`NoExecute`); without it the probe pod stays `Pending` | ||
| and never runs: | ||
|
|
||
| ```shell | ||
| kubectl run tcp-probe --rm -i --restart=Never --image=busybox:1.36 \ | ||
| --overrides='{"spec":{"nodeSelector":{"<gpu-node-label-key>":"<value>"},"tolerations":[{"operator":"Exists"}]}}' \ | ||
| -- sh -c 'nc -zv -w 5 <worker-pod-ip-or-svc> <PORT>' | ||
|
Comment on lines
43
to
+51
Contributor
There was a problem hiding this comment. 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- target file structure ---'
ast-grep outline docs/integrator/eks-dynamo-networking.md --lang markdown || true
printf '%s\n' '--- target lines and surrounding content ---'
sed -n '1,120p' docs/integrator/eks-dynamo-networking.md
printf '%s\n' '--- related networking and probe references ---'
rg -n -i -C 3 'GPU|system|nodegroup|node group|request port|event port|security.?group|tcp-probe|worker endpoint|frontend|worker' docs/integrator docs README.md 2>/dev/null | head -300Repository: NVIDIA/aicr Length of output: 32806 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- relevant tracked files ---'
git ls-files | rg -i 'dynamo|eks|manifest|validator|prometheus' | head -250
printf '%s\n' '--- Dynamo deployment and scheduling definitions ---'
rg -n -i -C 4 'dynamo-platform|frontend|worker|nodeSelector|toleration|tolerations|kv-events-config|5557|request.*port|service.*port|port:' recipes tests pkg manifests components 2>/dev/null | head -500
printf '%s\n' '--- exact target documentation line numbers ---'
nl -ba docs/integrator/eks-dynamo-networking.md | sed -n '1,115p'Repository: NVIDIA/aicr Length of output: 48555 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- smoke-test manifest ---'
sed -n '35,100p' tests/manifests/dynamo-vllm-smoke-test.yaml
printf '%s\n' '--- Dynamo component values and EKS overlays ---'
sed -n '1,220p' recipes/components/dynamo-platform/values.yaml
for f in recipes/overlays/gb200-eks-ubuntu-inference-dynamo.yaml recipes/overlays/h100-eks-ubuntu-inference-dynamo.yaml recipes/overlays/rtx-pro-6000-eks-ubuntu-inference-dynamo.yaml recipes/overlays/eks-inference.yaml; do
if [ -f "$f" ]; then
printf '\n--- %s ---\n' "$f"
sed -n '1,220p' "$f"
fi
done
printf '%s\n' '--- focused diff for the documentation ---'
git diff -- docs/integrator/eks-dynamo-networking.md
printf '%s\n' '--- exact target line numbers ---'
awk '{printf "%5d %s\n", NR, $0}' docs/integrator/eks-dynamo-networking.md | sed -n '35,70p'Repository: NVIDIA/aicr Length of output: 18517 🏁 Script executed: #!/bin/bash
set -e
python3 - <<'PY'
from pathlib import Path
import re, json
text = Path("docs/integrator/eks-dynamo-networking.md").read_text()
block = re.search(r"```shell\n(.*?)\n```", text, re.S).group(1)
override = re.search(r"--overrides='([^']+)'", block).group(1)
spec = json.loads(override)
target = re.search(r"nc -zv -w 5 (\S+) (\S+)", block).groups()
assert spec["spec"]["nodeSelector"] == {
"<gpu-node-label-key>": "<value>"
}
assert target[0] == "<worker-pod-ip-or-svc>"
print("probe node selector:", spec["spec"]["nodeSelector"])
print("probe target:", target[0])
print("probe port:", target[1])
print("result: source is constrained to the GPU nodegroup and destination is constrained to a worker endpoint; this does not exercise a GPU-to-system path")
PYRepository: NVIDIA/aicr Length of output: 281 🏁 Script executed: #!/bin/bash
set -e
python3 - <<'PY'
from pathlib import Path
import json, re
text = Path("docs/integrator/eks-dynamo-networking.md").read_text()
start = text.index("kubectl run tcp-probe")
block = text[start:text.index("```", start)]
override = re.search(r"--overrides='([^']+)'", block).group(1)
spec = json.loads(override)
target = re.search(r"nc -zv -w 5 (\S+) (\S+)", block).groups()
print("probe node selector:", spec["spec"]["nodeSelector"])
print("probe tolerations:", spec["spec"]["tolerations"])
print("probe target:", target[0])
print("probe port:", target[1])
assert "<gpu-node-label-key>" in spec["spec"]["nodeSelector"]
assert target[0] == "<worker-pod-ip-or-svc>"
print("result: the documented source is constrained to the GPU nodegroup and the documented destination is a worker endpoint; this is not a GPU-to-system probe")
PYRepository: NVIDIA/aicr Length of output: 460 Add cross-nodegroup probes. The current probe runs on the GPU nodegroup and targets a worker endpoint, so it does not test GPU-to-system or system-to-GPU traffic. After identifying the actual request and KV-event listeners, run probes from each nodegroup to an endpoint in the other nodegroup. 🤖 Prompt for AI Agents |
||
| ``` | ||
|
|
||
| The conformance validator's `ai-service-metrics` check adds a third requirement: | ||
|
|
@@ -64,8 +78,9 @@ SG rule below remains the reliable cluster-side guarantee. | |
|
|
||
| ## Required Security Group Rules | ||
|
|
||
| Allow ingress from the GPU node security group to the system node security | ||
| group on: | ||
| - TCP `<PORT>` - Dynamo request plane + KV events (dynamo-platform) — confirm exact port(s) on-cluster, see TODO above | ||
|
Comment on lines
+81
to
+83
Contributor
There was a problem hiding this comment. 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n -C 5 \
'authorize-security-group-ingress|<PORT>|5557|frontend|worker' \
docs/integrator/eks-dynamo-networking.md tests recipesRepository: NVIDIA/aicr Length of output: 50370 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- networking document ---'
sed -n '1,130p' docs/integrator/eks-dynamo-networking.md
printf '%s\n' '--- relevant endpoint and port declarations ---'
rg -n -C 4 \
--glob 'dynamo-vllm-smoke-test.yaml' \
--glob 'phases.sh' \
--glob 'eks-dynamo-networking.md' \
'kv-events-config|endpoint|port|PORT|frontend|worker|DYN_ROUTER_MODE' \
docs tests
printf '%s\n' '--- all Dynamo-related networking declarations ---'
rg -n -C 3 \
'dynamo|NATS|ZMQ|5557|4222|fromPort|toPort|authorize-security-group-ingress' \
docs tests recipesRepository: NVIDIA/aicr Length of output: 50370 🌐 Web query:
💡 Result: In the ai-dynamo Dynamo 1.4 architecture, these terms relate to the system's KV (Key-Value) cache routing and inter-component messaging mechanisms [1][2][3]. Dynamo uses a TCP-based request plane for communication between frontend and worker components [1][4]. For KV-aware routing, worker engines (such as vLLM or SGLang) publish raw KV cache events—which track cache state across data parallel ranks—over a ZMQ (ZeroMQ) socket [3][5]. The specific technical details are as follows: - ZMQ Port 5557: This is a standard default ZMQ endpoint used by Dynamo workers for publishing KV events [3][6]. Engines like vLLM are configured to publish these events to Citations:
🌐 Web query:
💡 Result: In the NVIDIA Dynamo framework, the request plane transport layer manages communication between components such as the frontend, router, and workers [1][2]. When using the default TCP request plane, the worker port is dynamically assigned by the operating system (using port 0 by default) to avoid port conflicts [1][3]. If a specific, static port is required—for example, to accommodate firewall rules—you can explicitly set the port by configuring the Citations:
🌐 Web query:
💡 Result: The term 5557 in the context of KV (Key-Value) events refers to the default TCP port used for ZMQ (ZeroMQ) PUB/SUB communication in systems like Dynamo, vLLM, and SGLang [1][2][3]. Key details regarding this setup include: 1. ZMQ PUB Port: LLM engines often utilize a ZMQ PUB socket (e.g., tcp://*:5557) to stream KV cache events, such as block stored, removed, or cleared events, to subscribers [1][2][4]. 2. Payload Structure: The messages published over this channel are typically formatted as a Msgpack payload consisting of a timestamp, a list of events, and a data parallel rank (dp_rank) [5][1]. 3. Data Parallel Rank (dp_rank): This field identifies the data parallel rank associated with the event [1]. In multi-rank configurations, engines may offset the base port (e.g., 5557) by the dp_rank to multiplex events from different ranks or handle them via distinct subscribers [2][6]. 4. Reliability: Because ZMQ PUB/SUB is inherently lossy, these systems often implement a supplementary mechanism, such as a secondary ROUTER socket or a sequence-number-based replay buffer, to allow consumers to detect gaps and request missing event data [2][4]. Citations:
🌐 Web query:
💡 Result: The Citations:
🏁 Script executed: #!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import re
doc = Path("docs/integrator/eks-dynamo-networking.md").read_text()
manifest = Path("tests/manifests/dynamo-vllm-smoke-test.yaml").read_text()
uat = Path("tests/uat/lib/phases.sh").read_text()
rules = re.findall(
r'(?m)^- TCP (\S+) - (.+)$',
doc,
)
aws = re.findall(
r'(?ms)aws ec2 authorize-security-group-ingress --group-id (\S+) \\\n'
r'\s+--protocol tcp --port (\S+) --source-group (\S+)',
doc,
)
endpoint = re.findall(r'"endpoint":"([^"]+)"', manifest)
placements = {
"frontend": bool(re.search(r'(?ms)- name: Frontend.*?type: frontend', manifest)),
"worker": bool(re.search(r'(?ms)- name: VllmDecodeWorker.*?type: worker', manifest)),
"worker_gpu_selector": bool(re.search(
r'(?ms)- name: VllmDecodeWorker.*?nodeSelector:', manifest
)),
}
print("document_ingress_rules:", rules)
print("document_aws_ingress_rules:", aws)
print("manifest_kv_endpoints:", endpoint)
print("manifest_worker_endpoint_binds_all_interfaces:",
any(e.startswith("tcp://*:") for e in endpoint))
print("manifest_component_shapes:", placements)
print("manifest_runtime_versions:",
sorted(set(re.findall(r'vllm-runtime:([^\s]+)', manifest))))
print("uat_worker_endpoint_present:",
bool(re.search(r'--kv-events-config.*?5557', uat, re.S)))
print("uat_runtime_versions:",
sorted(set(re.findall(r'vllm-runtime:([^\s]+)', uat))))
PY
printf '%s\n' '--- upstream request-plane port statements ---'
curl -fsSL https://docs.nvidia.com/dynamo/design-docs/communication-planes/request-plane \
| grep -Eio '.{0,100}(DYN_TCP_RPC_PORT|port 0|dynamic|worker port).{0,160}' \
| head -20 || trueRepository: NVIDIA/aicr Length of output: 879 🏁 Script executed: #!/bin/bash
set -euo pipefail
for url in \
https://docs.nvidia.com/dynamo/v-0-9-1/design-docs/request-plane \
https://docs.nvidia.com/dynamo/v-0-9-0/design-docs/request-plane \
https://raw.githubusercontent.com/ai-dynamo/dynamo/main/docs/integrations/kv-events-custom-engines.md \
https://raw.githubusercontent.com/ai-dynamo/dynamo/main/docs/components/router/kv-event-replay-comparison.md
do
printf '\n--- %s ---\n' "$url"
curl -fsSL "$url" \
| grep -Ein -C 3 \
'DYN_TCP_RPC_PORT|port 0|dynamic|listen|bind|connect|5557|endpoint|subscriber|publisher' \
| head -120 || true
doneRepository: NVIDIA/aicr Length of output: 50368 Add direction-specific, per-port security-group rules. The current 🤖 Prompt for AI Agents |
||
| - TCP `9090` - Prometheus (required for the `ai-service-metrics` conformance check) | ||
|
|
||
| The `9090` rule is required as a fallback guarantee: the orchestrator *prefers* | ||
|
|
@@ -94,9 +109,9 @@ aws ec2 describe-instances \ | |
| --query "Reservations[0].Instances[0].SecurityGroups[*].GroupId" \ | ||
| --output text | ||
|
|
||
| # 2) Allow Dynamo request/event-plane + Prometheus from GPU SG -> system SG | ||
| aws ec2 authorize-security-group-ingress --group-id <system-sg-id> \ | ||
| --protocol tcp --port <PORT> --source-group <gpu-sg-id> | ||
|
|
||
| aws ec2 authorize-security-group-ingress --group-id <system-sg-id> \ | ||
| --protocol tcp --port 9090 --source-group <gpu-sg-id> | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,11 +20,11 @@ | |
| # serving Qwen/Qwen3-0.6B (public, ungated, ~1.2GB). | ||
| # No HuggingFace token required. | ||
| # | ||
| # Uses the Dynamo 1.4+ Kubernetes path: Kubernetes-native discovery, with | ||
| # the request plane on TCP (default) and the KV event plane on ZMQ | ||
| # (default) — no bundled NATS relay in between. The frontend runs | ||
| # KV-cache-aware routing; the local vLLM engine publishes KV events over | ||
| # ZMQ and the KV router consumes them directly, end-to-end. | ||
|
Comment on lines
+23
to
+27
Contributor
There was a problem hiding this comment. 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n -C 5 \
'vllm-runtime:|DYN_(REQUEST|EVENT)_PLANE|kv-events-config|global\.nats\.install|defaultVersion' \
recipes testsRepository: NVIDIA/aicr Length of output: 35541 🏁 Script executed: set -euo pipefail
printf '%s\n' '--- smoke manifest ---'
cat -n tests/manifests/dynamo-vllm-smoke-test.yaml | sed -n '1,110p'
printf '%s\n' '--- Dynamo platform values ---'
cat -n recipes/components/dynamo-platform/values.yaml | sed -n '1,90p'
printf '%s\n' '--- related transport references ---'
rg -n -C 4 'DYN_REQUEST_PLANE|DYN_EVENT_PLANE|NATS_SERVER|kv-events-config|vllm-runtime|Dynamo 1\.4|ZMQ|NATS' \
recipes tests demos docs README.md 2>/dev/null || true
printf '%s\n' '--- repository state summary ---'
git status --short
git diff --statRepository: NVIDIA/aicr Length of output: 33983 🌐 Web query:
💡 Result: NVIDIA Dynamo is a distributed inference framework [1][2]. The vllm-runtime 1.2.1 refers to a specific version of the containerized environment used to run vLLM-based inference within the Dynamo ecosystem [3][4]. The parameters mentioned relate to Dynamo's communication planes, which define how different components (frontend, router, worker) interact [5][6]. 1. DYN_REQUEST_PLANE: This environment variable configures the request distribution transport, determining how RPC requests flow between the frontend, router, and workers [5][7]. - Valid values: tcp (default, optimized for speed), nats [5][7]. - Can be set via the --request-plane CLI flag [7][8]. 2. DYN_EVENT_PLANE: This environment variable configures the pub/sub layer used for near real-time event exchange, such as KV cache updates, worker load metrics, and sequence tracking [6]. - Valid values: nats (default), zmq [6][7]. - Can be set via the --event-plane CLI flag [7][8]. 3. NATS and ZMQ: These are the underlying messaging technologies used to implement the communication planes [6][9]. - NATS (used for both request and event planes) is a messaging system that provides durable messaging (JetStream) and is often used for events and distributed synchronization [5][6]. - ZMQ (ZeroMQ) is an alternative transport option specifically for the event plane, which does not require an external server, as workers dynamically discover each other [6]. In summary, these settings allow users to tune Dynamo's distributed architecture to their specific infrastructure requirements (e.g., choosing between ZMQ for easier, serverless event communication or NATS for more robust, durable messaging) [5][6][2]. Citations:
🌐 Web query:
💡 Result: In the Dynamo distributed inference framework, DYN_REQUEST_PLANE and DYN_EVENT_PLANE are environment variables used to configure separate communication channels for system operations [1][2]. DYN_REQUEST_PLANE This variable controls the transport mechanism for RPC requests moving between components (e.g., frontend, router, and worker) [1]. - Valid values: tcp (default) and nats [3][4][5]. - The tcp mode is considered the fastest and requires minimal infrastructure [1]. - The nats mode uses a NATS message broker for request distribution [1][6]. DYN_EVENT_PLANE This variable specifies the transport mechanism for distributing KV cache events and router synchronization messages [1]. - Valid values: zmq and nats [3][4][5]. - If left unset, the default behavior depends on the discovery backend: - Local backends (file, mem) default to zmq [7][8]. - Distributed backends (etcd, kubernetes) default to nats [7][8]. - The zmq option allows the router to function without external services like NATS or etcd [2]. - When using nats, the system uses a NATS server (defaulting to localhost:4222) for event publishing [1][2]. These two planes operate independently; for example, a system can be configured with a tcp request plane while using a nats or zmq event plane [1][2]. Citations:
🏁 Script executed: set -euo pipefail
python3 - <<'PY'
import json
import urllib.request
urls = [
"https://api.github.com/repos/ai-dynamo/dynamo/tags?per_page=100",
"https://raw.githubusercontent.com/ai-dynamo/dynamo/main/lib/runtime/src/distributed.rs",
"https://raw.githubusercontent.com/ai-dynamo/dynamo/main/lib/runtime/src/config/environment_names.rs",
"https://raw.githubusercontent.com/ai-dynamo/dynamo/main/components/src/dynamo/frontend/frontend_args.py",
"https://raw.githubusercontent.com/ai-dynamo/dynamo/main/docs/design-docs/event-plane.md",
"https://raw.githubusercontent.com/ai-dynamo/dynamo/main/docs/components/router/router-operations.md",
]
for url in urls:
print(f"\n--- {url} ---")
try:
with urllib.request.urlopen(url, timeout=20) as response:
data = response.read().decode()
if url.endswith("/tags?per_page=100"):
tags = json.loads(data)
print("\n".join(t["name"] for t in tags if "1.4" in t["name"] or "1.3" in t["name"] or "1.2" in t["name"] or "1.5" in t["name"]))
else:
for i, line in enumerate(data.splitlines(), 1):
if any(x in line for x in ("DYN_EVENT_PLANE", "DYN_REQUEST_PLANE", "event_plane", "request_plane", "kubernetes", "zmq", "nats", "default")):
print(f"{i}:{line}")
except Exception as exc:
print(type(exc).__name__, exc)
PY
printf '%s\n' '--- version and release references in repository ---'
rg -n -C 3 '1\.2\.1|1\.4\+|DYN_EVENT_PLANE|DYN_REQUEST_PLANE|event plane defaults|Kubernetes.*NATS|Kubernetes.*ZMQ' \
. --glob '!vendor/**' --glob '!node_modules/**' --glob '!build/**' --glob '!dist/**' 2>/dev/null || trueRepository: NVIDIA/aicr Length of output: 27985 🌐 Web query:
💡 Result: In NVIDIA Dynamo, the event plane transport used for near real-time event exchange (such as KV cache updates and worker load metrics) is determined by the configured discovery backend [1][2]. For deployments using Kubernetes as the discovery backend (which the Dynamo operator automatically configures for Kubernetes environments), the default event plane transport is NATS [2][3]. Key details regarding Dynamo event plane configuration: Transport Selection Logic: When the event plane transport ( Citations:
🌐 Web query:
💡 Result: Dynamo version 1.2.1 refers to a release of the NVIDIA Dynamo vLLM Runtime, a containerized framework designed for distributed large language model (LLM) inference [1][2]. As of August 2, 2026, version 1.2.1 is identified as the latest stable release tag in the NVIDIA NGC catalog for the vLLM runtime environment [1][3]. There are also experimental development builds under the 1.3.0 series (e.g., 1.3.0-cosmos3-dev.1) available [2][4]. Citations:
🏁 Script executed: set -euo pipefail
python3 - <<'PY'
import json
import urllib.request
urls = [
"https://api.github.com/repos/ai-dynamo/dynamo/tags?per_page=100",
"https://raw.githubusercontent.com/ai-dynamo/dynamo/main/lib/runtime/src/distributed.rs",
"https://raw.githubusercontent.com/ai-dynamo/dynamo/main/lib/runtime/src/config/environment_names.rs",
"https://raw.githubusercontent.com/ai-dynamo/dynamo/main/components/src/dynamo/frontend/frontend_args.py",
"https://raw.githubusercontent.com/ai-dynamo/dynamo/main/docs/design-docs/event-plane.md",
"https://raw.githubusercontent.com/ai-dynamo/dynamo/main/docs/components/router/router-operations.md",
]
for url in urls:
print(f"\n--- {url} ---")
try:
with urllib.request.urlopen(url, timeout=20) as response:
data = response.read().decode()
if url.endswith("/tags?per_page=100"):
tags = json.loads(data)
print("\n".join(t["name"] for t in tags if any(v in t["name"] for v in ("1.2", "1.3", "1.4", "1.5"))))
else:
for i, line in enumerate(data.splitlines(), 1):
if any(x in line for x in ("DYN_EVENT_PLANE", "DYN_REQUEST_PLANE", "event_plane", "request_plane", "kubernetes", "zmq", "nats", "default")):
print(f"{i}:{line}")
except Exception as exc:
print(type(exc).__name__, exc)
PY
printf '%s\n' '--- version and release references in repository ---'
rg -n -C 3 '1\.2\.1|1\.4\+|DYN_EVENT_PLANE|DYN_REQUEST_PLANE|event plane defaults|Kubernetes.*NATS|Kubernetes.*ZMQ' \
. --glob '!vendor/**' --glob '!node_modules/**' --glob '!build/**' --glob '!dist/**' 2>/dev/null || trueRepository: NVIDIA/aicr Length of output: 27985 🏁 Script executed: set -euo pipefail
fetch() {
url="$1"
printf '\n--- %s ---\n' "$url"
curl -ksSfL --max-time 30 "$url"
}
fetch 'https://api.github.com/repos/ai-dynamo/dynamo/tags?per_page=100' |
python3 -c '
import json, sys
for tag in json.load(sys.stdin):
name = tag["name"]
if any(x in name.lower() for x in ("1.2", "1.3", "1.4", "1.5")):
print(name, tag["commit"]["sha"])
'
for path in \
lib/runtime/src/distributed.rs \
lib/runtime/src/config/environment_names.rs \
components/src/dynamo/frontend/frontend_args.py \
docs/design-docs/communication-planes/event-plane.md \
docs/design-docs/event-plane.md \
docs/components/router/router-operations.md
do
fetch "https://raw.githubusercontent.com/ai-dynamo/dynamo/main/$path" 2>/dev/null |
rg -n -C 3 'DYN_EVENT_PLANE|DYN_REQUEST_PLANE|event_plane|request_plane|kubernetes|zmq|nats|default' || true
done
printf '\n--- upstream release references ---\n'
curl -ksSfL --max-time 30 \
'https://api.github.com/repos/ai-dynamo/dynamo/releases?per_page=100' |
python3 -c '
import json, sys
for release in json.load(sys.stdin):
name = release.get("name") or ""
tag = release.get("tag_name") or ""
if any(x in (name + " " + tag).lower() for x in ("1.2", "1.3", "1.4", "1.5")):
print(tag, name)
'Repository: NVIDIA/aicr Length of output: 755 Set the Dynamo event plane explicitly. With Kubernetes discovery, an unset 🤖 Prompt for AI AgentsSource: Learnings |
||
| # | ||
| # Usage: | ||
| # kubectl apply -f tests/manifests/dynamo-vllm-smoke-test.yaml -n dynamo-system | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Add a blank line before the fenced block.
markdownlint-cli2reports MD031 at Line 25. Put a blank line between the preceding blockquote and theshellfence.🧰 Tools
🪛 markdownlint-cli2 (0.23.1)
[warning] 25-25: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
🤖 Prompt for AI Agents
Source: Linters/SAST tools