Add isaacteleop.rig: a tmux based launcher that starts different process in panels#788
Merged
Merged
Conversation
Contributor
|
📝 Docs preview is not auto-deployed for fork PRs. A maintainer with write access to |
jiwenc-nv
changed the title
Collaborator
Author
|
/preview-docs |
Contributor
|
✅ Preview deployed: https://NVIDIA.github.io/IsaacTeleop/preview/pr-788/ |
Replace scripts/run_se3_demo.sh with a generic, wheel-shipped launcher
module that brings up a teleop rig — the configured set of CloudXR
runtime + producer plugins + consumer apps — in a tmux session from a
declarative YAML rig file. The same rig files serve demos, production
teleop, and data collection. The rig file is the single source of
configuration: params live in its params: block and are substituted
into every command that references them; there are deliberately no CLI
override knobs (edit the file and relaunch to change them).
- isaacteleop.rig (src/core/rig/python/): config.py (schema/loader:
unknown top-level and entry keys are hard errors, {key} substitution
from params with {python} -> sys.executable, tmux-safe name
validation, warn-only lint for Python consumers that would
self-launch a second CloudXR runtime), launcher.py (pane
orchestration behind a single injectable run_tmux seam), __main__.py
(positional rig YAML, --no-runtime, --kill; bare invocation prints
usage plus an example, never a stack trace). Public API: RigConfig,
load_rig_config, launch_rig, kill_rig.
- Pane orchestration: the runtime pane starts immediately;
producer/consumer panes wait for the runtime, load its env, then
auto-run their command; pane-id addressing; idempotent session reuse
with a kill hint; $TMUX switch-client vs attach; PYTHONPATH
forwarding; per-pane titles with session-scoped pane-border-status;
pre-attach instructions.
- Layout: tmux main-horizontal with a session-scoped main-pane-height
of 25% (RUNTIME_PANE_HEIGHT) — the runtime is pane 0 and becomes a
slim full-width strip on top (it only prints status lines and the
web-client URL); worker panes tile in the remaining 75%. Panes are
re-laid out tiled after every split so chained splits never hit
tmux's "pane too small" limit at any rig size; under --no-runtime
all panes are peer workers and simply stay tiled.
- Worker panes auto-load the CloudXR env, then auto-run: each pane's
spawn command is a POSIX wrapper (never typed input; stty echo
bracketing keeps machinery invisible) that polls for the
runtime_started sentinel (recreated only once the runtime actually
serves), sources <run-dir>/cloudxr.env, prints a status line and an
auto-run banner, and runs the rig command via sh -c with a scoped
'trap : INT' (Ctrl+C kills the app, not the pane; a command syntax
error cannot kill the wrapper). On command exit it prints the exit
status and drops to the user's shell with the same command pre-typed
(buffered via send-keys -l into its own tty while echo is off) — so
an app that exited early, e.g. started before the headset connected,
reruns with one Enter. On sentinel timeout (120 s) the command is
not run: the wrapper prints the manual-source remedy and pre-types
the command instead. The run dir follows the runtime command's
--cloudxr-install-dir when present, else $CXR_INSTALL_DIR, else
~/.cloudxr, mirroring the runtime's own EnvConfig resolution.
Workers behave identically under --no-runtime.
- --kill tears down a rig's session by rig file (kill_rig() also
exported for programmatic use); idempotent, rejects --no-runtime;
the session-reuse hint and pre-attach instructions point at --kill
first with the raw tmux one-liner as the alternative.
- rigs/se3_tracker.yaml: first rig file, annotated as the schema
exemplar (collection_id defined once under params and referenced by
both sides — a mismatch is silent no-data; NOTE about
--no-launch-cloudxr-runtime for Python TeleopSession consumers).
Rig files are checkout artifacts and are not shipped in the wheel.
- docs: Rig Launcher reference page (prerequisites, running a rig,
--no-runtime/--kill, auto-run + exit-status + Enter-to-rerun
behavior, rig YAML schema key reference, collection_id
single-sourcing rule, self-launching-runtime warning, troubleshooting
table), registered in the References toctree after the CloudXR
runtime page.
- Packaging: rig_python copy target wired into python_package and the
pyproject packages list; pyyaml>=6.0.3 added to the base
requirements.
- Tests: src/core/rig_tests/python — pytest cases against a recording
FakeTmux and the shipped rig file (including regression tests that
pane machinery is never typed and never echoes, that commands run
only when the runtime is ready, and that Ctrl+C kills the app not
the pane), registered as ctest entries; no tmux or headset needed.
- Delete scripts/run_se3_demo.sh (the runbook in the meta-repo docs/
is updated to the new invocation separately).
pytest 58/58, ctest rig-filtered 5/5, pre-commit clean, Sphinx -W
green; pane bring-up smoke-tested in real tmux under bash and zsh
(auto-run, exit-and-rerun, and timeout captures) and end to end
against a live session.
Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
Contributor
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis change introduces a YAML-driven Rig Launcher exposed through Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant RigCLI
participant RigConfig
participant Tmux
participant CloudXR
participant Workers
Operator->>RigCLI: Run python -m isaacteleop.rig rig.yaml
RigCLI->>RigConfig: Load and validate YAML
RigCLI->>Tmux: Create or attach to rig session
Tmux->>CloudXR: Start runtime pane
Tmux->>Workers: Start producer and consumer panes
CloudXR->>Workers: Publish runtime_started and cloudxr.env
Workers->>Workers: Load environment and execute commands
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
jiwenc-nv
requested review from
aristarkhovNV,
rwiltz,
xdaNvidia and
yanziz-nvidia
July 17, 2026 04:49
Collaborator
Author
|
/preview-docs |
Contributor
|
✅ Preview deployed: https://NVIDIA.github.io/IsaacTeleop/preview/pr-788/ |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/core/rig/python/CMakeLists.txt`:
- Around line 22-25: Update the add_custom_command in rig_python to create
RIG_OUTPUT_DIR with a cmake -E make_directory command before copying PYTHON_FILE
to OUTPUT_PATH, ensuring the destination exists during clean builds.
In `@src/core/rig/python/launcher.py`:
- Around line 183-187: Update the launcher command around the sentinel wait and
worker block to introduce a readiness flag initialized as unset, set it only
after sourcing the quoted env_file succeeds, and gate the runtime_started/worker
execution path on that flag rather than sentinel existence alone. Preserve the
existing timeout and failure-message behavior when the environment cannot be
loaded.
- Around line 295-297: In the launch flow around _create_session, synchronously
remove the prior runtime_started readiness sentinel after resolving
cloudxr_run_dir and before starting the managed runtime when no_runtime is
false. Preserve the existing no_runtime behavior, and ensure stale readiness
state cannot be observed by workers for the new session.
- Around line 231-293: The launcher currently performs plan substitution,
working-directory validation, and command checks before checking for an existing
tmux session. Move the _session_exists probe and existing-session reattachment
path earlier, after establishing the tmux runner, so it returns before
launch-only preflight such as plan construction, cwd validation, placeholder
checks, and command validation; preserve the existing --no-runtime messaging and
_goto_session behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 74ce0f56-baf1-43b2-81ad-ab90418c595a
📒 Files selected for processing (20)
docs/source/index.rstdocs/source/references/rig.rstrigs/se3_tracker.yamlsrc/core/CMakeLists.txtsrc/core/python/CMakeLists.txtsrc/core/python/pyproject.toml.insrc/core/python/requirements.txtsrc/core/rig/CMakeLists.txtsrc/core/rig/python/CMakeLists.txtsrc/core/rig/python/__init__.pysrc/core/rig/python/__main__.pysrc/core/rig/python/config.pysrc/core/rig/python/launcher.pysrc/core/rig_tests/CMakeLists.txtsrc/core/rig_tests/python/CMakeLists.txtsrc/core/rig_tests/python/conftest.pysrc/core/rig_tests/python/pyproject.tomlsrc/core/rig_tests/python/test_config.pysrc/core/rig_tests/python/test_launcher.pysrc/core/rig_tests/python/test_main.py
yanziz-nvidia
left a comment
yanziz-nvidia
left a comment
Contributor
There was a problem hiding this comment.
Reviewed by yanziz-reviewer-bot
Approved by yanziz
Summary
This PR adds isaacteleop.rig, a YAML-driven tmux launcher that replaces a manual three-terminal runbook with a single command: it starts the CloudXR runtime in a top pane, waits for its readiness sentinel, sources the CloudXR env into each producer/consumer pane, then auto-executes those panes' commands. The module ships in the isaacteleop wheel (adding a pyyaml>=6.0.3 dependency), includes a comprehensive test suite with an injectable tmux seam, and adds full RST documentation. All PR checklist items are unchecked and the Testing section is empty.
Legend: 🚫 Blocker · 💡 Suggestion · 🔍 Nit
| Finding | |
|---|---|
| 🚫 | src/core/rig/python/__init__.py:16-17 — Module docstring says panes are "pre-typed but NOT executed — press Enter after the headset connects," directly contradicting the actual behavior. _worker_pane_command auto-executes each pane as soon as the runtime_started sentinel appears; Enter is the rerun mechanism after an early exit. This is the first text in help(isaacteleop.rig) and IDE tooltips — wrong mental model here causes immediate confusion. |
| 💡 | src/core/rig_tests/python/CMakeLists.txt:12 — file(GLOB TEST_FILES ...) without CONFIGURE_DEPENDS means adding a new test_*.py won't be picked up by CTest until cmake is re-run manually. Add CONFIGURE_DEPENDS to the glob. |
| 🔍 | src/core/rig/python/config.py:179 — source.read_text() uses platform default encoding; on non-UTF-8 locales a rig file with non-ASCII characters will mis-decode or raise. Prefer source.read_text(encoding="utf-8"). |
Actionables (for bots — copy-paste-ready for AI)
Fix if it makes sense in context — these are agent-generated suggestions, not human-vetted obligations. Skip anything that's wrong, already addressed, or not worth the churn.
src/core/rig/python/__init__.py:16-17— Replace the stale "pre-typed but NOT executed — press Enter" docstring: describe the auto-execution flow (waits for sentinel, sources env, runs automatically; Enter reruns after early exit).src/core/rig_tests/python/CMakeLists.txt:12— AddCONFIGURE_DEPENDStofile(GLOB TEST_FILES ...).src/core/rig/python/config.py:179— Changesource.read_text()tosource.read_text(encoding="utf-8").
…entinel cleanup - rig/python/CMakeLists: create RIG_OUTPUT_DIR before copying so clean builds work on CMake < 3.26 (the repo floor is 3.20) - launcher: gate worker auto-run on cloudxr.env sourcing SUCCESS via a rig_env_ready flag, not on sentinel existence; [ -r ] guard prevents dash's fatal '.' on a missing file from killing the pane wrapper; distinct source-failure message (timeout message unchanged) - launcher: probe for an existing session before launch-only preflight — edits to a running rig's YAML never block reattachment (reattach needs only a loadable rig file and tmux) - launcher: clear a stale runtime_started sentinel before starting a managed runtime (workers probe within milliseconds; the runtime needs seconds to delete it itself); never under --no-runtime, where the external runtime owns the sentinel; an undeletable sentinel surfaces as a PreflightError with a remedy - rig/__init__: fix the stale 'pre-typed but NOT executed' docstring — panes auto-run; Enter is the rerun mechanism - rig_tests/CMakeLists: CONFIGURE_DEPENDS so new test files register with CTest without a manual reconfigure - config: read rig files as UTF-8; a non-UTF-8 file is a RigConfigError naming the real cause, never a traceback - tests: pin HOME in the autouse fixture so the suite can never unlink a real ~/.cloudxr sentinel; cover the reattach-despite-broken-plan path, the full sentinel ownership matrix, and the new messages - docs: extend step 4 wording and add a troubleshooting row for the source-failure message Tests: 66 passed (rig_tests via uv/pytest). Lint: pre-commit clean (SKIP=check-copyright-year). Clean-configure + rig_python target build verified from an empty build tree. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
|
/preview-docs |
1 similar comment
Collaborator
Author
|
/preview-docs |
Contributor
|
✅ Preview deployed: https://NVIDIA.github.io/IsaacTeleop/preview/pr-788/ |
yanziz-nvidia
approved these changes
Jul 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add isaacteleop.rig: YAML-driven tmux teleop-rig launcher
Bringing up a teleop demo today means juggling three terminals by hand — start the CloudXR runtime, wait, source ~/.cloudxr/run/cloudxr.env, start the producer plugin, start the consumer app, and keep the collection ids in sync between them. This PR replaces that runbook (and scripts/run_se3_demo.sh) with a launcher module shipped in the wheel that brings up a whole rig — runtime + producer plugins + consumer apps — in one tmux session from a declarative YAML file.
Usage
from the Teleop repository root, in the venv where the wheel is installed:
python -m isaacteleop.rig rigs/se3_tracker.yaml
When a CloudXR runtime is already running elsewhere:
python -m isaacteleop.rig rigs/se3_tracker.yaml --no-runtime
Tear the whole rig down (session + all processes):
python -m isaacteleop.rig rigs/se3_tracker.yaml --kill
What it does
addressing, per-pane titles.
teleop-rig.mp4
Fixes #(issue)
Type of change
Testing
Checklist
SKIP=check-copyright-year pre-commit run --all-filesgit commit -s) per the DCOSummary by CodeRabbit
New Features
Documentation