Skip to content

Add dVRK needle-pass environment to contrib#6570

Draft
chrisvoncsefalvay wants to merge 8 commits into
isaac-sim:developfrom
chrisvoncsefalvay:feat/dvrk-needle-pass-develop
Draft

Add dVRK needle-pass environment to contrib#6570
chrisvoncsefalvay wants to merge 8 commits into
isaac-sim:developfrom
chrisvoncsefalvay:feat/dvrk-needle-pass-develop

Conversation

@chrisvoncsefalvay

@chrisvoncsefalvay chrisvoncsefalvay commented Jul 16, 2026

Copy link
Copy Markdown

Description

This draft PR replaces #6527, which was closed rather than rewriting its main-based history. It is opened against develop in response to @kellyguo11's maintainer feedback, which asked that the contribution move to develop and that the task use the new isaaclab_tasks/contrib area.

Part of the dVRK Extended Cinematic Universe: best read alongside NVIDIA/IsaacTeleop#769.

This adds a contact-driven dVRK PSM needle-pass environment, its pinned PSM asset configuration, and paired-controller XR teleoperation. The needle is initialised once during reset; runtime transfer is qualified from measured contacts and load without attaching or directly repositioning the needle.

The task now lives under isaaclab_tasks.contrib.needle_pass and is registered as IsaacContrib-NeedlePass-dVRK-IK-Abs. The develop port also adds lazy runtime configuration imports, finite-input and controller-contract validation, fail-safe jaw and pose hold behaviour, and public teleoperation-pipeline coverage.

Main changes

  • Add the fixed-base dVRK Patient Side Manipulator asset configuration.
  • Add IsaacContrib-NeedlePass-dVRK-IK-Abs, with differential IK, contact observations, load-backed hand-off phases, rewards, termination criteria, and CUDA physics validation.
  • Add a task-specific IsaacTeleop pipeline that maps paired XR controllers to the environment's 18-dimensional action interface and holds safe commands when controller data is invalid.
  • Add asset-integrity preflight checks and focused CI coverage for the task, physics contracts, and the public IsaacTeleop output pipeline.
  • Document the dVRK CloudXR/OpenXR workflow and add changelog fragments for isaaclab_assets and isaaclab_tasks.

Dependency status

NVIDIA/IsaacTeleop#769 is still open, and its dVRK retargeter API is not yet part of a release. The normal package dependency accepts isaacteleop>=1.3.0,<2.0.0. Until a release in that range contains the dVRK API, the focused CI job and documented local-validation helper build the exact PR revision ca175df7afc8198cbba0592cd1b447b11a4f3165. The source-build helper can be removed after that API is released.

Validation

  • ./isaaclab.sh -f: all 22 pre-commit hooks passed.
  • ./isaaclab.sh -p tools/changelog/cli.py check develop: both modified packages have valid fragments.
  • Focused dVRK unit tests: 41 passed.
  • Exact pinned IsaacTeleop PR [Bug Report] --device_id can not work #769 public-pipeline tests: 4 passed.
  • Headless CUDA physics qualification: 3 selected tests passed, including 32-environment finite-action coverage.
  • Qualification MP4: non-empty, with its first frame decoded successfully.
  • Documentation build: 226 source files processed successfully.

Type of change

  • New feature (non-breaking change which adds functionality)
  • Documentation update

Screenshots

dVRK needle-pass environment dVRK needle hand-off

Checklist

  • I have read and understood the contribution guidelines.
  • I have run the pre-commit checks with ./isaaclab.sh -f.
  • I have made corresponding changes to the documentation.
  • My changes generate no new task warnings.
  • I have added tests that prove the fix is effective and the feature works.
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package.
  • My name is present in CONTRIBUTORS.md.

Big thanks to @kellyguo11 for the encouragement and advice on repo structure!

Move runtime imports behind lazy configuration strings and preserve
inherited IK behaviour without CUDA synchronisation. Validate physical
inputs and make invalid pose and jaw commands fail safe.

Cache phase constants and cover the new action semantics and
pre-application configuration loading.
Make the pinned PR 769 integration job blocking and let the source-build
helper select supported active Python environments safely. Accept the
unreleased 1.4 wheel without allowing a future major version.

Execute the public retargeting graph with controller samples and require a
non-empty, decodable qualification video.
@github-actions github-actions Bot added documentation Improvements or additions to documentation asset New asset feature or request infrastructure labels Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

asset New asset feature or request documentation Improvements or additions to documentation infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant