docs: add LeRobot and SO-101 getting-started guide

docs/source/device/joint_space.rst

@@ -69,6 +69,8 @@ The gripper is just another named DOF (conventionally ``"gripper"``). ``velocity
 and ``ee_pose`` are optional/reserved: the reference plugin and ``JointStateSource`` populate and
 surface joint **positions** only.
 
+.. _so101-leader-plugin:
+
 The SO-101 leader plugin
 ------------------------
 

docs/source/getting_started/lerobot/data_collection_real.rst

@@ -0,0 +1,185 @@
+.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+.. SPDX-License-Identifier: Apache-2.0
+
+Data Collection in Real
+=======================
+
+Record demonstrations on a physical SO-101 into a `LeRobot dataset
+<https://huggingface.co/docs/lerobot/en/index>`_, driving the follower with either teleop device
+(see :doc:`devices`). The example scripts live in ``examples/isaac_teleop_to_so101/`` in the
+`LeRobot <https://github.com/huggingface/lerobot>`_ repository: ``teleoperate.py`` drives the arm
+live, and ``record.py`` does the same while saving a dataset. Both take the same
+``--robot.*`` / ``--teleop.*`` flags; ``--teleop.type`` selects the device
+(``xr_controller`` | ``so101_leader``).
+
+Before you start
+----------------
+
+#. A working **SO-101 follower** — assembled, motors set up, and calibrated. See
+   `SO-101 support in LeRobot`_.
+
+#. The **isaac-teleop extra** installed (``isaacteleop`` ships on public PyPI; its
+   ``[cloudxr,retargeters]`` extras pull the CloudXR runtime bindings and the retargeter library):
+
+   .. code-block:: bash
+
+      uv pip install -e '.[isaac-teleop]'
+
+#. Run the scripts from the example directory, and log in to the Hugging Face Hub — recorded
+   datasets are pushed to the Hub by default (pass ``--dataset.push_to_hub=false`` to keep them
+   local):
+
+   .. code-block:: bash
+
+      cd examples/isaac_teleop_to_so101
+      huggingface-cli login
+
+Then follow the steps for your teleop device:
+
+.. tab-set::
+
+   .. tab-item:: XR controller
+
+      The controller pose drives the follower's end-effector through the clutch + IK pipeline,
+      streamed over CloudXR.
+
+      #. **Fetch the robot model.** The XR path solves inverse kinematics, so it needs the SO-101
+         URDF and meshes (downloaded into ``./SO101/``):
+
+         .. code-block:: bash
+
+            python download_assets.py
+
+      #. **Connect a headset.** Bring up CloudXR and connect your XR headset — follow the
+         :doc:`/getting_started/quick_start`.
+
+      #. **(Optional) Try teleoperation without recording.** A good way to check the setup before
+         committing to a dataset:
+
+         .. code-block:: bash
+
+            python teleoperate.py \
+                --robot.type=so101_follower \
+                --robot.port=/dev/ttyACM0 \
+                --robot.id=so101_follower_arm \
+                --teleop.type=xr_controller
+
+         Squeeze and hold the grip to engage the clutch and move the arm; the trigger controls the
+         gripper. Release the grip to pause.
+
+      #. **Record a dataset.** Add cameras and the dataset parameters:
+
+         .. code-block:: bash
+
+            python record.py \
+                --robot.type=so101_follower \
+                --robot.port=/dev/ttyACM0 \
+                --robot.id=so101_follower_arm \
+                --teleop.type=xr_controller \
+                --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
+                --dataset.repo_id=<hf_user>/<dataset_name> \
+                --dataset.single_task="Pick up vial from rack on the left side" \
+                --dataset.num_episodes=3 \
+                --dataset.episode_time_s=20 \
+                --dataset.reset_time_s=5
+
+      .. note::
+
+         **Customizing the reset pose.** On startup the XR path slews the arm to a built-in default
+         reset pose (a comfortable mid-range pose) before handing control to the clutch — you do
+         **not** need to record anything. To tailor it to your setup, back-drive the arm to the
+         pose you want and run ``python override_reset_pose.py``; it writes ``reset_pose.json``
+         (git-ignored, user-local), which takes priority over the default on the next run. Pass
+         ``--reset_to_origin=false`` to skip the slew and keep the arm where it is.
+
+   .. tab-item:: SO-101 Leader
+
+      A back-drivable SO-101 leader arm mirrored 1:1 to the follower. Its joints are streamed by
+      Isaac Teleop's ``so101_leader`` plugin, which the script launches for you.
+
+      #. **Build the so101_leader plugin.** It is part of Isaac Teleop's C++ source, not the
+         ``isaacteleop`` pip package, so build it from an Isaac Teleop checkout:
+
+         .. code-block:: bash
+
+            cmake -B build && cmake --build build --parallel && cmake --install build
+
+         The binary lands at ``install/plugins/so101_leader/so101_leader_plugin``. For details see
+         :ref:`so101-leader-plugin` and :doc:`/getting_started/build_from_source/index`.
+
+      #. **Calibrate the leader** so the leader and follower agree on each joint's zero and range.
+         This reuses the serial SO-101 leader's calibration (stored under ``so_leader/<id>.json``
+         and reused on every run):
+
+         .. code-block:: bash
+
+            lerobot-calibrate \
+                --teleop.type=so101_leader \
+                --teleop.port=/dev/ttyACM1 \
+                --teleop.id=so101_leader_arm
+
+      #. **(Optional) Try teleoperation without recording.** ``--launch_plugin`` spawns the plugin
+         after CloudXR is up; ``--teleop.port`` is the leader's serial port:
+
+         .. code-block:: bash
+
+            python teleoperate.py \
+                --robot.type=so101_follower \
+                --robot.port=/dev/ttyACM0 \
+                --robot.id=so101_follower_arm \
+                --teleop.type=so101_leader \
+                --teleop.port=/dev/ttyACM1 \
+                --teleop.id=so101_leader_arm \
+                --launch_plugin=/path/to/IsaacTeleop/install/plugins/so101_leader/so101_leader_plugin
+
+         Back-drive the leader arm by hand to move the follower.
+
+      #. **Record a dataset.** Same flags as teleoperation, plus the cameras and dataset parameters
+         (keep ``--launch_plugin`` so the plugin is started):
+
+         .. code-block:: bash
+
+            python record.py \
+                --robot.type=so101_follower \
+                --robot.port=/dev/ttyACM0 \
+                --robot.id=so101_follower_arm \
+                --teleop.type=so101_leader \
+                --teleop.port=/dev/ttyACM1 \
+                --teleop.id=so101_leader_arm \
+                --launch_plugin=/path/to/IsaacTeleop/install/plugins/so101_leader/so101_leader_plugin \
+                --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
+                --dataset.repo_id=<hf_user>/<dataset_name> \
+                --dataset.single_task="Pick up vial from rack on the left side" \
+                --dataset.num_episodes=3 \
+                --dataset.episode_time_s=20 \
+                --dataset.reset_time_s=5
+
+Recording controls
+------------------
+
+``record.py`` records ``--dataset.num_episodes`` episodes of ``--dataset.episode_time_s`` seconds
+each, with a ``--dataset.reset_time_s`` window between episodes to reposition the scene. While
+it is running, **press** these keys in the terminal where ``record.py`` is running — the example
+reads them from that terminal, so they work over SSH and in a plain terminal (Linux/macOS), with
+no desktop session required:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 22 78
+
+   * - Key
+     - Action
+   * - Right arrow →
+     - End the current episode early and save it.
+   * - Left arrow ←
+     - Discard the current take and re-record it.
+   * - Escape
+     - Stop after the current episode (already-saved episodes are kept).
+
+Set ``LEROBOT_KEYBOARD_BACKEND`` to override how keys are read — ``auto`` (the default; uses the
+terminal when one is attached, otherwise a global listener), ``stdin``, ``pynput``, or ``none``.
+The dataset is written under ``$HF_LEROBOT_HOME/<repo_id>`` and pushed to the Hub when recording
+finishes (unless ``--dataset.push_to_hub=false``). Next, train a policy on it:
+:doc:`training_groot`.
+
+.. _SO-101 support in LeRobot: https://huggingface.co/docs/lerobot/en/so101

docs/source/getting_started/lerobot/data_collection_sim.rst

@@ -0,0 +1,108 @@
+.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+.. SPDX-License-Identifier: Apache-2.0
+
+Data Collection in Sim
+======================
+
+Collect SO-101 demonstrations in simulation with `NVIDIA Isaac Lab
+<https://isaac-sim.github.io/IsaacLab>`_, on the cube-stacking task. You drive the simulated
+follower through Isaac Teleop (see :doc:`devices`) and record episodes to an HDF5 dataset.
+
+Two SO-101 stack tasks are registered in Isaac Lab:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 45 55
+
+   * - Task id
+     - Use
+   * - ``IsaacContrib-Stack-Cube-SO101-IK-Abs-v0``
+     - Absolute-pose IK + Isaac Teleop teleoperation (use this for data collection).
+   * - ``IsaacContrib-Stack-Cube-SO101-v0``
+     - Joint-position control baseline (no teleop).
+
+Before you start
+----------------
+
+.. important::
+
+   Both steps below are **required** — complete them first. The teleoperation and recording
+   commands later on will not work until you have.
+
+**Step 1 — Install Isaac Lab.** Follow the `Isaac Lab installation guide`_ to set up the ``Lab``
+repository, then run every script through its launcher: ``./isaaclab.sh -p <script> ...`` (or
+plain ``python`` inside the activated Isaac Lab environment). The SO-101 USD assets stream from
+the NVIDIA Nucleus server, so there is no manual asset download.
+
+**Step 2 — Set up CloudXR and connect a headset.** XR teleoperation needs CloudXR and a headset,
+the same as the real flow — follow the :doc:`/getting_started/quick_start` and the
+`CloudXR teleoperation in Isaac Lab`_ guide. CloudXR auto-launches by default; pick the profile
+with ``--cloudxr_env`` (``cloudxrjs`` for Quest/Pico, ``avp`` for Apple Vision Pro, ``none`` to
+disable). No physical headset? Open the CloudXR web client in a desktop browser, which emulates a
+headset.
+
+Collect Teleop Data
+-------------------
+
+.. tab-set::
+
+   .. tab-item:: XR controller
+
+      The controller pose drives the simulated follower's end-effector through the clutch + IK
+      pipeline, streamed over CloudXR — the same controls as on real hardware.
+
+      #. **(Optional) Try teleoperation without recording.** A good way to check the setup first:
+
+         .. code-block:: bash
+
+            ./isaaclab.sh -p scripts/environments/teleoperation/teleop_se3_agent.py \
+                --task IsaacContrib-Stack-Cube-SO101-IK-Abs-v0 \
+                --xr \
+                --viz kit
+
+         ``--xr`` enables the XR/CloudXR path and ``--viz kit`` opens the Omniverse Kit viewport.
+         Squeeze and hold the grip to engage the clutch and move the arm; the trigger controls the
+         gripper.
+
+      #. **Record a dataset.** ``record_demos.py`` runs the same teleoperation while saving
+         episodes to HDF5. It records ``--num_demos`` demonstrations, marking one successful after
+         ``--num_success_steps`` consecutive success frames:
+
+         .. code-block:: bash
+
+            ./isaaclab.sh -p scripts/tools/record_demos.py \
+                --task IsaacContrib-Stack-Cube-SO101-IK-Abs-v0 \
+                --dataset_file ./datasets/so101_stack_demos.hdf5 \
+                --num_demos 10 \
+                --step_hz 30 \
+                --xr \
+                --viz kit
+
+         The demos are written to the ``--dataset_file`` path in HDF5 format.
+
+   .. tab-item:: SO-101 Leader
+
+      .. admonition:: 🚧 Work in progress
+         :class: caution
+
+         Driving the simulated follower from an **SO-101 Leader** arm is **not yet supported in
+         Isaac Lab** — Isaac Lab has no joint-space leader device, and the SO-101 stack task is
+         wired only for the XR controller. Sim leader support is planned; until then, use the
+         leader arm on the real robot (:doc:`data_collection_real`).
+
+Convert to LeRobot Dataset
+--------------------------
+
+.. admonition:: 🚧 Work in progress
+   :class: caution
+
+   **Export to a LeRobot dataset.** Converting these sim HDF5 demos to the
+   :doc:`LeRobot dataset format <training_groot>` is **not yet provided** for the stack task. The
+   closest reference is the locomanipulation converter `convert_dataset.py`_ from the ``develop``
+   branch in Isaac Lab, which targets a different task and must be adapted.
+
+..
+   References
+.. _Isaac Lab installation guide: https://isaac-sim.github.io/IsaacLab/develop/source/setup/installation/index.html#isaaclab-installation-root
+.. _CloudXR teleoperation in Isaac Lab: https://isaac-sim.github.io/IsaacLab/develop/source/how-to/cloudxr_teleoperation.html
+.. _convert_dataset.py: https://github.com/isaac-sim/IsaacLab/blob/develop/scripts/imitation_learning/locomanipulation_sdg/gr00t/convert_dataset.py