Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docs/source/device/body_tracking.rst
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,14 @@ The ``all_joint_poses_tracked`` quality flag indicates whether every joint was
successfully tracked in the current frame. When it is false, consult individual
joint ``is_valid`` flags to determine which joints have valid poses.

For a minimal C++ reader see ``examples/schemaio/full_body_printer.cpp``, which
creates the tracker, queries the required OpenXR extensions, and prints the
joint data each frame through ``DeviceIOSession``. The Python equivalent is
``examples/oxr/python/test_full_body_tracker.py``. Running
``python -m isaacteleop.rig rigs/full_body.yaml`` starts the CloudXR runtime,
this printer, and the C++ MCAP recorder together in one tmux session (see
:ref:`rig-launcher`).

Troubleshooting
~~~~~~~~~~~~~~~

Expand All @@ -204,6 +212,11 @@ Troubleshooting
enterprise features (see above). Verify all PICO motion trackers are
paired, powered on, and calibrated. Confirm the PICO browser is up to
date.
- **The printer prints** ``[body tracking inactive]``. The headset is
connected but body tracking is unavailable: check that the Motion Trackers
are paired and calibrated and that the headset has enterprise body-tracking
support. A printer that never reaches ``[Step 4] Reading samples...`` is
still waiting for the runtime / headset instead.
- **Some joints report** ``is_valid: false``. The PICO runtime may
temporarily lose tracking for individual joints during fast movement or
partial occlusion. These joints will recover automatically once tracking is
Expand All @@ -217,6 +230,15 @@ same retargeting pipeline — no headset required during replay. See
:doc:`../references/mcap_record_replay` for the recording / replay API and
the ``record_full_body.py`` / ``replay_full_body.py`` example.

Recording is also available directly from C++: pass a ``McapRecordingConfig``
to ``DeviceIOSession::run()`` with the tracker mapped to the ``full_body``
channel base name. ``examples/mcap_record_replay/cpp/record_full_body.cpp``
demonstrates this — a file recorded there replays unchanged with
``replay_full_body.py``. The ``rigs/full_body.yaml`` rig includes a recorder
pane running this example; each :kbd:`Enter` rerun in that pane writes a fresh
timestamped take into ``examples/mcap_record_replay/recordings/``, where
``replay_full_body.py`` looks by default.

.. figure:: ../_static/full-body-replay.gif
:alt: Full body skeleton replayed from an MCAP recording in viser
:class: no-image-zoom
Expand Down
7 changes: 7 additions & 0 deletions docs/source/device/trackers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,13 @@ reads the PICO ``XR_BD_body_tracking`` extension directly.
- :code-file:`src/core/schema_tests/python/test_full_body.py`
- :code-file:`examples/oxr/python/test_full_body_tracker.py`

- Examples:

- :code-file:`examples/schemaio/full_body_printer.cpp`
- :code-file:`examples/mcap_record_replay/cpp/record_full_body.cpp`
- :code-file:`examples/mcap_record_replay/python/record_full_body.py`
- :code-file:`examples/mcap_record_replay/python/replay_full_body.py`

.. note::

``FullBodyTrackerPico`` remains available as a deprecated alias for
Expand Down
9 changes: 9 additions & 0 deletions docs/source/references/mcap_record_replay.rst
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,15 @@ additional source nodes (``HeadSource``, ``ControllersSource``, …) in
For a live browser view of **all** human DeviceIO trackers at once (hands, head,
controllers, and full body), see ``examples/deviceio_live_view/python/``.

A C++ recorder lives at ``examples/mcap_record_replay/cpp/``:

- ``record_full_body.cpp`` — records the ``full_body`` channel by passing a
``core::McapRecordingConfig`` to ``DeviceIOSession::run()``. It uses the
same channel base name as ``FullBodySource``, so the resulting file replays
with ``replay_full_body.py``. ``python -m isaacteleop.rig rigs/full_body.yaml``
runs it together with the CloudXR runtime and the full-body printer (see
:ref:`rig-launcher`).

Recording
^^^^^^^^^

Expand Down
12 changes: 8 additions & 4 deletions docs/source/references/rig.rst
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,10 @@ own:
- name: se3 printer (requires headset)
command: "install/examples/schemaio/se3_printer {collection_id}"

``rigs/full_body.yaml`` shows the other supported shape — no ``producers``
key, because its consumers read the tracking data directly from the runtime,
so there is no ``collection_id`` to rendezvous on.

Top-level keys:

.. list-table::
Expand Down Expand Up @@ -163,10 +167,10 @@ errors — a typo fails loudly at load time instead of misbehaving in a pane.

.. important::

Producers and consumers rendezvous on a shared ``collection_id``, and a
mismatch is **silent no-data** by design. Define it once under ``params``
and reference it as ``{collection_id}`` in every command — then one edit
in one place changes both sides together.
In a rig with both producers and consumers, the two sides rendezvous on a
shared ``collection_id``, and a mismatch is **silent no-data** by design.
Define it once under ``params`` and reference it as ``{collection_id}`` in
every command — then one edit in one place changes both sides together.

.. warning::

Expand Down
3 changes: 3 additions & 0 deletions examples/mcap_record_replay/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,8 @@

cmake_minimum_required(VERSION 3.20)

# Build C++ examples
add_subdirectory(cpp)

include(${CMAKE_SOURCE_DIR}/cmake/InstallPythonExample.cmake)
install_python_example(DESTINATION examples/mcap_record_replay/python)
24 changes: 24 additions & 0 deletions examples/mcap_record_replay/cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

cmake_minimum_required(VERSION 3.20)

find_package(Threads REQUIRED)

# Full body MCAP recorder example
add_executable(record_full_body
record_full_body.cpp
)

target_link_libraries(record_full_body PRIVATE
deviceio::deviceio_session
deviceio::deviceio_trackers
oxr::oxr_core
isaacteleop_schema
Threads::Threads
)

# Install
install(TARGETS record_full_body
RUNTIME DESTINATION examples/mcap_record_replay/cpp
)
161 changes: 161 additions & 0 deletions examples/mcap_record_replay/cpp/record_full_body.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

/*!
* @file record_full_body.cpp
* @brief Record a live full-body tracking session to an MCAP file using only the C++ API.
*
* Creates a FullBodyTracker, opens an OpenXR session with its required extensions, and
* passes a McapRecordingConfig to DeviceIOSession::run() — the tracker impl then writes
* MCAP samples during each session->update() call. This application does not launch the
* CloudXR runtime; start it (and connect the headset) before running, or use the full_body
* rig (rigs/full_body.yaml) to launch the runtime, printer, and recorder together.
*
* Usage:
* record_full_body [duration_seconds] [output.mcap | output_dir/]
*
* Defaults: 5 seconds -> full_body_<timestamp>.mcap in the current directory. When the
* output argument is a directory (existing, or with a trailing slash), it is created if
* needed and each run writes a fresh timestamped file inside it.
*
* The recording is written to the standard "full_body" channel, so any Isaac Teleop
* replay tooling can read it (see the MCAP record & replay documentation).
*/

#include <deviceio_session/deviceio_session.hpp>
#include <deviceio_trackers/full_body_tracker.hpp>
#include <oxr/oxr_session.hpp>
#include <schema/full_body_generated.h>

#include <chrono>
#include <cstdint>
#include <ctime>
#include <filesystem>
#include <iomanip>
#include <iostream>
#include <memory>
#include <sstream>
#include <string>
#include <thread>
#include <vector>

namespace
{

// Standard channel base name for full-body recordings. MCAP topics become
// "<base_name>/<sub_channel>", and replay tooling reads the "full_body" channel, so this
// must not change.
constexpr const char* MCAP_BASE_NAME = "full_body";

std::string default_output_path()
{
const std::time_t now = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
std::tm tm_buf{};
localtime_r(&now, &tm_buf);
std::ostringstream name;
name << "full_body_" << std::put_time(&tm_buf, "%Y%m%d_%H%M%S") << ".mcap";
return name.str();
}

// An existing directory (or a path with a trailing slash) receives a timestamped file
// inside it — created if needed — so repeated runs never clobber a previous take.
std::string resolve_output_path(const std::string& arg)
{
const std::filesystem::path path(arg);
if (std::filesystem::is_directory(path) || (!arg.empty() && arg.back() == '/'))
{
std::filesystem::create_directories(path);
return (path / default_output_path()).string();
}
return arg;
}

uint32_t count_valid_joints(const core::FullBodyPoseT& data)
{
uint32_t valid_count = 0;
for (uint32_t i = 0; i < core::FullBodyTracker::JOINT_COUNT; ++i)
{
if ((*data.joints->joints())[i]->is_valid())
{
++valid_count;
}
}
return valid_count;
}

} // namespace

int main(int argc, char** argv)
try
{
const double duration_s = (argc > 1) ? std::stod(argv[1]) : 5.0;
const std::string mcap_path = (argc > 2) ? resolve_output_path(argv[2]) : default_output_path();

std::cout << "[record] writing " << mcap_path << " for " << duration_s << "s" << std::endl;

// Step 1: Create the tracker
auto tracker = std::make_shared<core::FullBodyTracker>();

// Step 2: Get required extensions and create OpenXR session
std::vector<std::shared_ptr<core::ITracker>> trackers = { tracker };
auto required_extensions = core::DeviceIOSession::get_required_extensions(trackers);

auto oxr_session = std::make_shared<core::OpenXRSession>("McapFullBodyRecordExample", required_extensions);

// Step 3: Create DeviceIOSession with recording enabled
core::McapRecordingConfig recording_config{ mcap_path, { { tracker.get(), MCAP_BASE_NAME } } };

auto session = core::DeviceIOSession::run(trackers, oxr_session->get_handles(), std::move(recording_config));

// Step 4: Update the session for the requested duration. The tracker impl writes the MCAP
// sample during each update(); no explicit write call is needed here.
const auto start = std::chrono::steady_clock::now();
size_t frame_count = 0;
while (true)
{
const double elapsed_s = std::chrono::duration<double>(std::chrono::steady_clock::now() - start).count();
if (elapsed_s >= duration_s)
{
break;
}

session->update();

if (frame_count % 60 == 0)
{
const auto& tracked = tracker->get_body_pose(*session);
std::cout << "[record] t=" << std::fixed << std::setprecision(2) << elapsed_s << "s frame=" << frame_count;
if (tracked.data)
{
std::cout << " joints=" << count_valid_joints(*tracked.data) << "/"
<< core::FullBodyTracker::JOINT_COUNT;
}
else
{
std::cout << " [body tracking inactive]";
}
std::cout << std::endl;
}
++frame_count;

// Tick at ~60 Hz.
std::this_thread::sleep_for(std::chrono::milliseconds(16));
}

// Destroying the session closes the MCAP writer and emits the summary/statistics block that
// replay tooling relies on — exit normally rather than aborting.
session.reset();

std::cout << "[record] done — " << mcap_path << std::endl;
return 0;
}
catch (const std::exception& e)
{
std::cerr << argv[0] << ": " << e.what() << std::endl;
return 1;
}
catch (...)
{
std::cerr << argv[0] << ": Unknown error occurred" << std::endl;
return 1;
}
15 changes: 14 additions & 1 deletion examples/schemaio/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,19 @@ target_link_libraries(se3_printer PRIVATE
Threads::Threads
)

# Create full body tracker printer executable
add_executable(full_body_printer
full_body_printer.cpp
)

target_link_libraries(full_body_printer PRIVATE
deviceio::deviceio_session
deviceio::deviceio_trackers
oxr::oxr_core
isaacteleop_schema
Threads::Threads
)

# Create frame metadata printer executable
add_executable(frame_metadata_printer
frame_metadata_printer.cpp
Expand All @@ -62,6 +75,6 @@ target_link_libraries(frame_metadata_printer PRIVATE
)

# Install
install(TARGETS pedal_pusher pedal_printer se3_printer frame_metadata_printer
install(TARGETS pedal_pusher pedal_printer se3_printer full_body_printer frame_metadata_printer
RUNTIME DESTINATION examples/schemaio
)
Loading
Loading