RGBDSensor

RGBDSensor is a modular C++ toolkit for working with RGBD sensors such as Kinect v1, Azure Kinect, Femto Bolt, and others. It provides reusable libraries and practical sample applications for real-time visualization, multi-camera synchronization, intrinsic/extrinsic calibration, low-level raw data access, and more.

The project is built with CMake, supports modular configuration, and currently targets Windows (Linux support in progress). Devices and features can be enabled individually via CMake flags for flexibility and minimal dependencies.

---

✨ Core Features

• 🎥 Supports multiple depth sensors:
  • Kinect v1
  • Azure Kinect (incl. Body Tracking)
  • Femto Bolt
  • RealSense (optional)
  • Virtual depth sensor (for testing)
• 🔧 Tools for:
  • Depth and color stream visualization
  • Camera calibration (intrinsic + extrinsic)
  • Multi-camera array capture and synchronization
  • Raw frame dumping and processing
• 🧪 Sample projects and utilities with real sensor interaction
• 🛠️ Modular architecture using CMake + optional SDKs

---

📦 Requirements

• C++17-compatible compiler (MSVC / Clang / GCC)
• CMake 3.16+
• Windows 10+ (tested); Linux support in progress
• OpenCV (optional, recommended for visualization)
• Eigen (optional, for math & calibration)
• Sensor SDKs:
  • Kinect SDK v1.8
  • Azure Kinect SDK + Body Tracking SDK
  • Femto Bolt SDK
• DirectX SDK (Windows-only, for certain 3D viewers)

💡 Not all modules require all dependencies. You can enable or disable modules individually via CMake options.

---

🔧 Building the Project

1. Clone the Repository

git clone https://github.com/yourusername/RGBDSensor.git
cd RGBDSensor
git submodule update --init --recursive

2. Configure with CMake

mkdir build
cd build

Then run CMake with desired flags:

cmake .. -G "Visual Studio 17 2022" -A x64 ^
  -DSENSOR_ENABLE_KINECT_V1=ON ^
  -DSENSOR_ENABLE_KINECT_AZURE=ON ^
  -DBUILD_RGBD_SENSOR_SAMPLES=ON ^
  -DBUILD_RGBD_SENSOR_TOOLS=ON

🔧 You can toggle individual modules/sensors depending on your environment.

3. Build

cmake --build . --config Release

Or open the solution file with Visual Studio.

---

🚀 Samples Overview

Sample applications demonstrate sensor access, data visualization, camera array management, and more. Some key examples include:

• ✅ DepthSensorViewer: Real-time OpenGL-based RGBD visualization from a single sensor

• ✅ KinectAzureArray: Synchronize, visualize, and monitor multiple Azure Kinect devices

• ✅ KinectAzureRawArray: Access and dump raw images, point clouds, and calibration data from a camera array

Other available samples:

• CalibrateCameraArray — Estimate extrinsics of camera arrays
• WebCameraArray — Basic support for multi-USB webcam capture
• TestKinectCalib — Visualization/testing of calibrated Kinect setup

All samples are conditionally built via BUILD_RGBD_SENSOR_SAMPLES.

---

🛠 Tools Overview

Additional utilities for calibration, frame analysis, and export:

• KinectAzureArrayDump — Frame-by-frame dumping from an Azure array
• KinectAzureArrayAnalysis — Alignment and calibration evaluation
• DumpKinectAzureRawCameraFrame — Save raw Azure Kinect camera data
• KinectAzureArrayDumpNVJPEG — Save frames using NVJPEG (CUDA)

Enable via BUILD_RGBD_SENSOR_TOOLS.

---

📂 Directory Structure

RGBDSensor/
├── src/                       # Core libraries, drivers, and APIs
├── src/Samples/               # Sample applications (viewer, array, raw dump)
├── src/Tools/                 # Command-line tools and batch utilities
├── 3rdParty/                  # Core rendering and math utilities
├── cmake/                     # CMake module files
└── docs/                      # Additional documentation (coming soon)

---

📚 Documentation

Each sample includes its own README.md:

• DepthSensorViewer
• KinectAzureArray
• KinectAzureRawArray

Upcoming documents under /docs/:

• SDK installation guides
• Calibration pipeline overview
• Data format explanations

---

🧩 Integration in Other Projects

You can integrate RGBDSensor as a submodule in larger C++ projects:

git submodule add https://github.com/yourusername/RGBDSensor.git external/RGBDSensor

Then link it via:

add_subdirectory(external/RGBDSensor)

---

🤝 Contribution

We welcome feedback, issues, and contributions:

• ✅ Test on Linux / cross-platform fixes
• 🎯 Integration with more SDKs (e.g. ZED, Orbbec Gemini)
• 🚀 Performance optimizations (GPU offload, NVJPEG/DepthAI)

---

📄 License

This project is licensed under the MIT License. See the LICENSE file.

---

🙋 FAQ

Q: Does it support Linux? A: In progress. Most core logic is cross-platform, but some visualization and SDKs are Windows-only.

Q: Can I only build one sample or tool? A: Yes. You can toggle individual samples via CMake options or selectively include targets in the top-level CMakeLists.txt.

Q: Do I need OpenCV/Eigen? A: Not necessarily. Only specific tools or visualizations require them. CMake will detect and enable features automatically.

---