Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/unit_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,11 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install uv
uses: astral-sh/setup-uv@v6
- name: Install the library
run: |
pip install .
uv sync --frozen
- name: Run the unit tests
run: |
python3 tests/test_methods.py
uv run python tests/test_methods.py
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
__pycache__/
*.egg-info/
.venv/
imgui.ini
build/
146 changes: 146 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# AGENTS.md

Guidance for AI agents and contributors working in this repository.

## Project Overview

`stereodemo` is a Python package and GUI utility for comparing stereo depth estimation methods on rectified stereo pairs or live OAK-D camera input. The application loads image pairs, computes disparity maps with OpenCV, ONNX Runtime, or TorchScript-backed methods, and visualizes colored point clouds with Open3D.

Core code lives in `stereodemo/`. Sample datasets live in `datasets/`. Model conversion and capture helpers live in `tools/`. Tests live in `tests/`.

## Repository Layout

- `stereodemo/main.py`: CLI entry point, image discovery, method registration, and application startup.
- `stereodemo/visualizer.py`: Open3D GUI, point cloud rendering, method parameter controls, and source abstraction.
- `stereodemo/methods.py`: shared dataclasses and base `StereoMethod` interface.
- `stereodemo/method_*.py`: individual stereo or depth method adapters.
- `stereodemo/oakd_source.py`: live OAK-D capture support and OAK-provided disparity source.
- `stereodemo/utils.py`: model downloading and image padding helpers.
- `datasets/`: small checked-in stereo samples and calibration JSON files.
- `models/` and `dist/`: local/generated artifacts when present; do not treat them as canonical source unless explicitly asked.
- `tests/test_methods.py`: unittest-based inference regression tests.
- `pyproject.toml`: uv and setuptools packaging configuration.
- `setup.py`: minimal compatibility shim for older editable-install workflows.
- `build_release.sh`: release build script that temporarily symlinks `datasets/oak-d` into the package.

## Environment Setup

Use Python 3.8 or newer. CI currently uses Python 3.11.

Prefer `uv` for local development. Create or update the environment with:

```sh
uv sync
```

Then run project commands through `uv run` so they use the resolved environment:

```sh
uv run stereodemo datasets
uv run python tests/test_methods.py
```

CI installs from the uv lockfile with:

```sh
uv sync --frozen
```

Main runtime dependencies are declared in `pyproject.toml`: `numpy`, `opencv-python`, `open3d`, `torch`, `torchvision`, and ONNX Runtime. `depthai` is optional and only needed for OAK-D camera input; install it with `uv sync --extra oak` when needed. Keep `uv.lock` in sync when changing project dependencies, but do not churn it for unrelated source changes.

## Common Commands

Run the test suite:

```sh
uv run python tests/test_methods.py
```

Run the GUI on bundled datasets:

```sh
uv run stereodemo datasets
```

Run on one folder or explicit left/right files:

```sh
uv run stereodemo datasets/oak-d
uv run stereodemo left.png right.png --calibration stereodemo_calibration.json
```

Run with an OAK-D camera:

```sh
uv run stereodemo --oak
```

Build a release package:

```sh
uv build
```

For the project release flow, use `./build_release.sh` because it removes and recreates `stereodemo/datasets`, symlinks `datasets/oak-d` for package data, builds `dist/` with `uv build`, then removes the temporary package dataset folder.

## Testing Notes

The test suite instantiates multiple methods and checks median disparity and coverage against a fixed ETH3D sample. It uses `tempfile.gettempdir() / "models"` as the model cache.

Important implications:

- Tests may download model files from GitHub releases on first run.
- Network failures or missing model assets can fail otherwise valid code changes.
- CUDA OpenCV tests are skipped automatically when OpenCV CUDA is unavailable.
- Some methods prefer CUDA when available; preserve CPU fallback behavior unless intentionally changing device policy.
- Numerical changes should be justified. If an inference change intentionally alters output, update expected medians/coverage with a clear explanation.

For small edits that only touch docs, packaging metadata, or non-executed helper scripts, running the full inference suite may be unnecessary. For changes in `stereodemo/methods.py`, `stereodemo/main.py`, `stereodemo/visualizer.py`, or any `stereodemo/method_*.py`, run `uv run python tests/test_methods.py` when dependencies and model downloads are available.

## Coding Conventions

This codebase is small and mostly straightforward Python. Match the existing style in nearby files.

- Keep method adapters as subclasses of `StereoMethod`.
- Return `StereoOutput(disparity_pixels, color_image_bgr, computation_time, ...)` from `compute_disparity`.
- Treat images as OpenCV BGR arrays unless a model-specific preprocessing step explicitly converts to RGB or grayscale.
- Keep disparity values in pixels. Use `StereoMethod.depth_meters_from_disparity()` and `disparity_from_depth_meters()` for depth/disparity conversions.
- Add user-tunable options through `IntParameter` or `EnumParameter` so the Open3D UI can render controls.
- Cache loaded models/sessions inside method instances when practical; repeated UI parameter changes should not reload the same model unnecessarily.
- Keep downloads centralized through `utils.download_model()` or existing per-method URL maps.
- Do not introduce broad refactors while changing one method. These adapters carry method-specific preprocessing and scaling details.

## Data and Calibration

`FileListSource` discovers stereo pairs by looking for `*left*`/`*right*` or `im0`/`im1` image names under folders. Calibration defaults to `stereodemo_calibration.json` beside the left image unless `--calibration` is provided.

When adding dataset samples:

- Include left/right naming that `find_stereo_images_in_dir()` can discover.
- Include `stereodemo_calibration.json` when point cloud scale matters.
- Keep sample data small. Large datasets and generated model files should not be added casually.

## Model Assets

Most pretrained models are not package source. Methods download their required `.onnx` or `.scripted.pt` files into `Config.models_path`, which defaults to `~/.cache/stereodemo/models` for the CLI and a temporary models folder for tests.

When adding or changing a model-backed method:

- Add explicit URLs and expected filenames near the method implementation.
- Make the target input shape, scaling, and postprocessing obvious in code.
- Preserve deterministic output for the regression sample where possible.
- Avoid checking large generated model files into git unless the maintainer explicitly requests it.

## GUI and Runtime Caveats

The GUI uses Open3D and creates windows, so it may not run in headless environments. Prefer unit-level tests for automated validation and only run the GUI when a display is available.

OAK-D support requires `depthai` and hardware access. Do not make OAK-D required for normal installation, tests, or non-camera code paths.

## Packaging and Release Notes

Version numbers are duplicated in `pyproject.toml` and `stereodemo/__init__.py`; update both for a release.

`MANIFEST.in` excludes most sample datasets from source distributions. `pyproject.toml` package data expects `stereodemo/datasets/oak-d` to exist during release builds, which is why `build_release.sh` creates the temporary symlink.

Do not leave generated `stereodemo/datasets`, `build/`, `dist/`, or `*.egg-info/` changes in source edits unless the task is explicitly about release artifacts.
1 change: 1 addition & 0 deletions CLAUDE.md
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Making a new release

- Bump the version numbers in `setup.cfg` and `stereodemo/__init__.py`
- Bump the version numbers in `pyproject.toml` and `stereodemo/__init__.py`

```
./build_release.sh
twine upload dist/*
uv run twine upload dist/*
```

Username is always `__token__`
17 changes: 9 additions & 8 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
include LICENSE
include README.md
recursive-exclude * .vscode
recursive-exclude * .github
prune .vscode
prune .zed
prune .github
recursive-exclude * __pycache__
recursive-exclude datasets/drivingstereo *
recursive-exclude datasets/eth3d_lowres *
recursive-exclude datasets/kitti2015 *
recursive-exclude datasets/middlebury_2014 *
recursive-exclude datasets/opencv-sample *
recursive-exclude datasets/sceneflow *
prune datasets/drivingstereo
prune datasets/eth3d_lowres
prune datasets/kitti2015
prune datasets/middlebury_2014
prune datasets/opencv-sample
prune datasets/sceneflow
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,19 @@ https://user-images.githubusercontent.com/541507/169557430-48e62510-60c2-4a2b-87
python3 -m pip install stereodemo
```

For local development, install the locked environment with uv:

```
uv sync
```

Then run commands through uv:

```
uv run stereodemo datasets
uv run python tests/test_methods.py
```

## Running it

### With an OAK-D camera
Expand Down Expand Up @@ -121,4 +134,3 @@ I did not implement any of these myself, but just collected pre-trained models o
The code of stereodemo is MIT licensed, but the pre-trained models are subject to the license of their respective implementation.

The sample images have the license of their respective source, except for datasets/oak-d which is licenced under [Creative Commons Attribution 4.0 International License](https://creativecommons.org/licenses/by/4.0/).

14 changes: 6 additions & 8 deletions build_release.sh
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
#!/bin/zsh

set -e

rm -rf stereodemo/datasets
mkdir stereodemo/datasets
mkdir -p stereodemo/datasets
trap 'rm -rf stereodemo/datasets' EXIT

pushd stereodemo/datasets
ln -sf ../../datasets/oak-d .
popd
ln -sf ../../datasets/oak-d stereodemo/datasets/oak-d

rm -f dist/*
pip install build
python3 -m build

rm -rf stereodemo/datasets
uv build
60 changes: 58 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,62 @@
[build-system]
requires = [
"setuptools>=42",
"wheel"
"setuptools>=77",
"wheel",
]
build-backend = "setuptools.build_meta"

[project]
name = "stereodemo"
version = "0.6.2"
description = "Compare various stereo depth estimation algorithms on image files or with an OAK-D camera."
readme = "README.md"
requires-python = ">=3.8"
license = "MIT"
license-files = ["LICENSE"]
authors = [
{ name = "Nicolas Burrus", email = "nicolas@burrus.name" },
]
classifiers = [
"Programming Language :: Python :: 3",
"Operating System :: OS Independent",
]
dependencies = [
"numpy",
"onnxruntime>=1.10.0; sys_platform == 'darwin'",
"onnxruntime-gpu>=1.10.0; sys_platform != 'darwin'",
"opencv-python",
"open3d>=0.15.1",
"torch>=1.11.0",
"torchvision",
]

[project.optional-dependencies]
oak = [
"depthai",
]

[project.scripts]
stereodemo = "stereodemo:main"

[project.urls]
Homepage = "https://github.com/nburrus/stereodemo"
"Bug Tracker" = "https://github.com/nburrus/stereodemo/issues"

[dependency-groups]
dev = [
"build>=1.2.0",
"twine>=5.0.0",
]

[tool.setuptools]
include-package-data = false

[tool.setuptools.packages.find]
where = ["."]
include = ["stereodemo*"]

[tool.setuptools.package-data]
stereodemo = [
"datasets/oak-d/*.png",
"datasets/oak-d/*.json",
]
40 changes: 0 additions & 40 deletions setup.cfg

This file was deleted.

4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import sys
site.ENABLE_USER_SITE = "--user" in sys.argv[1:]

# Everything is defined in setup.cfg, added this file only
# to support editable mode.
# Project metadata lives in pyproject.toml. This file only keeps compatibility
# with older editable-install workflows.
import setuptools
setuptools.setup()
Loading
Loading