Skip to content

Conversation

@brysonjones
Copy link
Contributor

What this does

This PR adds support for an implementation of Multitask Diffusion Transformer Policy, which was shown in a demo of Boston Dynamics' Atlas robot performing whole-body manipulation tasks

I wanted to dive into the research of this method, and build an open-source implementation for the community to leverage and build from.

I will simultaneously be releasing a blog post that includes the details of this work as this gets merged in! (Note: the current blog post link is broken until being released)

How it was tested

  • I added a simple test script test_multi_task_dit_policy.py to validate any install and import errors
  • I've tested launching training runs on Modal with batch sizes of 320 on H200 and B200 GPUs
  • I've trained multiple policies for >60k training steps and >20 hours with this policy, and deployed them onto a Trossen WidowAI Arm (for which I have another PR open for to integrate)

How to checkout & try? (for the reviewer)

Run the test script:

uv run pytest tests/policies/test_multi_task_dit_policy.py -v

Train a policy:

lerobot-train \
  --policy.type=multi_task_dit \
  --dataset.repo_id={{your_dataset_name}} \
  --dataset.root={{your/dataset/path}} \
  --output_dir=outputs/train/multi_task_dit \
  --job_name=multi_task_dit_training_test \
  --policy.device=cuda \
  --batch_size=16 \
  --steps=10000 \
  --save_freq=1000 \
  --wandb.enable=true \
  --policy.repo_id=YOUR_HF_USERNAME/multi_task_dit_policy_test

@brysonjones brysonjones changed the title Feature/add multitask dit Feature/add multitask diffusion transformer policy implementation Nov 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant