Zerith ACT Adapter

This repository adapts ACT to Zerith data and runtime.

Reference implementation:

• ACT paper/codebase: https://github.com/tonyzhaozh/act
• Zerith SDK: https://github.com/inFpZero/zerith_public_sdk

Project Scope

• Train ACT on Zerith-format datasets.
• Run local robot inference through Zerith SDK wrappers.
• Keep the codebase focused on real training and deployment paths.

Core Files (Required)

Training path:

• imitate_episodes.py: training entrypoint.
• policy.py: ACT policy wrapper.
• utils.py: dataset loading and preprocessing.
• constants.py: task/dataset configuration.
• detr/: ACT model modules.

Inference path:

• robot_infer/inference_robot.py: robot inference entrypoint.
• robot_infer/utils/real_env_sdk.py: robot environment wrapper.
• robot_infer/utils/camera_sdk.py: camera input wrapper.
• robot_infer/lib/lib_h1_sdk_python.so: Zerith robot binary SDK.
• robot_infer/lib/camera_client.py: camera streaming client.
• robot_infer/lib/robot_pb2.py
• robot_infer/lib/robot_pb2_grpc.py

Packaging and environment:

• zerith_act_train.yaml
• setup.py

Directory Layout

zerith_act_open/
    imitate_episodes.py
    policy.py
    utils.py
    constants.py
    detr/
    tools/
        optional_debug/
    robot_infer/
        inference_robot.py
        zerith_act_robot.yaml
        utils/
            real_env_sdk.py
            camera_sdk.py
        lib/
            lib_h1_sdk_python.so
            camera_client.py
            robot_pb2.py
            robot_pb2_grpc.py
    run.sh
    zerith_act_train.yaml

Environment Setup

Use Conda (recommended).

Training environment:

conda env create -f zerith_act_train.yaml
conda activate zerith_act
pip install -e .
cd detr && pip install -e . && cd ..

Robot inference environment (on the robot host):

cd robot_infer
conda env create -f zerith_act_robot.yaml
conda activate zerith_act

The robot-side YAML is stored at:

• robot_infer/zerith_act_robot.yaml

Hardware note:

• Training GPU recommendation:
  • Recommended: NVIDIA RTX 4090 (24 GB VRAM).
  • Verified workable: NVIDIA RTX 3090 (24 GB VRAM).
  • The above training tests were run with batch_size=16.
• Inference GPU requirement:
  • Verified: NVIDIA RTX 5060 (8 GB VRAM) can run inference directly on Zerith H1Pro.
  • Use --device cuda (default in inference_robot.py) and verify CUDA is available in the selected environment.

Training

Before training, register your dataset/task in constants.py.

At minimum, add one entry in TASK_CONFIGS, for example:

• dataset_dir: your dataset root
• episode_len: action horizon per episode
• camera_names: camera key list used by your data

If you train simulation tasks (sim_*), add entries in SIM_TASK_CONFIGS as well.

Direct launch:

python imitate_episodes.py \
    --num_steps 40000 \
    --task_name coke \
    --ckpt_dir ./checkpoints/coke \
    --policy_class ACT \
    --kl_weight 10 \
    --chunk_size 50 \
    --hidden_dim 512 \
    --batch_size 16 \
    --dim_feedforward 3200 \
    --lr 2e-5 \
    --seed 0 \
    --eval_every 200000000 \
    --save_every 2000

Screen launcher:

sh run.sh imitate

Logs are saved to imitate.log by default. Default environment name is zerith_act.

Inference

Run robot inference:

cd robot_infer
python inference_robot.py \
    --ckpt_path ../checkpoints/coke/policy_best.ckpt \
    --stats_path ../checkpoints/coke/dataset_stats.pkl \
    --device cuda

Startup Checklist

Before running inference, confirm:

• Zerith SDK runtime is installed and compatible with lib_h1_sdk_python.so.
• Camera service/gRPC stream is available.
• Checkpoint and dataset stats paths are correct.
• The selected Conda environment matches your CUDA/PyTorch setup.