Fine tuning robotic foundation models version 2

Pipeline and troubleshooting techniques

Part 1

1. RT1 Simpler Episode Simulation environment google colab https://colab.research.google.com/drive/1yiIg36up4GLPlvdufx3r9c5aFT-ne0ns?authuser=3#scrollTo=nT8XbO31KzSX
2. Move or copy the saved episodes to the server

Part 2

4. Activate conda rlds env (environment_DS)
5. Change the name of dataset folder/*builder.py/class in *builder.py. Change the location of train/val folders for _split_generators function.
6. Run the builder with tfds build --overwrite from inside the dataset folder, and move/copy the ~/tensorflow_dataset/ to /data/dgx1/ms23ss/ (https://github.com/kpertsch/rlds_dataset_builder)

Part 3

8. Activate conda env2 (environment_FT)
9. Go to SimplerEnv/octo and run python ./scripts/finetune.py --config.pretrained_path=hf://rail-berkeley/octo-base --name='PickCokeCan' --config.save_dir='/home/ms23ss/finetune_saves/base_PickCokeCan' and editing the pretrained path, the name, the save directory and also editing the file finetune_config
10. In finetune_config.py, change the name to the dataset name and the data dir to /data/dgx1/ms23ss/ {so that data_dir/dataset_name is the dataset}
11. For the python call, change the name to be whatever I want to save the model as and the wandb run
12. Change the save directory and also ensure that it exists to save the checkpoints in
13. Move/Copy this checkpoint folder (a new one will be made called -- inside the save_dir/octo-finetune. Move/Copy it to google drive and delete all checkpoints except the last one and also keep the ‘state’ folder.

Part 4

15. Now load this checkpoint via octo inference notebook and change octopath to refer to the checkpoint (and make sure the model in the main function is Octo-custom if custom)

Trouble shooting

1. If the memory is too high for fine-tuning, do the following
   1. sudo apt-get install psmisc
   2. fuser -v /dev/nvidia*
   3. sudo kill -9 $PID
2. If memory is high but because someone else is using it (XLA/JAX issues show up):
   1. XLA_PYTHON_CLIENT_MEM_FRACTION=.4 && python ./scripts/finetune.py --config.pretrained_path=hf://rail-berkeley/octo-base --name='OctoPickCokeCan' --config.save_dir='/home/ms23ss/finetune_saves/base_OctoPickCokeCan'
   2. Can change the fraction based on memory availability