The error when using datasets with different trajectory lengths

[kavinwkp]

hello, I am using my collected dataset to train ACT, but the varying lengths of each trajectory in the dataset are causing errors as follows. Could you please advise on any methods to resolve this issue? Thank you.

Traceback (most recent call last):
  File "/home/kavin/Documents/PycharmProjects/act/imitate_episodes.py", line 445, in <module>
    main(vars(parser.parse_args()))
  File "/home/kavin/Documents/PycharmProjects/act/imitate_episodes.py", line 113, in main
    best_ckpt_info = train_bc(train_dataloader, val_dataloader, config)
  File "/home/kavin/Documents/PycharmProjects/act/imitate_episodes.py", line 367, in train_bc
    for batch_idx, data in enumerate(train_dataloader):
  File "/home/kavin/anaconda3/envs/aloha/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 634, in __next__
    data = self._next_data()
  File "/home/kavin/anaconda3/envs/aloha/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 678, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/kavin/anaconda3/envs/aloha/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch
    return self.collate_fn(data)
  File "/home/kavin/anaconda3/envs/aloha/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 264, in default_collate
    return collate(batch, collate_fn_map=default_collate_fn_map)
  File "/home/kavin/anaconda3/envs/aloha/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 142, in collate
    return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed]  # Backwards compatibility.
  File "/home/kavin/anaconda3/envs/aloha/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 142, in <listcomp>
    return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed]  # Backwards compatibility.
  File "/home/kavin/anaconda3/envs/aloha/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 119, in collate
    return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
  File "/home/kavin/anaconda3/envs/aloha/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 162, in collate_tensor_fn
    return torch.stack(batch, 0, out=out)
RuntimeError: stack expects each tensor to be equal size, but got [238, 24] at entry 0 and [211, 24] at entry 1

[crlz182]

Hi,

did you manage to find a solution for this one?

I recorded data for a simple picking task. The episodes are of different lengths of course. The approach I tested so far was to use the length of the longest episode. The other episodes are then just filled up with their last action and observation. In this way you have data of equal length. The results are ok because the robot moves towards the cube but it grasps next to it. I have also tested filling up the shorter episodes with end tokens (e.g. unrealistic joint angles) but this did not work for me.

I think filling up the episodes until a fixed length is the way to go. Let me know if that helps.