About goal point in training

[fengjiang5]

Hello, thank you for your excellent work. I'm really interested in these!

I have a question: in your training code, has_navi=True. However, in the flow section, there is code like this:

if self._config.has_navi:
  navi=gt_trajs[:,7:8,:2].clone().to(device)
  navi_feature=pos2posemb2d(navi,num_pos_feats=self._config.tf_d_model//2).squeeze(1)
  global_feature=trajectory_query.squeeze(1)

Is this using the ground-truth point?

Shouldn't it use the goal point?

[ZebinX]

@fengjiang5 Yes, in training, we have tried two different settings: using the gt goal point (end point of the gt trajectory) and the goal point predicted by the goal point construction module. The results of these are very similar, and the former is slightly better.

[frkmac3]

@fengjiang5 Yes, in training, we have tried two different settings: using the gt goal point (end point of the gt trajectory) and the goal point predicted by the goal point construction module. The results of these are very similar, and the former is slightly better.

So if I want to use this model in a real-world scenario, do I need to run inference twice? First to infer a goal point, and then to infer the trajectory?

[ZebinX]

@frkmac3 Yes, you should first infer a goal point, and then the trajectory.

[frkmac3]

@frkmac3 Yes, you should first infer a goal point, and then the trajectory.

have you tried training two stage jointly?

[ZebinX]

@frkmac3 No, we haven't tried this.