Training slowdown caused by frequent API calls

[DabinSheng]

In the training pipeline, the original implementation uses a locally deployed Qwen3-32B model for inference.
I replaced it with the Qwen API, and the training script runs correctly.
However, frequent API calls have significantly slowed down the training process.

In the paper, it’s mentioned that training the 4B model used 32 × H100 GPUs.
Did a large portion of that training time also include delays from model inference calls, or was a local acceleration or caching mechanism used to avoid this slowdown?

[wangyu-ustc]

Yes it could slow down the training process, thus you can set sub_sample_question_ratio=0.2 to sub-sample 20% of the questions to calculate the reward, which may slightly affect the performance but significantly improve the training efficiency.
We did not use any local acceleration, so we faced the same issue during the training. In my experiments, normally it takes around half of the time to do the inference and half of the time to run RL (including rollouts and gradient descent)