CI fails: ValueError: zero-size array to reduction operation maximum which has no identity #4035
Description
CI fails: ValueError: zero-size array to reduction operation maximum which has no identity when using use_transformers_paged=True in OnlineDPOTrainer: https://github.com/huggingface/trl/actions/runs/17548916927/job/49836630934?pr=4033
FAILED tests/test_online_dpo_trainer.py::TestOnlineDPOTrainer::test_training_with_transformers_paged_0_standard_prompt_only - ValueError: zero-size array to reduction operation maximum which has no identity
FAILED tests/test_online_dpo_trainer.py::TestOnlineDPOTrainer::test_training_with_transformers_paged_1_conversational_prompt_only - ValueError: zero-size array to reduction operation maximum which has no identity
When running OnlineDPOTrainer with use_transformers_paged=True, the training fails with a ValueError: zero-size array to reduction operation maximum which has no identity during the generation step.
Traceback:
tests/test_online_dpo_trainer.py:444: in test_training_with_transformers_paged
trainer.train()
.venv/lib/python3.13/site-packages/transformers/trainer.py:2328: in train
return inner_training_loop(
.venv/lib/python3.13/site-packages/transformers/trainer.py:2672: in _inner_training_loop
tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
trl/trainer/online_dpo_trainer.py:1265: in training_step
prompt_ids, prompt_mask, completion_ids, completion_mask = self._generate(model, prompts, images)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
trl/trainer/online_dpo_trainer.py:1115: in _generate
completion_ids = pad(completion_ids, padding_value=self.pad_token_id, padding_side="right")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
trl/trainer/utils.py:288: in pad
output_shape = np.max([t.shape for t in tensors], 0).tolist()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:3164: in max
return _wrapreduction(a, np.maximum, 'max', axis, None, out,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
obj = [], ufunc = <ufunc 'maximum'>, method = 'max', axis = 0, dtype = None
out = None
kwargs = {'initial': <no value>, 'keepdims': <no value>, 'where': <no value>}
passkwargs = {}
def _wrapreduction(obj, ufunc, method, axis, dtype, out, **kwargs):
passkwargs = {k: v for k, v in kwargs.items()
if v is not np._NoValue}
if type(obj) is not mu.ndarray:
try:
reduction = getattr(obj, method)
except AttributeError:
pass
else:
# This branch is needed for reductions like any which don't
# support a dtype.
if dtype is not None:
return reduction(axis=axis, dtype=dtype, out=out, **passkwargs)
else:
return reduction(axis=axis, out=out, **passkwargs)
> return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E ValueError: zero-size array to reduction operation maximum which has no identity
.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:86: ValueError