About pre-training of diffusion language model

[sglucas]

Hi, thank you very much for your great work! I would like to ask whether the pre-tarining code of diffusion language model has been tested and whether there is a checkpoint can be shared. Thank you very much in advance!

[kcz358]

Hi, sorry but I haven't tested DLLM training myself. In our latest code we've moved to transformers>=5.5.0, so the DLLM integration likely needs to be updated for compatibility. CC @JinjieNi and @BIGKnight who integrated these features — they might have more insight on what needs to change. Thanks to both of them for their help on this!