https://arxiv.org/abs/2104.08692
mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs (Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang Xian-Ling Mao, Heyan Huang, Furu Wei)
t5에 text2text pretraining task를 추가. 요즘 보면 MS가 lm을 꽤 열심히 하네요.
#pretraining #language_model