Skip to content

关于MLM pretraining时,做句子对Classfication的咨询? #21

@done520

Description

@done520

您好,想请教下句子对Pretraining,我看了Task/TaskForPretraining.py,是 MLM和NSP的组合任务,受到启发想咨询下,如果做句子对分类(即判断句子a和句子b是否属于同一类),是不是相应的调整一下句子对的处理(即模型输入token_type_ids改为[0] * (len(token_a_ids) + 2) + [1] * (len(token_b_ids) + 1)),用句子对label替换 nsp_label即可?还是说有其他的方法?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions