【摘要】 针对传统的BERT模型在使用中文语料进行预训练时无法获取词的信息问题，本文中在预训练阶段引入基于动态掩码的RoBERTa预训练模型；该预训练模型所生成的语义表示含有词的信息，能够表达句子丰富的句法和语法信息，并且能够对词的多义性建模，更适用于中文命名实体识别任务；同时通过字在上下文中能形成词的词向量对相应的字向量进行了增强；为了提高模型的效率，在序列建模层引入膨胀卷积神经网络。实验表明，该模型在多尺度上下文和结构化预测方面比传统CNN有更好的容量，在常用中文实体识别任务语料库上，验证了模型的有效性。更多还原

【Abstract】 Aiming at the problem that the traditional BERT model can not obtain word information when using Chinese corpus for pre-training, this paper introduced Roberta pre-training model based on dynamic mask in the pre-training stage. The semantic representation generated by the pre-training model contains word information could express rich syntactic and grammatical information of sentences, and could model the polysemy of words, which is more suitable for Chinese named entity recognition tasks. And the corresponding word vector was enhanced by the word vector that could form a word in the context. At the same time, in order to improve the efficiency of the model, the dilated convolution neural network was introduced into the sequence modeling layer. Experiments show that the proposed model has better capacity than traditional CNN in multiscale context and structured prediction. The effectiveness of the model is verified on the co mmon Chinese entity recognition task corpus.更多还原