【作者】 王超；

【导师】 陈阳； 姬东溟；

【作者基本信息】 东南大学 ， 电子与通信工程（专业学位）， 2019， 硕士

【摘要】 命名实体识别是信息抽取、机器翻译、知识图谱等应用领域的重要基础工具,在自然语言处理技术走向实用化过程中有着重要地位。在中文命名实体识别研究落后于英文命名实体识别研究以及深度学习兴起的背景下,本文面向企业图谱的构建这一目标,基于深度神经网络模型,深入研究了图谱构建过程中涉及的中文命名实体识别技术。论文的主要工作如下:在经典的BiLSTM-CRF命名实体识别模型的基础上融入Self-Attention,提出了SA-BiLSTM-CRF模型。实验证明该模型能够有效提取远距离依赖信息,有着更好的识别效果。并且发现BiLSTM与Self-Attention结合时使用双向结合的方式比单一结合的方式效果更好。验证了SA-BiLSTM-CRF模型中能够较好地通过LSTM学习到单词位置信息而不需要Positin Embedding。发现Multi-Head机制则能够提升模型效果,但是抽头过多容易过拟合。为了将SA-BiLSTM-CRF模型应用于标注语料较少的企业领域,针对标注语料少、深度网络模型难以训练的问题,分别基于迁移学习、自学习、主动学习、自学习与主动学习相结合四种思路提出了四种优化策略。实验验证了四种优化的有效性。同时,在基于迁移学习的优化策略中,发现最适合的迁移学习方式是迁移和微调整个模型的参数。在其他三种优化策略中,发现使用自学习与主动学习相结合的算法能够优势互补,优化效果要比单一使用其中一种算法的效果更好。在SA-BiLSTM-CRF模型的训练流程中,引入基于迁移学习优化策略和基于自学习与主动学习相结合的优化策略,提出了一个适用于企业领域的命名实体识别系统,并完成了企业图谱的构建。更多还原

【Abstract】 Named entity recognition is an important basic tool in the fields of information extraction,machine translation,knowledge graph,question and answer system,etc.It plays an important role in the process of natural language processing technology becoming practical.Based on the deep neural network model,the Chinese named entity recognition technology involved in construction process of enterprise knowledge graph is researched deeply in this work。Firstly,based on the classical BiLSTM-CRF named entity recognition model,the SA-BiLSTM-CRF model is proposed by incorporating the self-attention mechanism.Experiments show that the model can effectively extract long-distance dependence information,which has better recognition effect than BiLSTM-CRF model.it is found that bidirectional self-attention is better than sigle self-attention when combaining with BiLSTM.Otherwise,position embedding is not necessary for SA-BiLSTM-CRF model because of LSTM.The multi-head mechanism in Attention can improve the model effect,but too many heads will lead to overfitting.Secondly,aimed to apply the SA-BiLSTM-CRF model to the enterprise domain with less corpus and solve the problem that deep network model is difficult to train,four optimizations are proposed which are based on transefer learning,self-training,active learning,self-training and active learning.The experiment verified the effectiveness of the four optimizations.At the same time,in the optimization strategy based on transfer learning,it is found that the best transfer learning method that fits the experimental situation is fine-tuning the whole pre-trained model.Among others,it is found that the algorithm combining self-training and active learning can complement each other,and the optimization effect is better than the single one.Finally,Combining the SA-BiLSTM-CRF model with the transfer learning optimization strategy and the optimization strategy based on self-training and active learning,a named entity recognition system suitable for enterprise domain is proposed,and the enterprise knowledge graph is completed.更多还原