【作者】 徐进；

【导师】 郑凯；

【作者基本信息】 电子科技大学 ， 工程硕士（专业学位）， 2021， 硕士

【摘要】 随着数据时代的到来,互联网上产生了海量的文本信息。由于大量非结构化的文本因其结构松散,内容冗杂,无法直接提取关键内容,给数据管理以及分析和挖掘带来了挑战,因此,信息抽取作为从大规模海量文本中抽取结构化知识的技术受到广泛的关注。作为信息抽取的关键子任务,实体识别和关系抽取是知识图谱,智能客服等自然领域上游任务的基础工作,也是帮助法律、金融、医疗等垂直领域实现文本自动化的关键技术,其任务是从输入文本中识别命名实体的边界和类型,并且判断实体之间是否存在某种类型的关系。近些年来深度学习发展迅速,其强大的参数学习和特征提取能力弥补了传统的机器学习算法和人工构建特征的不足,更加有利于实体识别和关系抽取模型的构建。传统的管道式抽取将实体识别和关系抽取分成两个独立的子任务分别构建模型,因此会导致上游任务误差的累积,信息冗余以及子任务之间缺乏关联等问题。本文对基于深度学习的实体关系联合抽取展开研究,调研了国内外研究发展现状,针对现有模型存在的缺陷和不足,提出了两种联合抽取模型,具体的工作内容如下所示:1.提出了一种基于多层指针网络与多头选择机制的联合抽取模型。通过使用基于BERT的共享编码层建立实体识别和关系抽取子任务之间的依赖,使得模型能够同时进行两个子任务,降低了误差传播的累计。针对实体识别的实体嵌套问题和关系抽取时的关系重叠问题,分别使用了多层指针网络和多头选择机制,使得模型能够更好地处理复杂实体关系。为了提高关系抽取的精确度,模型中加入了全局关系预测的辅助任务,让模型学习到句子级别的关系信息。为了提高模型的泛化能力,模型训练时加入基于FGM的对抗训练。模型在数据集上取得了良好的表现,并且通过消融实验验证了各个部分的有效性。2.提出了一种融合BERT和改进多头选择机制的联合抽取模型。在前述模型的基础上,为了更好地利用BERT模型的语义表达,重新构建了基于BERT编码层的输出向量,并且通过实验分析了当前任务下不同方式构建的BERT输出向量的性能。为了更好地构建关系预测矩阵,使用双仿射计算代替原有的线性变换,提高了实体对之间关系特征的交互;通过实验验证了模型的有效性,并且与现有的联合模型进行对比实验,取得了更佳的结果。更多还原

【Abstract】 With the coming of the big data era,massive amounts of text information have been generated on the Internet.Because a large number of unstructured texts are loosely structured and have cumbersome content,they cannot directly extract key content,which brings challenges to data management,analysis and mining.Therefore,how to extract massive texts on a large scale has become a research hotspot.Information extraction came into being.Entity recognition and relation extraction are key subtasks in the field of information extraction,which are the basic work of many NLP tasks such as knowledge graphs and intelligent Question-Answer and have made outstanding contributions to vertical fields such as finance,law,and medical care.The main purpose is to identify the boundaries and types of named entities from the text and determine whether there is a certain type of relationship between entities.In recent years,deep learning has developed rapidly.Its powerful parameter learning and feature extraction capabilities make up for the shortcomings of traditional machine learning algorithms and artificial features,which are more conducive to the construction of entity recognition and relationship extraction models.Traditional pipeline extraction treats entity recognition and relation extraction as two independent subtasks,which has problems such as error propagation and accumulation,information redundancy,and subtask dependence.This paper conducts research on the joint extraction of entity and relation based on deep learning,investigates the status quo of research in this thesis,and proposes two joint extraction models especially for the shortcomings and deficiencies of existing models.The main works are listed as follows:1.A joint extraction model based on multi-layer pointer network and multi-head selection mechanism is proposed.By using the BERT-based shared encoding layer to establish the dependence between entity recognition and relation extraction,the model can perform two subtasks at the same time,reducing the accumulation of error propagation.Aiming at the entity nesting problem in entity recognition and the relation overlap problem in relation extraction,a multi-layer pointer network and a multi-head selection mechanism are used respectively,so that the model can better handle complex entity relations.In order to improve the generalization ability of the model,auxiliary tasks of global adversarial training are also used.The model has achieved good performance on the data set,and the effectiveness of each part is verified through ablation experiments.2.A joint extraction model combining BERT and multi-head selection mechanism is proposed.On the basis of the aforementioned model,the embedding output based on the BERT encoding layer is reconstructed in order to make better use of the semantic expression of the BERT model and the performance of the BERT output vector constructed in different ways under the current task is analyzed through experiments.In order to better construct the relationship prediction matrix,biaffine calculation is used to replace the original linear transformation,which improves the interaction of relation features between entity pairs;the effectiveness of the model is verified through experiments and compared with the existing joint model.Experiments have achieved better results.更多还原