【作者】 张翔；

【导师】 琚生根； 刘亮；

【作者基本信息】 四川大学 ， 工程硕士（专业学位）， 2023， 硕士

【摘要】 任务是自然语言处理(Natural Language Processing,NLP)领域的研究热点,它在搜索预览、知识提炼等场景下有着广泛应用。该任务的目标是利用计算机自动地从源文档或文档集合中抽取重要的句子组成摘要以概括全文所表达的语义信息,帮助人们从海量的文本数据中获取到关键的信息,从而辅助人类进行决策。通常包含编码器和句子打分器,编码器对输入的文档内容进行编码,学习文档的潜在语义表示,句子打分器计算相应的句子得分来决定抽取哪些句子作为摘要。目前实现抽取式摘要的主流方法是通过捕捉单词级别及句子级别的局部关系,计算得到句子的重要性得分从而选择句子。这种方法虽然取得了一定的效果,但得到的摘要质量不高,模型依然存在一些缺陷。第一,现有模型只对单词或句子间局部关系进行建模,忽略了原文的全局语义信息,模型容易受到局部非重要关系的干扰,从而影响模型表现。第二,现有的考虑摘要级别语义信息的模型虽然可以得到更适合作为摘要的句子集合,但是其基于先抽取句子然后进行句子重排序的两阶段过程,引入了额外的重排序器,极大地影响了模型推理速度并且导致了模型整体泛化性能较差,在现实世界中难以得到广泛应用。本文针对上述的两个问题分别进行了如下工作:(1)针对现有模型缺乏原文的全局语义信息的问题,本文提出了融入全局信息的抽取式摘要模型。它主要由两个模块组成,一个是捕捉文档潜在全局分布信息的全局信息提取模块,该模块将文档向量作为输入,使用方面抽取模型或神经主题模型建模文档的全局信息,然后通过对比学习(Contrastive Learning)进行优化;另一个模块是全局信息融入模块,它在句子级编码器中加入了全局注意力机制,将提取到的全局信息通过全局注意力融入到句子向量中,使模型具备全局语义感知的能力。(2)针对考虑摘要级别语义的两阶段模型推理速度慢且泛化性较差的问题,本文提出基于对比重排序和对抗权重扰动的抽取式摘要模型。该模型将句子抽取和句子重排序两个阶段融合为一个阶段,在对比学习框架下进行优化,并加入了对抗权重扰动机制增加模型的泛化能力。同时,利用标签期望算法从抽象式摘要数据集中生成抽取式标签信号,促使模型进行多参考摘要学习,也从另一个方面提升了模型对摘要级别信息的感知能力。(3)通过在CNN/Daily Mail等公开的文本摘要数据集上进行对比实验,验证了本文提出两个模型的有效性。同时,我们还对模型进行了一系列实验的探索,从多维度对模型进行了剖析。更多还原

【Abstract】 Extractive text summarization is a research hotspot in the field of Natural Language Processing(NLP),and it is widely used in scenarios such as search preview and knowledge extraction.The goal of this task is to use computers to automatically extract important sentences from source documents or document collections to form summaries to summarize the semantic information expressed in the full text and to help people obtain key information from massive text data,thereby assisting humans in decision-making.Extractive text summarization usually includes an encoder and a sentence scorer.The encoder encodes the content of the input document and learns the latent semantic representation of the document.The sentence scorer calculates the corresponding sentence score to determine which sentences to extract as summaries.At present,the mainstream method of extractive summarization is to select sentences by capturing the local relationship at the word-level and sentence-level,and calculating the importance score of the sentence.Although this method has achieved certain results,the quality of the summary obtained is not high,and the model still has some defects.First,the existing models only model local relationships between words or sentences,ignoring the global semantic information of the original text.The model is easily disturbed by local non-important relationships,which affects the performance of the model.Second,although the existing model considering summary-level semantic information can get a more suitable sentence set as a summary,it introduces an additional reranker based on the two-stage process of first extracting sentences and then reranking sentences.It greatly affects the reasoning speed of the model and the overall generalization performance of the model is poor,so it is difficult to be widely used in the real world.In this thesis,the following work is carried out for the above two problems:(1)Aiming at the problem that existing models lack the global semantic information of the original text,this thesis proposes an extractive summarization model that incorporates global information.It mainly consists of two modules,one is the global information extraction module that captures the potential global distribution information of the document,which takes the document vector as input,uses the aspect extraction model or the neural topic model to model the global information of the document,and then learns by contrastive learning for optimization;another module is the global information integration module,which adds a global attention mechanism to the sentence-level encoder,and integrates the extracted global information into the sentence vector through global attention,so that the model has global semantics the ability to perceive.(2)Aiming at the problem of slow reasoning and poor generalization of the two-stage model considering summary-level semantics,this thesis proposes an extractive summary model based on contrastive re-ranking and adversial weight perturbation.The model combines the two-stage process of extracting sentences and then reranking them into one stage,optimizes it under the framework of contrastive learning,and adds an adversial weight perturbation mechanism to increase the generalization ability of the model.At the same time,the oracle expectation algorithm is used to generate extractive label signals from the abstract summary dataset,which promotes the model to learn multi-reference summaries,and also improves the model’s perception of summary-level information from another aspect.(3)Through comparative experiments on public text summarization datasets such as CNN/Daily Mail,the effectiveness of the two models proposed in this thesis is verified.At the same time,we also conducted a series of experimental explorations on the model and analyzed the model from multiple dimensions.更多还原