【作者】 王兴龙；

【导师】 杜伟；

【作者基本信息】 吉林大学 ， 计算机技术（专业学位）， 2022， 硕士

【摘要】 随着移动互联网的快速发展,众多自媒体APP喷涌而出,大大丰富了人们的生活,但与此同时也积累了海量的文本信息。自动文本摘要技术能从繁杂的文本中提取出主旨信息,过滤无关内容,提升人们的工作效率。目前,序列到序列(Seq2Seq)模型成为自动文本摘要技术中的主流研究方向之一。Seq2Seq模型由编码器和解码器两部分所构成,它能够灵活地处理输入输出数据,应用非常广泛。虽然Seq2Seq模型在文本摘要领域中取得了一定的突破,但还有一些问题值得关注和研究。例如,关键词表示不全、单词或语句重复生成、未登录词、曝光偏差等问题。本文针对以上问题,提出一种基于关键词替换的多头注意力机制Seq2Seq模型,和基于改进注意力机制与强化学习的Seq2Seq模型生成方法。主要工作分为以下两部分:(1)实现了一种基于关键词替换的多头注意力机制Seq2Seq模型。主要内容包括:多头注意力机制,模型通过多头注意力机制建立编码器和解码器之间的联系,能够学习到文本在不同空间中的特征信息,解决了长距离依赖问题;关键词替换,该方法利用基于词向量的Text Rank算法对源文本进行关键词的提取,生成一个关键词集,而后对前面模型生成的摘要进行关键词替换,以形成最后的生成摘要,解决了关键词表示不全的问题。最后在万方科研数据集上进行实验,实验结果验证了模型的有效性。(2)实现了一种基于改进注意力机制与强化学习的Seq2Seq模型。主要内容包括:改进注意力机制,编码器采用编码内部注意力关对历史注意力高的部分进行惩罚,解码器采用解码内部注意力对历史预测出的单词进行惩罚,避免当前时刻生成新单词和前面有重复,解决了单词的重复生成问题;指针生成机制,通过使用指针生成器计算指针概率,再依据指针概率决定从输入文本中选择已有单词,还是从固定的词汇表中生成单词,缓解了未登录词出现的问题;强化学习,把生成摘要模型当成智能体,将输出生成摘要作为动作,将生成摘要与输入参考摘要对比的Rouge得分作为奖励,并采用策略梯度方法反复训练生成摘要模型。解决了曝光偏差问题。最后在LCSTS数据集上进行实验,实验结果验证了模型的有效性。更多还原

【Abstract】 With the rapid development of the mobile Internet,many self-media APPs have spewed out,which has greatly enriched people’s lives,but at the same time,massive amounts of text information have also been accumulated.Automatic text summarization technology can extract key information from complex texts,filter irrelevant content,and improve people’s work efficiency.At present,the sequence-tosequence(Seq2Seq)model has become one of the mainstream research directions in automatic text summarization technology.The Seq2 Seq model consists of an encoder and a decoder.It can flexibly process input and output data and widely used in many fields.Although the Seq2 Seq model has made some breakthroughs in text summarization,there are still some issues that deserve attention and research.For example,such as incomplete keyword representation,repeated words or sentences,unknown words,and exposure bias problem.Aiming at the above problems,this paper proposes a Seq2 Seq model based on multi-head attention mechanism and keyword replacement,and a Seq2 Seq model generation method based on improved attention mechanism and reinforcement learning.The main work is divided into the following two parts:(1)A Seq2 Seq model based on multi-head attention mechanism and keyword replacement.The main contents include: multi-head attention mechanism,the model establishes the connection between the encoder and the decoder through the multi-head attention mechanism,can learn the feature information of text in different spaces,and solves the problem of long-distance dependence;keyword replacement,The method uses the Text Rank algorithm based on word vector to extract keywords from the source text to generate a keyword set,and then replace the keywords generated by the previous model to form the final abstract,which solves the problem of incomplete keyword representation.Finally,experiments are carried out on the Wan Fang scientific research data set,and the experimental results verify the effectiveness of the model.(2)A Seq2 Seq model based on improved attention mechanism and reinforcement learning.The main contents include: the Intra-attention mechanism,the encoder uses the encoding internal attention to penalize the part with high historical attention,and the decoder uses the decoding internal attention to penalize the historically predicted words,in order to avoid generating the same word at the current moment.which solves the problem of repeated word generation;the pointer generation mechanism,it calculates the pointer probability by using the pointer generator,and then decides whether to select an existing word from the input text or generate a word from a vocabulary according to the pointer probability.which solves the problem of unknown words;reinforcement learning,it uses the generated summary model as an agent,uses the output generated summary as an action,and uses the Rouge score of the generated summary compared with the input reference summary as a reward,and uses the selfcritic strategy gradient method repeatedly training the generated summary model.which solves exposure bias problem.Finally,experiments are carried out on the LCSTS dataset,and the experimental results verify the effectiveness of the model.更多还原