【摘要】 目前，基于BERT预训练的文本摘要模型效果良好。然而，预训练模型内部使用的自注意力机制倾向于关注文本中字与字之间的相关信息，对词信息关注度较低，并且在解码时存在语义理解不充分的情况。针对上述问题，该文提出了一种基于BERT的语义增强文本摘要模型CBSUM-Aux(Convolution and BERT Based Summarization Model with Auxiliary Information)。首先，使用窗口大小不同的卷积神经网络模块提取原文中的词特征信息，并与输入的字嵌入进行特征融合，之后通过预训练模型对融合特征进行深度特征挖掘。然后，在解码输出阶段，将卷积之后的词特征信息作为解码辅助信息输入解码器中指导模型解码。最后，针对束搜索算法倾向于输出短句的问题对其进行优化。该文使用LCSTS和CSTSD数据集对模型进行验证，实验结果表明，该文模型在ROUGE指标上有明显提升，生成的摘要与原文语义更加贴合。更多还原

【Abstract】 BERT-based pre-trained text summarization models are featured by the self-attention mechanism capturing the relevant information between words in the text. To better capture the information in words themselves, we present a semantically enhanced text summarization model CBSUM-Aux(Convolution and BERT based summarization model with auxiliary information) based on BERT. First, the word information in the original text is extracted using convolutional neural network modules with different window sizes, and feature fusion is performed with the input word embedding. Then, in the decoding output stage, the word information after convolution is input to the decoder as decoding auxiliary information to guide the model decoding. Finally, the bundle search algorithm is optimized to avoid un-desired short sentence generations. Experiments on LCSTS and CSTSD datasets show that the proposed model improves ROUGE score, as well as a better match with the semantics of the original text.更多还原