【作者】 王旭；

【导师】 王能义；

【作者基本信息】 华中科技大学 ， 应用统计（专业学位）， 2022， 硕士

【摘要】 随着大数据与云计算技术的飞速发展,海量文本数据的获取变为可能,自然语言处理已经成为人工智能领域中最重要的方向之一。现如今的自然语言处理主要是通过大型的深度学习模型来对文本数据进行分析,近些年来,以注意力机制构建的深度模型在文本数据建模上不断刷新记录,成为了自然语言处理模型的主要发展方向。注意力机制相比于传统的循环神经网络结构具有可并行化、计算速度快、模型精度高等一系列优点,但是由于抛弃了循环神网络的顺序结构,注意力机制需要使用位置编码来建模文本的位置信息。近些年来越来越多基于注意力机制的模型被提出,而每个模型都具有其独特的位置编码方式。但是,几乎所有的位置编码都只考虑了输入文本是按照从前到后顺序阅读的情况,没有考虑输入文本是乱序的、非自然语言顺序阅读的文本的情况。本文在现有的绝对位置编码和相对位置编码的基础上进行了改进,提出了一种新型位置编码来解决乱序文本输入问题。具体的改进方法为结合文字在图片中的信息,通过图片特征建立从文字在序列中的原位置到文字在图片中特征排名的映射关系。其原理类似于将乱序的文本序列按照图片特征进行重新排序,通过将额外信息融入位置编码来降低模型识别乱序文本输入的难度。通过介绍一个乱序文本输入的实际任务场景:POI名称识别,来测试改进的效果。基于transformer模型将改进的位置编码应用在该任务中,并且设计了数据增强,对抗训练以及模糊匹配三种提升模型效果的算法。最后通过对比实验分析模型的分级得分和BLEU得分,来说明了本文提出的方法对模型效果的提升。更多还原

【Abstract】 With the rapid development of big data and cloud computing technology,the acquisition of massive text data has become possible,and natural language processing has become one of the most important directions in the field of artificial intelligence.Today’s natural language processing mainly analyzes text data through large-scale deep learning models.In recent years,deep models built with attention mechanisms have continuously refreshed records in text data modeling and have become the most popular natural language processing models’ main development direction.Compared with the traditional recurrent neural network structure,the attention mechanism has a series of advantages such as parallelization,fast calculation speed,and high model accuracy.However,because the sequential structure of the recurrent neural network is abandoned,the attention mechanism needs to use positional encoding to model text.location information.More and more attention-based models have been proposed in recent years,and each model has its own unique positional encoding.However,almost all positional encodings only consider the case where the input text is read in front-to-back order,and do not consider the case where the input text is out-of-order,text that is not read in natural language order.Based on the existing absolute position coding and relative position coding,this paper proposes a new position coding to solve the problem of out-of-order text input.The specific improvement method is to combine the information of the text in the picture,and establish a mapping relationship from the original position of the text in the sequence to the feature ranking of the text in the picture through the image features.The principle is similar to reordering the out-of-order text sequence according to the image features,and it reduces the difficulty of the model to recognize out-of-order text input by incorporating additional information into the position encoding.The improvement is tested by introducing a real task scenario of out-of-order text input: POI name recognition.Based on the transformer model,the improved positional encoding is applied to this task,and three algorithms to improve the model effect are designed: data augmentation,adversarial training and fuzzy matching.Finally,by comparing the classification score and BLEU score of the experimental analysis model,the improvement of the model effect of the method proposed in this paper is illustrated.更多还原