【作者】 张建国；

【导师】 李建平；

【作者基本信息】 电子科技大学 ， 工程硕士（专业学位）， 2021， 硕士

【摘要】 随着人工智能技术的不断进步,现有机器模型已经基本达到了感知智能,正朝着认知智能前进。自然语言处理是智能认知的基础,是学界和工业界的研究热点。为了满足社会对各种语言的需求以及世界各国日益频繁交流更加的便捷,价格低廉的机器翻译研究正在逐渐兴盛。随着深度学习技术的持续提高,机器翻译也逐渐融合了这些方法和策略,并且在多个任务中斩获不错的成绩。但是还存在一些不足的地方。首先,大多数的翻译模型都基于注意力机制,去解决双语间的词语对齐问题。然而基于softmax进行归一化计算的注意力,导致无关词之间也有少量的注意力分布。因此,如何获取更加精准的注意力分布至关重要。其次,大部分神经翻译模型都是以“编码-解码”结构为基础的。整个模型的翻译都依赖自回归机制,所以每生成下一个词都是基于已完成的词,导致了模型的解码效率不高以及不能获取译文的全局消息。最后,词向量是模型获取语义以及语法信息的根本所在,因而如何获得蕴含更全面语义和语法信息的词向量十分关键。基于上文所述的问题,本文主要进行如下几个方面的研究:1、针对翻译中注意力的精准对齐问题,本文采用了稀疏归一化的方法去代替常用的softmax归一化,并在以Transformer为基础框架的神经机器翻译系统上进行了试验论证。实验结果表明,通过稀疏预测词中最大权值的相关词,减少无关词的不必要权值分布,缓解了数据之间归纳偏差问题,增强了翻译系统的准确性和可解释性。2、针对Transformer在推理阶段解码时间随译文长度成平方增长的问题,本文采用了累加方式的平均注意力层去缓解此问题。此外在神经机器翻译模型中,只可以利用前序生成序列信息的问题,本文融合了推敲神经网络的思想,通过两次解码获取了相关生成语句的全局信息。实验结果显示,两次解码后翻译的句子更加连贯且句义更加完整。3、针对目前大多数模型采用基于单词的嵌入向量表示,提出了多表征融合的词向量,使用字符级别编码向量和单词级别编码向量直接拼接的方法。多表征融合的词向量可以有效的解决词表中的未出现词以及一些低频词,可以表达更加完整词义信息,直接影响整个翻译模型性能。通过实验结果可以看出,我们提出的融合方法和策略,有效的提升整体翻译模型的翻译效果与质量。更多还原

【Abstract】 With the continuous advancement of artificial intelligence technology,existing machine models have basically reached perceptual intelligence and are moving towards cognitive intelligence.Natural language processing is the foundation of intelligent cognition and a research hotspot in the academic and industrial circles.In order to meet the society’s needs for various languages and to make it more convenient for countries around the world to communicate with each other more and more frequently,low-cost machine translation research is gradually flourishing.With the continuous improvement of deep learning technology,machine translation has gradually integrated these methods and strategies,and achieved good results in multiple tasks.But there are still some shortcomings.First of all,most translation models are based on the attention mechanism to solve the problem of word alignment between bilinguals.However,the attention of normalized calculation based on softmax results in a small amount of attention distribution among irrelevant words.Therefore,how to obtain a more precise attention distribution is very important.Secondly,most neural translation models are based on the "encoder-decoder" structure.The translation of the entire model relies on the autoregressive mechanism,so each generation of the next word is based on the completed word,which leads to the low decoding efficiency of the model and the inability to obtain the global message of the translation.Finally,the word vector is the basis for the model to obtain semantic and grammatical information,so how to obtain the word vector containing more comprehensive semantic and grammatical information is very important.Based on the problems mentioned above,this thesis mainly conducts the following researches:1.Aiming at the problem of precise alignment of attention in translation,this thesis uses the sparse normalization method to replace the commonly used softmax normalization,and conducts experimental demonstrations on the neural machine translation system based on Transformer.The experimental results show that by sparsely predicting the related word with the largest weight in the word,the unnecessary weight distribution of irrelevant words is reduced,the problem of inductive bias between data is alleviated,and the accuracy and interpretability of the translation system are enhanced.2.Aiming at the problem that the decoding time of Transformer in the inference stage increases with the square of the translation length,this thesis adopts the cumulative average attention layer to alleviate this problem.In addition,in the neural machine translation model,only the preamble can be used to generate sequence information.This thesis integrates the idea of scrutinizing neural networks,and obtains the global information of related generated sentences through two decodings.The experimental results show that the translated sentences after two decodings are more coherent and the sentence meaning is more complete.3.In view of the fact that most of the current models adopt word-based embedding vector representation,a multi-characterization fusion word vector is proposed,and a method of using character-level coding vector and word-level coding vector to directly concatenate.The word vector of multi-characterization fusion can effectively solve the non-appearing words and some low-frequency words in the vocabulary,can express more complete word meaning information,and directly affect the performance of the entire translation model.It can be seen from the experimental results that the fusion methods and strategies we propose effectively improve the translation effect and quality of the overall translation model.更多还原