【作者】 李昊；

【导师】 何晓飞；

【作者基本信息】 浙江大学 ， 计算机科学与技术， 2019， 硕士

【摘要】 自然语言处理是人机交互领域的重要核心,是近些年人工智能领域快速发展的方向之一,吸引着研究人员的关注。在深度神经网络被提出后,尤其以序列到序列模型(Seq2Seq)为代表的模型被广泛应用在文本生成任务中,该方向相关的各项任务,如机器翻译、文本等,性能都得到了较大的提高。然而,受限于神经网络数据驱动的特点,相关的自然语言模型只能在构建在规模巨大的语料库上。一旦将已经预训练完毕的模型应用在其他语料,模型的效果会发生明显的削弱。因此,本文对自然语言模型中广泛使用的序列到序列(Seq2Seq)结构进行改进,增加了能够利用迁移学习的功能结构,使其能够利用先验知识或不同任务的预训练模型。相比于原始的Seq2Seq结构,该模型能够利用先验知识和已构建好的预训练模型,即使在小规模数据上也能够达到较好的效果。本文在若干著名的公开数据集和爬虫抓取的网络社区数据集上进行了实验,实验结果表明,本文提出的迁移学习Seq2Seq模型能够很好地利用先验知识以及自然语言任务中学习到的通用知识,在多项指标上均超过了现有的先进算法,实现了迁移学习在不同数据集、不同任务上的普适性。本文所衍生的部分研究被学术界认可并发表了相关的文章至顶会NIPS与IJCAI,在国际竞赛与相关企业实践中都取得了切实的成绩,证明了该研究的学术价值和工业价值。更多还原

【Abstract】 Natural language processing(NLP)is a core part of huma.n-computer interaction,and it,is one of the rapid development directions in the field of artificial intelligence in recent years,attracting the attention of researchers.After the deep neural network is proposed,sequence-to-sequence model(Seq2Seq)is widely used in text generation tasks.With the help of Seq2Seq model,the text generation tasks,such as machine translation and summarization get great improvements.However,since neural network relys on large datasets,these models could only be built on a large corpus.Once the pre-trained model is applied to other corpora,the effect of the model will be significantly weakened.Therefore,this thesis improves the Seq2Seq model by adding advanced semantic encoders that could utilize transfer learning,enabling models to use prior knowledge or pre-trained models for different tasks.Compared to the original Seq2Seq model,the transfer learning model could achieve better results even on small datasets.In this thesis,experiments are carried out on several well-known public datasets and crawler crawling datasets.The experimental results show that the proposed transfer learning Seq2Seq model could make good use of prior knowledge and general knowledge learned in NLP tasks.The model achieves the universality of transfer learning in different datasets and different tasks.更多还原