【作者】 马超；

【导师】 胡亮；

【作者基本信息】 吉林大学 ， 计算机技术（专业学位）， 2019， 硕士

【摘要】 近年来,随着机器学习和深度学习的迅速发展,对高质量的训练数据的需求越来越高。目前获取高质量数据的方式主要有搜索现有文献资料、查找开源数据集等,但到特定领域、面对特定问题时,常常难以直接找到有效的高质量数据,就需要在人工搜集的基础上再做进一步处理。这时会面对两种困境,一种是搜集到的数据中包含了少量的噪声数据,另一种是搜集到的数据中只有少量数据是需要的,其他大量的是噪声数据。对于这两种情况,都需要我们选取有效的方法把需要的数据挑选出来,剔除不需要的噪音数据。目前学术届和工业界的研究者已经提出了多种离群点检测方法来尝试解决这个问题,主要有基于频率的方法、基于统计的方法、基于深度或距离的方法和基于机器学习的方法等。这些方法对于结构化数据取得了较好效果,但当面对非结构化数据尤其是文本数据时,很多方法收效甚微。本文尝试将卷积神经网络引入文本离群点检测,并加以改进,具体工作有:1.分析了文本离群点的特点,提出了基于卷积神经网络的文本离群点检测方法。相较于循环神经网络按照时间步伐依次输入数据的特点,卷积神经网络的池化操作会丢失部分位置信息,更符合乱序文本检测的特点;同时其卷积操作很好的模仿了语言模型的n-gram方法。2.提出从目标数据到构建对照集,再到词向量预训练和增益训练、模型训练和迭代的完整流程方法。本文通过Xenc工具计算域外数据与域内数据的交叉熵并排序,并将排序后的数据按照一定比例有序挑选出对照集;采用了先用大语料预训练一个词向量模型,再添加域内语料进行增益训练的方式,权衡了词本身信息与词语使用场景之间的关系;采用了训练迭代的方式不断逼近预期效果。3.针对短文本的文本离群点检测,提出了融合词性特征的卷积神经网络模型,并设计实验验证其有效性。很多口语化语句的句法信息是错误的,但词法信息多数得以保留,引入词性信息可以更好的扩展信息维度,尤其是对指令性语句,作用更大。4.针对小数据集上的文本数据,提出了先对词语进行位置编码,并在位置编码后打乱词语顺序扩充数据集的方法,在尽量保留原始文本信息的同时有效扩充了数据量。通过实验验证了有效性。更多还原

【Abstract】 In recent years,with the rapid development of machine learning and deep learning,the demand for high quality training data is increasing.At present,the methods for obtaining high-quality data mainly include searching for existing literature materials,searching for open source data sets,etc.But when it comes to specific fields and facing specific problems,it is often difficult to directly find effective high-quality data,which needs to be based on manual collection.Do further processing.At this time,there are two kinds of dilemmas.One is that the collected data contains a small amount of noise data.The other is that only a small amount of data is needed in the collected data,and the other large amount is noise data.For both cases,we need to choose an effective method to select the required data and eliminate the unwanted noise data.Researchers in the academia and industrial circle have proposed a variety of outlier detection methods to solve this problem,including frequency-based methods,statisticalbased methods,depth or distance-based methods,machine-based learning methods and so on.These methods have achieved good results for structured data,but many methods have had little effect when faced with unstructured data,especially textual data.This paper attempts to introduce convolutional neural networks into text outlier detection and improve them.The specific work includes:1.The characteristics of text outliers are analyzed,and a text outlier detection method based on convolutional neural network is proposed.Compared with the characteristics of the recurrent neural network inputting data according to the time step,the pooling operation of the convolutional neural network will lose part of the position information,which is more in line with the characteristics of out-of-order text detection.At the same time,its convolution operation mimics the n-gram language model very well.2.This paper proposes a complete process approach from target data to building a control set to word vector pre-training and gain training,model training and iteration.In this paper,the Xenc tool is used to calculate the cross entropy of the extra-domain data and the intra-domain data,and the sorted data is sorted out according to a certain proportion.The word set vector model is pre-trained with large corpus,and then the domain corpus is added.The way of gain training balances the relationship between the information of the word itself and the scene of the word used;the method of training iteration is used to continuously approach the expected effect.3.For the text outlier detection of short text,a convolutional neural network model with morphological features is proposed,and the experiment is designed to validate it effectively.The syntactic information of many colloquial sentences is wrong,but the majority of lexical information is preserved.The introduction of part of speech information can better expand the information dimension,especially for directive statements.4.Aiming at the text on the small dataset,the method of first encoding the words and arranging the words to expand the data set after the position coding is proposed.The amount of data is effectively expanded while keeping the original text information as much as possible.The validity was verified by experiments.更多还原