基于混合神经网络的文本分类研究

【作者】 张倩；

【导师】 高建瓴；

【作者基本信息】 贵州大学 ， 信息与通信工程， 2023， 硕士

【摘要】 随着信息化时代的不断发展,承载着丰富信息的文本数据呈指数型增长,如何对结构复杂的文本信息进行分类处理并从中挖掘出利于社会各界使用的有效信息,是当下研究的热点与难点。神经网络方法因具有强大的特征表达能力以及大规模数据处理能力被广泛应用在文本分类任务中,但仍存在部分不足。如:卷积神经网络侧重提取文本的局部特征,但难以处理长文本结构信息以及捕获上下文语义关系;循环神经网络侧重提取文本长距离信息,但存在短期记忆问题,极易造成梯度消失现象。可见,单一的神经网络无法充分提取文本特征,从而导致分类效果不理想。针对上述问题,本文对文本分类中的文本表示、特征提取等技术展开研究,提出了基于混合神经网络的文本分类模型,主要研究内容如下:(1)针对单神经网络模型特征提取不充分的问题,提出基于自注意力机制和残差网络的混合神经网络模型(SAR-BC)。该模型首先通过Bi GRU-CNN提取输入文本的上下文语义特征以及局部特征,接着利用自注意力机制对卷积运算后的信息赋予权重,筛选出关键文本特征,最后对池化后的特征信息层归一化处理后引入残差网络,解决深度神经网络中的梯度消失以及网络退化问题。此外,在模型运算过程中使用了更为光滑的Mish函数作为非线性激活函数,以此提高模型泛化能力。通过在中、英文数据集上进行对比实验和消融实验得出,本文构建的基于自注意力机制和残差网络的混合神经网络模型性能指标均优于其他对比模型,证实该模型能提取到更全面的文本特征,从而提高分类性能。(2)针对中文文本存在向量表征过于单一以及分类准确率较低等问题,本文在第三章的基础上提出一种基于多特征融合的混合神经网络模型。该模型首先利用Word2vec获取词级别的原始语义向量,并通过词性特征、拼音字符特征以及五笔字符特征进行特征扩展,以此降低中文文本中同音词、错别字等问题带来的影响。接着将处理好的特征表示分别输入到SAR-BC模型中进行全局和局部双重特征提取,最后对输出的多种语义特征进行融合来丰富文本语义表示,并输入到全连接层中完成分类。通过在两个中文数据集上进行对比实验和消融实验得出,本文构建的基于多特征融合的混合神经网络模型性能指标均优于实验中其他对比模型,证实该模型能获取到更丰富的中文特征表示,从而提升模型性能。更多还原

【Abstract】 With the development of the information age,the text data carrying rich information is growing exponentially,and it is a hot and difficult research task to classify the complex text information and extract the effective information for the use of the society.Neural network methods are widely used in text classification tasks because of their powerful feature representation and large-scale data processing capability,but there are still some shortcomings.For example,convolutional neural networks focus on extracting local features of text,but it is difficult to handle long text structure information and capture contextual semantic relationships;recurrent neural networks focus on extracting long distance information of text,but there is a short-term memory problem and it is very easy to cause gradient disappearance.It can be seen that a single neural network cannot fully extract text features,which leads to unsatisfactory classification results.To address the above problems,this paper researches the text representation and feature extraction techniques in text classification,and proposes a hybrid neural network-based text classification model.The main research contents are as follows:(1)A hybrid neural network model(SAR-BC)based on self-attentive mechanism and residual network is proposed to address the problem of inadequate feature extraction of single neural network model.The model first extracts the contextual semantic features and local features of the input text through Bi GRU-CNN,then uses the self-attentive mechanism to assign weights to the convolutional information to filter out the key text features,and finally introduces the residual network after normalizing the pooled feature information layer to solve the problem of gradient disappearance and network degradation in deep neural networks.In addition,a smoother Mish function is used as the nonlinear activation function during the model operation to improve the generalization ability of the model.The performance of the hybrid neural network model based on the self-attentive mechanism and the residual network is better than other comparative models,which confirms that the model can extract more comprehensive text features and thus improve the classification performance.(2)To address the problems that Chinese text has too single vector representation and low classification accuracy,this paper proposes a hybrid neural network model based on multi-feature fusion on the basis of Chapter 3.The model first uses Word2 vec to obtain the original semantic vectors at the word level,and then extends the features with lexical features,pinyin character features and five-stroke character features to reduce the impact of homophones and misspellings in Chinese texts.The processed feature representations are then input to the SAR-BC model for both global and local feature extraction,and finally the output multiple semantic features are fused to enrich the semantic representation of the text and input to the fully connected layer to complete the classification.Through comparison experiments and ablation experiments on two Chinese datasets,the performance indexes of the hybrid neural network model based on multi-feature fusion constructed in this paper are better than those of other comparison models in the experiments,confirming that the model can obtain richer Chinese feature representations and thus improve the model performance.更多还原