【作者】 张淑芳；

【导师】 唐焕玲；

【作者基本信息】 山东工商学院 ， 计算机科学与技术， 2022， 硕士

【摘要】 文本分类作为自然语言处理领域基础且关键的任务,实现文档资源的梳理、归纳、分类,应用于信息过滤、舆情检测及新闻文档分类等场景。近年来,在算法复杂化和结构层次化的趋势下,依赖大数据的深度学习,其鲁棒性和泛化性的表现效果优异。目前,深度学习被广泛用于文本分类任务,并取得卓越成就。然而,对于一些特定领域,数据采集与数据标注的代价昂贵,当样本稀少或缺乏标注数据,使得有监督深度学习出现过拟合现象,模型难以取得良好性能。虽然数据增强和正则化方法可以缓解过拟合问题,但并没有得到完全解决。相反,人类却能够通过少量知识学习到事物的本质特征,快速泛化到新任务的识别预测过程。受此启发,研究者提出小样本学习,旨在从少量数据中学习到解决问题的办法。因此,基于小型数据集的小样本学习方法,成为解决此类问题的关键技术。所以,如何在训练数据匮乏或标注数据代价大的情况下,取得优异性能是目前小样本学习研究的难点。针对小样本学习中标注样本匮乏、噪声数据干扰,导致特征表示差、模型泛化性能低的问题。本文从有监督主题模型和多层次特征表示方法展开研究,重点研究内容如下:(1)本文使用元学习来模拟小样本任务,结合有监督主题模型(Supervised LDA,SLDA)和动态路由算法提出一种新的动态路由原型网络模型DRP-SLDA(Dynamic Routing Prototypical Network based on SLDA)。利用SLDA主题模型建立词汇与类别之间的语义映射,增强词的类别分布特征,从词粒度角度编码获得样本的语义表示。提出动态路由原型网络DR-Proto,通过提取交叉特征利用样本之间的语义关系,采用动态路由算法迭代生成具有类别代表性的动态原型,旨在解决特征表达问题。实验结果表明,在Few Rel、20newsgroup、Sogou数据集上,由于DRP-SLDA模型有效提取词的类别分布特征,且获取动态原型提高类别辨识力,从而能够有效提升小样本文本分类的泛化性能。(2)本文在元学习框架内,构建双路网络结构引入主题,提出一种基于SLDA的特征融合模型FI-SLDA(Feature Integration based on SLDA)。引入SLDA模型,细化粒度将样本映射到词义丰富的主题空间。构建双路特征融合网络,Hierarchical Network利用多层卷积神经网络提取局部特征,获得支持集样本特征表示;Cross Network兼顾支持集和查询集的样本交互特征,利用注意力机制获得文档集全局特征表示;融合不同网络提取的特征,获得特征分布鲜明的类别向量,从而使分类边界更清晰。实验表明,对比FI-SLDA模型和6种已有小样本学习模型,FI-SLDA模型显著提高分类正确率;在20newsgroup、Sogou数据集上设置噪声数据比例0%、10%、30%、50%,双路网络分别有效提取局部特征和全局特征,说明FI-SLDA模型充分考虑支持集和查询集对分类的重要性,从而提高小样本文本分类精度。更多还原

【Abstract】 Text categorization,as a basic and key task in the field of natural language processing,implements the sorting,induction and classification of document resources,and applies to information filtering,public opinion detection and news document classification and other scenarios.In recent years,with the tendency of algorithm complexity and structure hierarchy,deep learning which relies on big data has excellent performance of robustness and generalization.At present,deep learning is widely used in text categorization tasks with remarkable achievements.However,for some specific fields,the cost of data collection and data annotation is expensive.When samples are scarce or annotated data is lacking,supervised deep learning has over-fitting phenomenon,and it is difficult for the model to achieve good performance.Although data augmentation and regularization methods can alleviate the over-fitting problem,this problem has not been completely solved.On the contrary,humans can learn the essential characteristics of things through a small amount of knowledge,and quickly generalize to the recognition and prediction process of new tasks.Inspired by this,researchers propose few-shot learning,which aims to learn solutions to problems from a small amount of data.Therefore,few-shot learning methods based on small data sets have become the key technology to solve such problems.Hence,how to obtain ideal performance results in the case of insufficient training data or high quality annotation data is the difficulty of current few-shot learning research.Aiming at the problem of poor feature representation and low model generalization performance due to the lack of labeled samples and noise data interference in few-shot learning.This paper carries out research from supervised topic model and multi-level feature representation method,and focuses on the following research contents:(1)In this paper,meta-learning is used to simulate few-shot tasks,and a new Dynamic Routing Prototypical Network based on SLDA(DRP-SLDA)is proposed by combining supervised topic model and dynamic routing algorithm.SLDA topic model is used to build semantic mapping between words and categories to enhance word category distribution characteristics,and the semantic representation of samples is encoded from the perspective of word granularity.A Dynamic Routing Prototype Network(DR-Proto)is proposed,which utilizes the semantic relationship between samples by extracting cross-features,and uses dynamic routing algorithm to iteratively generate dynamic prototypes with representative categories,aiming to solve the problem of feature expression.The experimental results show that on the Few Rel,20 newsgroup,and Sogou data sets,the DRP-SLDA model can effectively extract the category distribution characteristics of words,and obtain the dynamic prototype to improve the category recognition ability,so as to effectively improve the generalization performance of few-shot text categorization.(2)In this paper,within the framework of meta-learning,a two-way network structure is constructed to introduce topic,and a Feature Integration model based on SLDA(FI-SLDA)is proposed.The SLDA model is introduced to map the samples to the semantic-rich topic space with fine-grained granularity.A two-way feature integration network is proposed.The Hierarchical Network uses multi-layer convolutional neural network to extract local features to obtain support set sample feature representation;The Cross Network takes into account the sample interaction features of the support set and the query set,and uses the attention mechanism to obtain the global feature representation of the document set;The features extracted from different networks are fused to obtain the category vector with distinct feature distribution,which makes the classification boundary clearer.The experimental results show that the FI-SLDA model significantly improves the classification accuracy,which compared with the 6 existing few-shot learning models;With the noise ratio of 0%,10%,30% and50% is set on 20 newsgroup and Sogou data sets,the local feature and global feature are extracted effectively by two-way network respectively,indicating that the FI-SLDA model fully considers the importance of support set and query set to classification,so as to actively improve the accuracy of text categorization.更多还原