数据挖掘系统及其应用研究——用关联特征提高朴素贝叶斯文本分类器的性能

【作者】 张利军；

【导师】 康慕宁；

【作者基本信息】 西北工业大学 ， 计算机软件与理论， 2003， 硕士

【摘要】 文本分类就是自动地将自然语言文本文档根据其内容分类到预定义好的类别中去。随着在线文本文档数量的快速增长，文本分类已经成为处理和组织文本数据的一种关键技术。文本分类技术可以用来对新闻分类，可以从互联网上发现有用信息，还可以用在搜索引擎中提高搜索效率。 朴素贝叶斯分类器由于其简单性及计算的有效性，一直在文本分类领域中占有很重要的地位。但是由于传统的朴素贝叶斯分类器以单个的单词作为特征，并且认为任意两个单词之间的出现与否是互不相关的，即单词之间是相互独立的，在现实世界中这种假设显然是不成立的，这在一定程度上影响了分类器的性能，如果能够减轻或消除这一假设的影响，就可以提高分类器的性能。 为了减轻朴素贝叶斯假设对分类器性能的影响，我们提出了关联特征的概念，关联特征是多个同时在一个文档中频繁出现的单词的集合。关联特征不但能够比单个的单词更好地描述一个概念，而且能够描述它所包含的单个单词（称之为原始特征）不能描述的概念。 在Apriori算法的基础上，我们提出了一种挖掘关联特征的算法ATM（Association Terms Mining），用来在训练文档集上挖掘频繁的关联特征。为了压缩特征空间，提高分类效率和性能，同时提出了对关联特征进行剪枝的冗余剔除算法和利用信息增益进行筛选的特征筛选算法，并以关联特征作为分类特征实现了我们的文本分类系统NBAT。 在Reuters21578文档集中10个类别上的测试结果表明，关联特征能够提高朴素贝叶斯文本分类器的性能。使用关联特征比不使用关联特征在Support=0.1时，宏平均BEP提高了6.9％，而宏平均F₁提高了12.2％。 在本文的结尾，指出了NBAT的主要不足，即挖掘效率低，当特征空间较大，或设置的支持度阈值比较低时，难以挖掘出所需要的关联特征，并提出了可能的解决方法。最后展望了以后的研究工作的几个方向。更多还原

【Abstract】 Text categorization is the task of automated assignment of natural language texts to predefined categories based on their content. With the rapid growth of online digital text data, text categorization has become one of the key techniques for handling and organizing text data. Text categorization techniques can be used to classify news stories, to find interesting information on the WWW, and to guide a user’s search through hypertext.Naive Bayes(NB) classifier has long been considered a core methodology in text categorization mainly due to its simplicity and computational efficiency. The main drawback of Naive Bayes classifier is that it uses single words as features, and it assumes that the probability of appearance/absence for a word is independent from the presence/absence of any other word given that the class is known. In fact, this assumption is, quite obviously, not verified in practice, so the performance of the Naive Bayes Classifier is reduced. If the assumption can be relaxed to a certain extent, then we can enhance the performance of Naive Bayes Classifier.In order to relax the strength of the assumption, we propose the concept of association terms. An association term is a set of single words(these single words are called as primitive terms) that occurrence together frequently. Association terms make better features than primitive terms. In many cases, an association term describes a concept better than its component words. In other cases, the concept is described only by it, not by its component words.We devised an simple algorithm based on Apriori, which can be used to mining association terms. In order to reduce the size of feature space and improve the efficiency and effectiveness of classification, two algorithms were proposed to select useful association terms: redundancy pruning algorithm and feature selection by information gain algorithm. By using association terms as features, we built our text classifier: NBAT(Naive Bayes text classifier by Association Terms).NBAT was run on the ten most populated categories from Reuters21578 dataset. The result of experiment showed that the performance of the Naive Bayes text classifier can be improved by using association terms. When using association terms as features, the increase in macro-averaged break-even point was 6.9% and inmacro-averaged Fl measure was 12.2%.Finally, we point out the weakness of our algorithm. The main weakness of our algorithm is the low mining efficiency, especially when the size of feature space is very large or the minimum support threshold is very low. A possible solution to this problem was suggested: using some association terms mining algorithm without generating candidate frequent itemsets, such as FP Tree etc. At the last, we presented some research work in this field that should be done in the future.更多还原