【作者】 何玉；

【导师】 冯剑琳；

【作者基本信息】 华中科技大学 ， 计算机软件与理论， 2006， 硕士

【摘要】 随着信息技术的不断发展,特别是Internet应用的普及,电子化文本数量迅速增长。为了快速的处理和查询所需要的文本信息,要求能够有效地组织文本。文本分类技术常被用于大量文本数据或超文本数据的组织,然而传统的文本分类技术已经难以满足当今分类任务高维度、大数量以及高可读性的要求。新近提出的一些基于关联规则的文本分类算法普遍采用这样的策略:文档中的单词被看作是项目(items);每一篇文档被看作是一个事务(transaction),即项目的集合,在此基础上进行频繁项集的挖掘。然而,在一篇文档中,能够表达一个完整语义的最小单位是句子,属于同一句子的单词集合往往比属于不同句子的单词集合语义联系要丰富得多。基于以上考虑,提出一种基于句子级频繁项目集的文本分类算法。研究了使用句子级别的最大/闭合关联规则来表示文档,并且根据从文档中挖掘出的最大/闭合关联规则来构造类的分类器以及使用分类器进行分类。此外,与传统的基于文档覆盖的修剪方式不同,它使用一种简单适中的方法进行剪枝,提高了修剪分类规则的速度。通过在新闻数据集Reuters和电子邮件数据Enron上的实验证明,这可以取得与其它几个著名算法媲美的分类效果,并且远远超过了目前文档级的关联规则分类算法。更多还原

【Abstract】 With the development of information technology, especially the popularization of Internet, the number of electric text grows rapidly. It is necessary to organize the information for faster and easier processing and searching. Text classification is often used to organize and manage large sets of text or hypertext documents, but most of the conventional algorithms can not meet the requirement of current classification task, such as high dimensionality, high volume and high readability.While previous association-rule-based text classification follows such an idea: a document is treated as a transaction and frequent itemsets are mined from the transaction database consisting of all documents. However the basic semantic unit in a document is actually a sentence. A group of words co-occurring in the same sentence are usually more meaningful than the same group of words spanning several sentences.Based on the above intuition, this paper presents a new frequent itemset based text classify method. It views a sentence rather than a document as an association transaction. It mine the maximal frequent itemset or close frequent itemset in the sentence level, and use these itemset to construct prefix trees and classify the test documents. classification speed can be improved greatly. It use a novel heuristic instead of the traditional method based on the document coverage to prune the association rules, it improve the speed of pruning rules.The effectiveness of this method has been demonstrated comparable to well-known alternatives and much better than current document-level association based methods on the Reuters and Enron corpus.更多还原