【作者】 伍建军； 康耀红；

【Author】 WU Jian-jun, KANG Yao-hong (College of Information Science & Technology, Hainan University, Hainan, Haikou 570228,China)

【机构】 海南大学 信息科学技术学院；

【摘要】 文本分类的一个关键点就是如何对高维的特征集进行降维。而特征降维的常用方法就是特征选择、特征抽取等。首先介绍了几种常见的特征选择和特征抽取方法,并对这些特征降维方法的优缺点进行了分析。最后结合 K-近邻分类算法对四种特征选择方法(文档频率、互信息、CHI统计法、相关系数法)进行了分类测试,同时通过测试分析,提出了一些改进的、可行的互信息评价函数,实验结果表明,采用这种方法,在同等训练语料的情况下,分类效果比基于传统的互信息评价函数的分类效果要好。更多还原

【Abstract】 The key point of text categorization is how to reduce the high dimension of the feature vectors. Feature selection and feature extraction is the main methods of feature reduction. This paper introduces five methods of feature selection and feature extraction. And then k-nearest neighbor is selected as the evaluating classifier to compare the performance of the four feature selection methods(including Document Frequency、Mutual Information、Chi-square、correlation coefficient) in text categorization. From the test result, a new improved method of FS is presented based on Mutual Information, and is proved to be effective by experiment.更多还原