代价敏感的支持向量机监督学习研究

【作者】 李钢；

【导师】 王蔚；

【作者基本信息】 南京师范大学 ， 教育技术学， 2007， 硕士

【摘要】 监督学习中的误分类代价问题是近年来机器学习技术走向现实的过程中产生的。为了更好地解决真实世界中的此类问题，代价敏感学习成为国际机器学习界的一个研究热点。支持向量机(Support Vector Machines，SVM)源于统计学习理论，是目前为止一种强有力的机器学习算法，但和传统的决策树、神经网络算法一样，SVM不具有代价敏感性，不能直接用于代价敏感学习。因此，设计代价敏感的SVM，并尽可能的提高其性能显得尤为重要。本文以支持SVM为主线，针对监督学习中错误分类引起的代价问题，对实现代价敏感的SVM以及提高其性能方面做了如下工作：1．在研究统计学习理论的基础上，指出了SVM在监督学习中优于其他学习算法的原因，并实现了一种标准SVM的训练算法—SMO算法。2．在实现SMO算法的基础上，通过样本类空间重构的方式，设计并实现了基于随机过取样、SMOTE、欠取样的三种代价敏感SVM。并且借鉴代价敏感神经网络集成方法，对三种代价敏感SVM分别进行了软集成和硬集成。在采用软集成时，针对SVM的特点采用了不同于神经网络的软集成方法。试验结果表明：总体性能上，基于欠取样的代价敏感SVM是一种非常好的方法；但是在数据集严重不平衡时，该方法是失效的，而软集成在此类数据集时性能最好。另外还得出了一些更详细的结论，为采用此类方式处理代价敏感问题提供了参考依据。3．针对Lin等人提出的代价敏感SVM(CSSVM)，进行了理论研究和实验测试，试验结果表明，虽然该方法可以有效降低总体错误分类的代价，但是模型中的参数选择的好坏影响了CSSVM算法的学习性能。针对自动选取最优参数问题，本文将遗传算法与代价敏感SVM结合起来，设计并实现了基于遗传算法的参数选择，提高了代价敏感SVM的性能。更多还原

【Abstract】 The problem of misclassification cost in supervised learning appears in the process of machine learning technology applied to the reality in recent years. In order to solve real-world problems in a better way, cost-sensitive learning has become one hot topic of current international machine learning community. Support Vector Machines(SVM) is an excellent machine learning algorithm based on statistical learning theory. However, SVM is not cost sensitive, like decision tree, artificial neural network(ANN) and other traditional algorithms. So, how to design cost sensitive SVM and to improve its performance become more import.This thesis studies the implementation of cost sensitive SVM and the methods to improve its performance. In detail, the main works are as following:1. Discuss statistical learning theory and find out the reason that SVM is more effective than other algorithms. Then, a training algorithm of standard SVM-SMO algorithm is implemented.2.Three cost sensitive SVM algorithms based on random over-sampling, SMOTE, and under-sampling are designed by reconstructing the space distribution of classes. Moreover, combination methods of above techniques i.e. hard-ensemble and soft-ensemble, are also designed. In our soft-ensemble, ensemble method is different from the method of ANNs. Experimental results on data sets suggest that cost sensitive SVM based on under-sampling is a good method on the whole, but on the seriously imbalanced data sets, this method is ineffective while soft-ensemble is more effective. Moreover, some more detail results is achieved. These results provide foundation for using this kind of cost sensitive SVM.3. From theoretical and empirical perspectives, one cost learning algorithm-CSSVM algorithm, proposed by Lin etc., is studied. Experimental results show that although CSSVM could reduce the whole misclassification cost, the parameters in the model influence learning performance of CSSVM. In order to select the best parameters automatically, this paper designs and implements the parameter selection based on genetic algorithm (GA) by combining GA and cost sensitive SVM.更多还原