基于不同损失函数的投影双子支持向量机算法研究

【作者】 刘杰；

【导师】 王学永；

【作者基本信息】 曲阜师范大学 ， 运筹学与控制论， 2023， 硕士

【摘要】 支持向量机(Support Vector Machine,SVM)是一种基于统计学习理论和结构风险最小化原则的机器学习方法,被广泛应用于许多领域.受SVM启发,学者们提出了多种非平行平面支持向量机算法,例如类内方差最小化的投影双子支持向量机(Projection Twin Support Vector Machine,PTSVM).鉴于PTSVM良好的推广性和分类表现,很多学者对其进行深入研究,并提出了多种改进模型.本文主要以提高PTSVM的计算效率及降低算法的噪声敏感性和重采样不稳定性为目的,构建了两个更为有效的改进模型.本文主要研究内容如下:(1)分析研究了分类和回归领域中几种比较流行的损失函数,包括0-1损失函数、Hinge损失函数、L1损失函数和Pinball损失函数.(2)为提高PTSVM的计算效率,本文提出了基于0-1损失函数的PTSVM模型(L0-1-PTSVM).对于该模型,首先建立它的最优性理论,得到其最优解与其近似稳定点之间的关系,进而使用交替方向乘子法进行求解.通过在每一步迭代中选择适当的工作集来降低算法的计算复杂度,从而加快迭代速度.在人工数据集和多个UCI标准数据集上验证了该算法的可行性和有效性.(3)为降低PTSVM的噪声敏感性和重采样不稳定性,本文提出了基于Pin-ball损失函 数的改进投影双子支 持向量机模型(Pin-IPTSVM),分别研究 了它的线性和非线性分类情况,并对算法的一些性质做了分析.该算法具有较好的泛化性能,并能较好的解决算法对噪声敏感以及对重采样不稳定的问题.在带有不同噪声比例的人工数据集和UCI标准数据集上验证了该算法的可行性和有效性.更多还原

【Abstract】 Support Vector Machine(SVM)is a machine learning method based on statistical learning theory and risk minimization principle,which is widely used in many fields.Inspired by SVM,scholars have proposed a variety of non-parallel hyperplane Support Vector Machine algorithms,such as the Projection Twin Support Vector Machine(PTSVM)via within-class variance minimization.Due to its good generalization and classification performance,many scholars have researched PTSVM deeply and put forward various improvement models.In order to improve the computational efficiency and reduce the noise sensitivity and re-sampling instability of PTSVM,two effective improved models have been built.The main research contents of this paper are as follows:(1)Several popular loss functions in classification and regression are analyzed and studied,including 0-1 loss function,Hinge loss function,L1 loss function and Pinball loss function.(2)In order to improve the computational efficiency of PTSVM,a model of PTSVM based on 0-1 loss function(L0-1-PTSVM)is proposed.First,the optimality theory of the model is established,and the relationship between the optimal solution and its proximal stationary point is obtained.Then,alternating direction method of multipliers(ADMM)is used to solve this problem.In order to speed up the iteration of the method,an appropriate working set is selected in each step to reduce the computational complexity.The feasibility and effectiveness of the proposed algorithm are verified on artificial datasets and several UCI benchmark datasets.(3)In order to reduce noise sensitivity and re-sampling instability of PTSVM,an improved Projection Twin Support Vector Machine based on Pinball loss function(Pin-IPTSVM)is proposed in this paper.The linear and nonlinear classification cases of Pin-IPTSVM are obtained,and some properties of the algorithm are analyzed.This algorithm has good generalization performance and can solve the problem of noise sensitivity and re-sampling instability.The feasibility and effectiveness of the proposed algorithm are verified on artificial datasets and multiple UCI datasets with different noise ratios.更多还原