【作者】 王洪波；

【导师】 赵光宙；

【作者基本信息】 浙江大学 ， 控制理论与控制工程， 2012， 博士

【摘要】 支持向量机(Support Vector Machine, SVM)是在统计学习理论基础上发展起来的新型机器学习算法。SVM采用结构风险最小化原则,同时最小化经验风险和置信范围,具有拟合精度高、选择参数少、推广能力强和全局最优等优势。SVM为解决小样本、高维数及非线性等问题提供了有效工具,已成为机器学习领域中研究热点之一并得到了广泛应用。SVM是针对二分类问题提出的,需要两种类别的样本作为训练样本。在实际应用中,有些领域几乎无法获取两类的样本或者代价极高,如敌我识别、攻击样本、卫星故障等,只能获取一个类别的样本,故只能利用这一类样本进行学习,形成数据描述从而实现分类,故出现了单分类算法。支持向量数据描述(Support Vector Data Description, SVDD)和一类支持向量机(one-class support vector machine, OCSVM)是SVM算法扩展成的单分类算法。在相同的高斯核函数作用下,两种算法完全等价,合称之为单分类支持向量机(1-SVM).“工欲善其事,必先利其器”。要使1-SVM能更好地应用于实际工程问题,首先需要解决1-SVM的训练或学习问题,其过程实际上是求解一个二次规划(OP)问题。本文以提高1-SVM的学习能力为目标,分别对1-SVM的粒子群优化学习、快速学习以及增量学习三个方面进行了研究,并提出了相应的解决方案,主要工作如下：提出将粒子群优化算法(PSO)的一种扩展算法——线性粒子群优化算法(LPSO)应用到1-SVM的学习上。为了解决粒子飞行到群体最优位置容易陷入停滞状态而过早收敛的问题,采用改变群体最优粒子飞行方式的策略,使群体最优粒子与其他粒子按照不同的方式飞行,改善了LPSO算法的收敛性能,并将此方法应用到LPSO学习1-SVM的过程中。LPSO为解决1-SVM的学习问题提供了新思路。针对1-SVM的大规模样本集的学习问题,受启发于随机取样算法在凸二次规划问题的成功应用,提出了一种基于随机取样算法的快速学习方法。随机选取大规模样本集的两个样本子,依据随机取样引理和推导出的随机结合定理,对两个子集自身的支持向量(极值点,extreme)和相互间违背KKT条件的样本(外点,violator)进行融合,最后学习出两个子集共同的新决策边界。以此类推,直到全部样本抽取和融合完毕。此方法将大规模样本分批抽取为小样本集并对每一个小样本集进行1-SVM学习,降低了1-SVM学习的内存空间和计算时间,是一种有效的快速学习方法。为了实现1-SVM的增量学习过程,分析了一类支持向量机(OCSVM)的几何表示特性,提出了一种基于德尔塔函数的增量学习方法。因为OCSVM的几何结构仅仅有一个分类超平面,在其分类超平面的决策函数上添加一个德尔塔函数能够形成新的决策函数,即一个新的分类超平面。根据新增样本求解德尔塔函数就是OCSVM的增量学习过程。受OCSVM二次规划问题的启发,分析得到德尔塔函数的优化问题同样是二次规划问题,并提出利用修正的序贯最小优化(SMO)算法进行求解。简要介绍了车牌识别系统的组成部分；分析了高清车牌识别的特点,并为车牌定位、字符分割和字符识别等核心技术提出了相应方案；将一对多的1-SVM多分类方法应用到字符识别,并利用提出的增量学习方法提高1-SVM的识别能力。最后,通过C++编程环境实现了识别过程,结果表明成功实现了高清车牌识别系统。更多还原

【Abstract】 Support Vector Machine (SVM) is a novel machine learning method based on the framework of Statistical Learning Theory. SVM minimizes the empirical risk and confidence interval simultaneously by using Structural Risk Minimization, which holds the advantages of high fitting accuracy, few parameters, good generalization and convergence to global optimum. SVM provides a valid tool in dealing with small-sample, high-dimensional and nonlinear problems, so it has become one of the hotpots of machine learning reseach and been widely applied in many areas.SVM is proposed for binary classification problems, and two categories of samples are necessary for classifier training; however, in some fields of practical applications, obtaining two categories of samples is almost impossible or extremely cost, for example, enemy-friend recognition, attack samples, satellite fault diagnosis, etc. Only one class of samples can be acquired, so the data description has to be learned from this type of samples for classification. The classification problem is named as single-class classification.Support Vector Data Description (SVDD) and One-class Support Vector Machine (OCSVM) are two algorithms of single-class classification extended by SVM. The two algorithms are completely equivalent if using the same Gaussian kernel function, and be together called as Single-class Support Vector Machine (1-SVM)."A workman must first sharpen his tools if he is to do his work well." To make1-SVM more effective in practical engineering problems, first of all, its training or learning problems should be solved, which is actually the process of solving a quadratic programming (OP) problem. In this paper, the goal is to enhance the1-SVM’s learning ability. Three aspects of particle swarm optimization learning, fast learning and incremental learning on1-SVM are respectively studied, and corresponding solutions are put forward. The main work is as follows:Linear Particle Swarm Optimization algorithm (LPSO), which is an extension of Particle Swarm Optimization (PSO) algorithm, is proposed to apply to1-SVM learning. Because the particles flying to the group optimal position are easy fall into stagnation, PSO and LPSO have the problem of premature convergence. In order to solve this, the strategy of changing the flight mode of group optimal particle is adopt, and group optimal particle flies in different ways with other particles. The algorithm improves the LPSO’s convergence, and is applied to the process of LPSO learing1-SVM. LPSO provides a new idea to solve1-SVM learning.For1-SVM’s learning problem of large-scale sample set, inspired by the successful application of Random Sampling Lemma (RSL) in convex quadratic programming problem, a Fast Learning (FL) method is proposed based on RSL. Two sample subsets are randomly selected from large-scale sample set. Based on RSL a and deduced Combining Lemma, the support vectors (extreme) and samples violating the KKT (violator) of each other are fused, then the common decision boundary of two subsets are produced. And so on, until the extraction and integration of all samples are completed. This method breaks the large-scale dataset into subsets at random and learns1-SVM to each subset. It reduces the memory space and computing time of1-SVM learning, so is an effective method of fast learning.In order to achieve incremental learning process of1-SVM, the geometry characteristics of OCSVM are analyzed, and an Incremental Learning (IL) method based on Delta Function is proposed. Because OCSVM has only one hyper-plane in its geometric structure, a new decision function (that is a new classification hyper-plane) will be formed when a Delta Function is added to the decision function of its classification hyper-plane. Solving the Delta Function according to the new sample is the process of Incremental Learning of OCSVM. Inspired by the quadratic programming problem of OCSVM, analyzing the optimization problem of the Delta Function is a quadratic programming problem too, and the modified sequential minimal optimization (SMO) algorithm proposed to solve this problem.The components of license plate recognition system are introduced in brief. The characters of high-definition license plate recognition are analyzed, and the corresponding solutions to core technologies of plate localization, segmentation and character recognition are proposed. One-against-all multi-classification based on1-SVM is applied to character recognition. Recognition ability of1-SVM is improved by the proposed incremental learning method. Finally, the system achieved through C++programming shows that it is successful.更多还原