【作者】 顾斌杰；

【导师】 潘丰；

【作者基本信息】 江南大学 ， 控制理论与控制工程， 2015， 博士

【摘要】 支持向量机(Support Vector Machines, SVM)是一种能够有效地解决小规模样本学习问题的机器学习方法,其理论基础是统计学习理论,近年来在数据挖掘、生物医学、图像处理、模式识别和人工智能领域应用广泛。样本的学习是SVM的核心问题之一。在实际的回归问题中,例如：生物反应过程、网络数据监控、金融数据分析和时间序列预测等,样本是在线环境下增量提供的,最为极端的情况是每次只增加一个新的样本。此时传统的批处理学习算法将会由于不具备实时性从而导致模型预测精度不高而且鲁棒性较差的问题,而增量式在线学习算法为解决上述问题提供了一种可行的途径。胪支持向量回归机(Support Vector Regression, SVR)是一种非常有效的回归学习方法,与ε-SVR相比,v-SVR的优点是参数ν能够控制支持向量的个数并能够自动调节不敏感损失函数的参数ε。然而,v-SVR的对偶问题要比ε-SVR的对偶问题复杂,因此目前为止还没有出现专门针对v-SVR的增量式在线学习算法的研究。本文在Cauwenberghs和Poggio提出的精确增量式在线C-支持向量分类机学习算法(称为C&P算法)基础上,重点研究了v-SVR的精确增量式在线学习算法及其可行性和有限收敛性,并将该算法应用于发酵过程的软测量,取得的研究成果如下：(1)针对v-SVR对偶问题存在的两个难题：一是箱约束不仅与惩罚参数而且还与训练样本集合的长度有关；二是与ε-SVR的对偶问题相比多出了一个不等式约束,提出了v-SVR的等价形式。通过将v-SVR原始问题的目标函数乘训练样本集合的长度并用等式约束代替不等式约束,解决了上述两个难题,为后续的研究工作奠定基础。(2)针对v-SVR的ν解路径算法存在的解路径不可行更新问题,提出了一种有效的ν解路径算法。基于v-SVR的等价形式及Karush-Kuhn-Tucker(KKT)条件,通过引入新的变量△g和附加项κΔρ的策略,有效地解决了在绝缘增量调整过程中存在的矛盾和异常,并最终经过有限次数迭代拟合出整个ν解路径。理论分析和仿真结果表明,该算法是可行且有效的。(3)为了解决v-SVR对偶问题的目标函数中增加的额外线性项而导致无法产生有效初始解的问题,以及直接将C&P算法应用于v-SVR在线学习存在的解路径不可行更新问题,基于v-SVR的等价形式及KKT条件,设计了一种精确增量式在线v-SVR学习算法。该算法包含三步：第一步是提前调整；第二步是宽松的绝缘增量调整；第三步是精确的恢复调整。理论分析证明了算法的可行性和有限收敛性。在基准测试集上的仿真结果进一步验证了理论分析的结论,而且与批处理学习算法相比具有更高的计算效率。(4)针对发酵过程中的一些重要生物状态变量难以在线测量的特征,将精确增量式在线v-SVR学习算法应用于谷氨酸分批补料发酵过程中菌体浓度和产物浓度的软测量,实现了菌体浓度和产物浓度的在线预估。仿真结果说明算法适合于发酵过程关键状态的在线预估。更多还原

【Abstract】 The Support Vector Machines (SVM) is a machine learning approach which can solve the learning problem of small scale samples effectively. The theoretical foundation of SVM is statistical learning theory. In recent years, SVM has been widely used in many fields including data mining, biomedical, image processing, pattern recognition, artificial intelligence. The learning of training samples is one of the key problems of SVM. In practical regression problems, such as bioprocess, network data monitoring, financial data analysis, time-series prediction and so on, the samples are usually provided incrementally in online scenarios, in the extreme case, one sample at a time. In these cases, the traditional batch learning algorithm will lead to lower model prediction accuracy and worse robustness due to lack of real-time, whereas the incremental online learning algorithm will offer a feasible path for solving the problems mentioned above.v-Support Vector Regression (SVR) is an effective learning approach for regression. In comparision to ε-SVR, it has the advantage of using a parameter von controlling the number of support vectors and adjusting the parameter ε of insensitive loss function automatically. However, the dual problem of v-SVR is more complex than the dual problem of ε-SVR. To date, there is no research on the incremental online learning algorithm which is specially designed for v-SVR. Based on the accurate incremental online C-support vector classification learning algorithm which was proposed by Cauwenberghs and Poggio (referred to as C&P algorithm), the accurate incremental online v-SVR learning algorithm as well as its feasibility and finite convergence, and the application of the proposed algorithm in the soft senor of fermentation process are mainly researched in this dissertation. The main works of this dissertation are listed as follows:(1) The equivalent formulation of v-SVR is proposed to address the two complications existed in the dual problem of v-SVR. The first one is that the box constraints are related to penalty parameter and the length of the training sample set; the second one is that the dual problem of v-SVR has an extra inequality constraint in comparison to the dual problem of ε-SVR. The two complications were addressed by multiplying the length of the training sample set and substituting the inequality constraints with inequality constraints. The equivalent formulation of v-SVR is the cornerstone of the following research works.(2) To solve the infeasible updating solution path problem of the v solution path for v-SVR, an effective v solution path for v-SVR is proposed. Based on the equivalent formulation of v-SVR and its Karush-Kuhn-Tucker (KKT) conditions, the strategy of introducing a new variable △(?) and an extra term κ△p could solve the contradictions and exceptions effectively during the adiabatic incremental adjustments. Finally, the proposed algorithm could fit the entire v solution path within finite number of iterations. Theoretical analysis and simulation results show that the proposed algorithm is feasible and effective.(3) To solve the problem of unable to generate an effective initial solution which is caused by the extra linear term introduced in the objective function of the dual problem of the v-SVR and the infeasible updating solution path problem when directly applying the C&P algorithm to online learning of v-SVR, an accurate incremental online v-SVR learning algorithm is designed based on the equivalent formulation of v-SVR and its KKT conditions. The proposed algorithm was composed by three steps:the first one was prior adjustments; the second one was relaxed adiabatic incremental adjustments; the third one was accurate restoration adjustments. Theoretical analysis proved the feasibility and finite convergence of the proposed algorithm. The simulation results on benchmark datasets further verify the conclusions of theoretical analysis. Furthermore, the proposed learning algorithm is of higher computation efficiency than batch learning algorithm.(4) Due to the fact that some important biological state variables are rather difficult to be online measured during fermentation, the accurate incremental online v-SVR learning algorithm was applied in the soft senor of biomass concentration and product concentration of glutamic acid fed-batch fermentation process, which realized the online estimation of biomass concentration and product concentration. The simulation results demonstrate that the accurate incremental online v-SVR learning algorithm is suitable for the online estimation of key state of fermentation process.更多还原