【作者】 刘学艺；

【导师】 李平；

【作者基本信息】 浙江大学 ， 控制科学与工程， 2013， 博士

【摘要】 随着信息技术的快速发展,基于数据的机器学习算法在工业过程建模中扮演着日益重要的角色,特别是对于那些结构复杂、机理不明的非线性时变系统。极限学习机(Extreme Leaning Machine, ELM)是最近几年才出现的一种新的机器学习算法。与传统的核学习算法相比,ELM不仅具有良好的泛化性能,还表现出训练速度快、算法简单易实现以及人为干预参数少等优势,在复杂系统建模、大规模样本学习以及实时在线预测等问题中表现出巨大潜力。目前,ELM理论和应用方面的研究正吸引着国内外越来越多的关注。本文围绕ELM算法的若干理论问题及其在高炉冶炼过程建模中的应用展开研究,得到的主要研究成果总结如下：(1)从两个新的角度——VC维和在不同训练样本规模下的算法性能——对ELM和支持向量机(SVM)做了比较研究。严格证明ELM的VC维以概率l等于其隐含层神经元个数。这为根据统计学习理论对ELM泛化性能的界进行理论估计提供了依据。而在不同训练样本规模下的比较研究表明：在较大训练样本集上,两者具有相近的泛化性能；但在小样本集上,前者的泛化性能及其稳定性要低于后者。这些结论是对当前现有研究结果的有益补充。(2)针对ELM的模型选择问题,完成了两方面的工作。一是针对回归和分类两种情形分别给出了ELM的快速留一交叉验证算法,并首先从理论上证明了其正确性,然后基于数值仿真验证了其有效性。二是提出了一种新的基于VC泛化界的ELM模型复杂性控制方法,仿真实验表明,在小样本情形下,该方法的性能明显优于其它4种经典统计模型选择方法。(3)基于Johnson-Lindenstrauss定理和仿真实验,首先指出了造成ELM算法训练和测试性能算法不稳定以及小样本情形下建模性能差的两个原因：随机投影的距离保持性能和随机特征向量各分量的分布稳定性；然后,通过输入层权重正交化,适当控制隐含层结点个数,并结合结构风险最小化原则,提出一种距离保持ELM算法(Distance Preserving ELM, DP-ELM),有效地解决了原ELM算法存在的以上两个问题；最后,仿真实验验证了其有效性。(4)为了增加ELM模型的透明性,研究了若干种不同类型的先验信息在ELM算法中的融合问题。首先,通过构造对称激活函数建立了对称ELM,并证明其具有以任意精度逼近任意有限样本的逼近能力,仿真实验表明,对称ELM需要更少的隐含层神经元个数和更短的训练时间,同时能够获得更高的泛化性能。其次,还给出了若干线性等式先验信息在ELM中的融合算法,具体包括：已知特定点函数值、特定点导数值以及各输出函数间依赖关系的先验信息,这些算法能有效地增加ELM模型透明性并改进其逼近性能。(5)在ELM算法框架下,对高炉冶炼过程的数学模型进行了探讨,分别建立了高炉铁水硅含量的数值预测模型、趋势预测模型以及烧结矿化学成分预测模型。首先,建立了高炉铁水硅含量的ELM和DP-ELM数值预测模型,并与BP神经网络和SVM算法做比较,分析了所建立模型的优缺点,并指出DP-ELM和SVM性能相近,优于ELM和BP神经网络；然后,基于ELM和DP-ELM分类器输出的后验概率属性建立了高炉铁水硅含量的趋势预测模型,实现了高炉炉温的变化趋势预测以及相应后验概率的估计；最后,根据烧结过程自身的特点,在ELM框架下建立了融合领域知识的烧结矿化学成分预测模型。仿真实验表明,在ELM这种黑箱建模框架下融合先验知识对于提高模型性能具有重要的作用。(6)针对ELM/DP-ELM算法对冗余变量敏感的问题,提出了基于留一误差梯度的DP-ELM特征伸缩算法(FS-DPELM),并据此探讨了高炉冶炼过程新的建模方法。首先,严格推导出特征伸缩因子和拉格朗日乘子关于留一交叉验证误差的梯度计算公式；然后,结合BFGS拟牛顿法给出了FS-DPELM训练算法。仿真实验表明,基于FS-DPELM算法能够进一步提高高炉铁水硅含量以及烧结矿化学成分的预测精度。此外,FS-DPELM的特征伸缩因子为特征选择提供了一种新的数量依据,特征筛选后的高炉铁水硅含量FS-DPELM预测模型具有更好的预测准确性和稳定性。更多还原

【Abstract】 With the development of information technology and data acquisition technology, data-based machine learning algorithms are playing an increasingly important role for industrial process modeling, especially for complex, mechanism-unknown, nonlinear and time-varying system. Extreme learning machine (ELM), an approach recently proposed for neural network training, can not only obtain generalization performance similar to or better than kernel-based methods, but also offer the advantages of low computational cost, good generalization ability, ease of implementation, and so on, which have shown great potential for complex industrial system modeling, large-scale problem learning and real-time online prediction. Its theory and application has recently become increasingly popular.In this thesis, the investigations are mainly focused on the theory of ELM and its application in blast furnace ironmaking process (BFIP) modeling. The main contributions of the dissertation are as follows:(1) A comparative analysis of basic ELM and SVM is performed from two viewpoints that are different from previous works, one is Vapnik-Chervonenkis (VC) dimension, and the other is their performance under different training sample sizes. It is shown that the VC dimension of ELM equals to the number of hidden nodes of ELM with probability one. Additionally, their generalization ability and computational complexity are exhibited with the training sample size changing, which show that ELM, compared with SVM, can obtain similar or better generalization ability for large sample case but generalize worse and unsteadily for small sample case. Achieved results can provide insight into the essential relationship between them, and can also serve as the complementary knowledge for their past experimental and theoretical comparisons.(2) Model selection of ELM model is considered from two different aspects. Firstly, a fast leave-one-out cross-validation algorithm (FLOOCV) for ELM with respect to both regression and classification problems is proposed, which can avoid training explicitly and just has the complexity of O(N) for a training data set with N points. Its validity is strictly proved, and the simulation results show its effectiveness. Secondly, a novel model selection method is proposed based on VC generalization bounds (VM). The experiments of VM and other4classic statistical model selection criteria show that, for small sample case, VM significantly outperforms other model selection criteria.(3) Based on Johnson-Lindenstrauss theory and simulations, it is pointed out that the reasons of the oscillation in ELM training and testing process, and the poor generalization ability for small sample case mainly lie in two aspects:one is distance preserving performance, and the other is the stability of the distribution of sample points in random feature space. Hence an improved training algorithm, distance preserving ELM (DP-ELM), is proposed based on Gram-Schmidt orthogonalization, hidden-neuron-number controlling and structure risk minimization. Simulation results verify the effectiveness of DP-ELM.(4) For the purpose of improving the transparency of ELM, several different kinds of prior information are considered to be incorporated into ELM. Firstly, symmetric ELM (S-ELM) is established by transforming the original activation function of hidden neurons into a symmetric one with respect to the input variables of the samples. In theory, S-ELM can approximate N arbitrary distinct samples with zero error. Simulation results show that, in the applications where there exists the prior knowledge of symmetry, S-ELM can obtain better generalization performance, faster learning speed, and more compact network architecture. Secondly, the linear-equality-constraint kind of prior information, such as constrains of objective values, constraints of derivative values, and connection between outputs. Simulation results show that the incorporating of prior information can effectively improve the performance of ELM algorithm.(5) ELM theories are used to model the BFIP for predicting silicon content in hot metal both numerically and qualitatively, as well as the sinter chemical composition. Firstly, the numerical prediction models are established based on ELM and DP-ELM. The comparisons with BP neural network (BPNN) and SVM show that DP-ELM outperforms BPNN and ELM, and achieves similar prediction precision with SVM. Secondly, a tendency prediction model is established based on the estimation ability of Bayesian posterior probability of the ELM/DP-ELM classifier, which can give not only the tendency prediction but also the corresponding posterior probability of the silicon content. Simulations on two blast furnaces (BFs) show that both the prediction correct rate and stability of posterior probability of DP-ELM is better than those of ELM. Finally, based on the analysis of the prior knowledge of sintering process, new prediction models incorporating these prior knowledges are proposed under the framework of ELM algorithm. Simulation results show that for the ELM algorithm, which is a kind of black-box modeling methods, incorporating a priori information can play a key role in improving its performance, especially for modeling the complex industrial processes.(6) A LOOCV-error-gradient-descent-based feature scaling DP-ELM (FS-DPELM) algorithm is developed to ovecome the sensitivity of ELM/DP-ELM to irrelevant variables, based on which, the modeling for prediction both the new silicon content and sinter chemical compositions is revisited. Firstly, the gradient of LOOCV error with respect to feature scaling factors and Lagrange multiplier is derived, and then a BFGS quasi-Newton method is used to optimize the parameters. Secondly, the FS-DPELM algorithm is also used to model the sintering process for predicting the sinter chemical compositions. Simulation results show that FS-DPELM can recognize the irrelevant parameters in BFIP and obtain better prediction than DP-ELM. In addition, the feature-scaling factors provide a novel approach for feature selection (FS). The futher simulation results show this FS method can further enhance the performance of FS-DPELM.更多还原