【作者】 张晶；

【导师】 冯林；

【作者基本信息】 大连理工大学 ， 计算机应用技术， 2016， 博士

【摘要】 随着互联网的发展,各类数据不断累积,如何从数据中获得有用的知识成为亟待解决的问题。BP神经网络(Back propagation neural networks)作为一种经典模式分类方法,成为获取信息的有效工具,并被广泛应用于各个领域。然而,BP神经网络采用迭代求解方式,存在计算速度慢、易获得局部最优解等问题。极端学习机(Extreme learning machine, ELM)与BP神经网络同为单隐层前馈神经网络(Single-hidden layer feedforward neural networks, SLFNs), ELM直接从输入层实现复杂的非线性映射,利用随机化参数的方法将迭代求解过程转化为线性方程组的求解过程,进而获得了更快的求解速度。同时,避免了易陷入局部最优解的问题,成为解决模式分类问题的有效途径。本文主要以ELM模型为理论基础,以提升模式分类速度、精度与稳定性为目标。针对ELM模型在不同分类应用中产生的问题进行了研究、分析,并取得了下述成果：(1)针对ELM对大规模数据难以实时分类的问题,本文提出并行在线极端学习机(Parallel online sequential extreme learning machine, POSELM)模型。首先,利用ELM理论得到隐层输出权值矩阵；然后,根据MapReduce框架并行计算特点,对矩阵分割后在不同工作结点上并行计算,以代替原有大规模矩阵累乘操作；最后,将计算结点上的结果累加后得到输出权值矩阵。该方法保证了在不损失计算精度前提下,将ELM的增量学习算法在MapReduce框架上进行拓展,从而实现数据的实时分类。(2)针对ELM分类中鲁棒性低的问题,提出相应的解决方案。ELM在建模时需全局数据参与运算,未考虑噪音样本对模型构建产生的影响,进而出现鲁棒性低的问题。针对该问题,提出局部极端机(Local extreme learning machine, LELM)模型。首先,获得测试样本的K近邻,根据近邻标签判断测试样本位置,并识别噪音训练样本；然后,提出监督聚类方法结合近邻法重建局部训练集；最后,针对新的训练集,构建局部分类器。LELM在充分考虑样本局部结构的同时,识别并移除噪音样本,从而降低噪音样本对模型构建产生的影响,提升模型鲁棒性。进一步,为了保证ELM增量分类方法的鲁棒性与实时分类的稳定性,将在线极端学习机模型(Online sequence extreme learning machine, OSELM)加以拓展,提出自组合核的在线极端学习机(Self-compounding kernel online sequential extreme learning machine, SCK-OSELM)模型。首先,提出自组合核方法,实现样本空间到非线性融合特征空间的转换。然后,借助稀疏贝叶斯原理对当前时间点参数稀疏。最后,将当前稀疏后得到的参数并入下一时刻运算。该方法将多个核空间的特征进行非线性融合,避免了隐层参数选择产生模型鲁棒性低的问题。(3)针对ELM对非平衡数据难以有效分类的问题,提出加权的在线极端学习机(Weights robust online extreme learning machine, WROSELM)模型。为解决数据类别分布非平衡性,该模型借助代价敏感学习理论生成局部动态权值矩阵,从而优化分类模型产生的经验风险,并通过对权值矩阵的分解,获得模型输出权值矩阵新的增量表达。同时,算法进一步考虑动态数据由于时序性改变造成的数据分布变化,而引入遗忘因子增强分类器对数据分布变更的敏感性。该方法对非平衡数据的分类问题具有较高的适应性,可实现对该类数据实时、有效的分类。(4)针对ELM对多特征数据难以有效分类的问题,提出多特征极端学习机(Multi-feature extreme learning machine, MFELM)模型。该模型利用ELM对单特征逐一建模,通过给出不同特征模型的权重迭代参数优化MFELM模型,最终获得使分类误差总和最小的特征组合系数与输出权值。在组合参数设计上,引入多次幂系数避免求解过程中算法退化为单特征的情况。同时,将MFELM拓展为核方法,提出基于多特征核的极端学习机(Multi-feature kernel extreme learning machine, MFKELM)模型,该模型避免了因多特征维度不同,产生MFELM难以计算的问题,使分类模型可利用不同特征同时为分类提供信息,提高分类精度。更多还原

【Abstract】 With the development of Internet, the amount of data is increasing at the speed of TB and PB per second. How to obtain valuable knowledge from data has become a hotspot problem. As a kind of classical pattern classification method, Back Propagation Neural Networks model (BP) is an effective tool to obtain information, and has been widely used in various fields. However, traditional BP algorithm is essentially an iterative solution method for the parameters optimization, which suffers from slow convergence and a local optimal solution problem. In order to solve above-menthod problems, extreme learning machine (ELM) has been proposed to train a single hidden layer feedforward neural networks (SLFNs) method, which is similar to BP. In ELM model, the nonlinear mapping is completed in the input layer, and the hidden nodes are randomly initiated and then fixed without iteratively tuning. Therefore, not only ELM has fast speed in solving pattern classification problems, but is also not easy to be trapped into local optimal solution. Meanwhile, this method becomes an effective way to solve the pattern classification problem. We aim to improve the classification speed, accuracy and stability based on ELM theory in this thesis. Research and analysis are conducted to solve these problems in different applications, and following achievements are obtained:(1) The efficiency of ELM is limited when executes the classification problem on the large-scale data in a single machine. In order to solve this problem, this thesis proposes a novel algorithm which is called Parallel Online Sequence Extreme Learning Machine (POSELM). The process of POSELM is as follows:firstly, calculate the output weight matrix of the hidden layer with the theory of ELM. Secondly, this matrix is partitioned into several matrix blocks based on the characteristics of the MapReduce framework so as to substitute the original large-scale matrix multiplication operation with calculation the matrix blocks in multi-work nodes in parallel. Finally, the values in calculation nodes are merged by the key values and we can acheive the output layer weight. Under the premise that the original calculation accuracy is guaranteed, we extend the online extreme learning machine algorithm to the MapReduce framework, and realize real-time classification in dynamic data.(2) ELM has lower robustness in the classification process due to the randomized theory. To solve the problem, this thesis proposes a novel algorithm that is called local extreme learning machine (LELM). The algorithm is summarized as follows:firstly, obtain K nearest neighbors of each testing sample to judge the location of the testing sample, and identify noisy samples. Secondly, combine the proposed supervised clustering method with the nearest neighbor method to reconstruct the local training set. Finally, establish the local classification model for the new training set. LELM obtains the local classification model by reconstructing the local training set, which has following advantages:①The local structure of samples is fully considered. ②The model obtains the balance between variance and bias. ③Reduce the impact of the noisy samples on the construction of the classification model, which improve the robustness and stability. Furthermore, in order to improve the robustness of incremental classification model, this thesis proposes a novel algorithm that is called Self-Compounding Kernel Online Sequential Extreme Learning Machine (SCK-OSELM) which is developed by OSELM (Online sequence extreme learning machine). The steps of this algorithm are as following:firstly, map the input samples into multiple kernel spaces to obtain different high-dimension features, and calculate the nonlinear combination of features. Secondly, introduce the prior distribution of training samples as a model weights to maintain the model generalization, and utilize super weight to make the posterior distribution of weights to zero, thus we can achieve the sparse parameters. Finally, incorporate the sparse obtained parameter into the next moment common operations. In SCK-OSELM method, the fused features of multi-kernel space avoids the problem of the randomly parameters, which improves the robustness for incremental classification. Meanwhile, the robust incremental learning process not only preserves the traceability of the model, but accelerates the calculation speed.(3) ELM is difficult to classify the imbalance dynamic data. In order to solve this problem above, this thesis proposes a novel algorithm which is called Weights Robust Online Extreme Learning Machine (WROSELM). The algorithm can be summarized as follows: Firstly, generate the local dynamic weighted matrix based on cost-sensitive learning theory in real-time, thereby optimize the empirical risk of the classification model. Then, decompose the weight matrix, and obtain the output weights by the new incremental expression. Finally, WROSELM takes into consideration the data distribution changing which is caused by temporal properties change of dynamic data, thus it introduces the forgetting factor to enhance the sensitivity of the classifier to the change of data distribution. This method gets a better adaptability to the problem of imbalance data classification, and obtains the real-time and efficient performance.(4) ELM is difficult to classify the multi-feature data. In order to solve this problem above, this thesis proposes a novel algorithm which is called Multi-feature Extreme Learning Machine (MFELM). The algorithm can be summarized as follows:firstly, nonlinearly fuse multi-features of the same sample. Secondly, obtain the output weights by the fused feature and the error of single-feature respectively. Finally, obtain the combination parameters that are the minimum of the current classification error-sum by the iterative calculation and the output weights. In order to avoid the phenomenon that only single-feature is used in solving process, the higher order coefficients is inducted. Meanwhile, enhances MFELM to the kernel method that is called multi-feature kernel extreme learning machine (MFKELM). It avoids the problem that multi-feature always locate in different dimensional spaces which challenges classification processing, thus the fused feature can be used to provide information for classification, and improve classification performance.更多还原