【作者】 刘松华；

【导师】 张军英；

【作者基本信息】 西安电子科技大学 ， 计算机应用技术， 2011， 博士

【摘要】 模式分类中的核方法在实际应用中获得了较为成功的应用,其良好的性能在于不仅能高效地对数据间存在的非线性关系进行模式分析,而且核方法本身建立在严格的统计分析基础之上,与线性统计分析方法一样有坚实的理论基础。但是核方法在处理大规模数据分类任务时存在如下问题:一方面,核方法的计算复杂度较高,其算法的设计和求解与训练样本个数有关,且常见的经典算法采用凸二次优化策略,对于大规模数据集,需要较高的时间和空间复杂度;另一方面,由于核空间为高维甚至无限维,其间样本具有多模式、多态性,因此相似性不便于描述。针对上述问题,本文主要探讨了核矩阵的低秩分解和核空间的相似性测度。一是从特征选择和矩阵分解的角度考虑如何学习较优的低秩近似核矩阵。二是从基于距离的度量来考虑核空间中高维数据的特性。在此基础上,结合已有算法分别进行了比较分析,用基于核矩阵低秩分解与信息能度量的核方法实现高维多模式对象的特征提取和模式分类,实验结果验证了算法的有效性。总的来说,本文的主要工作包括如下五个方面:1.针对核矩阵分解算法时间复杂度较高的问题,研究了如何对核矩阵进行低秩分解。常用的矩阵低秩分解算法均可以视为无监督算法,本文通过分析核矩阵中行/列与类别的相关性,结合已有的矩阵分解运算,提出了有监督的核矩阵低秩分解方法,最后给出核矩阵低秩近似误差界的期望值。实验证明核矩阵分解过程中,行/列的选取对分类效果有较大影响,在保证分类性能的前提下,本文算法能在一定程度上提高核机器学习效率,为大规模数据集中的应用奠定了良好的基础;2.核方法在低维数据中已经取得了较为成功的应用,然而在高维数据中,由于数据包含更为丰富的内在结构,因此常用的相似性测度如欧氏距离面临分类效果较低的困境。通过研究非距离的度量问题,提出了新的信息能度量,该方法满足距离的度量公理,且不仅适用于低维数据,同时可以有效挖掘高维数据中的相似性结构,实验结果验证了该相似性测度的正确性;3.研究了核空间中的特征提取问题。基于提出的信息能度量,结合梯度上升方法,提出了新的特征提取算法。该算法在应用于大规模数据集时,可以采用核矩阵的低秩近似分解来有效降低运算复杂度,且无须预先进行特征选择。同时信息能的梯度信息可以描述特征的聚集趋势,有利于指导分类任务;4.基于提出的信息能度量,通过研究已有的最近邻算法和其核化形式,对其进行改进,提出了新的核k-最近邻算法。该算法有效结合了近邻法和核方法的特性,有较好的物理意义,且能由此模型推出经典的互信息度量,因此具有良好的理论基础和推广能力;5.对已有的细胞表型图形态差异学习算法进行核化,提出新的基于信息能度量的核形态差异学习算法,并设计了一系列最优参数选择方案,保证了实验效果,为核方法在该领域的进一步应用奠定了基础。更多还原

【Abstract】 Kernel methods in pattern classification have achieved satisfactory results in practical application. Its high performance depends not only on their ability to analysis nonlinear relationships between data sets, but also strict statistical analysis which has a solid theoretical basis. However, the kernel machine will encounter many problems when deal with large-scale data sets. On the one hand, the kernel method has high computational complexity, because designing and solving is relate to the number of training samples and convex quadratic optimization strategies. On the other hand, the samples in high-dimensional or infinite-dimensional kernel space have multi modes and polymorphism, so it is difficult for similarity measure. As mentioned above, this thesis has focused on the theory and application for kernel machines.Firstly, we study the problem of learning low-rank approximation kernel matrices from the point of feature selection and matrix decomposition. Secondly, we consider the characteristics of high-dimensional data sets in kernel space using metric based on distance measure. Finally, we perform feature extraction and pattern classification based on low-rank kernel matrix and informative energy metric, experimental results verify the effectiveness of our proposed algorithms.All in all, the main contributions of this thesis are as follows:1. We study low-rank decomposition of kernel matrix for its high computational complexity. The traditional low-rank matrix decompositions can be regarded as unsupervised ones, we proposed supervised low-rank kernel matrix decomposition based on the correlation between row/column and class label. Then, the expectation of low-rank approximation error bound is given. Experimental results show that different selection of row/column has great impact on classification accuracy, and our algorithms are useful for large-scale data sets.2. Kernel methods have obtained successful application for low-dimensional data sets, but are unsuitable for high-dimensional data sets. The traditional similarity measure, such as Euclidean distance, faces lower classification performance because of the richer inner structure. We proposed a novel metric named as informative energy metric, which can solve the non-distance metric problem. The characteristic of our metric is that it meets with the distance metric axioms. So, our metric is not only suitable for low-dimensional data sets, but also high-dimensional data sets. Experimental results show the validity of our similarity measure.3. Feature extraction in kernel space is considered. We proposed a new feature extraction algorithm based on the informative energy metric, and been optimized using gradient ascent algorithm. The resultant low-rank kernel matrix is adopted for large-scale data sets. The main characteristics of our algorithm are that it can be used as feature selection, and the gradient information of the informative energy metric can describe the aggregate situation of features which is useful for classification tasks.4. We proposed a novel kernel k-nearest neighbor algorithm based on the informative energy metric. The main characteristics of the algorithm are that the mutual information metric can be derived from our model and it has solid theoretical basis and better generalization ability.5. The morphological difference algorithms based on kernel methods for cell image are considered. We proposed a new morphological difference learning algorithm based on the informative energy metric, and designed a series of parameter optimization strategy. Experimental results show that our algorithm is validity, and it conducted a solid foundation for further application of kernel methods in this field.更多还原