【作者】 张景祥；

【导师】 王士同；

【作者基本信息】 江南大学 ， 轻工信息技术与工程， 2015， 博士

【摘要】 迁移学习方法是一种新的机器学习框架,其放松了对训练数据和测试数据同分布假设的要求,能够利用不同领域的数据来帮助目标任务的学习,有效解决了训练数据和测试数据分布不一致的学习问题。本文以支持向量机为研究基础,以迁移学习为研究内容,以提高目标领域数据分类任务的精度为目标,针对迁移学习模式下的样本选择和特征重构方法的问题,从“迁移什么”,“怎么迁移”和“迁移多少”等视角提出几种适用于迁移学习的方法,主要研究成果包括:1.传统迁移学习方法大都是直接利用源领域中的数据来辅助完成目标领域的学习任务,而忽略了领域间互相学习的能力。针对此类问题,提出了一种具有协同约束的共生迁移学习方法(Collaborative Constraints based Symbiosis Transfer Learning,CCSTL)。在协同约束的基础上,引入共生迁移机制实现领域间的交替互动学习,进而强化了源领域和目标领域间的知识迁移程度,从而提高受训分类器的分类性能,实验结果表明了CCSTL算法的有效性。2.现有的特征迁移通过构造特征映射算法主要强调在领域共有特征间学习,然而其忽略了领域间特有特征的判别信息,使算法的适应性受到一定的局限。针对此问题,提出了一种融合异构特征的子空间迁移学习(The subspace transfer learning algorithm integrating with heterogeneous features,STL-IHF)算法。STL-IHF算法将领域数据的特征空间分成共享特征和特特征的组合,并基于经验风险最小框架理论将共享特征和特有特征共同嵌入到支持向量机的训练过程中。其在共享特征子空间上实现知识迁移的同时兼顾了领域特有的异构信息,增强了算法的适应性。3.针对许多迁移学习算法是通过核化转化为二次规划(QP)问题来求解,其计算核矩阵的空间和时间复杂度较高,对于大样本数据的训练计算开销则更大,限制了多源迁移学习方法的适应性。针对此问题,提出了一种基于共同决策方向矢量的多源迁移(common-decision-vector based multiple source transfer learning,CDV-MSTL)学习方法,该方法基于经验风险最小框架理论,以共同决策矢量为基准,通过核化将多个相似领域的决策方向矢量与共同决策方向矢量嵌入到支持向量机的训练过程中,有效的提高目标领域的分类器的分类性能。并进一步结合CVM理论提出了CDV-CVM快速分类学习算法。实验结果表明了CDV-MSTL和CDV-CVM算法的有效性。4.针对渐变概念漂移的分类问题,提出了一种自适应近邻投影均值差支持向量机(Adaptive Neighbor Projection Mean Discrepancy Support Vector Machine,NMD-SVM)算法。该算法引入再生核Hilbert空间中近邻投影均值差作为相邻分类器间差异的度量,在全局优化中融入了数据自身的分布特征,提高了算法的适应性。实验结果表明了NMD-SVM算法的有效性。5.特征选择也是模式识别和机器学习里一个重要研究领域,但现有的很多特征选择方法过分依赖特征加权且优化参数较多。针对这个问题,本文基于平方误差标准(ISE)核密度估计和随机置换理论,从特征空间存在单一和协同特征的现象出发,提出了一种单一特征的特征选择方法(FSKDE-RP)。以此为基础,通过拓展随机置换理论使之适合协同特征的场景,提出了多维协同特征选择算法(SFSKDE-MRP)对特征进行排序,最后利用KNN分类器的分类精度选择最优特征子集。实验结果表明了FSKDE-RP和SFSKDE-MRP算法的有效性。更多还原

【Abstract】 Transfer learning as a new learning framework of machine learning does not make the identical distribution assumption as traditional machine learning. Therefore,it uses the data in a different domain to help the target task to effectively solve the learning problem in which the training and test data are of different distributions.This paper,based on Support vector machine(SVM) method,take transfer learning as research content and improve the precision and speed of the target domain as its aim. Therefore,several transfer learning methods are proposed,aiming at the problem that the sample selection and feature reconstruction in the transfer learning mode from “what to transfer”,” how to transfer” and “how many to transfer” views. The main research results include:1. Transfer learning algorithms usually focus on reusing data of related domains to help solving the learning tasks in the target domain. However,these methods ignore the ability of mutual learning between domains. In this paper,a collaborative constraint based symbiosis transfer learning method(CCSTL) is proposed. Symbiotic transfer mechanism is used to implement mutual learning among domains along with the collaborative constraint. With the help of the iterative optimizations,the proposed method can realize knowledge transfer between the source and target domains. Experimental results on synthetic and real world datasets show the superior or comparable performance of the proposed algorithm compared with existing algorithms.2. The traditional transfer feature algorithms usually focus on learning by using the common features between the source domain and the target domain but ignore the discriminant information of the specific features of each domain,which makes the existing algorithms lack the adaptability to some extent. For this issue,in this paper a novel subspace transfer learning algorithm integrating with heterogeneous features(STL-IHF) is proposed based on the empirical risk minimum framework. The proposed method takes the feature space of each domain as a combination of the common features and the specified features,and further develops the algorithm based on the SVM framework by integrating with this feature combination. The derived algorithm can not only realize the transfer learning from the common features but also effectively leverage the specified features of each domain,which makes it have much better adaptability in the learning procedure.3. Many transfer learning algorithm are transformed into quadratic programming(QP) problem to solve,the adaptabilities of the multiple source transfer learning are restricted due to the large complexity of time and space consuming of the kernel matrix. A novel common-decision-vector based multiple source transfer learning(CDV-MSTL) and its fast classification learning method was proposed to solve this problem. Those decision vectors of domains and common decision vector are embedded in the process of training support vector machine(SVM) based on structural risk minimization principle,So the proposed method is of the good extension ability and the better accuracy. And Combined with the theory of CVM, realize training and classification of large sample quickly by the extension of the CDV to the CDV-CVM. The experimental results on simulation and real data set show the power of the proposed CDV-MSTL and CDV-CVM algorithms.4.At present,in order to handle with the problem of concept drift,a novel SVM-based algorithm which aims at classification of the concept drifting called NMD-SVM is proposed accordingly. The concept of the neighbor projection mean discrepancy in the reproducing kernel Hilbert space is defined to measure the discrepancy between adjacent sub-classifiers and the distribution of data are embed into the process of global optimization. The adaptability of classification algorithm can be enhanced. The experimental results in the simulation and real indicate the power of the proposed algorithm.5. Feature selection is an important problem in the pattern recognition and machine learning areas. But many of the existing methods overly depend on the feature weighting technique and have many optimized parameters. In this paper,According to the single and synergetic feature phenomena of the data set,we propose a novel supervised feature ranking method based on the integrated square error and random permutation. Making the developed random permutation theory to adapt synergetic feature phenomena, we propose a multidimensional collaborative feature selection algorithm and sort the features. At last,optimal feature subset selection is determined by the classification accuracy of the KNN algorithm. Experimental results on synthetic and real world datasets show the superior or comparable performance of the proposed algorithm compared with existing algorithms.更多还原