【作者】 倪彤光；

【导师】 王士同；

【作者基本信息】 江南大学 ， 轻工信息技术与工程， 2015， 博士

【摘要】 近年来,特征降维和模式分类方法作为模式识别研究领域的两大重要内容,受到广大学者的关注。特别是随着近期出现迁移学习框架的快速发展,使特征降维和模式分类方法的适用范围更加广泛。但是,传统的特征降维和模式分类方法仍然面临着一系列的问题,如精度不高,泛化能力弱,应用范围单一等。鉴于此,本课题在迁移框架下对特征降维和模式分类进行了相关研究,具体内容如下:1、以Relief算法为基础研究了基于迁移学习的特征选择方法。在迁移学习的应用场景下构造了基于间距最大化原理的最优化学习问题,而后运用优化理论对该函数进行了理论证明,最后对应于迁移环境下二类和多类的迁移学习问题,提出不同版本的迁移Relief算法。通过在人工集和真实数据集上的实验表明算法是有效的。2、研究面向演进数据流数据的分类方法,在有效利用相邻演进窗内数据间相似性信息的基础上,通过引入反例信息,构建了一种面向演进数据流的增强型演进分类器优化目标函数,从而推导出面向演进数据流的分类新方法。该方法在保有最大间隔原则和全局优化特性的同时,亦充分考虑反例信息对待解分类平面的影响。模拟和真实数据集上的实验表明了所提新方法的有效性。3、基于组概率的学习方法因其能够很好地保护数据的隐私性而成为近年来机器学习领域的研究热点。已有的组概率学习方法虽然取得了一定的效果,但是在模型训练时仅考虑单一的场景信息,如果在当前领域所采集的数据信息有限,则在当前领域下建立的分类模型泛化能力较差。针对此问题,提出了一种基于组概率和结构风险最小化模型的迁移组概率学习机(TGPLM)。该方法通过构造领域相似距离项来引入历史领域的先验知识,提出了针对类标签保护数据的增强型分类器优化目标学习准则,以期在有效利用当前领域数据类标签组概率信息的同时借鉴历史领域相关知识来指导当前领域下的学习任务。基于模拟、UCI及PIE人脸等数据集上的实验结果表明,本文所提之方法是有效的。4、为解决机器学习中的主观知识缺失问题,提出一种新的面向共享数据的迁移组概率学习机(TGPLM-CD)。该方法方法基于结构风险最小化模型,将源领域所含知识和目标领域的类标签组概率信息,特别是领域间的共享数据纳入学习框架中,进而实现了源领域和目标领域的知识迁移,因此在待研究领域数据信息不足的情况下提高了分类精确度。在大量数据集上的实验结果上表明了所提方法的有效性。5、在真实世界中往往无法得到数据的确切类别标签,为解决该场景下的分类学习问题,提出一种新的适用于不确定类标签数据的迁移标支持向量机方法(TSVM-UL)。该方法基于结构风险最小化模型,同时将源领域中所学知识,领域间的共享数据及目标领域中已标定的和不确定的数据纳入组概率学习框架中,进而实现了源领域和目标领域的知识迁移。在PIE人脸数据集和20Newsgroups数据集上的实验结果上表明了所提方法的有效性。6、在政治、欺诈检测和疾病诊断等领域,为了保护数据标签信息的隐私性,一种常见情况是仅知道少量已标记信息和未标记样本的比例信息,由此产生了一种人为的信息缺失。为解决该应用场景下的分类学习问题,提出一种新的部分类标签隐私保护的流形支持向量机(Support vector machine with manifold regularization and partially labeling privacy protection,SVM-MR&PLPP)。该方法将未标定数据的类标签比例信息纳入流形正则学习框架中,构造了一种分类器优化目标函数准则,提高了分类学习能力。而后为降低算法复杂度优化了目标函数,提出了适合大样本数据集的部分类标签隐私保护的大样本流形支持向量机(Scalable support vector machine with manifold regularization and partially labeling privacy protection,SSVM-MR&PLPP)实现了对大样本的快速训练和分类。大量人造和真实数据集上的实验结果表明了SVM-MR&PLPP和SSVM-MR&PLPP算法的有效性。更多还原

【Abstract】 Recently, feature selection and pattern classification, which are two of important tasks in pattern recognition, are attractting more and more attentions of the researchers. With the development of transfer learning technology, the range of traditional pattern recognition techniques is widely broadened. Many research results are used in data mining, image processing, speech recognition, fingerprint classification and medical diagnosis. However, feature selection and pattern classification based on transfer learning methods show drawbacks of low robustness and weak generalization ability to a certain extent. In order to solve the above problems, several issues are addressed in this dissertation:1. A new Relief feature weighting algorithm based on transfer learning is proposed. First, a new margin-based objective fuction integrating transfer learning information is proposed within the optimzation. Then by applying optimization theory, several useful theoretical results are derived from the proposed objective function. Finally, a set of transfer Relief feature weighting algorithms are developed for two-class data and multi-class data. As demonstrated by extensive experiments in artificial and UCI datasets, the proposed algorithms show the competitive performance to the state-of-the-art algorithms.2. A classification method for evolutionary data streams is proposed. By utilizing the similarity criterion between the data distribution within the adjacent evolutionary windows and the related knowledge of counterexamples effectively, a enhanced objective function for the optimization problem is proposed. Meanwhile, the solution for the optimization problem is also derived. Both the maximal margin criterion in each evolution window and global optimization of the whole evolutionary data stream are considered, the counterexamples are also fully utilized. The new method learns decision hyperplanes successfully. The experiments on the artificial and real datasets demonstrate the effectiveness of the method.3. Learning from group probabilities helps to protect the privacy of users and has become a hot topic in the community of machine learning. The traditional group probabilities based learning methods have gained certain success; however, they still fall short when the prior information is not fully provided. In order to solve this problem, a novel transfer learning method called transfer group probabilities based learning machine(TGPLM in abbreviation) is proposed by integrating group probabilities into the principle of structure risk minimization. In TGPLM, a novel learning criteria is proposed based on reusing the related domain knowledge by minimizing domain similarity distance, which makes the proposed TGPLM not only make full use of the group probabilities in the current scene, but also learn the existing useful knowledge in the history scene effectively. Experimental results on the artificial, UCI and PIE face datasets show the effectiveness of the proposed method.4. To address the problem of man-made information scarcity in the machine learning, a novel transfer learning machine based on group probabilities toward the common data, called TGPLM-CD, is proposed. The proposed method is based on structure risk minimization model, and considers both knowledge of source domain and class label group probabilities as well as the common data between source domain and target domain in the learning process, which realize knowledge transfer between source domain and target domain. Experiment results on extensive datasets show the effectiveness of the proposed method.5. The exact Labels of data are often unable to be known in the real world. To address the corresponding learning problem in the above scenarios, a novel transfer support vector machine for learning from data with uncertain labels(TSVM-UL) is proposed, the proposed method is based structure risk minimization model, and to learn considering knowledge of source domain and the common data between different domains, as well as labeled samples and probabilities of unlabeled sample of target domain, which realize knowledge transfer between source domain and target domain. Experiment results on PIE datasets and 20 Newsgroups datasets show the effectiveness of the proposed method.6. In areas like politics, improper content checking and disease diagnosis, only few labeled data and label proportions unlabeled observations may be known such that exact label information is protected, which results in a man-made scarcity of information. To address the corresponding learning problem in the above scenarios, a support vector machine with manifold regularization and partially labeling privacy protection, called SVM-MR&PLPP, is proposed. In SVM-MR&PLPP, a novel learning criteria is proposed by integrating the label proportions of unlabeled data into the manifold regularization framework, which improve the classification accuracy. Through transforming the optimization function, the complexity of the algorithm is reduced and a scalable support vector machine with manifold regularization and partially labeling privacy protection, called SSVM-MR&PLPP is proposed, which is efficient for large sample datasets. Experiment results on extensive datasets show the effectiveness of the proposed method.更多还原