【作者】 杨泽平；

【导师】 高大启；

【作者基本信息】 华东理工大学 ， 计算机应用技术， 2015， 博士

【摘要】 不平衡数据分类问题是机器学习领域内一个重要的研究课题,引起国内外学者的广泛关注。不平衡数据普遍存在于现实生活中,例如：基因表达数据、信用卡交易数据和医学数据等。现有分类器及其学习算法在解决不平衡数据分类问题时,由于只关注整体识别率而往往忽视少数类的识别率。如何改善类间样本的不平衡,增加少数类的识别率,同时兼顾多数类的准确度,是亟需解决的一个问题。本论文分析和比较了一些常用的分类模型与算法对不平衡数据集的学习和推广性能,从数据和算法两个层面重点研究基于神经网络的不平衡数据分类方法。本文的主要研究工作如下：(1)在分析了不平衡数据对神经网络及其误差反传(Back-propagation, BP)算法性能影响的基础上,提出了面向不平衡数据集的主动下采样算法。该算法可以自动去除远离边界的多数类样本,在保持整体分布特性近似不变的前提下,使得类别间不平衡程度得到改善。本文根据不平衡分类评价指标对主动下采样算法和现有采样方法进行了比较。实验结果表明,本文提出的下采样算法能够有效地减轻不平衡数据对BP算法性能的影响,提高少数类识别率且兼顾整体识别率。(2)传统采样方法在解决不平衡分类问题时容易导致类间样本重叠,现有数据清理算法在判断样本是否可删除时总是存在误删除现象。借鉴离群点检测算法的思想,结合采样方法,提出了边界噪声因子(Borderline Noise Factor, BNF)的概念,给出了基于边界噪声因子的数据清理算法。实验结果证明,基于边界噪声因子的数据清理算法和采样方法相结合可有效改善不平衡和类间样本重叠对BP算法性能的影响。(3)针对不平衡数据对极限学习机(Extreme Learning Machine, ELM)算法的影响,提出了量子行为粒子群优化极限学习机算法,该算法采用量子行为粒子群优化算法来优化极限学习机的网络结构,在经验风险和结构风险之间保持一个良好的平衡,并采用面向不平衡数据集分类器的评价标准作为优化算法的目标评价函数；实验结果证明,该算法不仅能够优化极限学习机的网络结构,且适用于不平衡数据集。(4)传统极限学习机对不平衡数据集分类时,采用相同的错分代价,导致少数类的识别率不高,针对这一问题提出了加权极限学习机的权值新策略；极限学习机算法随机选择输入权值和隐层阈值而导致隐节点大量冗余,加重了网络结构的复杂程度,影响了极限学习机的推广性能,因此提出了一种自适应剪枝算法来解决隐节点冗余问题,并采用隐节点输出向量正交投影距离和隐节点输出向量的范数这两种剪枝标准：将自适应剪枝算法与加权极限学习机相结合,提出自适应剪枝加权极限学习机算法。实验证明,自适应剪枝加权极限学习机算法能够有效地解决不平衡问题,且有很好的推广性能。(5)基因表达数据有着样本分布不平衡,基因维数大,样本维数小等特点,针对基因表达数据的特点,采用分类信息指数准则来进行初始基因的选择,然后利用主成分分析进行降维,同等条件下,采用不同的分类算法对实际的基因表达数据(结肠数据集、白血病数据集、蛋白质数据集等)进行了分析,结果表明本文所提出的这些不平衡分类算法能够提高基因表达数据的分类准确率。总之,本论文主要以经典的机器学习数据库不平衡数据集和基因表达数据为应用对象,从数据和算法两方面提出一系列学习方法,提高了神经网络对不平衡数据集的分类能力。更多还原

【Abstract】 The classification of imbalanced data is one of the most important issues in machine learning, which attracts wide attention of many scholars at home and abroad. Imbalanced datasets exist in a wide range of real applications in several fields such as gene expression data, credit transactions and medical diagnosis. Most of the existing classifiers ignore the minority classes for obtaining high overall classification accuracies when learning the imbalanced datasets. The problem of how to change the data distribution of dataset and improve the classification accuracy of minority classes while maintaining the overall classification performance needs to be solved.This thesis analyzes the cause that the performance of the traditional classifiers is hindered, compares several existing methods, and furthermore, puts forward some novel neural network based classification methods which focus on datasets and algorithms. The main contributions of this thesis are as follows.(1) Having analyzed the influences of imbalanced datasets on performance of back-propagation (BP) algorithm, this thesis presents an active under-sampling algorithm, which can automatically get rid of the samples of the majority classes which are far from the decision boundaries and reduce the degree of imbalance between classes while similarly keeping the density distributions of the whole training datasets. The approach can effectively improve the accuracies of minority classes while maintaining the overall performance by the experimental results, compared with the existing under-sampling methods.(2) The traditional sampling methods always lead to class overlapping for imbalanced data. What is more, the existing data cleaning methods often delete the non-noise samples. The borderline noise factor (BNF) is proposed based on outlier detection technology and the sampling method. Furthermore, a BNF-based data cleaning algorithm is given. The experimental results show that the method combining the cleaning algorithm with sampling is effective in correcting class imbalance and overlap and improving the performance of the BP algorithm.(3) A novel QPSO-ELM algorithm is put forward by studying the influences of performance of ELM algorithms for imbalanced datasets, which uses Quantum-behaved Particle Swarm Optimization (QPSO) to optimize the structure of ELM, achieves a good balance between the empirical and the structural risk and adopts G-mean as the fitness function. The experimental results show that the new algorithm can achieve a good performance with optimal structure in imbalanced datasets.(4) The costs associated with misclassifying examples among different classes are the same which are considered by ELM learning imbalanced datasets, so the accuracies of minority classes are low. A new ELM algorithm is proposed to adopt new weight values, which is more suitable for the imbalanced datasets than the traditional ELM. ELM would multiply the structural complexity of neural networks and impact the scalability, because the random selection of input weights and hidden biases usually results in the number of hidden nodes redundancy. The adaptive pruning algorithms are proposed to solve the hidden-node redundancy of ELMs, which use two pruning criteria, the orthogonal projection distances and the norms of output vectors of hidden nodes, respectively. The experimental results show that these algorithms are suitable for balanced datasets and have good generalization performance.(5) Gene expression data has some characteristics such as unbalanced data, high dimensions and small number of samples. IIC (Information Index to Classification) is adopted as the criteria to select genes, and then PCA is used to reduce the dimension. Finally, different methods are applied in the real gene expression datasets such as the colon datasets, the leukemia datasets, SRBCT datasets and protein datasets. The experimental results show that these algorithms based on imbalanced data can improve the classification accuracy of gene expression datasets.In sum, several learning methods consisting of the balance of training datasets and the modification of standard learning algorithms are proposed in this thesis, which improve the classification performance of neural networks for imbalanced datasets of UCI and gene expression data.更多还原