【摘要】 近年来,深度学习在诸多任务上展现了优异的性能,其一般基于海量数据并采用有监督的学习方式,依赖于完整的数据标签信息.然而在现实应用场景中,收集大量标签往往成本高昂.因此,如何利用未经充分标注的数据进行学习成为了当下的主要挑战.二分类问题中的从正例和无标签(Positive-Unlabeled,PU)样本数据进行学习,简称PU学习,即为其一.当前主流的PU学习算法需要准确无误的类别先验知识,但实际上类别先验通常难以获得,需要估计.已有的类别先验估计算法则主要面向传统的机器学习分类器进行设计,无法直接运用在大规模数据集上,因而不利于发挥深度学习在大规模数据集上的优势.为克服以上问题,本文提出了一个基于无监督混合模型的迭代式深度PU学习与类别先验估计框架.它利用了深度神经网络对正例和负例给出的预测分数具有不同的分布这一特性,使用双高斯成分的混合模型近似拟合预测分数的混合分布.其中,各个高斯分量分别代表了正类和负类的条件概率分布,混合权重系数代表了类别先验.结合半监督学习中的平均教师和温度锐化技术,所提框架在类别先验未知以及数据缺失负例监督的条件下,估计类别先验的同时进行PU数据上的深度学习,二者相互促进.在基准数据集MNIST、Fashion-MNIST、CIFAR-10和实际应用数据集Alzheimer上的实验结果验证了所提框架的有效性,准确率分别为94.66%、95.16%、89.98%和73.20%,该结果不仅超越了现有基于类别先验估计的PU学习算法,更可与基于真实类别先验的最前沿算法相媲美.更多还原

【Abstract】 Deep learning models have achieved superior performance with a large amount of data.Such success usually depends on complete label information in a fully-supervised training style.However,collecting all the labels can be very expensive.Consequently,weakly supervised learning,which aims at learning with incomplete,inexact,or inaccurate supervision,has attracted the machine learning community in the past decade.One of these real-world applications could be Positive-Unlabeled(PU) learning,where we have to train binary classifiers from a few positive examples with much more unlabeled data.Current state-of-the-art PU methods rely on the ground—truth class prior(i.e.,the proportion of the positive samples to the unlabeled data),which is hard to obtain and needs to be estimated in practice.While previous studies of class prior estimation mainly focused on traditional machine learning models,few could deal with a relatively large-scale dataset or be applied to deep learning scenarios.To solve such problems,we propose an iterative framework for deep PU learning and class prior estimation utilizing an unsupervised mixture model in the paper.Specifically,positive and negative classes are supposed to have distinct predicted score distributions intuitively.We investigate and demonstrate our intuition and approximate the score distributions by a Gaussian Mixture Model with two components.Each component represents a relevant class-conditional distribution,and the positive weight could be approximate to the ground truth class prior.We further incorporate techniques such as mean teacher and temperature sharpening from semi-supervised learning to stabilize the whole process and boost performance.Our proposed framework could estimate the class prior and learn from PU data simultaneously,achieving well-matched performance with other PU competitors based on the ground-truth prior.Experiments on three benchmark datasets(i.e.,MNIST,FashionMNIST,and CIFAR-10) and one practical application(i.e.,Alzheimer) validate the effectiveness of our framework.Our algorithm reaches the accuracy of 94.66%,95.16 %, 89.98%,and73.20%,respectively,with limited labels and unknown class prior.更多还原