【作者】 刘鑫；

【导师】 于剑；

【作者基本信息】 北京交通大学 ， 计算机科学与技术， 2023， 博士

【摘要】 以人工智能技术引领的新一代产业革命中,深度学习技术发挥了重要作用,在图像分类、人机对弈、无人驾驶等应用场景取得了重大突破。然而,目前深度学习模型的成功应用需要大量的训练数据,如果只有少量训练样本,复杂的深度学习模型很容易出现过拟合问题。这极大地限制了人工智能技术在数据匮乏或者数据获取成本很高领域(如医疗、军事等)中的应用。此外,相比于计算机,人类天生具备利用少量数据进行学习和快速适应新环境的能力。因此,如何缩小人工智能和人类智能之间的差距,进一步推动产业革命的发展成为人工智能领域的研究热点。在此背景下,小样本学习应运而生。小样本学习旨在让机器能够像人类一样,在仅有少量标注数据的情况下快速学习不同的任务。本文的研究聚焦于小样本图像分类问题,该问题的主要挑战在于目标任务中有标签的训练样本太少,通过最小化经验风险学习到的模型难以有效逼近假设空间内的最优模型,模型的估计误差增大,进而导致模型总的泛化误差增大,泛化性能下降。为了解决上述挑战,本文从数据、模型和优化三个层面,提出了三种小样本图像分类方法。本文的主要贡献如下:1.数据层面,本文提出一种自适应分布校准的小样本图像分类方法。该方法假设每个类服从高斯分布并且相似的类具有相似的分布信息,根据基类与新类之间的相似性自适应地从基类迁移适量分布信息以校准新类有偏的分布。通过从校准后的分布中采样,可以从源头上解决小样本分类任务由于缺乏训练数据导致泛化性能差的问题。自适应分布校准方法可以有效避免负迁移,提高校准分布和扩充样本的质量。此外,本文从理论上分析了所提方法的泛化误差界,并通过大量实验验证了该方法在小样本分类任务上的有效性。2.模型层面,本文提出了一种基于v MF损失函数的小样本图像分类方法。该方法利用插曲式训练机制,通过在辅助数据集上学习一个可迁移的低维度量空间来缩小目标任务的假设空间,从而提高模型在目标任务上的泛化性能。该方法首次探索度量空间的类结构信息与小样本分类性能之间的关系。发现一个有趣的现象:基类上度量空间的类内距离与新类上小样本分类精度之间存在着较高的相关性。基于此,本文引入球面上的高斯分布:冯·米塞斯-费希尔分布,提出了一种基于v MF损失函数的小样本分类方法。利用v MF损失函数进行学习可以得到一个类内更分散的度量空间,有效提高小样本分类方法的性能。在传统小样本分类任务和跨域小样本分类任务上的实验结果验证了所提方法的有效性。3.优化层面,本文提出了一种基于双层优化高效任务加权的小样本图像分类方法。该方法通过对训练任务加权,使得现有的基于元学习的小样本分类方法在训练任务和测试任务分布存在差异时,仍然能为测试任务学习一个好的初始点,提高模型的泛化性能。为了学习任务的权重,该方法将任务权重学习和小样本学习模型的元参数学习转化为一个双层优化问题。为了提高权重学习的效率,该方法通过引入可能影响任务权重的影响因素,提出一个高效任务加权模块,该模块可以在大大降低参数学习量的同时,有效提高小样本分类方法的性能。本文从理论上分析了该方法的泛化误差界,并在传统小样本分类任务和有领域外训练任务的小样本分类任务上验证了该方法的有效性。总的来说,为了解决小样本分类模型容易过拟合、泛化性能差的问题,本文从数据、模型和优化三个层面入手,通过提高扩充样本质量、学习一个更具迁移性的度量空间和更鲁棒的初始点,提出了三种小样本图像分类方法,可以有效提高小样本分类模型的泛化性能。更多还原

【Abstract】 In the new generation of industrial revolution led by artificial intelligence technology,deep learning has played a crucial role and achieved significant breakthroughs in applications such as image classification,human-machine gaming,and autonomous driving.However,the current success of deep learning models heavily relies on a large number of training samples.If only a small amount of training data is available,complex deep learning models are prone to overfitting issues.This limitation greatly hinders the application of artificial intelligence technology in data-scarce or high-cost domains such as healthcare and military.Moreover,compared with computers,humans are naturally capable of learning from a small amount of data and quickly adapting to new environments.Therefore,bridging the gap between artificial intelligence and human intelligence and further promoting the development of the industrial revolution have become research hotspots in the field of artificial intelligence.In this context,few-shot learning has emerged.Few-shot learning aims to enable machines to learn quickly from a small amount of data,akin to how humans learn.This research focuses on the problem of few-shot image classification,where the main challenge lies in the scarcity of labeled training samples for the target task.Models learned by minimizing empirical risk struggle to effectively approximate the optimal model within the hypothesis space,leading to increased estimation error and overall generalization error,resulting in degraded performance.To address this issue,this paper proposes three few-shot classification methods from the perspectives of data,model,and optimization,to improve few-shot classification models.The main contributions of this article are as follows:1.From a data perspective,this paper presents an adaptive distribution calibration method for few-shot learning.The primary goal of this method is to address the issue of poor generalization performance in few-shot classification due to limited training data by calibrating the biased distributions of novel classes through the adaptive transfer of distribution information from base classes.Our proposed method can effectively improve the quality of the calibrated distribution and augmenting the samples.Besides,this paper theoretically analyzes the generalization error bound of the proposed method and validates its effectiveness through extensive experiments.2.From a model perspective,this paper introduces a few-shot learning method based on the von Mises-Fisher Loss.The approach employs an episodic training mechanism to learn a transferable low-dimensional metric space on an auxiliary dataset,thereby narrowing down the hypothesis space for the target task and improving the model’s generalization performance.The method initially investigates the relationship between the class structure information in the metric space and the performance of few-shot classification.An intriguing observation emerges: a strong correlation exists between the intra-class distances in the metric space on base classes and the few-shot classification performance on novel classes.Building upon this observation,the paper introduces the von Mises-Fisher distribution and proposes the v MF loss for few-shot learning.The v MF loss can learn a more scattered intra-class embedding space,effectively improving the performance of few-shot learning.Experiments conducted on both traditional few-shot classification tasks and cross-domain few-shot classification tasks validate the effectiveness of the proposed method.3.From an optimization perspective,this paper introduces a bi-level optimization based efficient task re-weighting method for few-shot learning.The objective is to learn an effective initialization that can be rapidly adapted to testing tasks,even in the presence of a distribution gap between the training and testing tasks.This is achieved by re-weighting the meta-training tasks.To learn the weights for the metatraining tasks,this method formulates a bi-level optimization problem,wherein the meta parameters of few-shot learning model and the weights for meta-training tasks are jointly learned.Besides,we introduce a hypothesis that significantly reduces the required parameters by considering the factors that influence the importance of each meta-training task.This paper provides a theoretical analysis of the generalization error bound of the proposed method and validates its effectiveness on both traditional few-shot classification tasks and few-shot classification tasks with out-of-domain training.In summary,to address the issues of overfitting and poor generalization performance in few-shot classification models,this paper proposes three approaches at the data,model,and optimization levels.By improving the quality of augmented samples,learning a more transferable metric space,and obtaining a more robust initialization,these methods effectively enhance the generalization performance of few-shot classification models.更多还原