基于弱监督学习的细粒度图像识别算法研究

【作者】 刘洋；

【导师】 金忠；

【作者基本信息】 南京理工大学 ， 计算机应用技术， 2020， 硕士

【摘要】 细粒度图像识别近几年来在计算机视觉领域受到了广泛的关注,其任务是识别在外观信息以及语义信息都非常相似的物体子类,例如识别不同种类的“鸟”:如“棕头鸦雀”、“百耳画眉”,“赤腹山雀”等（或者识别不同种类的车:如“丰田”、“本田”、“路虎”等）,具有广泛的应用前景。与传统的图像分类相比,细粒度图像识别的不同和难点在于:一是类间差异非常细微（比如鸟的不同子类或车的不同子类）,不同子类对象间的差异主要体现在局部细节上;二是对于细粒度级别图像而言,类内差异大,即同一子类图像本身也具有形态、姿势、颜色、背景等较大差异。因此如何检测出具有分辨性的物体部件,以及如何更好的提取细粒度特征,成为当前细粒度识别领域亟待解决的难题。本文主要研究基于深度学习的细粒度图像识别算法。根据训练阶段是否使用除图像类别之外（如物体部位的边界框标注）的标签信息可以将细粒度图像分类算法分为两类:基于强监督学习的细粒度图像识别以及基于弱监督学习的细粒度图像识别。本文提出的两个算法致力于使用弱监督学习的方式使网络去学习定位物体中的分辨性区域,以及提取细粒度级别的特征。本文的主要研究内容如下:（1）提出了一种结合非局部和多区域注意力机制的细粒度图像识别算法。通过导航模块实现对鉴别性区域的弱监督定位。在此基础上,针对导航模块未考虑不同位置间的联系,通过引入非局部模块与导航模块进行结合,加强模型的全局信息感知能力。此外,通过分析非局部模块未建立特征通道间联系的缺陷,构建基于通道注意力机制的特征提取网络,使得网络关注更加重要的特征通道。在3个公开的细粒度图像识别库CUB-200-2011、Stanford Cars和FGVC Aircraft上的实验结果表明,结合非局部模块和通道注意力机制之后的模型较导航模块的基础上有明显的精度提升,并且高于多种对比算法。（2）提出了结合多层交叉双线性池化和视觉注意力机制的细粒度图像识别算法。该算法在多层交叉双线性池化（HBP）的基础上,在网络结构上做出了一些改进。首先,增加一个尺度的多层交叉双线性模块,即对两组提取不同尺度特征的卷积层分别进行多层交叉双线性池化,使用两种尺度能够利用上下文语义信息,丰富细粒度特征表达。此外,在级联（concat）两组层间交叉双线性池化特征之外,新增级联最后一个全局均值池化的输出特征作为最终的特征表示。最后,引入一种“区域打乱”的训练机制RCM以及一个对抗训练分支^[28]。RCM机制可以使得模型能够关注一些重要的局部区域,对抗训练分支用来降低区域打乱引入的噪声影响。RCM和对抗分支只在训练阶段引入小部分计算,而测试阶段可以完全移除,几乎不增加模型的复杂度。在3个公开的细粒度图像库上的实验结果表明,提出的算法的识别精度高于多种对比算法。更多还原

【Abstract】 In recent years,fine-grained image recognition has been widely concerned in the field of computer vision.Its task is to identify the object subclasses that are very similar in appearance information and semantic information,such as identifying different kinds of "birds",such as "Brown headed Brucea","hundred ear thrush","red bellied tit"(or identifying different kinds of cars,such as "Toyota","Honda","Lu Tiger " has a wide application prospect.Compared with the traditional image classification,the differences and difficulties of fine-grained image recognition are as follows: first,there are very small differences between classes(such as different subclasses of birds or different subclasses of cars),and the differences between different subclass objects are mainly reflected in the local details;second,for fine-grained image,there are large intra class differences,that is,the same subclass image itself has shape,posture,color,background,etc.Therefore,how to detect the object parts with resolution and how to extract the fine-grained features better has become an urgent problem in the field of fine-grained recognition.This paper mainly studies fine-grained image recognition algorithm based on deep learning.According to whether the label information other than image classification(such as boundary box annotation of object parts)is used in training stage,fine-grained image classification algorithms can be divided into two categories: fine-grained image recognition based on strong supervised learning and fine-grained image recognition based on weak supervised learning.In this paper,two algorithms are proposed to use weak supervised learning to make the network learn to locate the resolution region in the object and extract the finegrained features.The main contents of this paper are as follows:(1)A fine-grained recognition algorithm combining non-local and multi-area attention mechanism is proposed.Through the introduction of navigation module to achieve the weak surveillance location of the discriminative region.On this basis,for the navigation module does not consider the relationship between different locations,the non local module and the navigation module are introduced to enhance the global information perception ability of the model.Then,the feature extraction network based on channel attention mechanism is constructed to strengthen the connection between feature channels,which makes the network pay more attention to the more important feature channels.Experimental results on three open fine-grained image recognition libraries,CUB-200-2011,Stanford Cars and FGVC Aircraft,show that the model combined with non-local module and channel attention mechanism has a significant improvement in accuracy compared with the model using only navigation module,and is higher than a variety of comparison algorithms.(2)A fine-grained image recognition algorithm is proposed,which combines multi-level cross bilinear pooling and visual attention mechanism.In this algorithm,Res Net-50 is used as the reference network,and two multi-level cross bilinear modules of different scales are constructed on Res Net-50.The output characteristics of these two multi-level cross bilinear modules and the last global mean pooled output characteristics are used as the final feature representation.Secondly,a training mechanism of "region destruction" is introduced,that is,in addition to the original image,the original image after "destruction" is also input during the training,and an adversary loss is added to solve the noise introduced after image destruction.In this way,if the model wants to recognize an image correctly,it must "be forced" to pay attention to the object’s resolution area and detail features.This training method makes the model have "visual attention".The experimental results on three open fine-grained image databases show that the recognition accuracy of the proposed algorithm is higher than that of many comparison algorithms.更多还原