【作者】 张鹏；

【导师】 邵杰；

【作者基本信息】 电子科技大学 ， 计算机科学与技术， 2021， 硕士

【摘要】 基于自然语言描述的行人图像检索研究指的是给出一段自然语言形式的文本描述,从行人图像数据库中检索出与文本描述最相符合的行人图像。随着卷积神经网络和循环神经网络的快速发展,计算机已经具备高效提取图像与文本特征的能力。然而,仅仅提高视觉模型或者文本模型的特征提取能力对基于文本的行人图像检索研究的性能提升收效甚微。由于图像和文本特征分属于不同模态,如何减小不同模态之间的语义鸿沟成为提高基于文本的行人图像检索任务性能的关键。本文借鉴度量学习的思路,提出了一种能够融合文本和图像特征为一个新特征的方法并借助图像文本融合特征尝试在Softmax函数的基础上构建一个损失函数。该损失函数有助于在模型训练的过程中,增大同类文本图像特征对相似度的同时,不断减小不同类别的文本图像特征对的相似度。这就使得模型具有能够提取模态不变性的行人特征能力。此外,数据集中存在着样本不均衡的现象,即匹配文本图像对的数量远少于不匹配图像对的数量。这对模型性能会有消极影响。研究发现,图像文本对中存在着一些属于同一类别却具有较小的相似度,或者属于不同类别却具有较大相似度的图像文本对。针对这种现象,本文提出一种对不同图像文本对的赋权方法,该方法尝试在训练过程中为上述图像文本对赋予更大权重,以提高模型训练效率以及性能。本文的所提出的方法均在公开数据集CUHK-PEDES上做了验证实验并且证明了有效性。与该领域其他算法相比,本文所提出的方法在模型复杂度,可行性方面具有较强竞争力。更多还原

【Abstract】 Based on natural language description,pedestrian image retrieval research refers to giving a text description in form of natural language and retrieving the pedestrian image that is corresponding to the text description from the pedestrian image database.With the rapid development of recurrent neural network and convolution neural network,computers have been able to extract high-level features from images and text.However,only focusing on improving feature extraction capability has had little success in improving the performance.Since image and text features belong to different modalities,how to reduce the semantic gap between the different modalities become key to improving model performance.Inspired by deep metric learning,this thesis proposes a method which is capable of fusing text and image features into a new feature and with the integrated feature we try to build a loss function which is based on Softmax loss.Our loss function helps increase similarity between image and text of the same identity and reduce similarity of imagetext pair of different identities during the model training process.This makes the model capable of extracting the visual and textual feature of the property of modal-invariance.In addition,the situation of sample imbalance exists,which means the number of matched image-text pairs is far less than the number of unmatched image-text pairs.This will have a negative impact on our model performance.Our study found that there are imagetext pairs of the same category but having a smaller similarity and pairs belonging to different categories but having a greater similarity value.In response to this phenomenon,we propose a method to assign different weight to image-text pairs.In other words,this method attempts to make the model pay more attention to the abnormal image-text pairs so as to improve the model training efficiency and performance.Methods proposed in this thesis have been proven effective by extensive experiment results on public dataset CUHK-PEDES.Compared with other algorithms in the field,our methods in this thesis is more competitive in the aspects of model complexity and scheme feasibility.更多还原