【作者】 王鑫；

【导师】 刘芳芳；

【作者基本信息】 北京邮电大学 ， 新一代电子信息技术（含量子技术等）（专业学位）， 2025， 硕士

【摘要】 随着智能安防需求的日益增长,基于属性的行人搜索技术具有巨大的发展潜力,已成为计算机视觉与跨模态检索领域的研究热点。以往的行人搜索方法需要通过图像进行搜索,但当无法获取目标图像且仅有关于行人的文本描述时,此类方法便无法使用。基于属性的行人搜索可以根据行人的属性(如年龄、性别,穿着等)实现行人图像的快速搜索,在搜索失踪人员、嫌疑人等场景有关键作用。同时,相比于基于文本的搜索方式,成本更低且不引入冗余和噪声。但图像和属性存在的模态异构性也为基于属性的行人搜索带来挑战。首先,行人图像数据具有较大的类内差异和类间相似性,影响模型的鉴别能力。其次,真实场景中存在大量未在数据集中出现的样本,对模型的泛化性有较高要求。此外,图像模态和属性模态存在语义鸿沟,图像的表征维度和信息量相比于属性更高。最后,当前方法缺乏多模态的对应信息,不同模态的表征存在不对称的问题,限制了跨模态搜索的性能。本论文基于属性的行人搜索算法研究选题来自于北邮-电信视觉智能联合实验室项目。针对上述挑战,该研究首先通过特征分解和属性增强策略,分别提升模型的鉴别能力和泛化表现;其次通过引入多模态预训练模型的先验知识,弥补模态之间的语义鸿沟。本论文的主要研究内容和创新成果如下:(1)针对行人图像数据存在较大的类内差异和类间相似性,鉴别能力差,以及对数据集中未见类别的泛化性差的问题,提出基于特征分解和属性增强的行人搜索算法。构建有效的特征分解方法,将图像的全局特征在不同方向和尺度分解为细粒度的局部特征并和属性特征进行对齐,无需引入额外的计算,有效提升模型的鉴别能力;设计属性增强策略,在训练阶段随机丢弃部分属性特征,从而有效扩展属性组合,提升模型的泛化表现。实验结果表明,相比于基准算法,本算法的平均精度均值指标最高实现了14.6%的性能提升,并且能够提升泛化表现。(2)针对缺乏多模态对应信息导致搜索性能较差的问题,提出基于多模态预训练模型的行人搜索算法。通过引入多模态预训练模型隐含的跨模态关联先验知识,弥补模态之间的语义鸿沟。构建掩码属性预测任务实现隐式的局部特征关系学习,提升模型对细粒度特征的学习能力。设计基于属性交并比引导的对比损失,自适应调整不同属性组合在嵌入空间的分布,进一步提升搜索结果的准确性。实验结果表明,相比于基准算法,本算法的平均精度均值指标最高实现了 53.8%的性能提升。更多还原

【Abstract】 With the growing demand for intelligent security systems,attributebased person search techniques have emerged as a promising research area within computer vision and cross-modal retrieval.Conventional person search approaches rely heavily on image-based queries,which become ineffective when target images are unavailable and only textual attributes(e.g.,age,gender,or clothing)are accessible.In contrast,attribute-based person search enables efficient image retrieval by leveraging pedestrian attributes,proving particularly valuable in scenarios such as locating missing individuals or identifying suspects.Moreover,compared to textbased search methods,this approach offers a more cost-effective solution while minimizing redundancy and noise in the retrieval process.However,the modal heterogeneity between images and attributes also poses challenges for attribute-based person search.Firstly,pedestrian image data has significant intra-class differences and inter-class similarities,which affect the discriminative ability of the model.Secondly,there are a large number of samples in real scenarios that do not appear in the dataset,which requires high generalization of the model.In addition,there is a semantic gap between image modality and attribute modality,and the representation dimension and information content of images are higher compared to attributes.Finally,the current method lacks corresponding information for multiple modalities,and there is an asymmetry in the representation of different modalities,which limits the performance of cross-modal search.The research topic of this thesis on the attribute-based person search algorithm comes from the Beijing University of Posts and Telecommunications Visual Intelligence Joint Laboratory project.In response to the above challenges,this study first improves the discriminative ability and generalization performance of the model through feature decomposition and attribute augmentation,respectively;secondly,by introducing prior knowledge of multimodal pre-trained models,the semantic gap between modalities can be bridged.The main research content and innovative achievements of this thesis are as follows:(1)A person search algorithm based on feature decomposition and attribute augmentation is proposed to address the issues of significant intraclass differences and inter-class similarities in pedestrian image data,poor discriminative ability,and poor generalization of unseen categories in the dataset.Construct an effective feature decomposition method to decompose the global features of an image into fine-grained local features in different directions and scales and align them with attribute features without introducing additional calculations,effectively improving the model’s discriminative ability;Design an attribute augmentation strategy that randomly discards some attribute features during the training phase,effectively expanding attribute combinations and improving the model’s generalization performance.The experimental results show that compared to the benchmark algorithm,this algorithm achieves a performance improvement of up to 14.6%in the mean average precision(mAP)metric and can improve generalization performance.(2)Aiming at the problem of poor search performance caused by the lack of multimodal corresponding information,a person search algorithm based on a multimodal pre-trained model is proposed.By introducing multimodal pre-trained models with implicit prior knowledge of crossmodal correlations,the semantic gap between modalities can be bridged.Construct a mask attribute prediction task to achieve implicit local feature relationship learning,enhancing the model’s ability to learn fine-grained features.Design a contrastive loss guided by attribute intersection over union(IoU),adaptively adjust the distribution of different attribute combinations in the embedding space,and further improve the accuracy of search results.The experimental results show that compared to the benchmark algorithm,this algorithm achieves the highest performance improvement of 53.8%in the mAP metric.更多还原