【作者】 黄文彪；

【导师】 滕少华；

【作者基本信息】 广东工业大学 ， 计算机科学与技术， 2023， 硕士

【摘要】 现如今,越来越多的人可以轻松地在网络上发布和获取结构形式多样的数据,如文本、视频、音频、图像等。如何高效和准确地从这么多而复杂的多媒体数据中检索出所需的信息是一个亟待解决的研究问题。为此,一种基于哈希学习技术实现的跨模态哈希检索被提了出来。其思想是将在高维空间中相似的样本映射到低维的汉明空间中对应的相似二进制哈希码表示,然后就可以通过简单的异或运算来计算汉明距离,根据汉明距离的大小来排序就可以实现高效准确地近似搜索。因此,计算和存储成本大大降低。目前,跨模态哈希检索还没有成熟,还存在着一些不足:(a)一些方法将一个9)×(8维的标签矩阵进行成对计算以转化成一个9)×9)维的相似性矩阵,这样就导致需要较大的存储开销和时间代价来构造成对相似性矩阵和需要较大时间来训练哈希码的问题;(b)部分方法为了规避哈希码的离散优化难题,在优化过程中忽视哈希码的离散约束,先直接视为实数值来参与迭代更新,再设定固定阈值将其映射回离散值,这会导致量化误差大的问题;(c)有些方法为了保持哈希码的离散优化,直接将9)×(8维的0-1标签语义向量回归到离散哈希码中而不计算一组9)×9)维的相似性矩阵,这样虽然考虑了标签类别信息的学习,但是忽视了标签语义相似性信息的学习。在多标签数据集上训练时,这将会导致标签语义学习不充分的问题。(d)大部分方法没有考虑到样本特征语义信息对哈希码在学习标签语义信息上的辅助作用。标签特征的维度低,高度概括数据的语义信息;而样本特征的维度较高,能较全面地描述数据的内容。若将两者进行特征融合,将可更好地增强哈希码的高质量学习。本文将聚焦于寻求这些问题的解决方案。本文的主要研究内容如下:(1)针对不足(a)和(b),本文在第三章中提出了带松弛和标签语义引导的离散跨模态哈希。首先,该方法在建模标签语义相似性矩阵时,没有显式地构建成对相似性矩阵,而是将它隐式地表示成两个9)×(8维的单位化的标签矩阵相乘,然后引入两个松弛变量来充当离散哈希码的中间学习状态,采用矩阵分解技术和成对距离差异最小化方法让这两个松弛变量先去建模样本特征语义信息和标签语义信息的学习,再利用松弛变量来学习哈希码。因此,哈希码的学习过程不仅效率高,而且只需专注于两个松弛变量,避免哈希码的学习牵扯到很多的优化变量而导致学习过程发生抖动的问题,同时以离散值的方式让哈希码参与每一次迭代的优化过程,利于降低从实值到离散值带来的量化误差。(2)针对不足(c)和(d),本文在第四章中提出了一种标签与样本双语义增强的离散跨模态哈希。该方法先利用矩阵分解技术来对原始数据特征降维以学习低维特征语义信息,再通过成对距离差异最小化方法来让特征语义信息参与哈希码对于标签语义成对相似性信息的学习过程,同时将特征语义信息和标签语义判别性信息进行加权融合起来,进一步增强哈希码在语义上的学习。这就促使哈希码在学习标签语义相似性的同时,也能够关注学习标签类别语义之间、特征语义信息之间的区分性信息。(3)本文中提出的两个方法在3个公开的主流数据集上与经典的和近期的方法进行对比实验,不论是在检索精度,还是在训练效率上,都取得了具有竞争性的效果。更多还原

【Abstract】 Nowadays,more and more people can easily publish and access data in various forms on the Web,such as text,video,audio,images,etc.How to retrieve the required information efficiently and accurately from so many and complex multimedia data is an urgent research problem to be solved.For this purpose,a cross-modal hash retrieval based on hash learning technique is proposed.The idea is to map the similar samples in the high-dimensional space to the similar binary hash code representation in the low-dimensional Hamming space,and then the Hamming distance can be calculated by simple XOR operation,and the efficient and accurate approximate search can be realized by sorting according to the size of the hamming distance.As a result,computing and storage costs are greatly reduced.At present,cross-modal hash retrieval is not mature yet,and there are still some shortcomings:(a)Some methods explicitly transform9)×(8 dimensional semantic tag information matrix into9)×9)dimensional similarity matrix by performing pairwise calculations,which leads to the problem of requiring significant storage overhead and time cost to construct a similarity matrix and requiring significant time to train hash codes;(b)In order to avoid the discrete optimization problem of hash code,some methods ignore the discrete constraint of hash code in the optimization process,and first directly consider the real value to participate in the iterative update,and then set a fixed threshold value to map it back to the discrete value,which leads to large quantization error;(c)In order to maintain the discrete optimization of the hash code,some methods directly regress the 0-1 semantic vector of the9)×(8 dimension label to the discrete hash code without calculating a set of9)×9)dimensional similarity matrix.Although the learning of the label category information is considered,the learning of the label semantic similarity information is ignored.When training on multi-tag data sets,this results in inadequate tag semantic learning.(d)Most methods do not consider the auxiliary role of sample features in learning semantic information of tags.Tag features have low dimensions and highly summarize the semantic information of data;and the dimension of sample features is relatively high,which can comprehensively describe the content of the data.If the two are fused,it will better enhance the high-quality learning of hash codes.This thesis focuses on finding solutions to these problems.The main research content and contributions of this thesis are as follows:(1)In view of deficiencies(a)and(b),a discrete cross-modal hash with relaxation and tag semantics guidance is proposed in Chapter 3.First,when modeling the semantic similarity matrix of labels,this method does not explicitly construct a pair similarity matrix,but implicitly expresses it as the multiplication of two9)×(8 dimensional label matrices.Then,two relaxation variables are introduced to serve as the intermediate learning state of discrete hash codes.The matrix decomposition technology and the paired distance difference minimization method are used to let these two relaxation variables first model the learning of sample feature semantic information and tag semantic information,and then use the relaxation variables to learn the hash codes.Therefore,the learning process of hash codes not only has high efficiency,but also only needs to focus on two relaxed variables to avoid the problem of jitter in the learning process caused by many optimization variables involved in the learning of hash codes.At the same time,the hash code is involved in the optimization process of each iteration in a discrete value manner,which is conducive to reducing the quantization error brought by the transition from real value to discrete value.(2)In response to shortcomings(c)and(d),this thesis proposes a discrete cross modal hash with bilingual enhancement of tags and samples in Chapter 4.This method first uses the matrix decomposition technology to reduce the dimensions of the original data features to learn the semantic information of low dimensional features,and then the feature semantic information participates in the learning process of hash codes for tag semantic pairwise similarity information through the paired distance difference minimization method.At the same time,the feature semantic information and tag semantic discriminant information are weighted together to further enhance the semantic learning of hash codes.This enables hash codes to learn tag semantic similarity and pay more attention to the differentiated information between tag semantics and feature semantic information.(3)The two methods proposed in this thesis are compared with the classical and recent methods on three publicly available mainstream data sets,and both in terms of retrieval accuracy and training efficiency have achieved competitive results.更多还原