【摘要】 跨模态检索是指给定一种模态的查询词,返回与之语义相关的其他模态关联词的一种检索方法。现有工作主要集中监督式跨模态检索方法研究,而实际应用中样本标签少,样本标签获取成本高。为此,提出一种图约束的半监督对抗跨模态检索方法(SS-ACMR)。该方法通过对无标签样本建立图作为约束条件来学习公共子空间表示。具体而言,在对抗学习框架下:1)对无标签样本,根据样本之间欧式距离构建图,希望相似样本的公共子空间表示是相似的; 2)对有标签样本使用传统的对抗跨模态检索方法进行学习; 3)无标签样本和有标签样本在对抗学习框架下共同学习公共子空间的表示。Wikipedia数据集和NUSWIDE-10k数据集上的实验结果表明:本文的方法得到了和现有监督跨模态检索方法相当的检索结果,远好于现有半监督跨模态检索方法。更多还原

【Abstract】 Cross-modal retrieval is a retrieval method that gives a modal query word and returns other modal word related to its semantics. Current works mainly focus on supervised cross-modal retrieval methods,but in practical applications,there are fewer sample tags and the cost of sample tag acquisition is high. In this paper,a graph-constrained semi-supervised cross-modal adversarial retrieval method( SS-ACMR) is proposed. In this method,the common subspace representation is learned by constructing a graph by modeling the unlabeled samples as a constraint. Specifically,in the framework of adversarial learning,1) for unlabeled samples,the common subspace representation of similar samples is expected to be similar according to the Euclidean distance between samples; 2) for labeled samples,the traditional adversarial cross-modal retrieval method is used to learn; 3) for unlabeled samples and labeled samples,the representation of common subspace is jointly learned under the framework of adversarial learning. The experimental results on Wikipedia data set and NUSWIDE-10 k data set show that the proposed method achieved the comparable retrieval results as the existing supervised cross-modal retrieval methods,and is much better than the existing unsupervised cross-modal retrieval methods.更多还原