【摘要】 RGB-D显著目标检测旨在从配对的彩色图像与深度图像中识别最具视觉吸引力的目标,关键问题在于实现多模态特征与多尺度特征的有效融合.现有方法在RGB特征与深度特征融合过程中,模态互补信息表达、边缘细节保持及尺度间关联利用仍有进一步提升空间.为此,文中提出多模态注意力感知与相邻尺度建模的Transformer网络(Transformer Network with Multimodal Attention Perception and Adjacent-Scale Modeling, MATNet).首先,采用双分支金字塔池化Transformer编码器,分别提取RGB模态和深度模态的多层级特征,并在各层引入多模态注意力融合模块,联合通道注意力与空间注意力,增强模态互补信息表达和关键区域语义一致性.然后,构建相邻尺度建模模块,自上而下逐级聚合相邻尺度特征,有效融合高层语义信息与低层边缘纹理信息,提升显著目标的结构完整性与边界表征能力.最后,结合多尺度预测与监督机制,构建端到端检测框架.在5个公开数据集上的实验表明,MATNet在提升检测精度与边缘保持能力方面具有稳定性与有效性.更多还原

【Abstract】 RGB-D salient object detection aims to identify the most visually attractive objects from paired color images and depth images, and the key challenge is the effective fusion of multimodal and multiscale features. The existing methods still need the improvement in modal complementary information representation, edge detail preservation and utilization of cross-scale association during the fusion of RGB features and depth features. Therefore, a Transformer network with multimodal attention perception and adjacent-scale modeling(MATNet) is proposed. Multilevel RGB features and depth features are extracted by dual-branch pyramid pooling Transformer encoders. A multimodal attention fusion module is introduced at each stage. The modal complementary information representation and semantic consistency in key regions are jointly enhanced by channel attention and spatial attention. Then, an adjacent-scale modeling module is constructed to aggregate adjacent-scale features progressively in a top-down manner. High-level semantic information and low-level edge texture information are fused effectively. The structural integrity and boundary representation capability of salient objects are improved. Finally, an end-to-end detection framework is constructed by combining multi-scale prediction and the supervision mechanism. Experiments on five public datasets demonstrate that MATNet is effective and stable in improving detection accuracy and edge preservation capability.更多还原