【摘要】 语义分割是计算机视觉领域一项基本且具有挑战性的任务,最近的语义分割研究工作,着力于设计注意力机制和全局性模块,而在注意力机制中,亲和力矩阵是非常重要的部份。传统的亲和力矩阵是以注意力机制的一部分嵌入在神经网络中,通常作为特征融合的权重来使用。尝试直接将亲和力矩阵应用于注意力机制之外,与语义标签联系起来。首先定义标签亲和力矩阵,再在语义分割网络输出层计算亲和力矩阵,将两个矩阵结合起来,得到一个辅助的惩罚函数。标签亲和力矩阵可以视为一种结构上的监督信息,能辅助训练语义分割网络。在NYUv2数据集上的实验结果表明,惩罚函数有助于提高语义分割网络的精度,并且效果显著。更多还原

【Abstract】 Semantic segmentation is a basic and non-trivial task in computer vision. Many previous work focus on utilizing attention mechanisms and non-local models. In attention mechanisms, affinity matrix is very important. Traditional affinity matrix is an essential part of attention models embedded into the neural networks, used as feature fusion weights.In this paper, the affinity matrix is directly used out of attention mechanisms, and it is associated with labels. Firstly,the label affinity matrix is defined, and then the feature affinity is computed on the output layer of semantic segmenta-tions. Finally, with such two affinities, a novel loss function is defined, which uses label affinities as a kind of struc-tural supervision to help train segmentation networks. Extensive experiments on NYUv2 datasets demonstrate that the loss is effective in promoting semantic segmentation networks.更多还原