【作者】 谢文彬；

【导师】 张慧；

【作者基本信息】 清华大学 ， 软件工程（专业学位）， 2021， 硕士

【摘要】 语义分割任务是计算机视觉中的重要研究内容,广泛应用于自动驾驶、视频特效等领域。一般的语义分割深度模型训练需要获取像素级标注作为监督,然而这些像素级标注的获取成本非常昂贵。因此本文旨在研究基于轻量级标注的弱监督语义分割算法,使用获取成本更廉价的标注,如包围盒标注或图像级标签标注,完成语义分割模型的训练。在基于包围盒标注的设定下,本文提出了受限的前景分割网络(Constrained Foreground Segmentation Network,CFS)用于生成高质量原型。其它方法一般使用全尺寸全类别的语义分割网络优化传统方法生成的原型,而本文提出的CFS是输入区域被约束的类无关的分割网络,从而简化了优化问题。生成的原型经过矫正后可以作为语义分割模型训练时的监督。本文还提出了损失指导的自注意力模块(Loss-Guided Self-Attention Module,LGSA),使得语义分割网络具有较好的自我矫正能力。朴素的自注意力模块没有对关联矩阵的学习施加任何约束,而本文提出的LGSA在损失函数指导下,能够限制不同种类像素之间的信息传递,从而生成更合理的关联矩阵,进而不仅可以增强特征,还可以直接增强分割结果。消融实验验证了CFS和LGSA的有效性。本文提出的基于包围盒的弱监督语义分割算法在Pascal VOC 2012验证集上的mIoU指标可达72.4。为了进一步降低标注成本,本文还研究了基于图像级标签的弱监督语义分割算法。在分析了数据集中不可避免地存在数据耦合问题的基础上,提出了简单先验指导的孪生网络(Siamese Network with Simple Priors,SNSP),同时学习图像分类任务与类激活图矫正任务。通过引入极少量的简单先验,SNSP能缓解数据耦合的影响,并通过恰当的损失设计,减缓对简单先验的过拟合现象。SNSP中还包含了先验增强和类激活图矫正模块,用于优化生成的类激活图。这些类激活图合并后可以作为语义分割模型训练时的监督。消融实验验证了SNSP的合理性,并验证了LGSA在图像级标签场景下的有效性。朴素的自注意力模块会因为监督含有较多噪音,产生严重的信息错误传递,进而降低模型性能,LGSA则凭借更合理的关联矩阵,提升模型性能。本文提出的基于图像级标签的弱监督语义分割算法在Pascal VOC 2012验证集上的mIoU指标可达69.0。更多还原

【Abstract】 Semantic segmentation is an important computer vision application and is widely applied in many fields,e.g.,autonomous driving,video special effects.However,general deep semantic segmentation models require pixel-level annotations as supervisions,which are extremely expensive.Therefore,the paper aims to study weakly supervised semantic segmentation(WSSS)based on lightweight annotations,i.e.,using cheaper annotations to train semantic segmentation models.The used annotations include bounding boxes and image-level labels.When using bounding boxes as known annotations,the paper proposes Constrained Foreground Segmentation Network(CFS)to generate high-quality proposals.Other methods use full-size and multi-class semantic segmentation models to optimize proposals generated by traditional methods,while the proposed CFS is a region-restricted and class-agnostic segmentation model,which effectively simplifies the optimization.The generated proposals can be used as supervisions to train semantic segmentation models after correction.Besides,the paper proposes Loss-Guided Self-Attention Module(LGSA)to make semantic segmentation networks have the better ability of self-correction.Naive self-attention modules impose no constraints on the affinity matrix,while LGSA restricts the information exchange between different types of pixels under the guidance of a loss,thereby generating a more reasonable affinity matrix.The matrix can not only enhance the features but also the segments.The paper proves the effectiveness of CFS and LGSA through ablation study.The proposed WSSS method based on bounding boxes achieves70.2% mean Io U accuracy on the Pascal VOC 2012 validation set.To further reduce the cost of annotations,the paper explores WSSS based on image-level labels.By analyzing the inevitable data coupling challenges in the dataset,Siamese Network with Simple Priors(SNSP)is proposed.SNSP learns image classification and class activation map(CAM)correction together.By introducing a very small amount of simple priors,the data coupling phenomenon is reduced.With proper losses,SNSP won’t overfit the simple priors.Besides,a prior enhancement module and a CAM cor-rection module are proposed to optimize the generated CAMs.These CAMs can be used as supervisions for semantic segmentation models after merging.The paper verifies the effectiveness of SNSP through ablation study,and further verifies the effectiveness of LGSA based on image-level labels.The naive self-attention module leads to erroneous information exchange and performance degradation due to the noisier supervisions,while LGSA still improves the performance with a more reasonable affinity matrix.The pro-posed WSSS method based on image-level labels achieves 69.0% mean Io U accuracy on the Pascal VOC 2012 validation set.更多还原