【作者】 刘杰；

【导师】 黄艳；

【作者基本信息】 山东大学 ， 人工智能， 2023， 硕士

【摘要】 图像抠图是指从图像中提取软前景的过程,结果由包含每个像素位置处前景的不透明度水平(或α值)的透明度遮罩表示。区别于图像分割产生的α值为非0即1的硬前景,透明度遮罩的值是介于0和1之间的浮点数。作为一种基本的图像处理技术,图像抠图在图像、视频的编辑与合成、虚拟现实和增强现实、电影后期制作中有着许多重要的应用。由于图像抠图通常是一个不适定的问题,许多图像抠图算法除了图像之外还需要用户制作三分图等辅助输入来获得额外的指导。为了避免制作辅助输入耗费的大量劳动力,涌现了许多关于自动图像抠图方法的研究。大多数现有的自动图像抠图方法是通过设计网络来生成三值语义图并使用估计的语义图来指导抠图。随着抠图的应用范围扩大,用户需要面对复杂的场景变化。当抠图对象比较简单时,用户希望可以提高抠图效率,相反,当遇到复杂的抠图对象时,用户则倾向于牺牲效率来提高抠图精度。但遗憾的是,目前的抠图网络不具备灵活性。针对上述问题,本文首次尝试将图像抠图问题建模为多阶段任务。本文提出的多阶段自动抠图神经网络由三个新的应用注意力机制的阶段组成,每个阶段组成相似并且可以包含之前的阶段组成独立的网络,每个独立网络可单独生成透明度遮罩,而且精度逐步提高。通过上述方法,不仅给用户提供了一个无需辅助输入的全自动抠图网络,而且用户还可以在精度和效率之间灵活权衡。与此同时,发现目前的抠图方法普遍通过对真值透明度遮罩进行相同的形态学操作来为不同类型的前景定义统一的真值语义图。然而,由于自然图像前景具有复杂性和不统一性,统一的形态学操作可能会生成不合理的语义图。并且全自动抠图模型考虑到内存占用问题不会设计较大的网络去估计语义图,这就导致会产生不准确的语义图估计,进而导致语义图无法有效的指导后续的抠图任务。针对上述问题,本文提出广义二值掩码定义,将图片按照前景的类型分为显著和非显著两类,并且标记了 Adobe数据集中的图像作为真值语义图,与此同时广义二值掩码相较于三值语义图减轻了网络的目标难度,使得网络可以得到更准确的语义图估计。在此基础上,本文还提出了一个基于广义掩码的自动抠图网络,网络由两个子网构成,掩码子网利用轻量化模型从输入的彩色图像中估计广义掩码,然后将其与原始图像一起馈送到应用改进的Swin Transformer以及渐进式微调模块构成的抠图子网以生成最终的透明度遮罩估计。本文提出的两个模型都在很大程度上优于当前最先进的自动图像抠图模型。更多还原

【Abstract】 Image matting is the process of extracting soft foregrounds from an image.As a basic image processing technology,it has many important applications in editing and synthesizing images and videos,virtual reality and augmented reality,as well as film production.The result of matting is represented by an alpha matte that contains the opacity level of the foreground(as the value of α)at each pixel.Unlike the hard foreground generated by image segmentation where the a is either 0 or 1,the value of α in the alpha matte is between 0 and 1.Image matting is usually an ill-posed problem.Therefore,in addition to images,many image matting algorithms require users to make auxiliary inputs such as trimap(a three-value semantic map)to obtain additional guidance.In order to avoid the huge labor cost of making auxiliary input,many researches on automatic image matting methods have emerged.Most of the existing automatic image matting methods are based on the design of a network to generate asemantic trimap estimate and use the estimated semantic map to guide matting.The application scenarios of matting become more and more complex,and users need to face complex scene changes.When matting objects are simple,users hope to improve the efficiency of the matting process.On the contrary,when matting objects are complex,users tend to sacrifice efficiency to improve matting accuracy.Unfortunately,the current matting network is not flexible and cannot change with the change of application scenarios.In view of the above problems,we first try to model the image matting problem as a multi-stage task.The multi-stage automatic matting network we propose consists of three stages applying attention mechanism.Each stage is similar and can be combined with the previous stages to form an independent network.Each independent network can generate alpha matte estimation independently and the accuracy of the result is gradually improved.Through the above methods,we provide users with a fully automatic matting network without auxiliary input,and users can flexibly balance accuracy and efficiency.At the same time,we find that the current matting methods generally define a unified groundtruth semantic image for different types of foreground by performing the same morphological operation on the groundtruth alpha matte.However,due to the complexity and non-uniformity of the natural image foreground,unified morphological operations may generate unreasonable semantic images.In addition,the full automatic matting model will not design a large network to estimate the semantic map considering the memory occupation problem,which will lead to inaccurate estimated semantic map,and thus will not effectively guide the subsequent matting task.In response to the above problems,we propose the definition of generalized binary mask,which divides images into significant and non-significant categories according to the type of foreground,and labels the images in Adobe dataset as groundtruth semantic maps.At the same time,generalized binary mask reduces the difficulty of network objectives compared with three-value semantic maps,so that the network can obtain more accurate semantic map estimation.On this basis,we also propose an automatic matting network based on the generalized mask.The network is composed of two subnets.The mask subnet uses a lightweight model to estimate the generalized mask from the input color image,and then feeds it with the original image to the matting subnet constructed based on the improved Swin Transformer and progressive fine tuning module to generate the final alpha matte estimate.The two models we proposed are superior to the most advanced automatic image matting model to a large extent.更多还原