【作者】 李华；

【导师】 陈传波；

【作者基本信息】 华中科技大学 ， 软件工程， 2021， 博士

【摘要】 超像素分割是图像处理领域的一项基础性工作,其目的是根据像素内在特性,如颜色的一致性和纹理的相似性,将像素划分成若干个高语义层级的基元,并将其作为后续任务输入的最小基元。与离散的像素级基元不同,超像素可以产生具有语义感知意义的原子单元,这些原子单元可以贴合图像内容的边缘与纹理,并减少输入基元的数量。归功于其数据冗余少,表示效率高等优点,超像素被广泛应用于许多计算机视觉任务当中,比如显著性检测、目标跟踪、医学图像分割等,具有十分广阔的应用前景。尽管现有的大部分传统超像素分割算法都能够实现较好的分割效果,但是仍然面临着一些瓶颈与挑战:超像素块的形状结构基本上都是不规则图形,可能会给后续任务带来数据操作与存储上的负担;大多数超像素分割算法会丢失很多图像结构的细节信息,而这些对于某些任务来说可能是关键信息;面对日益增长的多源数据相关任务的需求,如何将传统的单幅图像的超像素分割算法与跨模态的任务相结合并发展也是亟需解决的难题。针对上述问题,以结构优化为导向,从结构化存储、结构保持和结构协同一致三个方向分别提出了相应的改进办法,以促进该领域的进一步发展,主要的研究内容包含以下几个部分:(1)针对超像素形状不规则带来的存储量大的问题,提出了一种结构化逼近的超像素分割算法。超像素分割的一个很明显的缺点就是超像素块形状不规则,这可能会导致后续操作的数据存储大量增加。超像素分割的初衷不仅是从图像的可见内容出发,还应从数据的存储和操作的角度考虑,从而实现基元的精简和分割效率的提升。将超像素分割建模为一个以正方形为基元的非对称分割问题,将语义感知的超像素记录在正方形层级上而不是像素点层级,节省存储空间。除此之外,为了使得规则形状的超像素单元更好地贴合图像边缘和轮廓,设计了一种组合优化策略来实现正方形和孤立像素的最佳组合,能够生成结构化逼近、边界清晰、语义信息全面的超像素。将该方法与一些最先进的超像素分割算法在公共测评基准上进行了比较,从定量和定性上验证了该方法的有效性。(2)针对图像细节结构与边缘难以保持的问题,提出了一种基于局部空间约束的子空间聚类的超像素分割算法。现有的超像素分割算法大多都面临一个问题是对局部细节信息分割的局限性,如细致的图像边缘及纹理等。在很多实际场景和工业应用中,这些局部细节信息往往是不能忽略的关键信息。如果要获取精细的细节信息,这些方法只能大幅增加超像素的数量,这样的处理方式通常会导致稀疏区域的大量数据冗余,所以如何平衡图像内容的细节信息和超像素的数量是超像素分割实际应用中所面临的另一挑战。将图像超像素分割问题建模为一个子空间聚类问题,每个超像素块可以视为一个具有相同本质属性的像素点的子空间集合,如相同的颜色值、空间信息、相似的纹理和边缘等。由于属于不同子空间的像素点往往属性也不一样,这样子空间聚类的解等同于超像素分割的方案。然后针对图像像素点之间的空间关联性,进一步设计了基于像素点邻近关系的一个空间正则化约束,提出了一种凸局部约束子空间聚类模型来生成内容感知的超像素块。与现有工作的多组对比实验结果验证了所提出的方法具备高效的性能,即可以用较少的超像素来获得更多的细节和语义信息。(3)为了解决超像素分割方法在双目图像中较难保持左右视图结构一致性的问题,提出了一个基于左右视图交互优化的双目立体图像超像素分割框架。对于立体视觉任务而言,目标是需要更加协同一致地得到左右视图的超像素分割结果,而不是简单地直接进行左右视图的独立分割,而这方面的研究还很少。考虑到左右图像视图的差异,首先根据视差将图像划分为匹配区域和非匹配区域,然后在匹配区域之间构建对应关系,以减轻遮挡造成的匹配误差。最后结合左右匹配一致性,提出了一种协同优化方案,以交互方式协调优化左右图像的匹配超像素,使匹配的双目超像素对更加一致和准确。定量和定性实验表明,与单幅图像的超像素分割相比,该框架在结构一致性和分割准确性方面都能取得较高的性能。综上所述,以超像素的结构优化为导向,从结构化存储、结构保持和结构协同一致三个方向分别对现有超像素方法存在的问题进行了改进,并且提出了相应的解决办法,缓解了相关问题存在的缺陷,促进了超像素分割领域的进一步发展。更多还原

【Abstract】 Superpixel segmentation is a basic task aiming at grouping pixels into some high-level primitives based on the intrinsic properties,such as coherent color and similar texture.Instead of discretizing pixel-level entities,superpixels can produce perceptually meaningful atomic units that adhere to object boundaries and reduce the number of primitives.As the basic processing unit,superpixel has been widely used in many tasks,such as saliency detection,object tracking,and medical image segmentation.The notable achievement of superpixel is contributed to its consistence with human visual recognition and less data redundancy.Most of the existing algorithms can achieve excellent segmentation performance and decrease the number of primitives.However,there still exist some drawbacks and bottleneck,for example,the irregular shape of superpixels,the loss of detailed information in superpixel segmentation,and the combination of conventional superpixel segmentation methods with multi-view images,and so on.In view of the above problems,this thesis puts forward corresponding improvement measures respectively to promote the further development of this field.The main research contents and work include the following parts.(1)For the irregular shape of superpixel,a superpixel segmentation method is proposed to generate approximately structural superpixels with sharp boundary adherence and comprehensive semantic information.One obvious disadvantage is that the shape of superpixels is irregular,which may induce the substantial increases in data storage of subsequent operations.The original intention of superpixel segmentation is to reduce the number of primitives not only from the visible image content,but also from the consideration of storage and operation.Therefore,generating the storage-efficient and regular-shape superpixels is a crucial issue.The superpixel segmentation is formulated as a square-wise asymmetric partition problem,where the semantic perceptual superpixels are recorded in square level to preserve abundant semantic information and save storage simultaneously.Moreover,in order to achieve regular-shape superpixel units to better adhere to image boundaries and contours,a combinatorial optimization strategy is devised to achieve an optimal combination of squares and isolated pixels.(2)A spatially constrained subspace clustering based superpixel segmentation model is proposed to preserve the detailed image content construction.Another obvious weakness of the existing superpixel methods is the limitation to adapt to the local image details,such as the content boundaries and object contours.And in many cases of practical problems and industrial tasks,the local details are the key information,which should not be ignored.For most methods,the detailed boundaries of image content are still hard to be well preserved,due to the lack of the prior knowledge about the shape and size of the superpixels in an image.These methods have to increase the number of superpixels and reduce the superpixel size to achieve the detailed boundaries.This may usually lead to large data redundancy in the sparse area,which is unwillingness for practical tasks.How to balance the detailed information and superpixel number is a challenging problem for superpixel segmentation to be applied in industrial tasks.To keep detailed image boundaries,a spatial regularization term to emphasize the spatial correlation is devised,and a spatially constrained subspace clustering based superpixel segmentation model is proposed to generate superpixels with more accurate and detailed boundaries,which is more appropriate for practical and industrial tasks.(3)To produce structural-consistency stereo superpixels,a left-right interactive optimization framework for stereo superpixel segmentation is proposed.In recent years,dual-camera system becomes more and more popular,which has been widely used in mobile phones and autonomous vehicles.Moreover,it turns out that stereo image pairs have better consistency with human perception scheme than a single image,and the information from the two views are complementary and correlative,which is conducive to scene representation and object modeling.However,the task of superpixel segmentation for stereo image pairs is a challenging new proposition,because the information consistency and difference between two viewpoints need to be considered jointly.For stereo images,the stereo vision task aims to obtain the superpixel segmentation results of the left and right views more cooperatively and consistently,rather than simply performing independent segmentation directly,but there is little research in this field.Considering the view difference between the left and right images,the images are divided into paired region and non-paired region according to the disparity,and construct a matching relationship between paired regions to alleviate the matching errors caused by occlusion.Then,combined with the left-right matching consistency,we propose a collaborative optimization scheme to coordinately refine the matched superpixels of the left and right images in an interactive manner,and enforce the matched superpixels in stereo pairs become more consistent and accurate.In conclusion,guided by the structural optimization of superpixels,the problems in the existing methods of superpixel segmentation are improved from three aspects: structural storage,structural preservation and structural coordination.The corresponding solutions alleviate the defects of related problems and promote the further development of the field of superpixel segmentation.更多还原