【作者】 杨坤；

【导师】 王明辉；

【作者基本信息】 四川大学 ， 工程硕士（专业学位）， 2021， 硕士

【摘要】 相对于立体视频,自由视点视频(Free-viewpoint Video,FVV)在倾入感和交互性方面更占优势,是未来数字视频应用的重点发展方向。因当前存储和带宽的制约,无法实现大量数据从FVV采集端到显示端的高效实时传送。虚拟视点绘制方法,作为FVV系统的核心算法,能将少量视点数据合成更多的视点数据,尤其是基于深度图的绘制(Depth-image-based Rendering,DIBR)方法,因具有带宽增量低、易于实现、执行速度快以及图像生成质量高等优势,被移动图像专家组(Moving Picture Experts Group,MPEG)应用于FVV系统中所使用的视点生成方法。DIBR存在虚拟视点质量等其它问题,解决好DIBR中的问题便能获得高质量的视觉体验效果。针对以上问题,论文围绕了DIBR性能优化(提高虚拟视点质量)展开研究,其主要研究内容和成果如下:(1)提出了一种基于颜色校正与深度图空域滤波的预处理方法,以防参考视点图像质量对虚拟视点绘制效果的影响。首先,该算法引用了一种基于块的颜色校正方法,以防止不同视点因位置、环境等不同而导致出现伪影现象;然后,该算法采用了论文优化的一种深度图空域滤波预处理方法,以修正深度图中存在的错误深度信息。该方法主要使用形态学闭运算、中值滤波以及导向滤波来提高深度图的准确性。(2)提出了一种优化的正向映射方法,以解决重叠问题和减少裂纹数量。该算法是基于Z-buffer和一投二方法,实验表明该方法能够消除重叠现象和减少裂纹数量。提出了一种优化的逆向映射算法,以防止逆向映射导致的错误裂纹填充点。该方法利用中值滤波和邻域内平均颜色差值来避免错误的裂纹填充点。提出了一种优化的邻域背景修补算法,以提高空洞修复的效果,实验表明该算法能够增强所获取加权像素点的可靠性。(3)提出了一种基于多路参考视点的虚拟视点绘制方法,以增强生成的视点图像质量。首先,根据参考视点分布特点,选定所需参考视点;然后,使用论文优化的图像预处理方法对参考视点图像执行预处理;其次,采用文中优化的正向映射及引用的突变点剔除方法来生成中间视点视图;然后,使用优化的逆向映射方法来消除裂纹;接着,采用基于基线距离的融合方式进行图像融合;最后,采用论文优化的邻域背景修补算法来修复空洞。通过实验验证,论文优化算法在增强虚拟视点质量的性能方面具有不错效果,提高了虚拟视点绘制效果以及视觉体验感受。更多还原

【Abstract】 Compared with stereoscopic video,Free-Viewpoint Video(FVV)has more advantages in the aspect of immersion and interactivity,which is the key development direction of digital video application in the future.Due to the constraints of current storage and bandwidth,it is impossible to realize the efficient real-time transmission of a large number of data from FVV acquisition end to display end.As the core algorithm of FVV system,virtual viewpoint rendering method can synthesize a small number of viewpoints into more viewpoints.Depthimage-based Rendering(DIBR)has been applied in FVV system by Moving Picture Experts Group(MPEG)because of its low bandwidth increment,easy implementation,fast execution speed and high quality of image generation.There are other problems in DIBR,such as the quality of virtual viewpoints.If the problems in DIBR are solved well,high-quality visual experience can be obtained.Aiming at the above problems,this paper focuses on the performance optimization of DIBR(improving the quality of virtual viewpoints),and its main research contents and results are as follows:(1)A preprocessing method based on color correction and depth map spatial filtering is proposed to prevent the influence of the quality of reference view image on the rendering effect of virtual view.Firstly,the algorithm uses a block-based color correction method to prevent artifacts from appearing in different viewpoints due to different locations and environments.Then,the algorithm adopts a spatial filtering preprocessing method optimized by the paper to modify the error depth information in depth map.This method mainly improves the accuracy of depth map by morphological closure operation,median filtering and guided filtering.(2)An optimized forward mapping method is proposed to solve the overlap and crack problems.The algorithm is based on z-buffer and the self-applicable one for two method.Experiments show that the method can eliminate the overlap phenomenon and effectively reduce the number of cracks.An improved inverse mapping algorithm is proposed to avoid a large number of wrong crack filling in the inverse mapping.The method uses median filter and average color difference in the field to eliminate the wrong mapping points.An optimized domain background repair algorithm is proposed to improve the effect of hole repair.Experiments show that the algorithm can enhance the reliability of the weighted pixels.(3)A virtual viewpoint rendering method based on multiple reference viewpoints is proposed to improve the quality of the generated viewpoint image.Firstly,according to the distribution characteristics of the reference viewpoint,the required reference viewpoint is selected;secondly,the optimized image preprocessing method is used to preprocess the reference viewpoint image;secondly,the optimized forward mapping and mutation point elimination method are used to generate the intermediate viewpoint view;then,the improved reverse mapping method is used to eliminate the cracks;and then,the base-based image preprocessing method is used finally,the optimized domain background repair algorithm is used to repair the hole.Through experimental verification,the optimized algorithm can achieve good results in performance,and improve the quality of virtual view image and visual effect.更多还原