【作者】 赵旭；

【导师】 孙怡；

【作者基本信息】 大连理工大学 ， 电子与通信工程（专业学位）， 2013， 硕士

【摘要】 自2010年11月kinect问世以来,它受到了众多学者的关注。到目前为止,kinect就被应用到了多个计算机视觉领域当中去,其中在跟踪和识别领域的应用最为广泛。然而,很少有基于kinect的匹配和分割技术方面的研究被提出,这主要是由于kinect深度图像中的各种误差影响限制了其在这些领域的应用,为了扩展kinect将来的应用范围,其深度图像修复技术将会越来越受到人们的重视。本论文从单帧与多帧的角度出发研究了kinect的深度图像修复技术,主要工作有以下几个方面。在单帧的情况下,本文通过彩色图像中的信息来修复深度图像。虽然Kinect能够同时生成深度图像和彩色图像,但是仔细观察不难发现这两幅图像中物体的边界是非对齐的,深度图像中物体的深度值会溢出物体的边界,本文将这些溢出的深度值称作干扰深度值。针对这种情况,本文提出了一个干扰消除方法,通过对彩色图像和深度图像同时进行canny边缘检测,将两幅图像边缘之间非对齐的深度像素点全部置为黑洞(即将该点的灰度值设置为0),这样就可以得到一幅消除掉干扰深度值后的深度图像。然后,在得到干扰消除的深度图像后,针对其具有大面积黑洞区域的特殊情况,本文结合迭代的思想,在充分考虑了彩色图像中的空域信息,颜色信息以及结构相似度(SSIM)信息的基础上,提出了一个迭代联合三边滤波器。通过用这个滤波器对黑洞区域进行黑洞填充,来得到高质量的深度图像。最后针对黑洞填充后的深度图像中存在的伪影,为了能够消除伪影却不破坏深度图像中的边界,本文将联合三边滤波器中的值域滤波器进行自适应处理,然后通过对深度图像进行滤波处理来消除伪影。在多帧的情况下,由于kinect的随机误差和精度会随着目标与kinect之间的距离而改变,从而导致目标距离kinect稍远的情况下,目标表面的深度值会随着时间而进行随机的跳变。为了解决这个问题,本文通过卡尔曼滤波算法在各个深度图像之间建立联系,通过上一时刻深度值预测出当前时刻的深度值,将当前时刻的深度预测值与当前时刻的深度值相结合来计算出当前时刻的深度最优值。实验表明,本文所提的方法不仅能够修复出高质量的深度图像,而且还能解决深度值随时间跳变的问题。更多还原

【Abstract】 The kinect has attracted attention of many scholars since it came out in November,2010. It has been applied to many computer vision fields so far, especially in the tracking and recognition fields. However, few research on matching and segmentation technology based on kinect are proposed. The reason that limits its application in these fields is the effect of the kinect’s various errors. Therefore, depth image inpainting technology of kinect will receive more attention in order to expand the applications of kinect in the future. This paper discusses the depth inpainting technology from the point of view of single-frame and multi-frame. The main works are as follows:In the case of single frame, this paper uses color image’s information to refine the depth image. Although the kinect is able to generate depth images and color images at the same time, the objects’boundary is not aligned between the two images. The depth values of the object in the depth images will overflow the object boundary. These values are called interference depth values in this paper. In the view of this situation, this paper proposes an interference cancellation method. It uses canny algorithm to detect the edges of the depth image and color image. Then the pixels between the edges of two images will be set to black hole (that is to say to set the gray value to zero). Finally we can get a depth image after interference cancellation.Considering the depth image after boundary alignment will have a large area of black hole, this paper combines with the thought of iteration. This paper proposes an iterative joint trilateral filter under the consideration of spatial information, color information and structural similarity (SSIM) of the color image to carry out black hole filling. This joint trilateral filter is used to fill the black hole area to obtain the high quality depth image. After filtering there will be artifacts in the depth image. In order to eliminate the artifacts without destroying the boundaries in the depth image, an adaptive processing range filter is implemented in the trilateral filter in this paper. And then filter the depth image with it to eliminate the artifacts.In the case of multiple frames, due to the random errors and precision of kinect will change with the distance between target and kinect, it will cause random jumpy-changing of depth values of the object surface when the object is a little far from the kinect. In order to solve this problem, this paper establishes a relationship between various depth images by kalman filter algorithm. It combines the predict depth value and the measurement depth value in the current moment to predict optimal current depth value. The experiment shows that the method proposed in this paper can not only obtain a high quality depth image, but also solve problem of depth values jumpy-changing with the time.更多还原