【作者】 周睿；

【导师】 夏丹；

【作者基本信息】 华中师范大学 ， 数字媒体技术， 2021， 硕士

【摘要】 近年来,随着手机成像质量的大幅提升,手机相机已逐渐取代专业相机成为人们日常图像拍摄的主力工具。通常手机相机多被用于拍摄风景照,然而受限于相机镜头的大小,获得图像有时无法满足人们对于宽视野风景图像的需求,此时需要利用图像拼接技术对拍摄的图像进行拼接处理。目前,几乎所有智能手机均已配备全景拍摄模式。手机全景拍摄模式具有拍摄导轨来引导进行图像拍摄,该模式下获得的图像具有视差小、尺度与旋转变化小、图像分辨率较高等特点。大部分手机全景拍摄模式中的图像拼接算法原理与AutoStitch算法相同,采用单视点变换模型进行拼接。此类算法存在两个问题:一、对于视差过大(如上下晃动幅度过大)场景会出现拼接失败问题;二、生成图像的整体自然性较差,图像四周边角部分易出现畸变且内部的直线结构变形严重。基于该模式下图像的特点,针对这些问题,本文以网格优化变形模型为框架,结合二进制特征提取算法与快速特征匹配算法,提出一种具有较好拼接效果同时具备一定实时性的手机图像拼接算法。本文主要的研究内容有:(1)提出了一种基于AKAZE-LATCH-GMS的特征提取与匹配算法。本文首先从拼接算法的实时性需求出发,通过大量文献调研与仿真实验对比,选择了更加快速、稳定的AKAZE-LATCH算法替代SIFT进行特征提取。为进一步提升特征提取与匹配的速度,将AKAZE-LATCH算法与暴力匹配算法、基于网格的运动统计方法结合,提出一种基于AKAZE-LATCH-GMS的高效、稳定特征提取与匹配算法。为提升算法的精度,本文在结合过程中进行了两点改进:一、将LATCH描述符中的三元组像素网格的划分与尺度空间参数相关联,提升算法的尺度不变性;二、在暴力匹配算法中引入双向匹配策略,提升特征匹配点对的正确率。仿真实验表明,相较于传统的SIFT+RANSAC特征提取匹配算法,提出的算法计算性能大幅提升,同时对光照、模糊变化等手机全景拍摄中常见的场景具有较好的鲁棒性。(2)提出一种具有线性结构保护的网格优化图像拼接算法。针对手机全景图像拼接算法效果不佳的问题,本文对基于网格变形优化模型的NISwGSP算法进行了研究。针对该算法缺乏对图像内部的直线结构保护问题,本文构造了一个直线约束项并将其添加到变换模型中,以解决图像中的线性结构变形问题。为减弱光照差异对最终图像效果的影响,使用基于人眼视觉感光特点构造的加权融合方法对图像进行融合。此外,本文还在算法中引入OpenMP多线程处理方案并对模型能量函数进行变形降维,实现算法效率的优化。与多个主流拼接算法相比,在大部分场景中,改进的算法具有更好的拼接效果。(3)提出一种基于改进AKAZE-GMS和网格优化的图像拼接算法。本文将提出的特征提取匹配算法与改进的NISwGSP图像拼接算法相结合,生成一种面向手机全景模式图像的拼接算法。针对大视差与运动环境下,手机全景模式的拼接结果出现鬼影与模糊问题,本文将提出的拼接算法与基于最优缝合线的融合算法结合,提供两种不同的融合模式以应对不同的拍摄场景。通过仿真实验,对生成图像从主观与客观两方面进行评估,表明提出的算法可较好地拼接手机拍摄的图像序列,生成的图像整体质量较高。此外,将提出算法与AutoStitch算法进行耗时比较与分析,验证了提出算法可基本满足实时性需求。更多还原

【Abstract】 In recent years,with the rapid improvement of mobile phone image quality,mobile phone camera has gradually replaced professional camera as the main tool of people’s daily photographing.Usually mobile phone camera is used to take landscape photos.However,limited by the size of the camera lens,the captured images can not always meet the needs of people for wide field of view landscape images.At this time,it is necessary to use image stitching technology to stitch the captured images.At present,almost all smart phones have already equipped with panoramic photograph mode.Mobile phone panoramic photograph mode has an instruction guide to guide the image shooting,which is easily affected by light conditions,motion conditions and other factors.Images captured in this mode has the characteristics of small parallax,small scale and rotation change,and high image resolution.The principle of image stitching algorithm in most mobile phone panoramic mode is the same as that of the AutoStitch algorithm,which is based on the single viewpoint transformation model.However,there are two serious problems in this kind of algorithm:first,this algorithm may fail to stitch for the scene with too large parallax,such as the scene that the shaking amplitude above and below is too large;secondly,the overall naturalness of the generated image is poor,the corners around the image are prone to distortion,and the internal linear structure is seriously deformed.To solve these problems,based on a large number of research and experiments,this paper takes the mesh optimization deformation model as framework and combine it with the binary feature extraction algorithm and fast feature matching algorithm,proposed a mobile phone panoramic image stitching algorithm with good stitching effects and certain real-time performance.The main research contents in this paper are as follows:(1)A feature extraction and matching algorithm based on AKAZE-LATCH-GMS is proposed.Based on the real-time requirements of mobile phone image stitching and through a large number of literature research and simulation experiments,we choose the faster and more stable algorithm of AKAZE-LATCH to replace SIFT for feature extraction.In order to further improve the speed of feature extraction and matching,an efficient and stable feature extraction and matching algorithm based on AKAZE-LATCH-GMS by combining AKAZE-LATCH algorithm with BF matching algorithm and grid based motion statistics method was proposed.Furthermore,in order to improve the accuracy of the algorithm,two improvements are made in the combination process:First,we associate the division of the triad pixel grid in the latch descriptor with the scale space parameters to improve the scale invariance of the algorithm;Secondly,we introduce the bidirectional matching strategy in the Brute-Force matching algorithm to improve the accuracy of feature matching point pairs.Simulation experiments show that,compared with the traditional SIFT+RANSAC feature extraction matching algorithm,the proposed algorithm not only greatly improves the computational performance,but also has better robustness to common scenes such as illumination,fuzzy changes and so on.(2)A mesh optimized image stitching algorithm with linear structure protection is proposed.In view of the poor stitching effects of the image stitching algorithm in mobile phone,we analyzed the NISwGSP algorithm which based on mesh deformation optimization model.To solve the problem that the algorithm lacks the protection of linear structure in the image,a linear constraint term is constructed and added into the model to solve the problem of linear structure deformation in the image.In order to reduce the influence of illumination difference on the final image effect,a weighted fusion method based on human visual sensitivity characteristics is used for image fusion.In addition,we also introduce the OpenMP multi-threading scheme into the algorithm and reduces the dimension of the model energy function to optimize the efficiency of the algorithm.Compared with several mainstream stitching algorithms,the improved algorithm performs well in most scenes.(3)An image stitching algorithm based on the improved AKAZE-GMS and mesh optimization is proposed.Combined the proposed feature extraction matching algorithm with the improved NISwGSP image stitching algorithm,we introduced a stitching algorithm for mobile phone panoramic image.Aiming at the problem that the mobile phone panoramic mode may produce stitching results with ghost and blur in large parallax and moving environment,we combine the stitching algorithm with the fusion algorithm based on the optimal stitching line,and provide two different fusion modes to deal with different shooting scenes.Experimental results show that the proposed algorithm can better stitch the image sequence captured by mobile phone,and the generated image has a visual effect.Through the subjective and objective evaluation of the generated image,the proposed algorithm has better stitching performance.In addition,the time-consuming comparison between the proposed algorithm and the AutoSitch algorithm show that the proposed algorithm can basically meet the real-time requirements.更多还原