【作者】 张丽霞；

【导师】 曾广平；

【作者基本信息】 北京科技大学 ， 计算机应用技术， 2020， 博士

【摘要】 由于图像采集设备的局限性,单一图像无法全方位展示场景的信息,因此,提出了图像融合的思想。图像融合是利用图像之间信息的冗余性和互补性,提取源图像中的“感兴趣的”特征信息,融合成信息全面的清晰的图像,进一步提高图像的空间感知能力、利于图像分析处理,更加适合于人类视觉识别和计算机后续检测、分类等工作。随着科技的进步,生活的快节奏,图像的多类型,使得图像融合的实时性、普适性成为迫切解决的关键问题。本文在参阅大量的中外文献的基础上,针对多聚焦图像、医学图像、红外与可见光图像的融合,从特征的精准度、方法的普适性和实时性三方面考虑,设计和实现了四种图像快速融合的方法。其主要研究内容和创新点如下:(1)针对图像的块效应问题,从邻域的内容相关性出发,提出了自适应差分进化算法。利用种群进化自适应调整缩放因子和交叉因子,实现了分区的自适应;通过启发式搜索策略,迭代求得分块的最佳尺寸。结合拉普拉斯金字塔变换,设计了基于自适应差分进化算法的多聚焦图像融合方法。近似系数利用自适应差分进化算法划分区域,利用SML计算区域的聚焦度并形成融合决策图,借助逐像素加权融合规则完成近似系数融合。细节系数采用了局部区域梯度能量取大与优化决策图相结合的融合策略完成融合。实验结果表明,本方法的融合结果画面清晰,消除了块效应和边缘的不连续性,在主观视觉效果和客观评价两方面均优于所选的方法。(2)针对多源图像特征的多样性,在自适应差分进化算法的基础上,从图像结构出发,提出了基于区域累加梯度对比度的活动级测量算法,精准提取图像的结构信息和边界信息。为了实现方法的实时性,设计了基于区域累加梯度对比度的多源图像快速融合方法(RAGC)。采用自适应差分进化算法计算图像区域分割的最佳尺寸,通过比较两输入图像对应像素点的区域累加梯度对比度构建融合决策图,借助形态学和图像引导滤波算法优化获得最终决策图,最后采用逐像素加权融合规则完成图像融合。实验结果表明,本方法特征提取精准,结构清晰,一定程度上解决了细节模糊问题和抑制了块效应,适合于多种类型图像的融合,具有一定的普适性;运行速度快,满足了实时处理的要求。(3)为解决RAGC方法在融合部分医学图像时出现少量的伪影问题,选用PCNN模型进行特征提取,根据神经元的点火总数来确定清晰度。为提高特征精准度,设计并实现了 PCNN模型所有参数的自适应设置,将图像的静态特征与PCNN模型所有参数动态链接,实现了不同图像,不同参数,不同特征。为加快执行速度,利用脉冲皮质模型简化了 PCNN模型,不仅降低了 PCNN模型的耦合度,而且减少了参数个数,由9个减少为5个。针对多模态图像,设计了 NSST域下参数自动设置的简化的PCNN模型的融合方法。高频系数利用参数自动设置的简化的PCNN模型计算各系数的点火总数完成融合。低频系数利用区域能量和梯度能量相结合实现融合。为充分验证方法的有效性,从多个角度设计了实验。实验结果表明,本方法的融合结果局部细节提取精准,画面清晰,对比度无失真,有效解决了伪影的问题,优于所选择的经典方法。(4)为了避免人工特征提取的局限性,采用了 CNN模型,利用大数据驱动自适应学习寻求图像特征,提升了特征的精准度,具有很强的泛化性。为了保证空间信息的完整性,提出了基于上采样操作的改进的CNN模型,由六层叠加小卷积操作组成。多层设计不仅扩大了感受野、保证了平移的不变性,而且减少了训练参数的个数,提升了运算速度。针对多聚焦图像,提出了基于改进的CNN模型的融合方法。输入图像经改进的CNN模型分离为聚焦区域和非聚焦区域,采用逐像素加权融合策略拼接聚焦区域完成图像的融合。实验结果表明,本方法的融合结果细节清晰、结构完整、对比度无失真,画面无伪影,有效避免了灰度不连续、人工噪音等问题。该方法在主观视觉效果和客观评价准则两方面都取得了优于传统方法的结果。本方法借助GPU,实现了并行计算,提升了执行效率。更多还原

【Abstract】 Due to the limitations of image acquisition equipment,a single image cannot fully display the information of the scene.Therefore,the idea of image fusion is proposed.Image fusion is to use the redundancy and complementarity of information between images to extract "interesting" feature information in the source image and merge it into a comprehensive and clear image,which further improves the spatial perception ability of the image and facilitates image analysis and processing.It is more suitable for human visual recognition and computer subsequent detection and classification.With the advancement of science and technology,the fast pace of life,and multiple types of images,the real-time and ubiquity of image fusion have become to be the key issues.In this paper,four fast image fusion methods are designed and implemented for the multi-focus image fusion,medical image fusion,infrared and visible image fusion,considering the three aspects of the accuracy of features,the ubiquity and real-time of methods.The main research contents and innovations are as follows:(1)In order to solve the problem of block artifact,an adaptive differential evolution algorithm is proposed based on the content relevance of neighborhood.The adaptation of the partition is realized by adaptively adjusted the scaling factor and crossover factor of population evolution algorithm.And the optimal size of the block is obtained iteratively by heuristic search strategy.Then,according to the Laplace pyramid transform,a multi-focus image fusion method(DE-LP)based on adaptive differential evolution algorithm is designed.The approximate coefficients are divided into regions by an adaptive differential evolution algorithm,the focus of the region is calculated by SML to form the fusion decision map,and the approximate coefficient is fused by pixel-by-pixel weighted rule.And the detail coefficient is fused by the fusion strategy combining the region gradient energy and the optimal decision map.Experimental results show that the DE-LP method is able to produce clear images without block artifacts as well as edge discontinuity,and is superior to the selected methods in both subjective visual effects and objective evaluation.(2)In view of the diversity of multi-source image features,the activity level measurement algorithm based on the regional accumulation gradient contrast is proposed to accurately extract the structure information and boundary information of the image,according to the adaptive differential evolution algorithm.In order to achieve the real-time of fusion method,a fast multi-source image fusion method(RAGC)based on the region-accumulation gradient contrast is designed.The optimal size of the region is calculated by the adaptive differential evolution algorithm,and the region-accumulation gradient contrast of each pixel is calculated.Then,the initial decision map is constructed by comparing the region-accumulation gradient contrast of the corresponding pixels of two images.With the optimization by morphology and image guided filtering algorithm,the final decision map is obtained.Finally,the input images are fused by pixel by pixel weighted rule.Experiments show that RAGC is capable of extracting feature accurately and the result is clear in structure,which solves the problem of detail blur and block artifacts to a certain extent.And RAGC is suitable for the fusion of various types of images and has a certain ubiquity.Running of RAGC is fast,which meet the requirements of real-time processing.(3)In order to remove a small amount of artifacts in the fusion results of some medical images by RAGC,PCNN model is adopted for feature extraction,in which the total of firing of neurons is to determine the clarity of image.the bigger the total of firing is,the clearer the image.In order to improve the accuracy of the features of multi-modal image,all parameters of PCNN model are dynamically linked with the static features of image,which realizes that different parameters of PCNN model are set in different images,and different features are extracted.Arming to improve the performance of execution,the PCNN model is simplified by the Spiking Cortical Model,which not only reduces the coupling of the PCNN model,but also reduces the number of parameters,from 9 to 5.For multi-modal images,a simplified PCNN model fusion method with automatic-setting parameter in NSST is designed.The high frequency coefficients are fused by the simplified PCNN model with automatic-setting parameter to calculate the total number of firing for each coefficient.The low frequency coefficients are fused by the combination of regional energy and gradient energy.Experimental results show that this method solves the problem of artifacts and is superior to the selected classic method.the fused image of this method is accurate in local details,clear,well contrast.(4)In order to avoid the limitations of artificial feature extraction,the CNN model is adopted to extract image features by big data-driven adaptive learning,which improves the accuracy of the features.For avoiding the loss of spatial information,an improved CNN model based on up-sampling is proposed,which consists of six layers of superimposed small convolution operations.The multi-layer design not only expands the receptive field,ensures the invariance of translation,but also reduces the number of training parameters,and improves the calculation speed.A fusion method based on improved CNN model is proposed for multi-focus images.The improved CNN model of the input image is separated into focus area and non-focus area,and the focus areas are intergraded by pixel-by-pixel weighted fusion strategy to obtain fusion image.Experimental results show that the fusion results of this method are clear in detail,complete in structure,no distortion in contrast,and no artifacts in the picture.This method effectively avoids grayscale discontinuity,artifacts and other problems,and it is better than traditional methods.With GPU,this method realizes parallels computing and improves execution efficiency.更多还原