【作者】 陈蓉；

【导师】 李翠华； 曲延云；

【作者基本信息】 厦门大学 ， 计算机科学与技术， 2019， 博士

【摘要】 单帧图像超分辨率重建技术利用图像处理和机器学习理论,实现用一张给定的低分辨率图像生成出一张具有良好客观质量和视觉效果的高分辨率图像。超分辨率图像可以被广泛应用于医疗图像处理、遥感图像处理、公共安全监控、数字娱乐、多媒体通讯等领域,超分辨率重建技术因此得到人们的广泛关注。近几年蓬勃发展的基于深度学习的图像超分辨率模型(即深度超分模型)在超分辨率重建问题上取得了良好成绩,大大超越了传统的超分辨率重建算法,成为计算机视觉领域的一个研究热点。已有的多数深度超分模型主要关注模型的保真项重建,忽视了对深度超分模型的正则化约束分析。针对该问题,本文研究正则化约束下的深度超分模型,通过图像内部的统计先验、超分模型重建的损失、上下文特征和判别器为正则化约束条件来提升超分模型的性能,提出了四个新的图像超分辨率重建算法或模型。本文的研究内容及主要创新点如下:1.针对已有深度超分模型未考虑输入图像内部的统计先验,提出了一个联合统计模型引导深度网络的超分重建算法。联合统计模型使用了两个图像内部统计先验:非局部相似性和局部平滑度,分别建模为两项全变分正则项。为了使深度模型的重建结果尽可能与预估的高分辨率图像相似,将深度网络重建约束作为第三项正则项。以联合统计模型为正则化约束条件,引导深度模型重建,并用可收敛的分裂的Bregman迭代算法优化全变分问题。在四个基准数据集上的定量和定性结果表明,所提算法比原深度模型有更高的峰值信噪比。与已有研究工作相比,该方法建立了联合统计模型和深度超分模型的统一框架,实现了传统超分算法与深度超分模型的联合求解。2.针对高倍率如4倍、8倍的图像边缘重建时存在严重的锯齿现象,提出一个基于残差记忆网络的超分辨率重建模型。将总变分损失和多尺度结构相似性损失结合为新的联合损失函数作为正则化约束条件,约束超分模型产生连续的边缘。用残差记忆模块作为基础模块,通过门限机制自主选择特征。所提方法使用更少的参数,在放大4倍、8倍上取得了良好的客观指标。与第一个研究内容不同,该方法重点研究用模型损失来约束深度超分模型。3.针对深度超分模型未能充分考虑各层特征之间的上下文关联信息,提出一个基于非局部多尺度融合的单帧图像超分辨率重建模型。该模型用宽激活残差模块扩宽激活层前的特征,用多尺度融合模块在多个尺度上融合特征,用非局部网络模块获取图像的全局特征,关注目标的核心区域,并用模型得到的上下文特征作为该模型的正则化约束项。在五个基准数据集上,所提算法取得了最好的定量和定性结果。与第二个研究内容相比,该方法是具有更深网络层数和更高重建性能的超分模型。在视频超分辨率重建、图像分割、目标检测任务上的实验对比结果显示,所提算法相比其它最新超分模型有一定的优越性。与已有研究工作相比,该方法用深度超分模型的上下文特征约束模型重建。4.针对面向视觉感知的图像重建未能平衡视觉感知和客观质量的问题,提出了一个联合注意判别器。在原判别器上使用密集通道注意力和跨层注意力,提升原判别器的判别性,将联合注意判别器作为深度超分模型的正则化约束条件,指导生成器产生具有更高客观质量的重建结果。与多种生成器结合后的实验结果表明,所提联合注意判别器比原判别器能更有效提升重建的客观指标和主观视觉效果。前三个方法的研究内容是以PSNR为导向的超分算法或模型,与前三个方法不同,该方法主要研究以视觉感知为导向的深度超分模型。与已有研究工作相比,该方法考虑了判别器对深度超分模型中的生成器的作用。更多还原

【Abstract】 Single image super-resolution reconstruction technology uses image processing and machine learning theory to generate a high-resolution image with good objective quality and visual effects from a given low-resolution image.The super-resolution images can be widely used in medical image processing,remote sensing image processing,public safety monitoring,digital entertainment,multimedia communication and other fields,and single image super-resolution has attracted widespread attention.The vigorous development of the image super-resolution model based on deep learning(the deep super-resolution model)in recent years has achieved good results which greatly surpasses the traditional image super-resolution algorithms,and has become a research hotspot of computer vision.Most of the existing deep super-resolution models focus on the reconstruction of the fidelity term of the model and ignoring the regularization constraint analysis of the deep super-resolution models.Aiming at this problem,this dissertation studies regularization for deep super-resolution models,and uses statistical priors in the image,loss of the superresolution model,the context features,and discriminator as regularization constraints to improve the performance of the model.Four new image super-resolution reconstruction algorithms or models are proposed.The research contents and main innovations of this dissertation are as follows:1.Aiming at the existing deep super-resolution model without considering the statistical priors in the input image,a joint statistical model-guided super-resolution algorithm for the deep network is proposed.The joint statistical model uses two internal statistical priors of the image:non-local similarity and local smoothness,they are described as two total variations regular term,respectively.In order to make the reconstruction result of the deep model as similar as possible to the estimated high-resolution image,the deep network reconstruction constraint is taken as the third regular term.The joint statistical model is used as a regularization constraint to guide the reconstruction of the deep model,and a convergent separated Bregman iterative algorithm is used to optimize the total variation problem.The quantitative and qualitative results on the four benchmark datasets show that the proposed algorithm has a higher peak signal-to-noise ratio than the original deep model.Compared with the existing research work,this method establishes a unified framework of the joint statistical model and deep super-resolution model,and realizes the joint solution of the traditional super-resolution algorithm and deep super-resolution model.2.Aiming at the problem of sawtooth existed in the high-magnification,such as 4x and 8x image edge reconstruction,a super-resolution reconstruction model based on residual memory network is proposed.Combine the total variation loss and the multi-scale structural similarity loss into a new joint loss function as a regularization constraint.The constraint super-resolution model generates continuous edges.The residual memory module is used as the basic module,and features are automatically selected through the threshold mechanism.The proposed method uses fewer parameters,and also achieves good objective indicators at 4x and 8x magnification.Different from the first research content,this method focuses on using the model loss to constrain the depth super-resolution model.3.Aiming at the deep super-resolution model failing to fully consider the contextual information between the features of each layer,a single image super-resolution model based on non-local multi-scale fusion is proposed.The model uses a wide activation residual module to widen the features before the activation layer,a multi-scale fusion module to fuse features on multiple scales,and a non-local network module to obtain the global features of the image,focusing on the core area of the target,and the obtained context features are used as regularization constraints of the model.The proposed model achieved the best quantitative and qualitative results on the five benchmark datasets.Compared with the second research content,this method is a super-resolution model with deeper network layers and higher reconstruction performance.Experimental comparison results on video super-resolution reconstruction,image segmentation,and target detection tasks show that the proposed model has certain advantages over other latest super-resolution models.Compared with the existing research work,this method uses the context feature constraint model reconstruction of the deep super-resolution model.4.Aiming at the problem that visual perception-oriented image reconstruction fails to balance visual perception and objective quality,a joint attention discriminator is proposed.We use dense channel attention and cross-layer attention on the original discriminator,to improve the discriminativeness of the original discriminator,and use the joint attention discriminator as a regularization constraint for the deep super-resolution model to guide the generator to produce higher objective quality reconstruction results.The experimental results combined with several generators show that the proposed joint attention discriminator can improve the reconstructional objective indicators and subjective visual effects more effectively than the original discriminator.The research contents of the first three methods are PSNR-oriented super-resolution algorithms or models.Unlike these methods,this method mainly studies the visual perception-oriented depth super-resolution model.Compared with the existing research work,this method takes into account the role of the discriminator on the generator in the deep super-resolution model.更多还原