【作者】 张凯；

【导师】 张磊； 左旺孟；

【作者基本信息】 哈尔滨工业大学 ， 计算机应用技术， 2019， 博士

【摘要】 随着便携式智能手机的普及,数字图像已成为人们感知、处理、分析和分享信息的重要载体。由于现实生活中复杂因素的影响,例如:成像系统的不完善、存储容量以及网络带宽的限制,数字图像在其形成、传输、存储过程中往往会引入噪声并且以较低分辨率的形式存在。图像去噪和图像超分辨率作为数字图像处理领域内经典而又十分活跃的底层视觉研究课题,其不仅能够改善图像的视觉感知质量,同时也可以提高后续高层语义分析的精确度。目前主流的图像去噪和超分辨率方法包括基于模型的方法和判别学习方法。一般来说,这两种方法都有其各自的优势以及缺点,例如:基于模型的方法可以灵活地处理各种不同的图像复原任务但是通常运行速度较慢并且需要设计复杂的图像先验来达到较好的性能;与此同时,以卷积神经网络为代表的判别学习方法虽然测试速度快以及由于端对端训练导致性能好,但是其应用范围受到特定退化模型的限制。针对上述问题,本文从函数回归、最大后验、优化算法的角度出发,设计了高效的去噪网络、提出了灵活的去噪和超分辨率网络、解决了复杂退化下的超分辨率重建。具体研究内容及创新点如下:(1)基于模型的去噪方法通常牺牲速度来提升性能,而目前的判别学习去噪方法虽然速度快但是性能差。受启发于近年来卷积神经网络在结构设计、学习算法和正则化方法方面取得的进展,研究了其在图像去噪方面的应用。首先,提出结合残差学习和批规范化来加速网络训练和提高去噪性能。其次,提出残差学习策略去除噪声图像中隐含的干净图像,并分析了其与基于最大后验推理引导的判别学习方法之间的联系。通过分析以及实验得出单个去噪网络模型可以处理更一般的图像去噪任务,如图像超分辨率和JPEG压缩图像复原。(2)现有的判别学习去噪方法大多学习单一模型处理特定噪声水平,并且不能灵活处理噪声水平空间不均匀的图像,因此限制了其实际应用价值。为此,提出了将噪声水平图也作为网络输入的快速、灵活去噪卷积神经网络。为了权衡去噪速度和性能,进一步提出在下采样的子图像进行去噪。相比较于现有的图像去噪方法,提出的深度去噪网络有如下性质:能够处理各种不同的噪声水平;可以指定噪声水平图来处理噪声水平空间不均匀的噪声;比BM3D有更快的CPU运算速度。(3)现有的卷积神经网络超分辨率方法主要假设低分辨率图像由高分辨率图像经过双三次降采样得到,因此当真实图像的退化过程不遵循该假设时,其超分辨率结果通常比较差。此外,现有的方法不能扩展到用单一模型解决多种不同的图像退化类型。为此,提出了维度拉伸策略使得单个卷积超分辨率网络能够将超分辨率退化过程的两个关键因素(即模糊核和噪声水平)作为网络输入。归因于此,提出的超分辨率网络模型可以处理多个甚至是退化空间不均匀的退化类型,为超分辨率实际应用提供了一种有效的解决方案。(4)基于模型的方法和判别学习方法都有其各自的优势以及缺点。最近的研究表明去噪先验结合变量分离技术可以嵌入到基于模型的方法中作为模块化的一部分来求解其它图像复原任务。因此,当去噪先验为深度卷积神经网络模型时,基于模型的方法将获得速度和性能优势。然而,结合快速判别去噪的图像复原方法研究仍然匮乏。为此训练了一组快速有效的深度去噪网络并结合半二次化算法用于求解其它图像复原问题,例如图像超分辨率。(5)目前主流的超分辨率方法主要基于双三次退化模型进行设计,如何改进现有方法处理任意模糊的低分辨图像仍待解决。与此同时,即插即用图像复原可以灵活地处理不同的图像复原任务。此方法的主要思想是将图像去噪先验嵌入其交替迭代优化算法,进而隐含地对图像先验进行建模。受此启发,提出了深度即插即用超分辨率网络来处理任意模糊的低分辨率图像。首先,设计了一种新的低分辨率图像退化模型,其优势是可以利用现有的盲去模糊方法估计模糊核。其次,为了优化新退化模型导出的能量函数,采用了变量分离优化算法求解得到相应的交替迭代解。该交替迭代解表明可以将基于双三次退化的超分辨率网络先验作为一个模块用于求解模糊图像超分辨率。更多还原

【Abstract】 With the popularity of portable smartphones,digital images have become an important carrier for people to perceive,process,analyze and share information.Due to the various factors in real life,such as the imaging system,the storage capacity and network bandwidth,digital images are often corrupted by noise and have low spatial resolution.Image denoising and single image super-resolution(SISR),as two classical and yet active low-level vision research topics in the field of digital image processing,can not only improve image visual perception quality,but also improve the accuracy of subsequent high-level semantic analysis.Generally,model-based methods and discriminative learning methods are the two dominant strategies for solving image denoising and image super-resolution.Typically,those two kinds of methods have their respective merits and drawbacks,e.g.,model-based methods are flexible for handling different image restoration problems but are usually time-consuming with sophisticated priors for the purpose of good performance;in the meanwhile,discriminative learning methods such as convolutional neural networks(CNN)have fast testing speed and usually have good performance due to end-to-end training but their application range is greatly restricted by certain degradation models.To address these issues,we in this dissertation will design an efficient denoising network,propose flexible denoising and super-resolution networks,and solve image super-resolution for complex degradation from the perspectives of function regression,maximum a posteriori and optimization algorithm.The main research contents and contributions are summarized as follows.(1)Model-based denoising methods generally achieve high performance at the sacrifice of computational efficiency.In the meanwhile,discriminative learning based denoising methods have shown promising results toward bridging the gap between computational efficiency and denoising quality,however,their performance are inherently restricted to the specified forms of prior.We take one step forward by investigating the construction of feed-forward denoising convolutional neural networks(Dn CNNs)to embrace the progress in very deep architecture,learning algorithm,and regularization method into image denoising.Specifically,residual learning and batch normalization are utilized to speed up the training process as well as boost the denoising performance.With the residual learningstrategy,Dn CNN implicitly removes the latent clean image in the hidden layers.We then further analyze the connection between Dn CNN and MAP inference guided discriminative learning method.Our analysis reveals and our experiments verify that it is possible to train a single Dn CNN model to tackle with several general image denoising tasks such as SISR and JPEG image deblocking.(2)Existing discriminative learning based denoising methods mostly learn a specific model for each noise level and lack flexibility to deal with spatially variant noise,which limits their applications in practical denoising.To address these issues,we present a fast and flexible denoising convolutional neural network,namely FFDNet,with a tunable noise level map as the input.The proposed FFDNet works on downsampled sub-images to speed up the inference.In contrast to the existing discriminative denoisers,FFDNet enjoys several desirable properties,including(i)the ability to handle a wide range of noise levels(i.e.,[0,75])effectively with a single network,(ii)the ability to remove spatially variant noise by specifying a non-uniform noise level map,and(iii)faster speed than benchmark BM3 D even on CPU without sacrificing denoising performance.(3)Existing CNN-based SISR methods mostly assume that a low-resolution image is bicubicly downsampled from a high-resolution image,thus inevitably giving rise to poor performance when the true degradation does not follow this assumption.Moreover,they lack scalability in learning a single model to deal with multiple degradations.To address these issues,we propose a dimensionality stretching strategy that enables a single convolutional super-resolution network to take two key factors of the SISR degradation process,i.e.,blur kernel and noise level,as input.Consequently,the proposed superresolver can handle multiple and even spatially variant degradations,which provides a highly effective solution to practical SISR applications.(4)While model-based methods and discriminative learning methods have their respective merits and drawbacks,recent works have revealed that,with the aid of variable splitting techniques,denoiser prior can be plugged in as a modular part of model-based optimization methods to solve other image restoration tasks.Such an integration induces considerable advantage when the denoiser is designed via CNN.However,the study of integration with fast deep denoiser prior is still lacking.To this end,we train a set of fast and effective CNN-based denoisers and integrate them into model-based optimization method with half quadratic splitting algorithm to solve other image restoration tasks such as SISR.(5)Existing CNN-based SISR methods are mainly designed for the widely-used bicu-bic degradation,there still remains the fundamental challenge for them to super-resolve low-resolution image with arbitrary blur kernels.In the meanwhile,plug-and-play image restoration has been recognized with high flexibility due to its modular structure for easy plug-in of denoiser priors.Inspired by this,we propose a principled formulation and framework by extending bicubic degradation based deep SISR with the help of plug-andplay framework to handle LR images with arbitrary blur kernels.Specifically,we design a new SISR degradation model so as to take advantage of existing blind deblurring methods for blur kernel estimation.To optimize the new degradation induced energy function,we then derive a plug-and-play algorithm via variable splitting technique,which allows us to plug any super-resolver prior as a modular part.更多还原