【作者】 刘阳；

【导师】 苏志勋；

【作者基本信息】 大连理工大学 ， 计算数学， 2021， 博士

【摘要】 近些年来,包括智能手机在内的许多成像设备已经普及人们的工作和生活,并沿伸出丰富的实际应用。受图像捕获过程中的复杂成像环境、图像存储与显示条件等因素的影响,获得的图像信息会退化。例如,有雾的图像,有雨的图像,低分辨率的图像或者破损的图像等等。图像复原的主要目标通过合适的算法,从退化图像复原相应的高质量图像。图像复原涵盖计算机视觉和图像处理中许多经典问题,这些问题在过去的几十年中引起了很多研究与关注。本文主要研究单幅图像去雾,图像去雨,图像超分辨率和图像修复问题。基于端到端的深度学习技术的方法极大地提高了图像复原的效果,这些方法主要依靠有效的网络结构从海量数据中来学习图像特征,从而建立退化图像与真值图像之间的映射关系。然而端到端的方式使得学习过程类似于黑箱,仅知道输入和输出图像的数据分布,却难以解释和刻画算法的中间过程。因此,解空间缺乏约束,不利于收敛至最优解,并且不易于指导进一步的网络结构的优化方向。图像复原问题是众所周知的不适定病态问题,为了使该问题适定,需要施加有效的约束。本学位论文依据图像的成像过程,建立图像退化物理模型与深度神经网络之间的内在联系。提出的方法利用物理模型引导神经网络的学习过程,帮助约束解空间,提高图像复原算法的效率和效果,并提供相应的可解释性。本论文的主要研究工作和贡献如下:1.本文提出了一种有效的图像去雾算法。基于图像去雾问题的退化物理模型,通过变量分离的形式,用梯度下降法迭代优化嵌入深度神经网络的变分模型并引导学习环境大气光、透射图和无雾图像相关的先验知识。这些先验知识结合受到退化物理模型约束的数据项,在每次迭代过程不断改善并生成准确的环境大气光、透射图和高质量的无雾图像。本文的方法结合了图像去雾的物理成像性质以及深度学习方法的特性。大量的实验结果表明,所提出的算法在基准数据集和真实世界图像中均优于最新方法。2.根据有雨图像的退化物理模型中雨图的特性,提出了一种基于雨方向约束深度神经网络的简单而有效的非成对数据学习的图像去雨方法。该算法主要由半监督学习部分和知识蒸馏部分组成。半监督部分根据已有的层分离原则估计雨图并重建无雨图像。雨方向约束被提出帮助学习雨图。结合有雨图像的退化物理模型合成了新的成对的真实场景数据,从而将非成对的无监督问题转化为成对的有监督问题,同时仅需要输入两个非成对的真实有雨图像集和无雨图像集,因此方法的整个学习过程是非成对学习的。本文提出了两个新的去雨数据集,称为Rain Direction和Real3000,以帮助验证所提出方法的有效性。定性和定量实验结果均表明,在基准数据集和真实世界图像上与最新方法相比,该方法具有良好的性能。3.本文提出了一种简单有效的图像超分辨率算法,该算法通过推导低分辨率图像的退化物理模型,提出用像素替换对深度神经网络施加图像成像约束。提出的算法首先使用深度神经网络来估计中间高分辨率图像,利用已知的模糊核对中间图像进行模糊处理,然后将未抽取位置处的像素值替换为低分辨率图像中相应像素的值。像素替换过程的输出严格满足图像成像模型,并通过相同的深度神经网络以级联方式进一步完善。所提出的框架以端到端的方式与现有的前馈深度神经网络联合训练,以实现超分辨率并在实践中快速收敛。大量的实验结果表明,所提出的算法与最新方法相比具有良好的效果。4.与许多方法假定损坏区域的位置已知不同,提出了一种有效的盲图像修复算法,可以从损坏的输入中直接恢复清晰的图像,而不需要输入蒙板图像。通过建立图像修复问题的退化物理模型与残差学习策略的内在联系,所提出的方法可以自动修复损坏的区域,并保持未损坏区域信息的真实性。同时引入了一种编码器和解码器网络结构,以挖掘破损区域周围多尺度的图像邻域特征,帮助网络重建损坏区域的图像信息。定性和定量实验都证明本文的算法可以处理多种形状区域的损坏,和最先进的方法相比,取得了良好的效果。更多还原

【Abstract】 In recent years,many imaging devices including smart phones have gone deeply into the work and life of people,and have extended a wide range of practical applications.In the process of capturing an image,due to the complicated imaging environment,image storage and display conditions,the actual image information is usually lost and degraded.For example,hazy im-ages,rainy images,low-resolution images,corrupted images,and etc.The main goal of image restoration is to restore corresponding high-quality images from degraded images via appropriate algorithms.Image restoration involves many classical problems of computer vision and image process-ing,which have attracted much research attention over the decades.This thesis mainly focuses on single image dehazing,image deraining,image super-resolution and image inpainting prob-lems.Recent methods based on end-to-end deep learning techniques have greatly improved the efficiency and effectiveness of image restoration.These methods mainly rely on effective net-work structures and a large amount of image data to learn image features so that the mapping relationship between degraded images and clear images can be achieved.However,the end-to-end learning approach makes these methods like a black box,only knowing the input and output data,and it is difficult to explain which step of the algorithm works.Therefore,the solution space lacks constraints,making it less effective to convergence to the optimal solution,and it is not easy to guide further optimization direction of the network structures.Image restoration problem is a well-known ill-posed problem.To make the problem conditioned,effective constraints need to be imposed.This thesis establishes the internal connection between the physical model and the deep neural network based on the imaging process.The proposed methods use the physical model to guide the learning process of the neural networks,help constrain the solution space,improve the efficiency and effect of the image restoration algorithm,and provide corresponding interpretability.The main research work and contributions of this thesis are as follows:1.We propose an effective image dehazing algorithm.Based on the degraded physical model of the image dehazing problem,the gradient descent method is used to iteratively optimize the variational model in a variable separation manner,whitch guides the embedded deep neural networks to learn deep prior knowledge of atmospheric light,transmission maps and haze-free images.These prior knowledge are combined with data items constrained by the degraded phys-ical model to continuously improve and generate accurate atmospheric light,transmission maps and high-quality haze-free images in each iteration.The method combines the physical imaging properties of image dehazing and the characteristics of deep learning methods.Extensive exper-imental results show that the proposed algorithm peforms favorably againt the state-of-the-art methods in both benchmark data sets and real-world images.2.We present a simple yet effective unpaired learning-based image rain removal based on deep neural networks by exploring the properties of rain maps.The proposed algorithm mainly consists of a semi-supervised learning part and a knowledge distillation part.The semi-supervised part estimates the rain map and reconstructs the derained image based on the well-established layer separation principle.To facilitate rain removal,we develop a rain direction regularizer to constrain the rain estimation network in the semi-supervised learning part.With the estimated rain maps from the semi-supervised learning part,we synthesize pseudo paired data from clear images based on the superimposition model and develop an effective knowledge distillation method to explore the useful information from the synthesized data for better rain removal.We propose two new rainy datasets,named Rain Direction and Real3000,to validate the effectiveness of the proposed method.Both quantitative and qualitative experimental results demonstrate that the proposed method achieves favorable performance against state-of-the-art methods in benchmark datasets and real-world images.3.We present a simple yet effective image super-resolution algorithm that imposes an im-age formation constraint on the deep neural networks via pixel substitution.The proposed algo-rithm first uses a deep neural network to estimate intermediate high-resolution images,blurs the intermediate images using known blur kernels,and then substitutes values of the pixels at the undecimated positions with those of the corresponding pixels from the low-resolution images.The output of the pixel substitution process strictly satisfies the image formation model and is further refined by the same deep neural network in a cascaded manner.The proposed framework is trained in an end-to-end fashion and can work with existing feed-forward deep neural networks for super-resolution and converges fast in practice.Extensive experimental results show that the proposed algorithm performs favorably against state-of-the-art methods.4.Different from existing methods that usually make some assumptions on the corrupted regions,we present an efficient blind image inpainting algorithm to directly restore a clear im-age from a corrupted input and do not need mask information.The proposed algorithm could restore corrupted information and preserve uncorrupted information by establishing the connec-tion between the degraded physical model and residual learning strategy.By introducing an encoder-decoder network architecture,neighboring multi-scale features are extracted to help re-construct the image information in the corrupted regions.Our algorithm can predict the missing information in the corrupted regions,thus facilitating the clear image restoration.Both quali-tative and quantitative experimental demonstrate that our algorithm can deal with the corrupted regions of arbitrary shapes and performs favorably against state-of-the-art methods.更多还原