【作者】 王斌；

【导师】 梁宇栋；

【作者基本信息】 山西大学 ， 计算机科学与技术， 2023， 硕士

【摘要】 图像增强技术是指一些改善图像质量,提高图像可视性和可识别性的技术。图像增强技术在航空、摄影、电子商务等诸多领域得到了广泛应用。其中,基于深度学习的图像增强方法是近年来的研究热点,该类方法利用神经网络模型学习从低质量图像到高质量图像的映射关系,能够完成多种复杂的图像增强任务,如图像去雾、图像去雨与低光增强等任务。但是基于深度学习的图像增强算法存在诸多缺点,例如对合成数据集过度依赖,对真实场景的泛化性与域迁移性差等一系列问题。本文针对基于深度学习的图像增强算法遇到的以上问题展开了相关研究,本文首先提出了一个基于亮度调节的无监督图像增强算法用于图像去雾与低光增强任务,有效缓解了算法对监督数据的依赖性高与算法泛化性差的问题。而恶劣天气下低质图像的亮度信息与图像的深度信息紧密相关,即图像的深度信息与光线散射、光线反射与光照不足等现象存在内在关联。本文基于图像深度信息使用自监督学习技术进一步克服图像去雾任务对监督数据的依赖,并提高了特征表征能力;使用动态深度学习模型技术,构建了层级动态网络模型,有效提升了图像增强算法的自适应性。具体地,本文主要研究内容如下:(1)在低质图像降质问题中,亮度偏离如图像偏亮及偏暗是较为常见的图像降质现象。基于全监督学习的图像增强方法面临训练数据难以获取或获取成本过高,训练数据和应用场景不一致的困境。本文针对以上难点,提出一种能够克服数据依赖与基于亮度自适应的无监督图像增强方法。方法针对图像去雾与低光增强任务,设计了一个采用通道与像素注意力的深度卷积网络,对低质图像进行无监督修复。训练中采用了亮度饱和度、空间一致性、照明平滑度与伪标签监督等多种无监督损失函数,在保证增强图像与输入图像一致性的同时,调节图像的亮度偏离程度,进而提高图像质量。实验结果表明本方法量化指标与视觉效果不仅优于传统的图像增强方法与基于无监督学习的图像增强方法,甚至优于近年的全监督图像增强方法。本文所提出的无监督图像去雾与低光增强算法,能够有效调节图像的亮度偏离程度,重建亮度正常、细节清晰与对比度较好的增强复原图像,较为有效地克服了当前底层视觉任务数据难以获取的难题,提升了算法在实际应用中的适用性。(2)现有的基于深度学习的图像去雾方法通常依赖于有监督学习,需要大量的雾霾-清晰图像对,这些图像对的收集难度大或者成本昂贵。当训练的雾霾-清晰图像对不足或训练使用的雾霾图像与实际场景中的雾霾图像不同时,学习到的模型去雾性能可能会显著下降。本文针对如何克服监督数据依赖和如何提高对真实雾霾图像的特征表示能力等两个关键问题,提出了两阶段的基于自监督学习的图像去雾与模型自适应算法,首先基于预训练的深度估计网络利用传统的大气光散射模型生成大量的雾霾-清晰图像对,用于去雾网络的自监督预训练;其次针对真实雾霾图像进行自监督的模型自适应,即将上阶段预训练好的网络针对真实雾霾图像进行模型自适应。在自监督适应学习过程中,采用了不遗忘学习策略与对比学习来兼顾算法针对合成数据和真实数据的性能。实验结果表明,本文提出的方法与最先进方法相比表现出色,并且模型计算非常高效,处理一张4K图像只需23毫秒。(3)在各种真实场景的图像增强应用中,图像的退化往往是非均匀或非齐次的,且具有多样性,这给推理阶段具有固定参数的深度网络带来了挑战,而动态深度网络可以根据输入调整模型结构或参数。受动态深度网络的启发,本文提出了一种基于深度信息引导的分层动态机制用于图像增强,从局部到全局调整模型参数和自适应学习特征,并有效保持动态区域的空间邻接性。具体地,方法依次应用了通道-空间级、结构级和区域级动态组件。通道-空间级组件采用通道注意力和空间注意力关注特征的不同通道和空间位置间的差异;结构级组件应用可变形卷积通过增加采样位置对几何变换进行建模,更好地描述变化的局部特征。本文利用了暗通道先验(DCP,Dark Channel Prior)进行深度信息引导,提出了一种新颖的区域级动态的区域划分方法,用于为动态特征生成空间连续的掩码,进而有效的保持了图像的局部一致性。所提出的方法在多个图像增强任务即图像去雾、图像去雨和低光图像增强上达到了最先进的性能,并生成了视觉上令人满意的图像。更多还原

【Abstract】 Image enhancement refers to a certain number of techniques that improve image quality and enhance image visibility and recognizability.Image enhancement techniques have been widely used in many fields such as aviation,photography,e-commerce and so on.Among them,deep learning-based image enhancement methods are a hot research topic in recent years.These methods use neural network models to learn the mapping relationship from low-quality images to high-quality images,and can accomplish a variety of complex image enhancement tasks,such as image dehazing,image deraining and low-light enhancement.However,deep learning-based image enhancement algorithms have many drawbacks,such as over-reliance on synthetic datasets,poor generalization to real scenes and poor domain migration,and a series of other problems.This paper researches the above problems encountered by deep learning-based image enhancement algorithms.Firstly,we propose an unsupervised image enhancement algorithm based on luminance adjustment for image dehazing and low-light enhancement tasks,which effectively alleviates the problems of high dependence on supervised data and poor generalization of the algorithm.The luminance information of low-quality images under bad weather is closely related to the depth information of images,i.e.,the depth information of images is intrinsically related to the phenomena of light scattering,light reflection and insufficient illumination.In this paper,we use self-supervised learning techniques to further overcome the dependence on supervised data and improve the feature characterization capability of image dehazing tasks based on image depth information;we use dynamic deep learning model techniques to build a hierarchical dynamic network model,which effectively improves the self-adaptability of image enhancement algorithms.Specifically,the main research of this paper is as follows:(1)In the low-quality image degradation problem,luminance shift such as image brightness and darkness is a more common image degradation phenomenon.Fully supervised learning-based image enhancement methods face the dilemma that training data are difficult to obtain or too expensive to obtain,and the training data and application scenarios are not consistent.In this paper,we propose an unsupervised image enhancement method that can overcome data dependency and luminance-based adaptation to address the above difficulties.The method designs a deep convolutional network using channel and pixel attention for image dehazing and low-light enhancement tasks and performs unsupervised restoration of low-quality images.Various unsupervised loss functions such as luminance saturation,spatial consistency,illumination smoothing and pseudo-labeling supervision are used in the training to adjust the degree of luminance deviation of the image while ensuring the consistency of the enhanced image with the input image,and thus improving the image quality.The experimental results show that the quantitative index and visual results of this method are not only better than traditional image enhancement methods and unsupervised learning-based image enhancement methods,but even better than recent fully supervised image enhancement methods.The unsupervised image dehazing and low-light enhancement algorithm proposed in this paper can effectively adjust the luminance deviation degree of the image and reconstruct the enhanced recovered image with normal luminance,clear details and good contrast,which can overcome the difficulty of obtaining data for the current low-level vision task more effectively and improve the applicability of the algorithm in practical applications.(2)Existing deep learning-based image dehazing methods usually rely on supervised learning,which requires a large number of haze-clear image pairs that are difficult or expensive to collect.When the trained haze-clear image pairs are insufficient or the haze images used for training are different from the haze images in real scenes,the performance of the learned model for dehazing may be significantly degraded.In this paper,a two-stage self-supervised learning-based image dehazing and model adaptation algorithm is proposed to address two key issues,including how to overcome the supervised data dependence and how to improve the feature representation capability of real haze images.Second,self-supervised adaptation of the model is performed for real haze images,i.e.,the pre-trained network is adapted for real haze images in the previous stage.In the self-supervised adaptation process,a no-forgotten learning strategy and contrast learning are used to balance the performance of the algorithm for synthetic and real data.Experimental results show that the proposed method performs well compared with state-of-the-art methods and the model computation is very efficient,taking only 23 ms to process a 4K image.(3)In various image enhancement applications for real scenes,the degradation of images is often non-uniform or non-simultaneous and has diversity,which poses a challenge to deep networks with fixed parameters in the inference stage,while dynamic deep networks can adjust the model structure or parameters according to the input.Inspired by dynamic deep networks,this paper proposes a hierarchical dynamic mechanism based on depth information guided for image enhancement,adjusting model parameters and adaptive learning features from local to global,and effectively maintaining spatial adjacency of dynamic regions.Specifically,the method applies channel-space-level,structure-level,and region-level dynamic components sequentially.The channel-space-level component uses channel attention and spatial attention to focus on the differences between different channels and spatial locations of features;the structure-level component applies deformable convolution to model geometric transformations by increasing sampling locations to better describe changing local features.This paper utilizes the dark channel prior(DCP)for depth information guidance and proposes a novel region-level dynamic region-division method for generating spatially continuous masks for dynamic features,which in turn effectively maintains the local consistency of the image.The proposed method achieves state-of-the-art performance on several image enhancement tasks,including image dehazing,image deraining and low-light image enhancement,and generates visually satisfactory images.更多还原