利用隐扩散模型的光学遥感影像去云方法

【作者】 胡浩；

【导师】 李佳田；

【作者基本信息】 昆明理工大学 ， 摄影测量与遥感， 2024， 硕士

【摘要】 针对云层遮盖会导致影像中的地表信息缺失问题,本文提出一种利用隐扩散模型的光学遥感影像去云方法,利用隐扩散模型强大的生成能力实现云层的自动去除。在此基础之上,利用DPM-Solver++搭建隐扩散模型,以轴向深度卷积模块和Shift MLP为核心构建轻量级图像分割网络,并将其用作隐扩散模型的噪声估计网络,以降低模型的计算负担并加快采样速度。主要工作内容如下:(1)提出一种利用隐扩散模型的光学遥感影像去云方法(LDMCR),基本过程为:首先、利用感知压缩模型在无云遥感影像上学习得到一个隐空间,建立与原像素空间等效的隐空间感知。然后,在隐空间中加入有云影像作为条件,引导扩散模型生成无云影像,并利用本文提出的类U-Net互协方差自注意力噪声估计网络(NE-UTViT)进行噪声估计,NEUTViT包括跳跃连接、互协方差注意力机制和门控线性单元,可有效地利用低级特征,并显著降低计算负担,提高网络非线性表征能力,实现更准确的噪声估计。最后、在前向过程中引入相似结构约束损失,缓解模型生成的随机性,指导模型生成更接近源影像的无云影像。(2)为验证本文提出的NE-UTViT网络各组成模块和联合损失函数的有效性,在SEN12MS-CR Winter dataset上进行消融实验,将XCA、SA与GLUFFN、MLP进行组合,得到三种不同的NE-UTViT变体,对比不同组合对去云效果的影响。实验表明互协方差注意力和门控线性单元对降低网络计算复杂度、增强模型去云能力是有效的。探索感知压缩模型的不同下采样因子对隐空间构建与去云效果的影响,此外、对比单一L2损失,联合损失可以增强模型的全局结构恢复能力,提高模型生成的去云影像质量。(3)以DMP-Solver++为基础构建轻量级隐扩散去云模型(Light-LDMCR),提高模型的采样的速度。遵循对称编码器-解码器架构并结合轴向深度卷积模块和Shift MLP模型,提出一种轻量级图像分割网络AMLU-net,并将其用作隐扩散模型中的噪声估计网络。AMLUnet网络在编码阶段前期采用深度轴向卷积以扩大模型的感受野,同时减少网络参数量。网络编码阶段后期采用Shift MLP以专注于局部偏置并减少计算复杂度。在网络设计中考虑模型维度对参数量和计算量的影响,采用更少的参数设计,网络中的每个阶段通道数对比Unet大幅降低。(4)本文的图像分割网络在ISIC2018 Dataset测试集上的F1分数、IoU指数分别达到89.78、81.76。相较Unet、Unet++、ResUNet、MedT、Trans Unet,F1分数分别提升6.8%、5.7%、4.9%、2.8%、0.9%,IoU指数分别提升9.6%、8.8%、8.1%、2.7%、1.5%。目前最优模型Unext的参数量为1.47M,Gflops为0.57。与Unext相比,本文网络具有相近的图像分割性能,但是参数量更少,只需1.43M,而Gflops仅为0.34,相较Unext减少40.3%,推理速度为20ms,相较Unext提高20%。通过对三种不同配置的模型进行实验,综合考虑模型性能、参数量、模型复杂度,M型通道配置(16、32、128、160、256)为最佳选择。此外,还通过实验验证了三种不同批量大小和轴向深度卷积核大小对模型性能的影响。最后,在两个去云数据集上的实验结果表明,本文提出的Light-LDMCR相比LDMCR在模型参数量、采样速度、Gflops均有大幅降低的情况下,仍具有较好的去云能力,平均SSIM和平均PSNR分别达到0.750、26.127,0.781、28.158。更多还原

【Abstract】 Aiming at the problem that cloud cover will lead to the lack of surface information in the image,this thesis proposes a cloud removal method for optical remote sensing images using the hidden diffusion model,which uses the powerful generation ability of the hidden diffusion model to realize the automatic removal of clouds.On this basis,the DPM-Solver + + is used to build the hidden diffusion model,and the lightweight image segmentation network is constructed with the axial deep convolution module and the Shift MLP core,which is used as the noise estimation network of the hidden diffusion model to reduce the computational burden of the model and accelerate the sampling speed.The main work is as follows :(1)An optical remote sensing image cloud removal method based on latent diffusion model(LDMCR)is proposed.The basic process is as follows:(1)The perceptual compression model is used to learn a hidden space on the cloudless remote sensing image,and the hidden space perception equivalent to the original pixel space is established.(2)In the hidden space,cloud images are added as conditions to guide the diffusion model to generate cloudless images,and the U-Net-like cross-covariance self-attention noise estimation network(NE-UTViT)proposed in this thesis is used for noise estimation.NE-UTViT includes skip connection,cross-covariance attention mechanism and gated linear unit,which can effectively utilize low-level features,significantly reduce the computational burden,improve the nonlinear representation ability of the network,and achieve more accurate noise estimation.(3)Similar structure constraint loss is introduced in the forward process to alleviate the randomness of model generation and guide the model to generate cloudless images closer to the source image.(2)In order to verify the effectiveness of each component module and joint loss function of the NE-UTViT network proposed in this thesis,ablation experiments were performed on the SEN12MS-CR Winter dataset.XCA and SA were combined with GLUFFN and MLP to obtain three different NE-UTViT variants,and the effects of different combinations on cloud removal were compared.Experiments show that cross-covariance attention and gated linear units are effective in reducing network computational complexity and enhancing the cloud removal ability of the model.Explore the image of the different downsampling factors of the perceptual compression model for the construction of the hidden space and the effect of cloud removal.In addition,compared with the single L2 loss,the joint loss can enhance the global structure recovery ability of the model and improve the quality of the cloud removal image generated by the model.(3)Based on DMP-Solver + +,a lightweight latent diffusion cloud removal model(LightLDMCR)is constructed to improve the sampling speed of the model.Following the symmetric encoder-decoder architecture and combining the axial deep convolution module and the Shift MLP model,a lightweight image segmentation network AMLU-net is proposed and used as a noise estimation network in the hidden diffusion model.The AMLU-net network uses deep axial convolution in the early stage of the coding stage to expand the receptive field of the model and reduce the number of network parameters.In the later stage of network coding,Shift MLP is used to focus on local offset and reduce computational complexity.In the network design,the influence of model dimension on parameter quantity and calculation amount is considered,and less parameter design is adopted.The number of channels in each stage of the network is greatly reduced compared with Unet.(4)The F1 score and IoU of the image segmentation network in this thesis on the ISIC 2018 Dataset test set reached 89.61 and 81.80,respectively.Compared with Unet,Unet + +,ResUNet,MedT and Trans Unet,the F1 score increased by 6.8 %,5.7 %,4.9 %,2.8 % and 0.9 % respectively,and the IoU index increased by 9.6 %,8.8 %,8.1 %,2.7 % and 1.5 % respectively.At present,the parameter of the optimal model Unext is 1.47 M,and the Gflops is 0.57.Compared with Unext,the network in this thesis has similar image segmentation performance,but the number of parameters is less,only 1.43 M,Gflops is only 0.34,which is 40.3 % lower than Unext,and the inference speed is 20 ms,which is 20 % higher than Unext.Through experiments on three different configuration models,considering the model performance,parameter quantity and model complexity,M-type channel configuration(16,32,128,160,256)is the best choice.In addition,the effects of three different batch sizes and axial depth convolution kernel sizes on the performance of the model are verified by experiments.Finally,the experimental results on two cloud removal datasets show that the Light-LDMCR proposed in this thesis still has better cloud removal ability than LDMCR when the number of model parameters,sampling speed and Gflops are greatly reduced.The average SSIM and average PSNR are 0.750,26.127,0.781 and 28.158,respectively.更多还原