【作者】 张龙；

【导师】 万毅；

【作者基本信息】 兰州大学 ， 物理学·无线电物理， 2025， 博士

【摘要】 单图像超分辨率技术是计算机视觉技术发展的重要分支,其在医学影像、安防监控和娱乐设备等多个现实场景均具有一定的研究和应用价值。随着深度学习技术的快速发展,基于神经网络的超分辨率方法吸引了广大研究者们的关注并取得显著进展。相比于传统的超分辨率方法,基于神经网络的超分辨率方法在特征表达方面具有独特的优势,因而通常具有更优的性能和更强的泛化能力。然而,在探索高质量图像超分方法的过程中,网络深层化所带来的执行效率等问题对其在实际应用中的部署带来严重阻碍。为此,轻量级超分辨率模型的设计逐渐成为超分领域的研究热点。本文工作主要围绕卷积神经网络(CNNs)和Transformer两类主流神经网络的轻量化展开,所提出的模型在图像恢复质量和执行效率上较现有技术均取得明显的性能提升。具体而言,本文的主要贡献和创新点总结如下:(1)本文提出一种基于部分卷积残差网络(PCRN)的轻量级超分辨率(SR)模型,PCRN的核心是通过使用部分卷积(PConv)代替标准卷积的方式大幅度降低模型推理过程中广泛存在的冗余卷积操作,从而减少不必要的模型参数,以达到降低模型推理复杂度的目的。此外,通过对Pixel Shuffle函数的理论分析及实验验证,我们认为Pixel Shuffle操作会导致特征在空间和通道维度发生混叠使得原卷积核的感受野分散,从而破坏特征的空间结构信息并影响到局部信息的提取。因此,我们在基础残差模块中引入了逐点卷积(PWConv)以抑制这一不利现象。实验结果表明,PCRN在同等量级的超分辨率模型中表现优异,并在多个公开数据集上取得良好的恢复效果。(2)为在保持图像恢复效果的同时进一步降低PCRN的计算复杂度,本文引入重参数化技术(Reparameterization)对PCRN的结构进行优化并提出部分卷积重参数化网络(PCRep N)。首先,我们对PCRN中的部分卷积残差模块(PCRB)进行调整以便融合卷积层和批归一化层(Batch Normalization)。其次,我们在优化后的PCRB模块中引入多分支结构以扩展模型的特征提取能力,并基于此提出部分卷积重参数化模块(PCRep B)。值得注意的是,PCRep B中的多分支结构将在推理阶段借助重参数化技术融合为一个单一卷积层。此外,我们进一步优化了PCRN中的注意力模块,通过结合轻量化的空间注意力模块ESA和通道注意力模块ECA进一步提升PCRep N的性能。实验结果表明,PCRep N在保持原PCRN性能的基础上显著降低模型复杂度,展现出其在性能和效率上的双重优势。(3)在众多基于神经网络的超分辨率方法中,当前Transformer类型的网络在超分效果上的表现通常优于CNN类型。然而,在追求高超分辨率性能的同时往往需要堆叠大量基础模块来构建超分模型,这在一定程度上导致了模型参数量的急剧增加。为解决这一问题,本文设计了一种高效的多分支Token混合器模块(MBTM),并基于MBTM构建轻量级Transformer超分辨率网络MBMT。MBTM的设计理念是通过适当扩展网络宽度大幅度降低网络模块的堆叠,进而降低模型计算复杂度。MBTM包含三条分支:Self-Attention分支用于提取全局特征,CNN分支用于捕获局部特征,自适应Token混合器(ATM)分支用于扩展全局特征。实验结果表明,多分支的设计思路有效融合了全局与局部特征,并在高效单图像超分辨率任务中表现出优异的性能。(4)为进一步降低Transformer类型超分辨率网络的计算复杂度,本文提出一种基于Transformer与Mamba混合架构的轻量级超分辨率模型Tran Mamba。相较Transformer,Mamba架构的优势在于其核心模块状态空间模型(SSM)能够捕获特征长程依赖且只具有线性复杂度。为了兼具Transformer架构的性能优势和Mamba架构的效率优势,Tran Mamba采用了Transformer聚合模块(TAB)与Mamba聚合模块(MAB)交替堆叠的策略构建模型的主框架以保证模型的高图像恢复质量和低计算复杂度。此外,鉴于重参数化技术在PCRep N中的成功应用,本文设计了重参数化门控前馈网络(Rep SGFN)作为TAB与MAB中的前馈网络层以增强模型的非线性特征表达能力。实验结果表明,Tran Mamba在多个数据集上相较于其他轻量级超分辨率模型取得了当前已发表方法中最先进的性能。更多还原

【Abstract】 Single Image Super-Resolution(SISR)is an important branch of computer vision technology development,which has certain research and application value in various real-world scenarios such as medical imaging,security monitoring,and entertainment devices.With the rapid development of deep learning technology,SR methods based on neural networks have attracted the attention of researchers and made remarkable progress.Compared to traditional super-resolution methods,neural network-based SR methods have unique advantages in feature representation,and therefore typically have better performance and stronger generalization ability.However,in the process of ex-ploring high-quality image SR methods,issues such as execution efficiency caused by network deepening pose serious obstacles to their deployment in practical applica-tions.Therefore,the design of lightweight SR models has gradually become a research hotspot in the field of SR.This study mainly focuses on the lightweight development of two mainstream neural networks:Convolutional Neural Networks(CNNs)and Trans-formers.The proposed model achieves significant performance improvements in image restoration quality and execution efficiency compared to existing technologies.Specif-ically,the main contributions and innovations of this paper are summarized as follows:(1)This paper proposes a lightweight SR model based on Partial Convolutional Residual Network(PCRN).The core of PCRN is to significantly reduce the redun-dant convolution operations widely present in the model inference process by replacing standard convolutions with partial convolutions(PConv),thereby reducing unneces-sary model parameters and achieving the goal of lowering the complexity of model inference.In addition,based on further analysis and experiments on the Pixel Shuffle function,we have found that the Pixel Shuffle operation can cause feature aliasing in both spatial and channel dimensions,leading to the dispersion of the receptive field of the original convolution kernel,thereby disrupting the spatial structural information of the features and affecting the extraction of local information.Therefore,we introduce Pointwise Convolution(PWConv)in the basic residual module to suppress this adverse phenomenon.The experimental results show that PCRN performs well in SR models of comparable complexity and achieves good restoration results on multiple publicly available datasets.(2)In order to further reduce the computational complexity of PCRN while main-taining the image restoration effect,this paper introduces reparameterization technology to optimize the structure of PCRN and proposes a Partial Convolution Reparameteriza-tion Network(PCRep N).Firstly,we adjust the Partial Convolutional Residual Block(PCRB)in PCRN to fuse convolution layers and batch normalization(BN)layers.Sec-ondly,we introduce a multi-branch structure in the optimized PCRB module to enhance the model’s feature extraction capability,and based on this,we propose a Partial Con-volution Reparameterization Block(PCRep B).It is worth noting that the multi-branch structure in PCRep B will be fused into a single convolution layer during the inference phase using reparameterization techniques.In addition,we further optimize the at-tention module in PCRN by combining the lightweight spatial attention module ESA and channel attention module ECA to enhance the performance of PCRep N.The ex-perimental results show that PCRep N significantly reduces model complexity while maintaining the original PCRN performance,demonstrating its dual advantages in per-formance and efficiency.(3)Among numerous neural network-based SR methods,Transformer type net-works typically perform better than CNN type networks in terms of SR performance.However,while pursuing high resolution performance,it often requires stacking a large number of basic modules to construct SR models,which to some extent leads to a sharp increase in the number of model parameters.To address this issue,this paper proposes an efficient Multi-branch Token Mixer(MBTM)and constructs a lightweight Transformer-based SR network,MBMT,based on MBTM.The design concept of MBTM is to significantly reduce the stacking of network modules by appropriately expand-ing the network width,thereby reducing the computational complexity of the model.MBTM consists of three branches:the Self-Attention branch for extracting global fea-tures,the CNN branch for capturing local features,and the Adaptive Token Mixer(ATM)branch for expanding global features.The experimental results show that the multi-branch design approach effectively integrates global and local features,and ex-hibits excellent performance in efficient single image super-resolution tasks.(4)To further reduce the computational complexity of Transformer type SR net-works,this paper proposes a lightweight SR model Tran Mamba based on a hybrid ar-chitecture of Transformer and Mamba.Compared to Transformer,the advantage of Mamba architecture lies in its core module,State Space Model(SSM),which can cap-ture long-range dependencies of features and only has linear complexity.In order to combine the performance advantages of Transformer architecture and the efficiency ad-vantages of Mamba architecture,Tran Mamba adopts a strategy of alternately stacking Transformer Aggregation Modules(TAB)and Mamba Aggregation Modules(MAB)to construct the main framework of the model,ensuring high image restoration qual-ity and low computational complexity of the model.In addition,considering the suc-cessful application of reparameterization technology in PCRep N,this paper designs a Reparameterized Spatial-Gate Feed-Forward Network(Rep SGFN)as the feedforward network layer in TAB and MAB to enhance the nonlinear feature expression ability of the model.The experimental results indicate that Tran Mamba achieves state-of-the-art performance compared to other published lightweight SR models on multiple datasets.更多还原