【作者】 何铮；

【导师】 杨晓敏；

【作者基本信息】 四川大学 ， 模式识别与智能系统， 2022， 硕士

【摘要】 单幅图像超分辨率隶属于低级计算机视觉任务,其目的在于将给定的低分辨率图像经过重建算法恢复出对应的高分辨率图像。目前,图像超分辨率技术已经被应用于许多实际任务之中,如安防监控,卫星遥感,医学成像等,因此如何求解出视觉质量好,拥有可靠细节的高分辨率图像一直都是该领域的热点研究问题。自从深度学习技术普及以来,卷积神经网络凭借其强大的特征表征能力被应用于各类计算机视觉任务之中。在图像超分辨率领域,基于卷积神经网络的方法也在各大基准数据集上取得了远超传统算法的重建性能。然而,一些图像超分辨率研究仅将重心集中在提高模型的重建准确度方面,所搭建的网络模型一般都有着庞大的参数量和计算量。对于这类方法,由于其并未在重建性能和模型复杂度之间取得良好的权衡,导致其在一些低算力设备和实时场景上的应用受限。因此,如何设计出轻量又高效的图像超分辨率算法也是目前该领域的研究热点之一。针对于现有的轻量级超分辨率算法,尽管其中多数能够在维持较低模型复杂度的情况下保证重建结果的性能,但是仍然有着可进行改进和优化的空间:(1)对于图像超分辨率任务来说,在不同场景中的物体常常有着不同的尺寸及纹理信息,如何增强对这些不同尺寸物体的感知及重建能力是十分重要的。因此,图像超分辨率模型应当尽可能多地关注到图像中的多尺度信息,从而提高模型的重建性能。而这些多尺度信息的获取往往依赖于模型拥有充分的感受野。然而,现有的大多数轻量级图像超分辨率模型在局部模块中只是采用单通路的结构,并不能有效地利用来自不同感受野的多尺度特征。尽管这些方法有着较少的参数和计算量,但是其特征表征能力和对图像中大尺寸物体的恢复能力却十分有限。(2)在图像的重建过程中,若是能够获取图像中各像素之间的依赖关系,最终的重建结果会得到有效的改善。虽然可以通过堆叠卷积层来获取这种像素间的依赖关系,但是这会给模型带来高额的参数负担和难以优化的问题。目前,非局部注意力机制的应用可以有效地获取图像中的像素级依赖关系而仅仅带来很少的参数量。然而,对于一般的非局部注意力模块来说,它对输入图像或特征尺寸的大小十分敏感,随着输入尺寸的增大,它将导致网络计算代价和显存消耗激增。显然,一般非局部注意力模块对于轻量级图像超分辨率任务是不适用的,其存在着较大的改进空间。(3)对于现有的轻量级超分辨率算法而言,其改进方面主要是设计出既高效又轻量的结构。虽然这种高效化结构的设计对改善重建性能的影响比较显著,然而深度学习领域中的一些模型压缩的方法在图像超分辨率领域应用非常有限。对于正常训练得到的轻量级神经网络,其实际上能够学习到提取更复杂特征的能力,而利用知识蒸馏的方法能够在不改变级其网络结构的前提下接近这一目标,故该类型的方法在轻量级超分辨率算法的应用上仍然有很大的探索空间。针对上述的研究背景,本文设计了如下的算法:(1)设计了一种基于串行-并行级联的轻量级多尺度特征整合网络,该网络由若干个基于并行级联机制的多尺度特征整合模块串行连接而成。多尺度特征整合模块内采用了多通路的结构,并且通过多通路间的并行级联操作,使其不仅获得了较大范围的感受野,并且整合了多尺度的感受野信息。具体来说,图像中的低级特征在该模块中能够实现由粗到细的特征提取。此外,通过主干网络中的串行级联机制,使得网络中的局部和全局信息能够得到充分的利用和交互,从而有利于特征在网络中的传递,进一步增强其特征表征能力。最终,大量的实验证明了该网络相比于同类型方法在参数量、计算量和重建性能方面取得了更好的权衡。(2)设计了一个轻量级混合非局部注意力模块,该模块通过将所设计的非局部空间注意力机制和通道注意力机制有效结合,使其能够高效地获取特征空间和通道维度像素级别的关联信息。并且为了在资源受限的情况下提高重建性能,还设计了渐进式学习策略,该策略通过合理分配网络的计算资源实现高效的特征提取,从而改善网络的超分辨率结果。另外,混合非局部模块相对于一般的非局部注意力模块,能够在不影响重建性能的同时能够极大减少计算代价和显存消耗,证明了本算法的高效性。最终,还通过实验验证了该模块的可移植性,并将其嵌入了其他轻量级超分辨率网络中。(3)设计了一种高效的自适应中间层特征蒸馏算法,并将该算法与经典的超分辨率模型结合来实现高效模型压缩的目的。相比于一般的中间层特征蒸馏算法,该算法能够利用教师模型所有的中间层特征,并自适应地对这些特征进行选择。最终,这些特征可以作为学生模型训练时的监督来改善重建性能。并且本算法针对该监督策略设计了一种基于自注意力机制的损失函数。此外,该算法还在学生模型的重建层使用教师的重建结果来进行额外的监督。最终的实验结果证明,该算法与经典图像超分辨率模型的结合不仅能够将其参数量大幅度压缩,而且其重建性能比正常训练的模型更加优越。更多还原

【Abstract】 Single-image super-resolution(SISR)belongs to the low-level computer vision task,which aims at recovering a low-resolution(LR)image to a corresponding high-resolution(HR)version.Currently,image super-resolution has been implemented to many applications,such as security surveillance,remote sensing,and medical imaging.However,there exist an ill-posed problem that a low-resolution image can be degraded from different high-resolution versions,so how to reconstruct a high-resolution image with satisfactory visual quality and reliable details has always been one of the frontier and spotlight fields.Since the rapid development of deep learning(DL),convolutional neural networks(CNNs)have been applied to diverse computer vision tasks due to the powerful feature representation capabilities.In the field of SISR,CNN-based methods have also achieved superior reconstruction performance than that of traditional algorithms on all benchmark datasets.However,some CNN-based methods have focused on improving the reconstruction accuracy,and hence the network models built by these methods are generally large in number of parameters and computational cost.For such methods,their applications are limited on some mobile devices and real-time scenarios because they can not make a good trade-off between reconstruction performance and model complexity.Therefore,how to design lightweight and efficient SISR algorithms is one of the current research hotspots in this field.For the current lightweight superresolution algorithms,although they can obtain competitive reconstruction results while maintaining low model complexity,there is still room for improvement and optimization:(1)For SISR tasks,different objects of the inputs usually have different scales and texture information,and it is important to enhance the ability to perceive and reconstruct these multi-scale objects.Therefore,the SISR model should focus on as much multi-scale information as possible,and the collection of such information relies on the sufficient receptive fields of the model.However,most of the existing lightweight SISR networks only utilize a single-path structure,which can not effectively utilize the multi-scale features from different receptive fields.Although these lightweight SISR networks also have a low computational cost,their feature characterization capability and reconstruction capability for large-size objects are limited to some extent.(2)In the reconstruction process of images,the final reconstruction results will be effectively improved if the pixel-wise feature dependencies can be captured.Although such dependencies can be obtained by stacking deep CNNs,it will generate high computational burden and make the network difficult to optimize.Currently,the application of non-local attention modules can solve this problem while bringing only a small number of parameters.However,for the general non-local attention module,it will bring about a surge of computational and memory consumption as the input size increases.Therefor,such inefficient non-local attention module is not suitable for lightweight SISR tasks,and there is much room for improvement.(3)For the existing lightweight SISR algorithms,the improvement aspect mainly focuses on designing the efficient and lightweight sturctures.Although the design of efficient network structures has significant impact on improving the reconstruction performance,some model compression methods still can be reconsidered for SISR.For the lightweight networks trained from scratch,they are actually capable of learning to extract more abstract features,and using knowledge distillation(KD)methods can bring the lightweight networks close to this goal.Thus,there is still much room for exploring the application of knowledge distillation-based methods to lightweight SISR algorithms.To this end,the thesis has proposed the following methods:(1)A lightweight multi-scale feature integration network(MFIN)based on serialparallel cascading mechanism(SCM&PCM)is proposed,which consists of several multi-scale feature extraction modules(MFEM).The MFEM adopts a multi-pathway structure and is cascaded in a parallel way so that it not only obtains multiple ranges of receptive fields,but also integrates the multi-scale feature.Specifically,the input lowlevel features can be reused in a coarse-to-fine way.In addition,the SCM in the backbone enables the local and global features to be fully utilized,thus facilitating the propagation of features from the lower to the higher layer and further enhancing the feature representation capability of the network.Extensive experiments demonstrate that the proposed method achieves a favorable trade-off in terms of number of parameters,computational cost and reconstruction performance compared to other lightweight SISR methods.(2)A lightweight hybrid non-local attention module(HNEN)is proposed,which enables to obtain the dense and pixel-wise dependencies by effectively combining the designed non-local spatial attention mechanism(Row&Column-AM)with the channel attention mechanism(Channel-AM).And in order to improve the reconstruction accuracy under the constraint resource,a progressive learning strategy(PLS)is also proposed,which is able to achieve efficient feature extraction by properly allocating the computational resources of the network.It is worth mentioning that the proposed method ccan greatly reduce the computational and memory consumption compared to the general non-local module,which demonstrates the efficiency of our proposed algorithm.Finally,the experiments prove that the proposed non-local module can not only be embedded into other lightweight super-resolution networks,but also further improve the performance of lightweight networks while properly increasing the complexity.(3)An efficient adaptive intermediate feature distillation(AIFD)method is proposed and combined with the classical SR model to achieve the efficient model compression.Compared with the general mediate feature distillation method,the proposed one is able to utilize all the mediate features of the teacher model and adaptively select these features,which can eventually be regarded as the supervision in the training of the student model.It is worth mentioning that the proposed mediate feature distillation is supervised by a loss function,which is designed by a self-attentive mechanism.In addition,the method uses the reconstruction results of the teacher modle for additional supervision.Plenty of experimental results demonstrate that the combination of the AIFD with the classical image SR model is not only able to compress the network size significantly,but also superior to that of the student model trained from scratch in the reconstruction accuracy.更多还原