【作者】 李梅；

【导师】 张二虎； 石争浩；

【作者基本信息】 西安理工大学 ， 印刷包装技术与设备， 2022， 博士

【摘要】 半色调技术是指将连续调图像转换为二值图像但视觉上近似连续调图像的技术,其广泛应用于印刷、出版、纺织、电子显示等领域。对于这些大量存在的半色调图像,若要再次利用,一般需要先将其转换为连续调图像。逆半色调技术就是将半色调图像转换为连续调图像的一项特殊的数字图像恢复技术,其在图像处理、图像印刷再现及机器视觉等任务中具有重要的学术研究意义及实际应用价值。由于半色调过程中不可避免的存在着图像信息损失,逆半色调是一个病态的问题。同时,半色调噪点去除与细节恢复互为矛盾,不同类型半色调图像存在着网点形状及分布的不一致性,扫描半色调图像无成对图像监督信息。这些挑战性的问题,成为逆半色调技术中需要攻克的技术难点。本文针对这些问题,研究基于深度学习的逆半色调技术。本文的主要工作和贡献包括:1)构建了逆半色调研究用的图像数据集为了推动逆半色调方法的研究,构建了一个公开的半色调图像数据集,该数据集包括多类型数字半色调图像数据集MDHD和扫描半色调图像数据集SHD。MDHD包含了连续调图像和相应4种半色调图像,共23445幅图像,分为训练数据集(2375张)、验证数据集(500张)和测试数据集(20570张)。SHD包含8000幅扫描半色调图像,分为训练数据集(6400张)、验证数据集(800张)和测试数据集(800张)。MDHD和SHD中包含了人物、植物、动物和建筑等生活中常见的图像,目标种类多。MDHD和SHD的创建为本文多类型半色调图像的恢复提供了保障,也为其他研究人员进行相关研究奠定了坚实的基础。2)提出一种基于多阶段多分辨率增强的单类数字半色调图像复原方法现有方法在去除半色调噪点时存在图像细节恢复不够清晰、复原图像颜色有一定差异的问题。针对这些问题,基于不同子网络侧重恢复不同图像信息的思想,本文提出了一种多阶段多分辨率单类数字半色调图像复原网络(MM-Net)。首先,利用多分辨率卷积神经网络实现噪点的去除;然后,结合稠密残差块和细节损失函数实现细节增强;最后利用图像高级语义信息实现包括色彩恢复的全局调整。在三个公开数据集上采用峰值信噪比、结构相似度和色差三个指标进行方法评价。实验结果表明,相比于已有逆半色调方法,所提出的MM-Net,无论是在客观指标还是主观可视化效果方面,都取得了最好的结果:在结构相似度方面提高了 0.02-0.11,在峰值信噪比方面提高了 1.61-6.8dB,在色差方面减少了 0.48-2.73。3)提出一种实现多类型数字半色调图像高质量恢复的网络模型目前的逆半色调方法大多都是针对单一类型的数字半色调图像进行复原的,尚不能兼顾不同类型数字半色调图像的噪点去除和细节恢复。为了解决这个问题,研究中提出了一种基于多尺度生成对抗网络的逆半色调方法(MS-GAN)。首先,提出通过多尺度特征提取模块来丰富当前像素点与周围邻域的关系。其中,通道注意力机制自适应地对融合后的特征权重进行重新分配,从而有选择性地关注更为重要的信息。然后,应用细节增强网络增强生成图像细节。最后,采取多尺度鉴别器进一步优化逆半色调图像,增强逆半色调图像的视觉感知效果。实验结果表明,针对Bayer有序抖动法(BDD)生成的半色调图像,本方法恢复得到的逆半色调图像峰值信噪比值提高了 0.25-0.61dB,结构相似度提高了 0-0.03;针对Knuth点分散法(KDD)生成的半色调图像,采用本方法恢复得到的逆半色调图像峰值信噪比提高了 1.14-1.28dB,结构相似度提高了 0.03-0.04;针对迭代法(DBS)生成的半色调图像,采用本方法恢复得到的逆半色调图像峰值信噪比提高了 0.45-0.72dB,结构相似度提高了 0-0.01;只有在处理Floyd-Steinberg误差扩散法(FSDD)生成的半色调图像时,效果稍逊于MM-Net。4)提出一种融合Transformer的扫描半色调图像阶段式逆半色调复原方法针对扫描半色调图像受纸张、油墨影响,退化情况复杂且不存在对应标签图像问题,为了得到高质量的扫描逆半色调图像,基于问题简化的思想,提出基于融合Transformer的阶段式逆半色调方法(ST-Net)。首先,通过无监督退化网络实现连续调图像向扫描半色调图像的转化,生成类扫描半色调图像。然后,通过应用成对数据集训练融合Transformer逆半色调网络模型生成扫描逆半色调图像。实验结果表明,与现有扫描逆半色调方法相比,由融合Transformer的阶段式逆半色调方法得到的扫描逆半色调图像在主观效果和客观指标上都取得了较好的结果:在MOR(Mean Opinion Rank)方面降低了0.67-2.28,在清晰度方面提高了 1.97-11.74,在 FID(Fréchet Inception Distance)方面减少了 1.80-10.96。更多还原

【Abstract】 The halftoning technology refers to the technique of converting a continuous tone image into a binary image which visually approximate a continuous tone image and widely used in the following fields such as printing industry,publishing,textile,electronic display,etc.Generally,these widely existing halftone images have to be first converted into continuous tone images in order to be used again.Inverse halftoning technology is a special digital image restoration technology that converts halftone images into continuous tone images.Therefore,researches on inverse halftoning method have important values both for academic research and practical application in image processing,image printing reproduction and machine vision.Due to the inevitable loss of image information in the process of halftoning,inverse halftoning is an ill-posed problem.At the same time,halftone dot noise removal and detail restoration are contradictory to each other.Different types of halftone images have inconsistency in dot shape and distribution,and scanned halftone images can not provide image supervision information.These challenging problems become the technical difficulties to be overcome in inverse halftoning method.Aiming at these problems,this dissertation studies deep learning based inverse halftoning methods.The main work and contributions are summarized as follows:1)Generates an halftone image dataset for studying inverse halftoningTo promote the study of inverse halftoning,we generate a publicly halftone image dataset including a multi-type digital halftone image dataset(MDHD)and a scanned halftone image dataset(SHD).The MDHD contains continuous tone images and their corresponding 4 types of halftone images with a total of 23445 images,which are divided into training dataset(2375),validation dataset(500)and test dataset(20570).The SHD contains 8,000 scanned halftone images divided into training dataset(6400 images),validation dataset(800 images)and test dataset(800 images).Images contained in both MDHD and SHD are common in life,including a variety of objects,such as human images,plant images,animal images,building images,etc.The creation of SHD and MDHD provides a guarantee for the restoration of multi-type halftone images in this dissertation,and also lays a solid foundation for other researchers to carry out related research.2)Proposes a multistage and multi-resolution network for single-type digital halftone image restorationThe existing methods have the problems that the image details are not clear enough when removing halftone noise,and the restored image color is different.To solve these problems,a multistage multi-resolution network(MM-Net)is proposed based on such a prior that different subnetworks focus on restoring different image information.Firstly,the multi-resolution convolutional neural network is used to achieve dot removal,and then both the dense residual block and detail loss function are combined to achieve detail enhancement.Finally,the image high-level semantic information is used to achieve global adjustment including color restoration.Three indexes of the Peak Signal-to-Noise Ratio,the Structural Similarity and the color difference are used to evaluate the method on three public datasets.Our experimental results show that compared with the existing inverse halftoning methods,the proposed MM-Net achieves the best results in both objective indexes and subjective visualization performance:the structural similarity is improved by 0.02-0.11,the peak signal-to-noise ratio is improved by 1.61-6.8dB,and the color difference is reduced by 0.48-2.73.3)Proposes a high quality restoration model for multi-type digital halftone imagesMost of the current inverse halftoning methods were aimed at the restoration of the singletype of the digital halftone image,which could not take into account the halftone noise removal and detail restoration for different types of digital halftone images.To solve this problem,an inverse halftoning method based on multi-scale generative adversarial network(MS-GAN)is proposed.Firstly,a multi-scale feature extraction module is employed to enrich the relationship between the current pixel and its surrounding neighborhood.The channel attention mechanism adaptively redistributes the weight of the fused features,so as to selectively pay attention to more important information.Then,the detail enhancement network is applied to enhance the image details.Finally,a multi-scale discriminator is used to further optimize the inverse halftone image to enhance the visual performance of the inverse halftone image.Experiments show that for the halftone images generated by BDD,the peak signal-to-noise ratio of the restored inverse halftone images is improved by 0.25-0.61dB while the structural similarity is improved by 0-0.03.For the halftone image generated by KDD,the peak signal-to-noise ratio of the restored inverse halftone image is improved by 1.14-1.28dB and the structural similarity is improved by 0.03-0.04.For the halftone image generated by DBS,the peak signal-to-noise ratio of the restored inverse halftone image is improved by 0.45-0.72dB while the structural similarity is improved by 0-0.01.Only when processing halftone images generated by FSDD,the performance of MS-GAN is slightly inferior to MM-Net.4)Proposes a Staged Transformer-fused inverse halftoning method for Scanned halftone image restorationIn view of the complex degradation of scanned halftone images affected by paper and ink,and the absence of corresponding label images,in order to obtain high-quality scanned inverse halftone images,a Staged Transformer-fused inverse halftoning method(ST-Net)is proposed based on the idea of problem simplification.Firstly,an unsupervised degradation network is trained to transform a continuous tone image into a scanned-like halftone image.Then,the Transformer-fused inverse halftoning model is trained by applying pairwise datasets to generate scanned inverse halftone images.Experimental results show that compared with the existing scanned inverse halftoning methods,the scanned inverse halftone images obtained by the Staged Transformer-fused inverse halftoning method achieve better results in both subjective effect and objective indexes:the Mean Opinion Rank is reduced by 0.67-2.28,the Clarity is improved by 1.97-11.74,and the Frechet Inception Distance is reduced by 1.80-10.96.更多还原