【作者】 王晓娜；

【导师】 匡翠方；

【作者基本信息】 浙江大学 ， 信息传感及仪器， 2021， 硕士

【摘要】 智能手机已经成为一件日常用品,它在人类生活中的作用越来越重要,它的功能越来越完善,外观也在不断改进。而近些年来,“全面屏”成为手机的一个研究热点,手机屏幕不断扩大,但是手机前置摄像头仍旧无法至于屏幕下方,也因此无法实现真正的全面屏。由于手机屏幕无法做到完全透明,其中的电路走线和其他不透明部分会对光造成遮挡,由于屏幕结构产生的狭缝更会使光透过屏幕时发生严重衍射现象,使物体在手机图像传感器上的成像模糊不清。解决这个问题,进而使屏下得到的图像重新恢复清晰,具有重要意义。本论文根据图像复原的基本原理,通过反卷积、深度学习以及图像修复等方法,对手机屏下摄像头拍摄的图像进行进一步处理,以期处理后的图像能够尽可能地接近在屏上拍摄得到的图像效果,使手机屏下成像真正成为现实。本文的主要内容及创新点如下:1.将反卷积算法应用于手机屏下成像的图像恢复中。使用光纤耦合LED,标定手机前置摄像系统的点扩散函数(point spread function,PSF),作为反卷积的先验。针对单张图中噪声过高问题,改进了传统反卷积方法,通过转换拍摄得到图像的颜色空间,对转换后得到的强度通道进行反卷积处理提高清晰度,对颜色通道进行高斯滤波处理降低噪声。并对反卷积之后的图使用非局部均值(Non-local means,NL-means)算法去噪,进一步改善视觉效果。从仿真结果可知,经过处理后的图像,信噪比能够提高10%左右,结构相似性指数也能提高0.2左右。2.将深度学习中的生成对抗网络(Generative Adversarial Networks,GAN)引入到手机屏下成像恢复中。由仿真结果可知,经过GAN网络后,退化图像的信噪比能够提升20%左右。同时,通过拍摄得到的屏上-屏下图像对构建数据集,对GAN网络进行训练。实验证明,训练后的网络对屏下拍摄的图像能取得相当程度的信噪比提高。3.针对拍摄高亮度场景时产生的饱和衍射斑,引入基于快速行进(Fast Marching Method,FMM)的修复算法。通过提取图像中饱和的位置,将其与PSF卷积获得修复掩膜。再使用FMM修复算法对掩膜确定的位置进行图像修复。通过对实际高亮度图像的修复结果可见,这种方法能够在不影响图像整体视觉效果的情况下,成功去除饱和衍射斑。更多还原

【Abstract】 The smartphone has become a daily essential,and its role in human life is becoming more and more important.Its functions and appearance are constantly improving.In recent years,"full screen" has become a research hotspot of mobile phones,and the front camera of mobile phones has become the last obstacle to a truly full screen.Since the screen of a mobile phone cannot be completely transparent,the circuit traces and other opaque parts in it will block the light,and the slits produced in them will cause serious diffraction of light after passing through the screen,which greatly reduces image quality on the sensor.It is of great significance to solve this problem to restore the clarity of the under-screen image.In this article,based on the theory of deconvolution,deep learning and image inpainting,the under-screen image is processed to be as close as possible to the up-screen image.Realization the imaging under the mobile phone screen is possible.The main content and innovations of this article are as follows:1.Deconvolution algorithm is applied to the image restoration of under-screen imaging of mobile phone.Calibrate the point spread function(PSF)of the front camera system of the mobile phone by fiber-coupled LED.To solve the problem of excessive noise in a single image,the traditional deconvolution method is improved by converting the color space of the image,deconvoluting the intensity channel to improve clarity,and filtering color channels to reduce noise.Non-local means(NL-means)is further used to process the image after deconvolution.This processing not only improves the signal-to-noise ratio of the image,but also obtains a better visual effect.It can be seen from the simulation results that the peak signal-to-noise ratio(PSNR)of the processed image can be increased by about 10%,and the structural similarity index(SSIM)can also be increased by about 0.2.2.Generative Adversarial Network(GAN)is introduced to the recovery of under-screen image.As shown in the simulation results that the peak signal-to-noise ratio of the degraded image can be increased by about 20%after trained GAN network.The dataset is constructed by the captured on/under-screen image pairs to train the GAN network.The experiment proves that the trained network can achieve a good recovery effect on the under-screen image.3.An inpainting algorithm based on Fast Marching Method(FMM)is introduced to deal with the saturated diffraction spots generated when capturing high-brightness scene.The repair mask is obtained by extracting the saturated position in the image,convolving it with the PSF.It can be seen from the inpainting result of the actual image that this method can successfully remove the saturated diffraction spots without affecting the overall visual effect of the image.更多还原