【作者】 陈卓；

【导师】 旷海兰；

【作者基本信息】 武汉理工大学 ， 电子科学与技术， 2019， 硕士

【摘要】 在计算机视觉领域,基于人脸图像的合成与分析一直是热门的研究课题。人脸表情合成在影视娱乐、人工智能产业应用广泛,自动化妆技术在美妆行业有着巨大的应用前景。虽然人脸图像合成技术在近几年得到了较大的发展,但在表情迁移与妆容迁移方面,仍存在难点与挑战:人脸表情因人而异,是面部肌肉与纹理共同变化的结果,要在保留原始身份的前提下自然清晰地模拟表情的变化并不容易;妆容风格更是千变万化,没有规律可循,同时还受到年龄、种族的影响。因此,在人脸表情迁移与妆容迁移的研究上仍需探索与创新。本文主要利用生成式对抗网络(Generative Adversarial Networks,GAN)来研究人脸表情迁移与人脸妆容迁移,主要工作内容如下:(1)提出一种基于GAN的人脸多表情迁移网络模型(Multi-expression Generative Adversarial Networks,MEGAN)。在网络设计上,MEGAN结合变分自动编码器(Variational Autoencoder,VAE),利用VAE的编码特性,将携带表情信息的图片编码为潜在向量,提取人脸的身份特征;将表情标签融入潜在向量,输入到生成器合成人脸图片;判别器不仅区分真假图片,并计算表情标签的分类损失,促使模型通过标签指导的方式输出指定表情的人脸图像;设计一组联合损失函数,包括对抗损失、图像重建损失、循环一致性损失、标签分类损失以及高斯先验损失,在保留大部分面部特征的前提下,同时输出多种清晰自然的表情迁移图片。通过实验对比与分析,MEGAN在表情迁移的准确度,以及部分损失函数曲线的比较上,要优于参照方法,在主观用户调查方面,MEGAN也能得到一定的认可。(2)提出一种基于GAN的人脸妆容迁移网络模型(Facial-makeup Generative Adversarial Networks,FMGAN),该模型能够对人脸图片进行自由的妆容变换。该方法在MEGAN的基础上,增加内容编码器与风格编码器,分别将图片编码为内容向量与风格向量,解开图片的面部特征与妆容风格;训练过程中使用潜在回归(latent regressor)的方式使风格向量得以表达,实现实例级别的转换;引入自适应实例归一化层(Adaptive Instance Normalization,AdaIN)将妆容风格注入到人脸图像中,实现快速妆容转换;结合人脸分割算法与泊松融合技术,采用分割-融合的方式对人脸面部进行处理与合成,使细节部位的妆容更加精细;利用风格向量的连续性以及生成器的非线性,实现模型的多样性输出。实验结果表明,FMGAN能够合成视觉效果真实的美妆与素妆图片,在妆容合成的色彩误差上,要优于参照方法,并能实现多模式的输出。(3)设计并实现了一个基于妆容迁移的虚拟美妆系统。该系统可以服务于各类美妆零售店或体验区,将人脸妆容迁移方法应用在实际场景中,不仅可以进行全脸的妆容迁移,同时也能在嘴唇区域、眼部区域单独进行妆容变换。通过该系统用户可以进行良好的试妆体验,商家也能够更好的进行产品销售以及客户拓展。更多还原

【Abstract】 In the field of computer vision,the synthesis and analysis of face images has always been a hot research topic.Facial expression transfer is widely used in film entertainment and artificial intelligence industries,and automatic makeup technology has great application prospects in the beauty industry.Although facial image synthesis technology has been greatly developed in recent years,there are still difficulties and challenges in facial expression transfer and facial makeup transfer: facial expressions vary from person to person,and are the result of changes in facial muscles and textures.It is not easy to naturally and clearly simulate changes in expressions while preserving the original identity;makeup style is always changing and has no rules to follow,it is also affected by age and nationality.Therefore,research and innovation are still needed in the study of facial expression transfer and makeup transfer.In this paper,the author mainly uses Generative Adversarial Networks(GAN)to study the synthesis of face images,including facial expression transfer and facial makeup transfer.The main research work in this paper is as follows:(1)Based on GAN,a multi-expression Generative Adversarial Networks(MEGAN)is proposed.In the network design,MEGAN combines the Variational Autoencoder(VAE),using the coding characteristics of VAE to encode the image carrying the expression information into a latent vector to extract the identity of the face.The emoticon label is integrated into the latent vector,and input to the generator to generate the face image;the discriminator not only distinguishes the true and false images,but also calculates the classification loss of the expression labels,urging the model to output a face image of the specified expression in a label-guided manner;design a set of joint loss functions,including adversarial loss,image reconstruction loss,cycle consistency loss,label classification loss,and Gaussian prior loss.It allows model to output a variety of clear and natural expression transfer images while preserving most of the facial features.Through experimental comparison and analysis,MEGAN is superior to the reference methods in the accuracy of expression transfer and the comparison of partial loss function curves.In the subjective user survey,MEGAN can also get certain recognition.(2)A GAN-based facial makeup transfer method(FMGAN)is proposed,which can perform makeup transformation on face images.This method is an improvement of MEGAN on the network model.The encoder includes a content encoder and a style encoder,respectively encoding the image as a content vector and a style vector,separates the facial features of the image from the makeup style;In the training process,style vectors can be expressed by means of latent regressor to achieve instance-level transformation.The Adaptive Instance Normalization(AdaIN)is introduced to inject the makeup style into the face image to achieve fast makeup transformation;combining face segmentation algorithm and Poisson fusion technology,segmentation-fusion method is used to process and synthesize face images,so that the makeup of detail parts is more refined;due to the continuity of the style vector and the nonlinearity of the generator,the model can automatically output images of different makeup styles.The experimental results show that FMGAN can generate realistic makeup and de-makeup images,which is superior to the reference methods in the color error of makeup synthesis,and can achieve multi-model output.(3)Designed and implemented a virtual makeup system.The system can be used in a variety of makeup retail stores or experience areas,applying the face makeup transfer method to the actual scene,not only for the whole face makeup transfer,but also for the makeup change in the lip area and the eye area.Through this system,users can carry out a good cosmetic trial experience,and merchants can also better sell products and expand customers.更多还原