【作者】 黄勇；

【导师】 屈代明；

【作者基本信息】 华中科技大学 ， 电子与通信工程， 2020， 硕士

【摘要】 眼球凝视估计是利用脸部图像中的眼部区域对眼睛的凝视方向与凝视点进行估计,判断其注意焦点的一种检测方法,其应用范围广具有重大经济价值。相比于早期的物理建模方法,基于深度学习方法的眼球凝视估计在准确率、稳定性、方便性等方面具有显著优势。目前,基于深度学习的单一凝视估计研究要么仅估计凝视点要么仅估计凝视方向,其性能还有提升的空间。基于此,本文主要研究将凝视点估计与凝视方向估计整合至同一个算法中,实现更为精确的凝视点和凝视方向估计。首先,构建本文实验所需的实验数据集。鉴于现有公开的多视图数据集的缺乏与其特征表示的不足,而精确的凝视点估计需要眼睛位置信息,多视图图像可以间接获得准确的眼部位置信息。因此本文采用3个校准过的相机同时拍摄人脸图像,对相机采集到的多视图图像使用Adaboost方法进行人脸检测,对检测到的人脸使用DMFRLMS方法进行脸部特征点检测,最后利用眼部区域特征提取算法从脸部特征点中提取眼部特征,得到的眼部区域图像作为凝视点数据集。对于凝视方向数据集本文采用直接公开的单视图MPIIGaze数据集。其次,构造对凝视点与凝视方向估计数据集进行特征提取的局部共享网络。在同一个局部网络中既可以实现对用于凝视点估计的多视图数据集的特征提取,也可实现对用于凝视方向估计的MPIIGaze数据集的特征提取,该局部共享网络引入了残差结构,避免了深层卷积神经网络训练中的梯度消失。凝视点与凝视方向特征提取共享一个局部网络,保证性能的同时减少了网络整体的开销,加速训练,缩短训练时间。最后,提出多视图多任务学习(MTL)框架,实现对凝视点与凝视方向的同时预测。针对凝视方向预测,对左眼和右眼提出了共面约束;针对凝视点预测,采用三视图数据输入间接引入眼睛位置信息,设计了一个跨视图池化模块;最后将凝视点与凝视方向的各自的代价函数整合至一个代价函数,加入正则化项防止网络训练出现过拟现象。实验结果表明,基于局部网络共享的多视图多任务凝视估计算法在凝视点与凝视方向两项指标上比当前主流方法领先。更多还原

【Abstract】 Eye gaze estimation is a detection method that uses the eye area in a face image to estimate the gaze direction and gaze point of the eye to determine its focus of attention,and has a wide range of applications of great economic value.Eye gaze estimation based on deep learning method has significant advantages in terms of accuracy,stability,convenience and so on compared to earlier physical modeling methods.Currently,single gaze estimation studies based on deep learing method either estimate only gaze point or only gaze direction,and their results have limitations.Based on this,this paper focuses on the integration of gaze point estimation and gaze direction estimation into a algorithm to achieve accurate gaze point and gaze direction estimation simultaneously.First of all,this work constructes the experimental data set required for our expetiment.In view the lack of existing publicly available multi-view datasets and the inadequacy of their feature representation,and the fact that accurate gaze point estimation requires eye position information,multi-view images can indirectly acquire accurate eye position information.Therefore,this work use thress calibrated cameras to take face images as the same time,using Adaboost method for face detection for multi-view images captured by these cameras,DMFRLMS method for face feature point detection for detected faces,and finally using eye area feature extraction algorithm to extract eye features from face feature points,and the resulting eye area images are used as gaze point data set.For the gaze direction data set,this work directly use the public data set for MPIIGaze.Next,this work constructs local shared networks for feature extraction of gaze point and gaze direction estimation data sets.Feature extraction of both the multi-view data set for gaze point estimation and the MPIIGaze data set for gaze direction estimation can be implemented in the same local shared network,which introduces a residual structure and avoids the gradient disappearance in deep convolutional neural network(CNN)training.The gaze point and gaze direction feature extraction share a local network,ensuring performance while reducing overall network overhead,accelerating training and shortening training time.Eventually,a multi-view multitask learning(MTL)framework is proposed to achieve simultaneous prediction of gaze point and gaze direction.For gaze direction prediction,coplaner constraints are proposed for left and right of eye.For gaze point prediction,a cross-view pooling module is designed using three-view data input to indirectly introduce eye position information.In the end,the respective cost function of gaze point and gaze direction are integrated into one cost function,and regularization terms are added to prevent overfitting in network training.The experimental results show that the multi-view multitask gaze estimation algorithm based on local network sharing is state-of-the-art the current mainstream methods on two indicators of gaze point and gaze direction.更多还原