【作者】 陈志华；

【导师】 金杰； 印二威；

【作者基本信息】 天津大学 ， 工程硕士（专业学位）， 2021， 硕士

【摘要】 近年来,增强现实(Augmented Reality,AR)技术迅猛发展,多种AR头戴式显示器(AR头显)涌现市场,与此伴随的是人机交互方式的深刻变革。依靠鼠标、操纵手柄等工具的传统交互式方法已无法满足人们对智能化交互的需求,研究者开始探索以人为中心的全新人机交互方法。其中,基于手势识别和手部姿态估计的手势交互凭借其自然、灵活的特性,成为面向增强现实的首选交互式方法。本文针对增强现实场景的实际交互需求,对手势识别及手部姿态估计方法进行研究。研究内容主要包括硬件设计、数据集创建、识别算法和姿态估计算法设计以及模型的应用。具体如下:针对手势识别任务,本文设计了一款数据手套,该手套基于惯性测量单元(Inertial Measurement Unit,IMU)可实时捕捉手部的运动数据,并通过蓝牙进行数据传输,可工作于任意区域,弥补了相机视角狭窄的缺陷。本文基于该数据手套创建手势动作数据集,并提出了一种手势识别算法(Gesture Recognition Network,GR-Net),该算法利用卷积神经网络和门控循环单元共同学习手势数据的空间特征和时序特征,从而实现了手势识别。最后,通过实验验证了GR-Net算法的有效性,并将其应用于虚拟无人机操控场景,增强了交互沉浸感。针对手部姿态估计任务,本文基于AR头显RGB相机采集第一人称手部姿态图像并创建手部姿态数据集。然后提出了一种手部姿态估计算法(Hand Pose Estimation,HPE-Net),该算法利用堆叠沙漏网络学习手部关键点特征,并通过图卷积神经网络进一步加强关键点间的连接关系,从而实现手部关键点位置的精确估计,为细节化手势交互提供基础。最后,通过对比实验验证了HPE-Net算法的优越性,并将其应用于虚拟物体移动场景,增加了交互灵活性。更多还原

【Abstract】 In recent years,with the rapid development of augmented reality(AR)technology,a variety of AR head-mounted displays emerge in the market.This is accompanied by a profound change in the way of human-computer interaction.Traditional interactive methods such as mouse and control handle can no longer meet people’s demand for intelligent interaction.Researchers begin to explore a new human-computer interaction method centered on human.Gesture interaction has become the preferred interaction mode for augmented reality due to its natural and flexible characteristics.Gesture recognition and gesture estimation have attracted wide attention due to their great academic significance and practical application value.This thesis focuses on gesture recognition and hand pose estimation for the actual interaction requirements of augmented reality scenarios.The research mainly includes hardware design,data set creation,algorithm design and model application.Details are as follows:For the task of gesture recognition,a data glove is designed,which collects hand movement data based on Inertial Measurement Unit(IMU)and transmits the data via Bluetooth.The glove can work in any area,which compensates for the camera’s narrow viewing Angle.Based on the data glove,this thesis creates a gesture dataset,and proposes Gesture Recognition Network(GR-Net).The algorithm realizes gesture recognition by learning spatial and temporal features of dynamic gesture data through convolutional neural network and gated recurrent unit.Finally,the effectiveness of GRNET algorithm is verified by experiments,and the algorithm is applied to the virtual UAV control scene to enhance interactive immersion.For the task of hand pose estimation,this thesis collects first-person hand pose images based on AR head-display RGB camera and creates a dataset of hand pose.Then,the Hand Pose Estimation(HPE-Net)algorithm is proposed.The algorithm uses stacked hourglass network to learn the features of hand key points,and further learns the connection relation between key points through graph convolution neural network to estimate the position of hand key points.This algorithm provides the basic theoretical support for gesture interaction.Finally,the superiority of HPE-NET algorithm is verified by comparative experiments,and it is applied to virtual object movement scene to increase the interactive flexibility.更多还原