基于双目视觉的三维人体姿态估计

【作者】 张平；

【导师】 潘海朗；

【作者基本信息】 南京理工大学 ， 电子与通信工程（专业学位）， 2020， 硕士

【摘要】 三维人体姿态估计作为人类行为动作识别的关键环节在游戏、体育训练以及人机交互等多个领域都有着广泛的应用前景。目前常见的三维人体姿态估计需要佩戴较为昂贵的辅助设备,运动目标一定程度上会受到这些设备的影响不能够灵活地运动,而且有些运动捕捉设备不能在室外等复杂场景使用。因此本文将成本更低,场景使用更灵活的双目视觉和二维姿态估计相结合来实现三维人体姿态估计。针对基于深度学习的OpenPose算法提取二维关节点时,左右图像同一关节点会出现不匹配的情况,导致恢复出来的三维姿态不能够准确的反应人体真实姿态,本文提出了一种改进方法,该方法采用可变形模型拟合人体四肢,既能够准确定位人体四肢关节点,同时也满足左右图像同一关节点相匹配,从而提升了人体三维姿态估计的准确性和真实性。本文在三维人体姿态估计领域主要开展了以下研究工作:(1)设计实现双目立体视觉系统,并调用开源不久的OpenPose算法提取人体二维关节点恢复出三维坐标实现三维人体姿态估计,该系统能够对采集的图像和视频进行处理。(2)针对OpenPose方法提取关节点不准确和左右图像同一关节点不匹配,从而导致恢复的三维姿态不能够准确的反映真实姿态,本文提出一种基于双目视觉的三维人体姿态估计改进方法。该方法得到的二维关节点结合双目视觉系统恢复的三维姿态能够更准确的反映了人体真实姿态。(3)利用无人机携带双目立体视觉系统,拍摄实际场景完成了多人复杂场景的三维姿态估计,证明了本文方法在复杂场景下的可行性,也验证了本文所设计实现的双目立体视觉系统良好的鲁棒性和实用性。更多还原

【Abstract】 As the key part of human action recognition,3D human pose estimation has a wide range of applications in games,sports training,and human-computer interaction.At present,the common 3D human pose estimation requires the wearing of relatively expensive auxiliary equipment.To some extent,the moving target will be affected by these equipments and cannot be flexibly moved,and some motion capture devices cannot be used in complex scenes such as outdoors.Therefore,this paper uses more cheap and flexible binocular vision and 2D pose estimation to achieve 3D human pose estimation.Aiming at the 2D joint points extracted by the OpenPose algorithm,the same joint points of the left and right images may not match,resulting in the restored3 D pose cannot accurately reflect the true human posture.This paper proposes an improved method.Methods Fitting human limbs with deformable model can not only accurately locate the joint points of human limbs,but also meet the same joint points of the left and right images,thereby improving the accuracy and authenticity of 3D human pose estimation.The following research work has been carried out in 3D human pose estimation:(1)Design and implement a binocular stereo vision system,and call the open source OpenPose algorithm to extract the 2D joint points and recover the 3D coordinates to achieve 3D human pose estimation.The system can process images and videos.(2)Aiming at the inaccurate joint points extracted by OpenPose and the mismatch between the same joint points of the left and right images,the restored 3D pose cannot accurately reflect the true pose,and an improved 3D human pose estimation method based on binocular vision is proposed.The 2D joint points obtained by this method combined with the 3D posture restored by the binocular vision system can more accurately reflect the true posture of the human body.(3)Utilizing the binocular stereo vision system carried by the drone,shooting the actual scene to complete the 3D pose estimation of the complex scene of multiple people,proving the feasibility of the method in this complex scene,and verifying the binocular designed and implemented in this paper Stereo vision system has good robustness and practicality.更多还原