【作者】 曾强；

【导师】 陈尚宾； 刘谦；

【作者基本信息】 华中科技大学 ， 生物医学工程， 2023， 博士

【摘要】 虚拟康复训练基于虚拟现实(Virtual Reality,VR)技术和仿真技术进行本地或远程治疗,已经被广泛应用于各种患者群体。在虚拟康复训练系统中,实时动作捕捉发挥着至关重要的作用,它不仅能够为用户提供高度逼真的沉浸式体验,还能够对患者的康复运动过程进行全程记录跟踪,从而有效监测和辅助动作训练。由于传统的动作捕捉系统集成到VR系统中会导致较高的延迟,基于稀疏VR设备的动作捕捉方法因其延迟低的优点成为VR应用中的研究热点。这类方法通常基于六个VR设备采集人体的根节点(盆骨)和末端节点(头、双手和双脚)的位姿信息,并通过求解逆运动学(Inverse Kinematics,IK)问题计算其它未知的关节信息。然而,VR设备的稀疏性导致准确性难以保证,面对VR设备的遮挡问题,鲁棒性较差。因此,为了解决实时性、准确性和鲁棒性三个关键问题,本文重点研究了实时、高精度动作重建算法及其在实时动作捕捉系统中的应用。主要研究内容如下:第一,针对IK算法动作重建误差较大的问题,本文提出了姿态估计-阻尼最小二乘(Pose Estimation-Damping Least Square,PE-DLS)算法,PE-DLS基于解析IK估计脊柱和四肢的动作姿态,结合关节约束快速计算最优解析解。然后基于姿态估计结果计算参考动作,并利用DLS进一步优化动作重建结果。公开动作数据集上的测试结果表明,PE-DLS的动作重建误差大约比DLS低25%,比Final-IK低54%,比BioIK低58%,同时计算时间仅需1.65 ms。PE-DLS相比主流的IK算法具有更高的准确性,同时保证了较好的实时性。PE-DLS算法创新性地结合了解析IK和DLS数值优化方法,并有效整合了关节约束对重建结果加以限制,从而实现了高精度、实时的人体动作重建。第二,为了处理VR设备的遮挡问题,本文提出了深度姿态追踪(Deep Tracking Poser,DTP)算法。DTP基于Transformer构建深度学习模型,能够根据稀疏的关节数据端到端地估计全身关节旋转。为了优化深度学习模型,本文提出了基于随机噪声的遮挡数据模拟方法,并合成了遮挡数据集AMASS-OCC。DTP首先使用AMASS数据集进行训练,然后基于AMASS-OCC进行优化。AMASS-OCC上的测试结果表明,DTP的重建误差大约比未优化的DTP0低44%,比TransPose低51%,比DLS低79%,DTP的抖动大约比DTP0低70%,比TransPose低81%,比DLS低90%,DTP同样具有较低的计算时间(2.49 ms)。DTP在遮挡情况下能够重建准确、自然的结果,鲁棒性较好。第三,实现了一套基于稀疏VR设备的实时动作捕捉系统。该系统基于六个VR设备采集人体根节点和末端节点的位姿信息,并通过整合PE-DLS和DTP方法,实现稳定的、高精度的实时动作重建。然后在动作重建之后引入了物理优化,基于刚体动力学约束减轻抖动,结合库仑摩擦约束避免滑步问题,进一步提高实时动作捕捉系统的准确性和稳定性。系统实例的测试结果表明,该系统能够稳定、准确地捕捉并重建人体动作,其系统延迟小于9 ms,足以满足大部分虚拟康复训练应用的实时性要求,并且只需要用户穿戴少量设备,在舒适度和便携性等方面具有较大优势。综上所述,本文围绕基于稀疏VR设备的实时动作捕捉中存在的关键问题,从动作重建算法和系统层面进行了改进和优化。该研究有助于在VR环境中构建稳定、高精度的实时动作捕捉系统,在虚拟康复训练等现代康复医疗领域具有广阔的应用前景。更多还原

【Abstract】 Virtual rehabilitation,based on Virtual Reality(VR)technology and simulation techniques,has been widely applied to various patient populations for local or remote therapy.In virtual rehabilitation systems,real-time motion capture plays a crucial role.It not only provides users with a highly immersive and realistic experience but also allows for the complete tracking and recording of the patient’s rehabilitation exercise process,effectively monitoring and assisting the motion training.Due to the higher latency associated when integrating traditional motion capture systems into VR systems,methods based on sparse VR devices have become a research hotspot in VR applications,due to their low-latency advantages.These methods typically involve capturing the poses of the root joint(pelvis)and end joints(head,hands,and feet)using six VR devices and solving inverse kinematics(IK)problems to estimate the unknown joint information.However,the sparsity of VR devices makes ensuring accuracy challenging,and they exhibit limited robustness when confronted with occlusion issues.Therefore,to address the key issues of real-time performance,accuracy,and robustness,this study focuses on the critical algorithm for realtime and high-accuracy motion reconstruction and its application in real-time motion capture systems.The main research are as follows:Firstly,to address the issue of large motion reconstruction errors in IK algorithms,this thesis proposes the Pose Estimation-Damping Least Square(PE-DLS)algorithm.PE-DLS leverages analytical IK to estimate the poses of the spine and limbs,and incorporates joint constraints to rapidly compute optimal analytical solutions.Then,it calculates the reference motion based on the estimated poses and utilizes DLS to further optimize the results.Testing results on the public motion datasets demonstrate that the motion reconstruction error of PE-DLS is approximately 25% lower than that of DLS,54% lower than that of Final-IK,and 58% lower than that of Bio-IK,with a computation time of only 1.65 ms.PE-DLS offers higher accuracy compared to mainstream IK algorithms while maintaining excellent realtime performance.PE-DLS innovatively combines analytical IK and DLS numerical optimization,while incorporating joint constraints to restrict the reconstruction results,resulting in high-accuracy and real-time motion reconstruction.Second,to address the occlusion issue in VR devices,this thesis proposes the Deep Tracking Poser(DTP)algorithm.DTP utilizes a Transformer-based deep learning model to estimate full-body joint rotations end-to-end based on sparse joint data.To optimize the deep learning model,this thesis proposes a random noise-based occlusion data simulation method and synthesize the occlusion dataset called AMASS-OCC.DTP is initially trained on the AMASS dataset and then fine-tuned using AMASS-OCC.Testing results on AMASS-OCC demonstrate that DTP’s motion reconstruction error is 44% lower than that of DTP0(without fine-tuning),51% lower than that of TransPose,and 79% lower than that of DLS.DTP’s jitter is approximately 70% lower than that of DTP0,81% lower than that of TransPose,and 90% lower than that of DLS,with a low computation time of 2.49 ms.DTP is capable of producing accurate and natural motion reconstruction results under occlusion,showcasing good robustness.Thirdly,a real-time motion capture system based on sparse VR devices is implemented.The system captures the pose information of the root and end joints of the human body using six VR devices,and achieve stable and highly accurate real-time motion reconstruction by integrating the PE-DLS and DTP.Then,physical optimization is introduced after motion reconstruction to reduce jitter through rigid body dynamics constraints and prevent sliding issues through Coulomb friction constraints,thereby further improving the accuracy and stability of the real-time motion capture system.Experimental results on the system prototype show that the system can capture and reconstruct human motion accurately and stably with a latency of less than 9 ms,meeting the real-time requirements of most virtual rehabilitation training applications,and it requires users to wear small devices,offering advantages in terms of comfort and portability.In summary,this thesis focuses on the key issues in real-time motion capture based on sparse VR devices and proposed improvements and optimizations in algorithms and systems.This research contributes to the development of stable and highly accurate real-time motion capture systems in VR,with promising application in modern rehabilitation medicine,such as virtual rehabilitation training.更多还原