【作者】 孙硕；

【导师】 潘博；

【作者基本信息】 哈尔滨工业大学 ， 机械电子工程， 2020， 硕士

【摘要】 与传统腹腔微创手术相比,腹腔微创手术机器人进行手术具有创伤更小,术后恢复更快,便于实施等优点。建立微创手术机器人虚拟操作训练仿真平台对提高医生操作水平、缩短学习曲线、试验新的医疗器械与控制策略具有重要意义。当前市面上存在的手术操作训练仿真系统提供了一个手术操作训练的环境,但都没有对机器人本体进行建模和仿真,本文基于开源的第一代达芬奇手术机器人d VRK(da Vinci Research Kit,d VRK),在仿真软件VREP中对机器人进行建模仿真,对提高医生操作技能,研究机器人的控制策略具有重要意义。此外,深度学习和强化学习领域的突破,赋予了机器人更多的智能,使机器人自主操作学习成为可能,本文基于建立的仿真平台研究机器人自主学习,对提高机器人手术效率具有重要意义。首先,本文基于D-H参数建立了d VRK机器人运动学模型,建立了机器人仿真平台。在分析主手与从手构型的基础上,通过雅可比矩阵制定了“异构”式主从运动映射策略。为了增强操作的沉浸感,本文采用VR头盔作为机器人内窥镜持镜臂的控制端和图像输出端,操作者可以通过VR头盔控制持镜臂末端内窥镜的运动,并获取内窥镜的视野。在仿真平台结构的设计上,本文采用两个独立线程分别处理运动映射和图像传输,提高了操作的流畅性。基于上述建立的仿真平台,建立了用于强化学习训练的任务环境。针对在稀疏奖励函数下机器人在环境中学习效率低的问题,基于深度确定性策略梯度DDPG(Deep Deterministic Policy Gradient,DDPG)并结合HER(Hindsight Experience Replay)的思想设计了HER算法,有效解决了机器人在环境中学习效率低的问题。针对在多步骤(multi-steps)任务环境中,机器人搜索空间大,搜索缓慢的问题,将模仿学习中的行为克隆(Behavior Cloning,BC)与强化学习结合,设计了BC+DDPG算法,提高了机器人在多步任务环境中完成任务的成功率。最后,针对上述建立的仿真平台进行了主从运动控制实验,理论轨迹和实际轨迹之间最大误差2.3mm。设计了两种任务测试环境并对无操作经验的测试人员进行实验,证明主从控制策略符合操作直觉,易于操作。针对机器人自主学习,设计了两种任务环境,并在两种任务环境分别对提出的算法进行了实验,验证了算法的有效性。更多还原

【Abstract】 Compared with the traditional abdominal minimally invasive surgery,the abdominal minimally invasive surgery robot has the advantages of less trauma,faster postoperative recovery,and easier implementation.The establishment of a virtual operation training simulation platform for minimally invasive surgery robot is of great significance to improve the operation skills of doctors,shorten the learning curve,and test new medical devices and control strategies.At present,the surgical operation training simulation system used provides a platform for surgical operation training,but it does not model and simulate the robot body.Based on the first generation of open-source da Vinci Surgical Robot(d VRK),this paper models and simulates the robot in the software V-REP,which is of great significance to improve doctors’ operation skills and research on robot’s control strategy.In addition,the breakthrough in the field of deep learning(DL)and reinforcement learning(RL)endows robots with more intelligence and makes it possible for robots to operate and learn autonomously.Based on the established simulation platform,this paper studies surgical task automation,which is vital to improving the efficiency of surgery.Firstly,based on the D-H parameters,this paper establishes the kinematics model of d VRK robot and the robot simulation platform.On the basis of analysis of the configuration of master hand and slave robot,a "heterogeneous" master-slave motion mapping strategy is developed through Jacobian matrix.In order to enhance the sense of immersion,a VR helmet is used as the control end and image output end of the robot endoscope arm.The operator can control the movement of the endoscope at the end of the arm through the VR helmet,and obtain the vision of the endoscope.Two independent threads are used to deal with motion mapping and image transmission respectively,which improves the smoothness of operation.Based on the simulation platform,a task environment for reinforcement learning training is established.In order to solve the problem of low learning efficiency of robot in the environment under sparse reward function,the paper designs HER algorithm based on Deep Deterministic Policy Gradient(DDPG)and the idea of Hindsight Experience Replay(HER),which effectively solves the problem of low learning efficiency of robot in the environment.Aiming at the problem of large searching space and slow search in multi-steps environment,the BC + DDPG algorithm is designed by combining Behavior Cloning(BC)in imitation learning with reinforcement learning,which improves the success rate of the task in the multi-steps environment.Finally,the master-slave motion control experiment is carried out for the simulation platform.The maximum error between the theoretical trajectory and the actual trajectory is 2.3mm.Two kinds of task testing environments are designed and tested for the operators without experience.It is proved that the master-slave control strategy is intuitive and easy to operate.Two kinds of task environments are designed for autonomous learning of robot.Experiments are carried out in the two task environments to verify the effectiveness of the algorithms.更多还原