【作者】 李锋；

【导师】 严浙平；

【作者基本信息】 哈尔滨工程大学 ， 交通信息工程及控制， 2008， 硕士

【摘要】 AUV(Autonomous Underwater Vehicle,简称AUV)作为一种高技术手段,在海洋环境监测、海底资源调查、科学考察、危险环境作业和打捞救生等方面起到了至关重要的作用。随着执行任务的复杂性日益增加,单AUV在大范围内作业的时效性、鲁棒性和柔性等方面就表现出明显不足。需要多个AUV构成系统来共同完成任务,而多AUV的协调控制成为关键问题,本论文主要讨论了一种新的多AUV协调控制方法、系统设计及仿真。本文首先介绍了多AUV技术的发展动态以及课题研究的意义,然后根据多AUV协调控制系统的需要建立AUV运动模型、多智能体Q学习模型,在此基础上建立了面向任务的多AUV系统结构。对比分析了近几年的几种重要的多智能体强化算法,然后提出了一种新的多智能体Q学习算法,这种算法结构简单,能够大大简化状态空间,加快收敛速度。试验表明,本文提出的多智能体Q学习方法是有效的。然后对多智能体Q学习算法实现多AUV协调的系统进行了设计,用多个仿真实例说明了多智能体Q学习算法在多AUV系统中的应用,试验证明,此本文提出的多智能体Q学习协调算法在满足Nash均衡的同时,避免了研究多个均衡点同时存在的问题,收敛速度快并且非常有效。更多还原

【Abstract】 Autonomous Underwater Vehicle (AUV) plays an important role in marine environment monitoring, seabed resources investigating, science inspecting, dangerous environment exploring, rescue and salvage and so on. Along with the mission become more complicatedly, the single AUV obviously shows insufficient at effectiveness, robustness and flexibility in a large-scale operating mission. It is necessary to coordinate with the help of Multi-AUVs operating together, and cooperation is the key techniques of Multi-AUVs . In this paper a cooperative strategy, a Multi-AUVs cooperative system’s design and it’s simulation are discussed.Firstly, the trend of multiple AUVs development and the research meaning are presented. Then a AUV dynamic model, multiagent Q-learning model is designed based on the need of multiple AUVs system, and a new architecture of AUV based on mission is proposed. Then, several single agent and multiagent reinforcement learning algorithms proposed in recent years are investigated, compared and analyzed deeply in this paper. And a muliagent Q-learning algorithm is proposed. This algorithm involves simple procedures and easy computations, and can guarantee good learning convergence. Experiment results of multi-AUV’s coordination and control show that this algorithm is effective.Then, a Multi-AUVs cooperative system based on muliagent Q-learning algorithm is desinged. Several simulation experiment show the application of muliagent Q-learning algorithm in multi-AUV’s coordination. And the result show this algorithm can converge to Nash equilibria, avoids resolving complex multiple Nash equilibria problem, and it is effective and converges well.更多还原