【作者】 李波；

【导师】 郑延斌；

【作者基本信息】 河南师范大学 ， 计算机软件与理论， 2016， 硕士

【摘要】 多agent系统的研究是当今人工智能和自动化控制领域的最前沿方向。多agent系统在各行各业中都表现出了极大的应用性,其自身所具有的分布性、鲁棒性强以及良好的协作性和适应性等优点是单个agent系统所不具备的。在实际应用中,多agent一般工作在未知动态环境中,环境中各种动、静态障碍物的状况是agent所不知道的,在处理这些突发情况的时候就要求agent具有较强感知环境和适应环境的能力,强化学习的无环境模型学习能力使agent具有了自学习和在线学习的能力,得到了越来越多研究者们的重视。但是,强化学习最大的缺陷就是遇到复杂任务的时候会出现“维数灾难”的问题。为了解决强化学习的“维数灾难”问题提出了分层强化学习算法,它以半马尔科夫决策为基础通过“抽象机制”把整个学习任务划分成不同层次的子任务,对状态空间降维,来解决“维数灾难”问题。其经典的算法有HAM、MAXQ和Option。最后,本文运用分层强化学习的思想来解决多agent系统中路径规划和编队控制问题,其主要工作概括如下:(1)从路径规划算法收敛速度慢及效率低、适应性差的角度出发,提出了一种基于分层强化学习及人工势场的多agent路径规划算法。首先,对环境中的人工势场进行离差标准化处理构建环境的先验知识,以此可以得到一个目标点具有最大势能,障碍物区域势能值为零的单调递增的曲面。最后,多agent以构建的先验知识为基础,利用分层强化学习的思想,使算法具有任务分层和良好的在线学习能力及自动划分子任务的能力,从而更加适应未知动态环境中路径规划任务。算法依次在出租车问题和中视典三维仿真平台中进行了验证,结果显示多agent对未知环境适应性强,算法的收敛速度快且稳定。(2)针对现阶段多agent编队控制中常出现的环境适应性差、agent无自学习能力和收敛速度慢等问题,本文提出了一种基于分层强化学习及CMAC神经网络的多agent动态编队方法。首先,在多agent动态编队中,引入“抽象机制”把整个任务分为根任务协作层,动作子任务选择层和基本动作执行层三个任务层次对状态空间降维和学习任务分解。其次,利用CMAC神经网络可以作为状态泛化方法和分层强化学习中的Q-学习方法相结合,通过状态变量的分割降级来减少CMAC空间存储量,再利用若干降级后CMAC分别逼近学习状态的Q函数来实现连续状态的泛化,加快算法的学习速率。最后,算法在中视典三维仿真平台验证其可行性,在matlab中证明了算法收敛速度快且稳定。更多还原

【Abstract】 Now the research of multi agent system is the most advanced direction in the field of artificial intelligence and automation control.It has a great application in all walks of life,this is due to their own distribution,large redundancy and robustness,as well as good cooperation and adaptability and other advantages which a single agent system does not have.In the actual situation,multi agent were generally working in unknown dynamic environment,which all kinds of dynamic and static obstacles couldn’t be known by agent,Therefore,it needs to have a strong sense of the environment and adaptability,when dealing with these unexpected situations.Agent had the ability of self-learning and online learning when the reinforcement learning ability on a no environmental model of learning.And this phenomenon has been paid more and more attention by researchers.But the biggest flaw of reinforcement learning was the problem of "dimension disaster" when it came to a complex task.The learning task was divided into different sub tasks through the "abstraction" approach with the semi Markov decision basis,in order to reduced state space dimension,and solve the "Curse of dimensionality" problem.Its classical algorithms were HAM,MAXQ and Option.Finally,this paper used the idea of hierarchical reinforcement learning to solve the problem of path planning and formation control of multi agent system,The main work of this thesis can be summarized as follows :(1)In this paper,a multi agent path planning algorithm was proposed which was based on the hierarchical reinforcement learning and artificial potential field,from the point of view of slow convergence and low efficiency and poor adaptability of path planning algorithm.Firstly,a target point with the maximum potential barrier potential value is zero monotonically increasing surface could be got by using the artificial potential field method after the difference to the standardization of environmental construction of prior knowledge.Finally,Multi agent made it possible that the algorithm had the task of hierarchical and good online learning ability and automatic classification of molecular tasks,which based on building a priori knowledge with the idea of hierarchical reinforcement learning.The algorithm has been verified in the experiment of the taxi problem and in the typical 3D simulation platform.The results showed that the algorithm has a fast convergence speed and stability for multi agent.(2)In view of the current stage of multi agent formation control,there were many shortcomings,such as poor environmental adaptability,agent self-learning ability and slow convergence speed.In this paper,a multi agent dynamic formation method was proposed,based on hierarchical reinforcement learning and CMAC neural network.Firstly,the whole task was divided into the root task collaboration layer through the "abstract mechanism",which help to reduce the dimension of the state space and the task decomposition of the task selection layer and the basic action layer of the three tasks in n multi Agent dynamic formation.Secondly,the CMAC neural network can be used as a state generalization method and Q-learning method in hierarchical reinforcement learning,the CMAC space storage was reduced by partitioning the state variable.Then,by using several CMAC functions,the Q functions of the degraded state are approximated by the state of philosophy to realize the generalization of the continuous state,and to speed up the learning rate of the algorithm.Finally,the algorithm was proved to be feasible in the 3D simulation platform code,and fast and stable in MATLAB.更多还原