【作者】 陈雪江；

【导师】 杨东勇；

【作者基本信息】 浙江工业大学 ， 计算机应用技术， 2004， 硕士

【摘要】 机器人技术的发展使机器人的能力不断提高，机器人应用领域和范围不断扩大。从自动化的装配工作到深海探测乃至火星探测等任务都有机器人的身影。对于有些任务来说，单个机器人的能力是有限的，因此必须把多个机器人组织起来才能完成特定的任务。对于由大量自主机器人组成的群体，如果采用由设计者提供合适的控制参数使之形成期望的协作行为，那是非常困难的。因此，整个机器人群体及群体中的每个机器人都必须具有学习能力，从而优化系统性能，适应环境变化。 强化学习是近几十年来迅速发展起来的一类机器学习算法。强化学习不需要有先验知识，而是通过与环境的不断交互获得知识，改进行为策略，具有自学习的能力。强化学习由于自身特点，在自主机器人研究领域中受到了国内外越来越多的重视。 基于马尔科夫过程的强化学习作为一种在线学习方式，能够很好地应用于单智能体环境中。由于强化学习理论的限制，在多智能体系统中马尔科夫过程模型不再适用，因此不能把强化学习直接用于多智能体的协作学习问题。本文提出了多智能体协作的分布式两层强化学习方法。该方法主要通过在单个智能体中构筑两层强化学习单元来实 摘要现，第一层强化学习单元负责学习智能体的联合任务协作策略，第二层强化学习单元负责学习在本智能体看来是最有效的行动策略。最后用3个智能体协作抬起圆形物体的实验来验证该方法，实验结果表明所提出方法比集中式强化学习方法更好， 在基于强化学习的多智能体环境下，评价一个智能体行为的效果和其他智能体的行为是密切相关的，如果每个智能体都能够通过预测其他智能体的行为再来采取自己的行为，则必将大大促进整个系统的协作。在传统强化学习方式中引入组合动作的基础上，本文提出了一种基于行为预测的多智能体强化学习方法，研究了对其他智能体行为进行预测的几种可行方法。实验结果表明，引入组合动作的传统强化学习方法是收敛的，行为预测方法能有效促进协作。更多还原

【Abstract】 With the development of robotics, the capability of a robot is becoming more powerful. The application of robotics has extended into many domains, from automatic assembly to deep sea exploration, mars exploration and so on. There are some specialized tasks which single robot has not enough power to complete, but multiple robots can be organized to complete. It is very difficult for the designer to provide proper control parameters which can be used to help the multi-robot system consisting of many autonomous robots work cooperatively. Therefore, it is highly desirable for multi-robot system and each robot to be able to learn control parameter values in order to optimize their task performance, and to adapt to dynamic environment.Reinforcement learning, a kind of machine learning algorithm, has receieved much attention in recent ten years. Reinforcement learning does not need priori knowledge and improves its behavior policy with knowledge obtained by interaction with the environment. So reinforcement learning has the ability of self-learning.Reinforcement learning has been applied to single agent environment successfully. Due to the theoretical limitation that it assumes that an environment is Markovian, traditional reinforcement learning algorithms cannot be applied directly to multi-agent system. In this paper, two-layer reinforcement learning method for multi-agent cooperation is presented. This method is realized by adding two-layer reinforcement learning unit to every agent. The first layer is for learning globalcooperation game, and the second layer is for learning efficient action game in its own view. An experiment that three agents raise a round object by cooperation is made to test the efficacy of the method. The experiment results show that the agents using this method cooperate better than the agents using center reinforcement learning.In multi-agent system based on reinforcement learning, the evaluation to the behavior of a robot depends on the other agents’ behaviors closely. If each agent takes its action after it predicts the other agents’ actions self-consciously, the performance of the whole system will be better. In this paper, introducing joint-action to the traditional reinforcement learning, a new multi-agent reinforcement learning algorithm based on behavior prediction is presented and several methods for predicting other agents’ behaviors are discussed. The experiment results show that joint-action is applied in the traditional reinforcement learning successfully and the cooperation process can be speeded by behavior prediction.更多还原