【作者】 吴睿；

【导师】 黄岚；

【作者基本信息】 吉林大学 ， 工程硕士（专业学位）， 2018， 硕士

【摘要】 深度强化学习是一种新的算法,它结合了深度学习和强化学习,以实现从感知到行为的端到端的学习。就像人一样,直接通过深度神经网络输入感知信息,如视觉和输出动作,而不需要手动操作。通过深度强化学习训练的代理人能够完全自主地学习一种或多种不同的技能。近两年机器学习迅速发展,深度强化学习作为机器学习的一个分支,发展尤为猛烈。深度强化学习旨在解决计算机从感知到控制的问题,基于深度强化学习算法创造出的成果已经逐渐应用于我们的日常生活,在计算机视觉,语音识别,自然语言处理等相关领域的发展取得重大突破,但是这些领域所研究的问题,都仅仅是让计算机认识和理解环境,而没有解决人工智能最核心的问题-决策控制。需要向计算机输入感知信息的例如计算机视觉等感知问题,计算机可以根据感知信息理解和判断控制问题,并由计算机判断并输出正确的行为要求。为了使计算机能够做出适当的决策控制,需要计算机具备一定的“思考”的能力,以便计算机能够通过学习获得解决各种问题的能力,而这正是通用人工智能研究的目标。智能体的行为都可以归纳为智能体与环境进行交互,智能体观察环境,根据自身的状态及所观察的环境做出动作,更改环境,并将反馈返回给智能体。因此,核心问题是如何构建可以与环境进行交互的智能体。深度强化学习就是将善于提供学习机制的深度学习,和可以为智能体提供学习目标的强化学习进行优势结合,使得智能体具备学习复杂能力的潜力。本文首先使用深度强化学习提出的第一个算法DQN进行实验,将智能体放在贪吃蛇游戏环境中进行博弈训练,智能体在训练时将不会得到除图像像素和分数之外的任何信息,智能体必须自己学习,并直接使用输入和分数来制定最佳行动策略。之后,本文加入课程学习策略,将课程学习思想与DQN算法相结合,再次对智能体进行博弈训练,对比实验结果。通过实验表明,使用深度强化学习算法DQN训练的智能体在与对手的博弈训练后可以获得更高的分数和更长的存活时间,使用课程学习策略则可以大大提高智能体的训练速度,进一步提升智能体的表现。更多还原

【Abstract】 Deep Reinforcement learning is a brand new algorithm that combines deep learning and reinforcement learning to achieve end-to-end learning from perception to action.Simply put,just like human beings,inputting perceptual information such as vision,and then outputting the action directly through deep neural networks without any hand-crafted work in between.Deep reinforcement learning has the potential to enable an agent to learn one or even more of a variety of skills completely autonomously.The deep reinforcement learning is a branch of the rapid development of the field of deep learning in the past two years.The goal is to solve the problem of computer control from perception to decision-making,so as to realize common artificial intelligence.With Google Deep Mind Team as its leader,deep-learning-based algorithms have made breakthroughs in computer vision,speech recognition,natural language processing and other fields,and the related technologies have matured and entered into our lives.However,the issues studied in these areas are simply to enable computers to perceive and understand the environment.At the same time,decision-making control is the core issue to be solved in the field of artificial intelligence.Perception problems such as computer vision require input of perceptual information to the computer,the computer can understand,and the decision-making control problem requires the computer to judge and output according to the perceptual information and output the correct behavior.To enable computers to make good decision-making and control requires that the computer have some "thinking" ability to enable the computer to master the ability to solve various problems through learning,and this is exactly the goal of universal artificial intelligence research.Agent behavior can be attributed to the interaction with the environment.The agent observes the environment,and then outputs the action based on the observation and its own state.As a result,the environment changes and the feedback is returned to the agent.Therefore,the core issue is how to construct such an agent that can interact with the environment.Deep Reinforcement Learning combines deep learning with reinforcement learning.Deep learning is used to provide a learning mechanism,reinforcing learning provides the learning goals for deep learning.This makes Deep Reinforcement Learning possess the ability to construct complex intelligence.In this paper,we use the first deep reinforcement learning algorithm DQN to conduct experiments,put the agent in the game of gluttonous snake environment for training,and the agent will not get any information except image pixels and scores during training,agents must learn by themselves and use inputs and scores directly to develop the best strategy of action.After that,we joined the curriculum learning strategy,combining the idea of course learning with DQN algorithm,and again training the agent on the game,comparing the experimental results.Experiments show that the agent using DQN algorithm can get higher scores and longer survival time after training with the opponent,and the use of the course learning strategy can greatly improve the training speed of the agent and further enhance the performance of the agent.更多还原