【摘要】 深度分层强化学习是深度强化学习领域的一个重要研究方向,它重点关注经典深度强化学习难以解决的稀疏奖励、顺序决策和弱迁移能力等问题.其核心思想在于:根据分层思想构建具有多层结构的强化学习策略,运用时序抽象表达方法组合时间细粒度的下层动作,学习时间粗粒度的、有语义的上层动作,将复杂问题分解为数个简单问题进行求解.近年来,随着研究的深入,深度分层强化学习方法已经取得了实质性的突破,且被应用于视觉导航、自然语言处理、推荐系统和视频描述生成等生活领域.首先介绍了分层强化学习的理论基础;然后描述了深度分层强化学习的核心技术,包括分层抽象技术和常用实验环境;详细分析了基于技能的深度分层强化学习框架和基于子目标的深度分层强化学习框架,对比了各类算法的研究现状和发展趋势;接下来介绍了深度分层强化学习在多个现实生活领域中的应用;最后,对深度分层强化学习进行了展望和总结.更多还原

【Abstract】 Deep hierarchical reinforcement learning(DHRL) is an important research field in deep reinforcement learning(DRL). It focuses on sparse reward, sequential decision, and weak transfer ability problems, which are difficult to be solved by classic DRL. DHRL decomposes complex problems and constructs a multi-layered structure for DRL strategies based on hierarchical thinking. By using temporal abstraction, DHRL combines lower-level actions to learn semantic higher-level actions. In recent years, with the development of research, DHRL has been able to make breakthroughs in many domains and shows a strong performance. It has been applied to visual navigation, natural language processing, recommendation system and video description generation fields in real world. In this study, the theoretical basis of hierarchical reinforcement learning(HRL) is firstly introduced. Secondly, the key technologies of DHRL are described,including hierarchical abstraction techniques and common experimental environments. Thirdly, taking the option-based deep hierarchical reinforcement learning framework(O-DHRL) and the subgoal-based deep hierarchical reinforcement learning framework(G-DHRL) as the main research objects, those research status and development trend of various algorithms are analyzed and compared in detail. In addition, a number of DHRL applications in real world are discussed. Finally, DHRL is prospected and summarized.更多还原