【摘要】 异步深度强化学习能够通过多线程技术极大地减少学习模型所需要的训练时间。然而作为异步深度强化学习的一种经典算法,异步优势行动者-评论家算法没有充分利用某些具有重要价值的区域信息,网络模型的学习效率不够理想。针对此问题,文中提出一种基于视觉注意力机制的异步优势行动者-评论家模型。该模型在传统异步优势行动者-评论家算法的基础上引入了视觉注意力机制,通过计算图像各区域点的视觉重要性值,利用回归、加权等操作得到注意力机制的上下文向量,从而使Agent将注意力集中于面积较小但更具丰富价值的图像区域,加快网络模型解码速度,更高效地学习近似最优策略。实验结果表明,与传统的异步优势行动者-评论家算法相比,该模型在基于视觉感知的决策任务上具有更好的性能表现。更多还原

【Abstract】 Asynchronous deep reinforcement learning(ADRL) can greatly reduce the training time required for learning models by adopting the multiple threading techniques.However,as an exemplary algorithm of ADRL,asynchronous advantage actor-critic(A3 C) algorithm fails to completely utilize some valuable regional information,leading to unsatisfactory performance for model training.Aiming at the above problem,this paper proposed an asynchronous advantage actor-critic model with visual attention mechanism(VAM-A3 C).AM-A3 C integrates visual attention mechanism with traditional asynchronous advantage actor-critic algorithms.By calculating the visual importance value of each area point in the whole image compared with the traditional Cofi algorithm,and obtaining the context vector of the attention mechanism via regression function and weighting function,Agent can focus on smaller but more valuable image areas to accelerate network model decoding and to learn the approximate optimal strategy more efficiently.Experimental results show the superior performance of VAM-A3 C in some decision-making tasks based on visual perception compared with the traditional asynchronous deep reinforcement learning algorithm.更多还原