【摘要】 强化学习是机器学习中一个重要的分支，随着深度学习的发展，深度强化学习逐渐发展为强化学习研究的重点。因应用广泛且实用性较强，面向连续控制问题的无模型异策略深度强化学习算法备受关注。同基于离散动作的Q学习一样，类行动者-评论家算法会受到动作值高估问题的影响。在类行动者-评论家算法的学习过程中，剪切双Q学习可以在一定程度上解决动作值高估的问题，但同时也引入了一定程度的低估问题。为了进一步解决类行动者-评论家算法中的高低估问题，提出了一种新的随机加权三重Q学习方法。该方法可以更好地解决类行动者-评论家算法中的高低估问题。此外，将这种新的方法与软行动者-评论家算法结合，提出了一种新的基于随机加权三重Q学习的软行动者-评论家算法，该算法在限制Q估计值在真实Q值附近的同时，通过随机加权方法增加Q估计值的随机性，从而有效解决了学习过程中对动作值的高低估问题。实验结果表明，相比SAC算法、DDPG算法、PPO算法与TD3算法等深度强化学习算法，SAC-RWTQ算法可以在gym仿真平台中的多个Mujoco任务上获得更好的表现。更多还原

【Abstract】 Reinforcement learning is an important branch of machine learning.With the development of deep learning, deep reinforcement learning research has gradually developed into the focus of reinforcement learning research.Model-free off-policy deep reinforcement learning algorithms for continuous control attract everyone’s attention because of their strong practicality.Like Q-learning, algorithms based on actor-critic suffer from the problem of overestimations.To a certain extent, clipped double Q-lear-ning method solves the effect of the overestimation in actor-critic algorithms, but it also introduces underestimation to the lear-ning process.In order to further solve the problems of overestimation and underestimation in the actor-critic algorithms, a new learning method, randomly weighted triple Q-learning method is proposed.In addition, combining the new method with the soft actor critic algorithm, a new soft actor critic algorithm based on randomly weighted triple Q-learning is proposed.This algorithm not only limits the Q estimation value near the real Q value, but also increases the randomness of the Q estimation value through randomly weighted method, so as to solve the problems of overestimation and underestimation of action value in the learning process.Experiment results show that, compared to the SAC algorithm and other currently popular deep reinforcement learning algorithms such as DDPG,PPO and TD3,the SAC-RWTQ algorithm has better performance on several Mujoco tasks on the gym simulation platform.更多还原