【摘要】 针对智能移动终端设备的资源有限性和未来网络架构需要云边协同能力等问题，提出了一种基于多智能体强化学习的任务卸载策略，通过部署Cybertwin智能体为用户设备所需卸载的任务合理分配资源，在保证终端设备的服务质量(QoS)要求的同时，最小化整个计算网络的总成本。首先联合设计Cybertwin智能服务代理、计算任务分配以及网络通信与算力等多维异构资源配置构建随机对策的马尔可夫博弈过程(MGP),使执行总延迟和总能耗之和最小。其次考虑到需要处理随机时变网络与动态资源请求的高维连续动作空间，采用了一种基于多智能体双延迟深度确定策略梯度(MATD3)的深度强化学习协同框架求解。仿真实验结果表明：与常见的单智能体学习算法和启发式方案相比，本文提出的MATD3方法具有较好的性能，在平均执行成本方面分别降低了25.61%和35.79%,在任务卸载率上分别提高了39.13%和77.76%。更多还原

【Abstract】 To address the limited resources of intelligent mobile terminal devices and the need for cloud-edge collaboration capability in future network architectures, a task offloading strategy based on multi-agent reinforcement learning is proposed, which minimizes the total cost of the entire computing network by deploying Cybertwin intelligences to reasonably allocate resources for the tasks to be offloaded by user devices while ensuring the quality of service(QoS) requirements of end devices. Firstly, we jointly design Cybertwin intelligent service agent, computing task allocation, network communication, computing power and other multidimensional heterogeneous resource allocation to construct a Markov game process(MGP) of stochastic game, so as to minimize the sum of total execution delay and total energy consumption. Secondly, considering the need to deal with the high-dimensional continuous action space with stochastic time-varying networks and dynamic resource requests, a deep reinforcement learning collaborative framework based on multi-agent twin delayed deep deterministic policy gradient algorithm(MATD3) is used to solve the problem. The simulation results show that compared with the common single-agent learning algorithms and heuristic schemes, the MATD3 method proposed in this paper has better performance, with 25.61% and 35.79% reduction in average execution cost and 39.13% and 77.76% improvement in task offloading rate, respectively.更多还原