【摘要】 针对空天地一体化网络信息物理系统模型复杂、很难获得网络拓扑先验知识和模型化假设的特点，研究其基于深度强化学习的垂直切换策略。首先，综合考虑系统稳定性、切换开销和网络使用成本约束，将垂直切换策略问题建模为约束马尔可夫决策过程（CMDP），并给出保证可行解存在的充分条件；其次，提出约束-近端策略优化（CPPO）算法解决该问题，并在基站侧引入分布式强化学习机制加速训练收敛。相较于基准策略，仿真验证了所提垂直切换策略的优越性和有效性。更多还原

【Abstract】 The vertical handover policy of space-air-ground integrated cyber-physical systems based on deep reinforcement learning was studied, in which the challenges of complicated network model and difficulties in acquiring prior knowledge for network topology and model were addressed. By jointly taking the system stability, handover cost and network-using cost into account, the vertical handover policy problem was modeled as a constraint Markov decision process(CMDP), and a sufficient condition to ensure the existence of a feasible solution was derived. Furthermore, a constraint-proximal policy optimization(CPPO) algorithm was proposed to solve the CMDP, and also the distributed learning scheme at base station sides was introduced to accelerate the speed of converging. Simulation results verify the validation and superiority of the proposed vertical handover policy as compared with the baselines.更多还原