【摘要】 包含超大规模分布式能源、负荷以及冷热电联产的泛在电力物联网的综合能源系统是未来发展趋势.由于泛在物联将给电网带来强的随机扰动问题,传统的自动发电控制(automatic generation control, AGC)方法已无法满足如此大规模复杂综合能源系统的频率稳定.机器学习是解决复杂能源系统AGC强随机扰动的一种有效方法.然而这种超大规模的泛在物联将给AGC求解带来维数灾问题.本文针对DDQN-AD(double deep Q networkaction discovery)算法中经验缓存机制构建问题,提出了一种基于比例优先级采样机制的深度强化学习算法PRDDQN-AD(prioritized replay DDQN-AD),以解决机器学习中多维状态-动作对的维数灾问题,进而解决泛在电力物联网综合能源系统模式下的随机扰动问题.对源网荷储协同的两区域综合能源系统模型和集成了大量源、网、荷、储及冷热电联产的多区域泛在电力物联网综合能源系统模型进行仿真.结果表明,与改进前的DDQNAD算法相比, PRDDQN-AD能够提升训练样本的质量,具有良好的学习效率和泛化性能,能够解决维数灾问题;与其他智能算法相比,其收敛速度和控制性能均有明显提升,可获得区域最优协同控制.更多还原

【Abstract】 The integrated energy systems are developing in the direction of ubiquitous power Internet of Things(IoT). The main feature is the large-scale integration of distributed energies, loads, and cogenerations, which usually brings random disturbances to the systems,thus causing frequency stability control problems, where cannot be effectively addressed by the traditional automatic generation control methods. The recently developed machine learning approach provides potential solutions for complex systems with random disturbances. However, when this approach is applied to the ultra-large-scale ubiquitous power IoT systems, the dimensionality related problem arises, and it should be solved. In this paper, a deep reinforcement learning algorithm is developed for the frequency stability control of the ultra-large-scale ubiquitous power IoT systems with random disturbances. The developed algorithm is based on the idea of a proportional priority sampling mechanism and the prioritized replay DDQN-AD(PRDDQN-AD) strategy. In this work,both the two-region integrated energy system model and the multi-regional ubiquitous power IoT integrated energy system model are adopted in simulation and analysis; these models include a large number of sources, loads, energy-storage units, and grids. Simulation and comparison results show that the training quality of samples, learning efficiency, and generalization performance of the strategy are improved by using PRDDQN-AD. The strategy has a fast convergence speed, and thus can successfully solve the dimensionality problem.更多还原