【摘要】 目前对于分布式能源集群调度的研究大多局限于单一场景，同时也缺少高效、准确的算法。该文针对以上问题提出了一种基于进化算法经验指导的深度强化学习（EARL）的分布式能源集群多场景调度方法。分别对分布式能源集群中的电源、储能、负荷进行个体建模，并基于个体调度模型建立了包含辅助调峰调频的多场景分布式能源集群优化调度模型；基于进化强化学习算法框架，提出了一种EA-RL算法，该算法融合了遗传算法（GA）与深度确定性策略梯度（DDPG）算法，以经验序列作为遗传算法个体进行交叉、变异、选择，筛选出优质经验加入DDPG算法经验池对智能体进行指导训练以提高算法的搜索效率和收敛性；根据多场景调度模型构建分布式能源集群多场景调度问题的状态空间和动作空间，再以最小化调度成本、最小化辅助服务调度指令偏差、最小化联络线越限功率以及最小化源荷功率差构建奖励函数，完成强化学习模型的建立；为验证所提算法模型的有效性，基于多场景的仿真算例对调度智能体进行离线训练，形成能够适应电网多场景的调度智能体，通过在线决策的方式进行验证，根据决策结果评估其调度决策能力，并通过与DDPG算法的对比验证算法的有效性，最后对训练完成的智能体进行了连续60 d的加入不同程度扰动的在线决策测试，验证智能体的后效性和鲁棒性。更多还原

【Abstract】 At present, the research on distributed energy cluster scheduling is mostly limited to a single scenario and lacks efficient and accurate algorithms. Aiming at these problems, this paper proposed a multi-scenario scheduling method for distributed energy clusters based on evolutionary algorithm experience-guided deep reinforcement learning(EA-RL). The individual models of power supply, energy storage and load in distributed energy cluster were established, respectively. Based on the individual scheduling model, a multi-scenario distributed energy cluster optimal scheduling model including auxiliary peak regulation and frequency modulation was established. Based on the framework of evolutionary reinforcement learning algorithm, an EA-RL algorithm was proposed. The algorithm combines genetic algorithm(GA) and deep deterministic policy gradient(DDPG) algorithm. The empirical sequence was used as the individual of genetic algorithm for crossover, mutation and selection. The high-quality experience was selected to join the DDPG algorithm experience pool to guide the training of the agent to improve the search efficiency and convergence of the algorithm. According to the multi-scenario scheduling model, the state space and action space of the multi-scenario scheduling problem of distributed energy cluster were constructed. Then, the reward function was constructed by minimizing the scheduling cost, the deviation of the auxiliary service scheduling instruction, the over-limit power of the tie line and the power difference between the source and the load, and the reinforcement learning model was established. To validate the effectiveness of the proposed algorithm and model, offline training of scheduling agents was conducted based on multi-scenario simulation cases, resulting in agents capable of adapting to various grid scenarios. Verification was carried out through online decision-making, and their scheduling decision-making capabilities were assessed based on decision outcomes. The validity of the algorithm was further verified through comparison with the DDPG algorithm. Finally, the trained agents undergo 60 consecutive days of online decision-making tests incorporating varying degrees of disturbances to validate their posterior effectiveness and robustness.更多还原