【作者】 廖一芃； 施嘉璇； 刘鹃； 马玉敏；

【Author】 Yipeng Liao;Jiaxuan Shi;Juan Liu;Yumin Ma;College of Electronic and Information Engineering,Tongji University;Shanghai Research Institute for Intelligent Autonomous Systems,Tongji University;

【机构】 同济大学电子与信息工程学院； 同济大学,上海自主智能无人系统科学中心；

【摘要】 在智能制造中,生产车间调度是优化生产过程、提高生产效率的关键环节。传统的集中式调度在处理大规模任务时常遇到性能瓶颈和扩展性问题。因此,本文提出一种基于多智能体强化学习的分布式调度方法,将调度决策功能下放至单元级,优化生产资源分配。首先为车间制造单元构建基于深度递归Q网络(DRQN)的调度智能体,设计了描述车间单元结构状态的局部状态空间;然后在车间级构建混合网络,设计全局奖励函数,通过多智能体强化学习QMIX算法协调单元调度智能体间的合作实现单元性能和车间整体性能的优化;最后,通过仿真实验证明了基于QMIX的分布式调度方法(QMIX-DSM)相较于基于DRQN的集中式调度方法(DRQN-CSM)在最大完工时间和机器利用率指标上的优越性,同时在不同问题规模上表现出更好的泛化能力。更多还原

【Abstract】 In intelligent manufacturing,production shop scheduling is the key link to optimize the production process and improve the production efficiency.Traditional centralized scheduling often encounters performance bottlenecks and scalability problems when dealing with large-scale tasks.Therefore,a distributed scheduling method based on multi-agent reinforcement learning is proposed in this paper.The scheduling decision-making function is delegated to the unit level,and the production resource allocation is optimized.Firstly,a scheduling agent based on Deep Recurrent Q-network(DRQN) is constructed for each manufacturing unit in the shop floor,and a local state space describing the structural state of workshop cells is designed.Then,a mix network is constructed at the workshop level,and the global reward function is designed.The multi-agent reinforcement learning QMIX algorithm is used to coordinate the cooperation between the unit scheduling agents to optimize the unit performance and the overall workshop performance.Finally,simulation experiments prove that the distributed scheduling method based on QMIX(QMIX-DSM) is superior to the centralized scheduling method based on DRQN(DRQN-CSM) in the maximum completion time and machine utilization index,and shows better generalization ability in different problem scales.更多还原