【作者】 张海峰；

【导师】 梁华刚；

【作者基本信息】 长安大学 ， 工程硕士 控制工程（专业学位）， 2021， 硕士

【摘要】 云制造是一种基于云计算、物联网以及大数据等新兴技术发展起来的面向服务的制造模式。它集成了分布式的制造资源并进行集中管理,以向消费者提供按需制造服务,从而促使资源的全面共享,提高资源的利用效率。云制造中一个重要问题是组合多个不同的制造服务构成增值服务以满足用户多阶段的复杂制造需求,即服务组合问题。在确定了子任务加工顺序的服务组合中使用启发式和元启发式算法不仅会降低算法的灵活性,而且会出现当任务或服务的规模发生变化、服务不可用时需要人工调节算法参数等问题。因此,论文通过引入深度强化学习(Deep Reinforcement Learning,DRL)算法来解决面向单复合任务的制造云服务组合问题。DRL算法是一种人工智能算法,其结合了深度学习对复杂环境的感知能力和强化学习对问题的决策能力,该算法通过学习不同服务组合之间固有的模式和规则,从而寻找一种可解决云制造服务组合(Cloud manufacturing service composition,CMfg-SC)问题的有效方案。深度强化学习算法分为两种类型:基于离散变量的算法和基于连续变量的算法。在CMfg-SC问题中,不仅需要考虑当前子任务对应的最优服务,同时需要考虑在满足任务服务质量(Quality of Service,Qo S)约束的情况下,实现面向该任务的组合结果的整体Qo S最大化。因此,论文将基于连续变量的深度确定性策略梯度(Deep Deterministic Policy Gradient,DDPG)算法引入,以解决云制造环境中的服务组合问题。首先,提出了一种考虑物流服务质量的CMfg-SC模型,包括单个复合任务模型、资源模型及物流服务模型,并结合相应的组合指标。其次给出了模型的目标函数及约束条件,并依据该模型建立了马尔科夫决策过程(Markov Deecision Progress,MDP)。最后采用DDPG算法来求解仿真案例下的CMfg-SC问题。为了验证该算法的有效性以及面向复杂的制造任务时当生产制造过程中包含多种不确定性的情况下算法的应变能力,通过实验依次分析了算法在仿真环境下的有效性、当服务不可用时算法的自适应性及当任务和服务的规模发生变化时算法模型的可扩展性能。为了直观验证算法的性能,论文以蚁群算法(Ant Clony Optimization,ACO)作为基准,并通过与深度Q学习(Deep Q-Network,DQN)算法做对比,来验证本文所提算法在各方面的性能。实验结果表明,论文提出来的方法不仅能有效解决大规模的CMfg-SC问题。同时,在动态制造环境下,当服务不可用时,论文所提算法相较于基于值的DQN算法有着更好的自适应性。最后,分别对任务集与资源集进行扩展并实验验证,其结果表明,使用DDPG算法可以有效解决在动态变化环境下面向单复合任务的制造云服务组合问题,而使用蚁群算法来解决这类问题的时候,不仅需要人为调参,无法有效扩展,且通过论文所提算法训练得到的模型,更加适用于解决云制造环境下的服务组合问题。更多还原

【Abstract】 Cloud manufacturing is a service-oriented manufacturing model based on emerging technologies such as cloud computing,Internet of Things,and big data.It integrates distributed manufacturing resources and conducts centralized management to provide consumers with on-demand manufacturing services,thereby promoting comprehensive resource sharing and improving resource utilization efficiency.An important issue in cloud manufacturing is to combine value-added services composed of multiple different manufacturing services to meet the multistage complex manufacturing needs of users,that is,the problem of service composition.The use of heuristic and meta-heuristic algorithms in the service composition that determines the processing order of subtasks will not only reduce the flexibility of the algorithm,but also cause problems such as manual adjustment of algorithm parameters when the scale of the task or service changes and the service is unavailable.Therefore,the paper introduces deep reinforcement learning(DRL)algorithms to solve the single composite task-oriented manufacturing cloud service composition problem.Deep reinforcement learning algorithm is an artificial intelligence algorithm that combines deep learning’s ability to perceive complex environments and reinforcement learning’s ability to make decisions on problems.The algorithm learns the inherent patterns and rules between different service combinations to find an effective solution to the problem of cloud manufacturing service composition.Deep reinforcement learning algorithms are divided into two types: algorithms based on discrete variables and algorithms based on continuous variables.In the cloud manufacturing combination problem,not only the optimal service corresponding to the current subtask needs to be considered,but also the overall quality of service(Qo S)for the combined result of the task needs to be considered in the case of meeting the Qo S constraints of the maximize task requirements.Therefore,the paper introduces the Deep Deterministic Policy Gradient(DDPG)algorithm based on continuous variables to solve the service composition problem in the cloud manufacturing environment.First,a cloud manufacturing service combination model that considers the quality of logistics services is proposed,including a single composite task model,a resource model,and a logistics service model,combined with corresponding combination indicators.Secondly,the objective function and constraint conditions of the model are given,and Markov Deecision Progress(MDP)is established based on the model.Finally,the DDPG algorithm is used to solve the cloud manufacturing service composition problem under the simulation case.In order to verify the effectiveness of the deep deterministic policy gradient algorithm and the ability to adapt to complex manufacturing tasks and the large amount of uncertainty in the manufacturing process under the cloud manufacturing environment,this paper sequentially carried out the effectiveness,adaptability and robustness of the algorithm.In order to verify the performance of the algorithm intuitively,this article uses the ant colony algorithm as a benchmark and compares it with the Deep Q-Network(DQN)algorithm to verify the performance of the algorithm used in this article in all aspects.The experimental results show that the deep deterministic policy gradient algorithm can effectively solve the problem of large-scale cloud manufacturing service composition.At the same time,in a dynamic manufacturing environment,when the service is unavailable,the algorithm proposed in the paper has better adaptability than the value-based DQN algorithm.Finally,the task set and resource set are expanded and experimentally verified.The results show that the use of DDPG algorithm can effectively solve the problem of manufacturing cloud service composition to single composite tasks in a dynamically changing environment,and the use of ant colony algorithm to solve this kind of problem When there is a problem,it not only requires manual parameter adjustment and cannot be effectively expanded,but the model trained through the algorithm proposed in the paper is more suitable for solving the service composition problem in the cloud manufacturing environment.更多还原