【作者】 王茜；

【导师】 殷国栋； 董钊志；

【作者基本信息】 东南大学 ， 车辆工程（专业学位）， 2021， 硕士

【摘要】 主动悬架系统因对不同路面与外界扰动的实时适应能力,具有较强的车辆平顺性控制潜力,近年来得到了汽车厂商与学者的广泛研究。传统基于模型的控制理论一方面依赖于数学模型的精度,另外一方面为了保证系统的实时性通常采用线性模型,这导致其在实际悬架控制中具有较大的局限性。而强化学习方法基于数据驱动,且不依赖于严格的数学模型,在主动悬架控制中具有较强的应用潜力。本论文为解决传统悬架控制方法的参数或工况适应性差问题,围绕主动悬架的深度强化学习控制方法开展研究,设计主动悬架深度Q神经网络(DQN)强化学习算法与面向半车的主动悬架深度确定性策略梯度(DDPG)算法,并进行随机道路条件的车辆悬架控制仿真测试,以期减小车身振动,提高车辆行驶平顺性。本文的研究内容包括:首先,建立主动悬架二自由度、半车与整车的动力学模型与路面激励模型,搭建用于与强化学习智能体交互的车辆悬架系统仿真环境,构建悬架状态-动作空间对应的系统动态响应数据集。其次,构建主动悬架深度强化学习控制问题,设计面向减速带的主动悬架DQN算法,提出车身加速度、轮胎动行程与悬架动挠度最小化的奖励函数,研究学习率、折扣因子、神经网络架构等参数对悬架控制策略训练效果的影响,以优化强化学习训练速度与悬架控制性能,求解兼顾舒适性和操纵稳定性的主动悬架最优控制策略。仿真结果表明,相比于传统悬架控制策略,基于DQN的主动悬架算法具有更好的平顺性与工况适应能力。然后,为提高强化学习在自由度更高(更大状态-动作空间)的半车主动悬架训练中收敛速度,提出基于DDPG的主动悬架控制策略,综合考虑车身加速度、俯仰角加速度等半车悬架性能指标设计奖励函数,进行随机路面条件下的控制策略训练与平顺性仿真试验。仿真结果表明,基于DDPG的主动悬架控制策略相比于DQN算法具有较快的收敛速度,不同道路条件与行驶车速的测试结果验证了所提算法的泛化性能。最后,为验证本文所设计的主动悬架深度强化学习控制策略的可行性,搭建基于dSPACE实时仿真系统的主动悬架硬件在环仿真试验平台,利用MicroAutoBox模拟二自由度悬架系统、路面激励和强化学习控制算法,以电磁作动器为执行器输出主动控制力,试验结果证明所提出的深度强化学习控制算法可以有效提高汽车的平顺性。更多还原

【Abstract】 The active suspension system has a strong potential for vehicle ride comfort control due to its real-time adaptability to different road surfaces and external disturbances.In recent years,it has been extensively studied by automobile manufacturers and scholars.On the one hand,traditional model-based control theory relies on the accuracy of the mathematical model,and on the other hand,linear model is usually used to ensure the real-time performance of the system,which leads to its great limitations in the actual suspension control.The reinforcement learning method is based on data-driven and does not depend on strict mathematical models,and has strong application potential in active suspension control.In order to solve the problem of poor adaptability to the parameters or working conditions of traditional suspension control methods,Research on the deep reinforcement learning control method of active suspension was conducted,and Deep Q-network(DQN)reinforcement learning algorithm of active suspension and half-car active suspension-oriented Deep Deterministic Policy Gradient(DDPG)algorithm were designed.The simulation test of vehicle suspension control under random road conditions was carried out,in order to reduce vehicle vibration and improve vehicle ride comfort.The research contents of this thesis include:Firstly,the dynamic models of the 2-DOF,semi-vehicle and full-vehicle active suspension and road excitation model were established.The vehicle suspension system simulation environment for interaction with reinforcement learning agents was built,and the system dynamic response data set corresponding to the suspension state-action space was constructed.Secondly,active suspension deep reinforcement learning control problem was constructed,an active suspension DQN algorithm for speed bumps was designed,a reward function that minimizes body acceleration,tire dynamic travel and suspension dynamic deflection was proposed.The influence of learning rate,discount factor,neural network architecture and other parameters on the training effect of suspension control strategy was studied,so as to optimize the reinforcement learning training speed and suspension control performance,and solve the optimal control strategy of active suspension considering both comfort and handling stability.Simulation results show that,compared with the traditional suspension control strategy,the DQN-based active suspension algorithm has better ride comfort and adaptability to working conditions.Then,in order to improve the convergence speed of reinforcement learning in the semicar active suspension training with higher degrees of freedom(greater state-action space),an active suspension control strategy based on DDPG is proposed,which comprehensively considers vehicle body acceleration,pitch acceleration,etc.Reward function for vehicle suspension performance index was designed,control strategy training and ride comfort simulation test under random road conditions were conducted.Simulation results show that the DDPG-based active suspension control strategy has a faster convergence rate than the DQN algorithm.The test results of different road conditions and driving speeds verify the generalization performance of the proposed algorithm.Finally,in order to verify the feasibility of the active suspension deep reinforcement learning control strategy designed in this thesis,an active suspension hardware-in-the-loop simulation test platform based on the dSPACE real-time simulation system was built.Micro AutoBox was used to simulate the 2-DOF suspension system,road excitation and the reinforcement learning control algorithm.The electromagnetic actuator was used as the actuator to output active control force.The experimental results prove that the proposed deep reinforcement learning control algorithm can effectively improve the ride comfort of the vehicle.更多还原