【摘要】 为预防工业场景下人员不安全生产行为所导致的安全事故，解决光线不佳、视野受限和遮挡等干扰情况下单一视觉模态动作识别效果不佳的问题，提出一种基于自适应证据推理(S-ER)策略，融合视频信息和惯性测量元件(IMU)信息的人员不安全行为决策方法。首先，构建基于注意力机制的多任务三维卷积模型(M-C3D)，分析视频信息，运用融合注意力机制的一维卷积神经网络(1D-CNN)处理IMU信息;其次，运用证据推理(ER)策略实现决策级融合，并通过萤火虫优化算法构建不同环境条件下证据权重和可靠度的优化集合，确保视频和传感器模态信息的权重能够根据环境情况自适应调整;最后，通过德克萨斯大学达拉斯分校的多模态人员行为数据集(UTD-MHAD)和中原工学院的多模态人员行为数据集(ZUT-MHAD)验证模型的有效性。结果表明:在存在干扰的工业场景中，S-ER方法的识别准确率最高可达98.53%，较传统多模态融合方法及单模态识别方法的最高值提升17.52%。更多还原

【Abstract】 In order to reduce the accidents in industrial scenarios which were caused by workers’ unsafe operation behaviors,meanwhile improve the performance of visual-based action recognition methods in industrial scene with poor lighting,limited field of view and occlusions,an improved decision-making strategy based on self-adaptive ER( S-ER) was introduced in this paper. This strategy could integrate video information and inertial measurement unit( IMU) information effectively. It firstly analyzed video information and IMU information with attention mechanism-based multi-task convolutional 3D( M-C3D)model as well as one-dimensional convolutional neural network( 1D-CNN) fused with attention mechanism,then ER theory was introduced to achieve decision-level fusion,where the set of evidence weights and reliability under different environmental conditions was optimized through the firefly optimization algorithm for improving the recognition accuracy and robustness of the model. The effectiveness of the proposed algorithm was verified on the public dataset Multimodal Human Action Dataset from University of Texas at Dallas( UTD-MHAD) and the self-built dataset Multimodal Human Action Dataset from Zhongyuan University of Technology( ZUT-MHAD). The results show that the identification results of S-ER for workers ’ unsafe behaviors in complex industrial scenarios can reach up to 98. 53%,which is17. 52% higher than the maximum value of traditional multimodal fusion methods and single-modality recognition methods.更多还原