【作者】 王睿；

【导师】 雷印杰；

【作者基本信息】 四川大学 ， 信息与通信工程， 2022， 硕士

【摘要】 行为早识别,是指针对摄像头捕捉到的一段影像的前一部分序列进行完整动作的分析和预测的过程。然而,现在的行为早识别任务聚焦于单一预测,忽略了不完整的部分序列可能有不同的运动趋势,并可能发展为不同的未来轨迹这一事实。因此,本文兼顾到多种可能性,即输出多个有可能的动作,以预测完整的事件或动作。这样,本文的结果才是真实可靠且符合实际的合理预测。本文针对只能观测到前序序列的骨骼数据,注重考虑实际情况下存在的多种结果,设计并提出了几种多样性行为早识别方法,并为提高准确率进行了不同角度的改进,设计了相关对应的损失函数。最后,这些方法提供了多样化、适当和明智的行动类别或行动轨迹,从而能够使得终端用户及时正确地作出反应,从而作出最终决策。主要研究内容如下:(1)本文构建了对应的多样化数据集。为了造就更加符合实际的、合理的多样性行为早识别数据集,除了对原有的数据进行了归一化处理并探究了相关的数据处理方法,分别对骨骼数据样本在不同的观测比下(0.1-1)进行相对序列比较,得出了所有观测比下最合理的多样性行为标签集。(2)基于现有行为早识别方法对数据进行有效地时空信息学习,针对输入的原始骨骼数据,分别处理成时差分坐标骨骼数据和结构差分骨骼数据,并建立能够处理时空多样性行为识别和早识别算法框架。(3)在捕捉时空信息的基础上,设计多分支结构多样性行为早识别方法。网络参数随机化并共享部分网络能够保证网络效果的同时减少参数量。利用“最优化损失函数”,训练网络所有分支的输出自发地对应标签集,使得网络分支学习到不同而多样的动作类;最后设计了序列译码器辅助生成部分序列的全序列,有助于网络进行弱监督学习。该算法具有实现简单、训练时间短、鲁棒性强和适应实际情况变化的特点。(4)在捕捉时空信息的基础上,设计了基于深度特征插值的可用于弱监督的多样性行为早识别方法,实现了单一网络下在参数完全共享的同时通过调整模式能够实现多样性预测。在训练时采用随机打乱过的有标签样本和无标签样本骨骼数据分批次交替输入,针对不具有监督信息的骨骼数据采用全序列监督优化分类结果;同时,将代表不同深度特征信息插值,输出对应的不同动作类结果,被称为模式转换器的模式转化。该算法具有模型小、测试时间短、鲁棒性强和准确率高的特点。更多还原

【Abstract】 Human early action recognition,refers to recognizing and analyzing a partial sequence which captured by camera before it is fully performed.However,methods now focus on deterministic early action recognition ignoring an incomplete partial sequence may have different movement trends and could develop as different future trajectories.Therefore,we need to consider multiple possibilities,outputting multiple possible actions corresponding to the complete event or action.In this way,it is reasonable and realistic to output diverse classes for early action recognition in this case considering uncertainties and diversities.In this thesis,aiming at the skeleton-based data that can only be observed in the preorder sequence,and paying attention to the various results existing in the actual situation,several early recognition methods of diversity behavior are designed and proposed,and the corresponding loss function is designed to improve the accuracy.Finally,these methods provide a variety of appropriate and informed categories or trajectories of action that enable the end user to respond correctly and in a timely manner to make the final decision.The main research contents are as follows:(1)This thesis constructs corresponding diversified data sets.In order to make a much more realistic and rational action prediction data sets,this thesis explores the related data processing method in addition to the original data normalization processing.Skeleton data samples under different observation ratios(0.1-1)are relatively compared to corresponding full skeleton sequence,until the whole dataset matches diversified action set which are the most similar and reasonable.(2)Based on the existing early action recognition methods which could effectively capture spatio-temporal information,the inputted skeleton sequences are respectively processed into time-difference coordinate skeleton data and structuredifference skeleton data.Meanwhile,an algorithm framework is established to deal with spatio-temporal diversity of action recognition and early action recognition.(3)Diversified early action recognition based on multiple branches is designed.Firstly,the parameters are randomized and partly shared,thus the number of parameters is reduced while the effect of the network is guaranteed.Secondly,the "nearest loss function" is used to train all the branch outputs of the network to spontaneously match the action set,so that the branches of the network can learn different and diverse action classes.Finally,a sequence decoder is designed to help generate construct full sequences of partial the sequence,which could realize semi supervised setting.The algorithm has the characteristics of simple implementation,short training time,strong robustness and adaptability to actual situation changes.(4)Diversified early action recognition based on mode selector is designed.Different from the former one,this method fully shares parameters in a single network by mode conversion.According to alternatively input random batches with/without class labels,the network could still be optimized under semi-supervised skeleton sequence.The algorithm has the characteristics of small model,short test time,strong robustness and high accuracy.更多还原