【作者】 孙玮；

【导师】 徐超； 王一夫；

【作者基本信息】 天津大学 ， 电子信息， 2023， 硕士

【摘要】 在计算机视觉领域中,人体动作识别一直是一个重要的研究课题,动作识别技术被广泛地应用于工作、娱乐和生活的各个领域,具有广阔的应用前景。然而当前多数识别方法不能充分地提取动作的时空特征,具体表现为在时间维度上难以捕获时序中的远距离依赖关系,在空间维度上感受野固定难以提取信息量丰富的特征信息。此外,当前多数方法难以精确关注到与识别动作最相关的特征。为了有效提取动作特征并捕捉关键时空信息,本文分别从以下两方面展开工作。第一,本文提出了一种多尺度时空特征增强图卷积网络(MS-STEGCN)。网络通过不同大小的滤波器对人体骨架动作数据提取多尺度特征,来增强骨架序列的特征表示。网络的基本组成模块包括两个子模块,分别为多尺度空间特征增强模块(MS-SEB)和多尺度时间特征增强模块(MS-TEB)。MS-SEB对一帧内的骨架图使用不同尺度的图卷积来提取多尺度空间信息。MS-TEB对多帧的骨架序列使用不同卷积核大小和膨胀系数的时序卷积来提取多尺度时间信息。第二,本文提出了一种基于多重注意力机制的图卷积网络(MA-GCN)。该网络是在MS-STEGCN的基础上集成了多个注意力机制实现的。网络包括三种注意力机制模块,分别为多尺度注意力模块、空间注意力模块和时间注意力模块。多尺度注意力模块从通道维度对不同尺度的动作特征进行权重分配;空间注意力模块从空间维度自适应地学习骨架图的关键拓扑特征;时间注意力模块从时间维度捕捉骨架序列中的关键帧。MA-GCN在多尺度特征融合的基础上集成了多重注意力机制,进一步增强了动作的特征表示。本文从多尺度时空特征和多重注意力机制的角度对动作识别算法进行研究,并将所提出的MS-STEGCN和MA-GCN网络在NTU-RGB+D和Kinetics数据集上进行了一系列实验。实验结果表明本文提出的网络模型与当前先进方法相比均有提升,验证了模型的有效性和优越性。更多还原

【Abstract】 In the field of computer vision,human action recognition has been an important research topic,and action recognition technology is widely used in various fields of work,entertainment and life,with broad application prospects.However,most current recognition methods cannot adequately extract the spatio-temporal features of actions,specifically in the temporal dimension,it is difficult to capture the long-distance dependencies in the temporal sequence,and in the spatial dimension,it is difficult to effectively capture the features of both fine-grained and coarse-grained actions in the perceptual field fixation.In addition,most current methods have difficulty in precisely focusing on the features most relevant to the identified actions.In order to effectively extract action features and capture key spatio-temporal information,this thesis works on the following two aspects,respectively.First,this thesis proposes a multi-scale spatio-temporal feature-enhanced graph convolution network(MS-STEGCN).The network extracts multiscale features from human skeleton action data by filters of different sizes to enhance the feature representation of skeleton sequences.The basic components of the network include two sub-modules,namely,the multi-scale spatial feature enhancement module(MS-SEB)and the multi-scale temporal feature enhancement module(MS-TEB).MS-SEB extracts multi-scale spatial information by using different scales of graph convolution for skeleton maps within a frame.MS-TEB uses temporal convolution with different convolution kernel sizes and expansion coefficients for skeleton sequences in multiple frames to extract multi-scale temporal information.Second,this thesis proposes a graph convolution network(MA-GCN)based on multiple attention mechanism.The network is implemented by integrating multiple attention mechanisms on the basis of MS-STEGCN.The network includes three attention mechanism modules,namely,the multiscale attention module,the spatial attention module and the temporal attention module.The multiscale attention module assigns weights to action features at different scales from the channel dimension;the spatial attention module adaptively learns key topological features on the skeleton graph from the spatial dimension;and the temporal attention module captures key frames in the skeleton sequence from the temporal dimension.MA-GCN integrates multiple attention mechanisms based on multiscale feature fusion to further enhance the feature representation of actions.In this thesis,we investigate action recognition algorithms from the perspectives of multiscale spatio-temporal features and multiple attention mechanisms,and we conduct a series of experiments on the proposed MS-STEGCN and MA-GCN networks on NTURGB+D and Kinetics datasets.The experimental results show that the proposed models in this thesis are both improved compared with the current state-of-the-art methods,which validate the effectiveness and superiority of the models.更多还原