【作者】 张潇；

【导师】 常发亮；

【作者基本信息】 山东大学 ， 控制科学与工程， 2022， 硕士

【摘要】 随着时代的发展,车辆向着智能化的方向发生改变,智能驾驶和自动驾驶技术受到了广泛的关注。车载相机作为车辆感知外界环境的重要传感器之一,具有成本低、收集数据简单方便等优点。在智能驾驶辅助系统中,识别车辆前方发生的交通事件能够对驾驶者进行提醒;在自动驾驶系统中,识别车辆前方发生的交通事件能够帮助车辆更好地进行运动规划。因此,基于车载相机拍摄的数据实现交通事件识别功能具有极为重要的意义。当前交通事件识别的研究主要在道路监控相机的视角下进行,相比之下,车载相机由于跟随车辆运动,存在拍摄视频背景不断发生变化的缺点,这为基于车载相机的交通事件的准确识别带来了困难。本文针对在车载相机多样化背景下交通事件难以准确识别的问题提出了视频分类和视频描述两种方法,实现了交通场景下交通事件的视频分类和视频描述,提高了多样化背景下交通事件视频分类识别和描述识别的准确度,本文主要的工作和创新点如下:(1)针对交通事件视频分类识别任务和交通事件视频描述识别任务数据集匮乏的问题,本文从其他交通数据集和本实验室智能驾驶车辆录制的数据中选取交通视频片段,分别构建了交通事件视频分类识别数据集和交通事件视频描述识别数据集。交通事件视频分类识别数据集包含了车载相机记录的680个交通视频片段,包括前方正常行驶、前方交通拥堵、前方车辆变道和前方行人经过四类。交通事件视频描述识别数据集包含了车载相机记录的800个交通视频片段,每个视频片段标注了 7个英文描述,涵盖了道路上常见的超车、变道、行人、停车、交通事故等交通场景。(2)针对多样化交通背景下交通事件视频分类识别困难的问题,本文提出了一种基于检测目标引导的交通事件视频分类网络。首先,本文在残差网络的基础上添加检测目标引导模块以去除复杂多样的交通背景,使网络更好地关注交通场景中的交通参与者;然后,本文将注意力机制引入特征提取的过程中,在残差网络中加入通道注意力和空间注意力,从而赋予网络关注更重要的特征的能力;最后,本文为网络设计了时序建模模块,使用ConvLSTM和时移操作赋予网络捕捉视频帧间时序特征的能力。该网络通过去除交通背景和时空建模有效地提高了多样化交通背景下交通事件视频分类的准确率。(3)视频描述技术可以增强车辆对交通场景的理解,从而提高车辆的通行效率和安全性。针对交通领域中难以对车载相机录制的背景多变的交通视频进行准确描述的问题,本文提出了一种应用于交通领域的基于上下文推理引导的交通事件视频描述网络。该网络以编码器解码器为基础框架,首先,对交通场景中的交通参与者和交通背景进行分离,使用编码器对交通视频进行编码;然后,本文在解码器中加入软注意力模块以更有效地利用编码器编码出的特征,同时引入层归一化提高解码器网络的性能;最后,为了使解码器能够结合上下文对描述进行推理,本文设计了上下文推理引导模块,增强了网络利用上下文生成描述的能力。该网络通过去除交通背景和上下文引导训练有效地提高了交通事件视频描述的准确性。本文分别在构建的交通事件视频分类数据集和交通事件视频描述数据集上对提出的方法进行了验证,从识别准确率和视频描述指标上与其他方法进行了对比,证明了本文提出方法的有效性。更多还原

【Abstract】 With the development of the times,vehicles are changing in the direction of intelligence,and smart driving and autonomous driving technologies are receiving a lot of attention.As one of the important sensors for sensing the external environment,in-vehicle cameras have the advantages of low cost and simple and convenient data collection.In intelligent driving assistance systems,the recognition of traffic events in front of the vehicle can alert the driver;in autonomous driving systems,the recognition of traffic events in front of the vehicle can help the vehicle to plan its movement better.Therefore,it is of great importance to implement traffic event recognition based on the data captured by in-vehicle cameras.Current research on traffic event recognition is mainly carried out from the perspective of road surveillance cameras.In contrast,in-vehicle cameras have the disadvantage of constantly changing video backgrounds as they follow the movement of vehicles,which makes it difficult to accurately identify traffic events based on in-vehicle cameras.This paper proposes two methods of video classification and video description for the problem of difficult accurate recognition of traffic events in the diverse background of in-vehicle cameras,realizing video classification and video description of traffic events in traffic scenes and improving the accuracy of video classification recognition and description recognition of traffic events in diverse backgrounds.(1)To address the lack of datasets for the traffic event video classification recognition task and the traffic event video description recognition task,this paper selects traffic video clips from other traffic datasets and data recorded by intelligent driving vehicles in this laboratory,and constructs the traffic event video classification recognition dataset and the traffic event video description recognition dataset respectively.The traffic event video classification recognition dataset contains 680 traffic video clips recorded by in-vehicle cameras,including four categories:normal traffic ahead,traffic congestion ahead,vehicle changing lanes ahead and pedestrians passing ahead.The traffic event video description recognition dataset contains 800 traffic video clips recorded by in-vehicle cameras,each video clip is labeled with 7 English descriptions,covering common traffic scenarios such as overtaking,lane changing,pedestrian,parking and traffic accidents on the road.(2)To address the problem of difficult traffic event video classification recognition in diverse traffic backgrounds,this paper proposes a traffic event video classification network based on detection target guidance.First,this paper adds a detection target guidance module to the residual network to remove the complex and diverse traffic backgrounds,so that the network can better focus on the traffic participants in the traffic scenes;then,this paper introduces the attention mechanism into the feature extraction process by adding channel attention and spatial attention to the residual network,so as to give the network the ability to focus on more important features;finally,this paper designs a temporal Finally,a temporal modelling module is designed for the network,using ConvLSTM and time-shifting operations to give the network the ability to capture temporal features between video frames.The network effectively improves the accuracy of video classification of traffic events in diverse traffic contexts by removing the traffic context and temporal modelling.(3)Video description techniques can enhance vehicle understanding of traffic scenes,thereby improving vehicle access efficiency and safety.To address the problem that it is difficult to accurately describe traffic videos recorded by in-vehicle cameras with diverse backgrounds in the traffic domain,this paper proposes a contextual inference-guided traffic event video description network applied to the traffic domain.The network is based on an encoder-decoder framework.First,the traffic participants and the traffic context in the traffic scene are separated and the traffic video is encoded using an encoder;then,a soft attention module is added to the decoder to make more effective use of the features encoded by the encoder,and layer normalization is introduced to improve the performance of the decoder network;finally,in order to enable the decoder to combine the description with the context Finally,in order to enable the decoder to reason about descriptions in context,a contextual inference bootstrap module is designed to enhance the network’s ability to generate descriptions using context.The network effectively improves the accuracy of traffic event video descriptions by removing the traffic context and context-guided training.The proposed method is validated on the constructed traffic event video classification dataset and the traffic event video description dataset respectively,and compared with other methods in terms of recognition accuracy and video description metrics to demonstrate the effectiveness of the proposed method in this paper.更多还原