【作者】 刘涛；

【导师】 赵占锋；

【作者基本信息】 哈尔滨工业大学 ， 通信工程（含宽带网络、移动通信等）（专业学位）， 2023， 硕士

【摘要】 近年来,低光照、雨雪等复杂场景下的多目标跟踪技术受到了广泛关注。然而,主流的可见光相机跟踪方案在这些复杂场景下的工作效果并不理想。因此,本文借鉴可见光多目标跟踪的基本思想,根据红外图像特点作出相应改进,提出了一种基于检测的红外多目标跟踪算法,在各种具有挑战性的场景下取得了准确、鲁棒的跟踪效果。接着,针对单一传感器存在的感知局限性问题,以改进的红外目标跟踪算法为基础,设计了一种红外相机与毫米波雷达融合的多目标跟踪方案,有效提升了跟踪的可靠性和全面性。具体开展的研究工作如下:(1)研究了红外目标检测和红外特征提取方法。首先,以YOLOv5模型在可见光数据集上训练得到的权重为基础,在红外数据集上进行迁移学习,从而得到适用于红外行人、汽车目标的检测模型,并展现出了优秀的检测效果。接着,针对基于可见光图像构建的表观特征提取模型或无法挖掘到红外同类目标细粒度特征的问题,设计了一种基于孪生网络的红外表观特征提取模型,表现出了良好的红外行人识别能力,较宽残差网络具有一定的跟踪性能优势。(2)研究了基于检测的红外多目标跟踪算法。首先,考虑到将表观特征关联置于高优先级会对红外多目标跟踪的精度和速度产生不利影响,提出了一种运动特征优先的关联机制,并同时利用了高、低置信度的目标检测结果。接着,为加快算法运行速度,在表观关联环节中提出了一种特征抽样提取与更新的策略。最后,针对当前多目标跟踪算法将连续失配容忍帧数设为定值或降低目标长时遮挡后被恢复跟踪概率的问题,提出了一种自适应的连续失配容忍帧数策略。实验结果表明,相较于其他跟踪算法,提出的基于检测的红外多目标跟踪方法在各种复杂场景下取得了更加准确、鲁棒的跟踪效果。(3)研究了基于红外与雷达融合的多目标跟踪方案。首先,针对毫米波雷达原始输出数据存在冗余目标、干扰目标等问题,提出了一系列数据预处理操作。接着,对红外相机与毫米波雷达的时间同步和空间配准进行研究和实践。最后,提出了一种红外与雷达决策级融合的多目标跟踪方案,并设计了一种长时目标关联策略。实验结果表明,提出的红外和雷达融合跟踪方案能克服单一传感器的感知局限性,显著减少漏报次数,提升跟踪的可靠性与全面性。更多还原

【Abstract】 In recent years,multi-object tracking technology in complex scenarios such as weak light,rain and snow has received widespread attention.However,the tracking effect of the tracking method based on visible light camera is not desirable when faced with these complex scenarios.Therefore,we draw on the idea of visible light multi-object tracking and make corresponding improvements according to the characteristic of infrared images,and propose an infrared multi-object tracking method based on detection,which achieves accurate and reliable tracking effects in various challenging scenarios.Furthermore,in order to overcome the limitation of the single sensor,based on the improved infrared object tracking method,we design a multi-object tracking solution that integrates infrared camera and millimeter wave radar,which effectively improves the reliability and comprehensiveness of the tracking.The specific research work of the thesis is as follows:(1)Research on the infrared object detection and infrared feature extraction method.Firstly,we get the pre-trained weights of the YOLOv5 model on the RGB dataset and perform the transfer learning on the infrared dataset,so as to obtain a detection model which is suitable for infrared pedestrians and cars and shows great detection effects.Secondly,considering that the appearance feature extraction models built based on RGB images may not be able to learn fine-grained features of infrared similar objects,we design an infrared appearance feature extraction model based on twin network,which shows excellent infrared pedestrian classification ability and has some tracking performance advantages compared with Wide Residual Net.(2)Research on the detection-based infrared multi-object tracking method.Firstly,considering that prioritizing appearance feature association may adversely affect tracking accuracy and speed,we prioritize the motion feature in association and use both high and low confidence detection results.Secondly,in order to speed up the operation of the algorithm,we propose a strategy of feature sampling extraction and update in the appearance association.Finally,the current multi-target tracking algorithm sets the threshold of continuous lost frames to a fixed value,which may reduce the probability of the tracking through longer periods of occlusion.To solve this problem,we propose an adaptive continuous mismatch tolerant frame strategy.Experimental results show that compared with other tracking methods,the proposed infrared multi-object tracking method achieves more accurate and robust tracking performance in a variety of complex scenarios.(3)Research on the multi-object tracking solution based on the fusion of infrared and radar.Firstly,considering that the original output data of millimeter wave radar have redundant objects and interference objects,we propose a series of data preprocessing operations to solve this problem.Secondly,we study the time synchronization and spatial alignment of infrared cameras and millimeter wave radar and conduct relevant experiments.Finally,we propose a multi-object tracking solution based on the fusion of infrared and radar at the decision level,and we also design a long-term object association strategy.Experimental results show that the proposed fusion tracking solution can overcome the perception limitations of the single sensor,significantly reduces the number of false negatives,and improves the reliability and comprehensiveness of tracking.更多还原