【作者】 李哲；

【导师】 张虎林； 周传刚；

【作者基本信息】 太原理工大学 ， 通信工程（含宽带网络、移动通信等）（专业学位）， 2023， 硕士

【摘要】 由于科学技术的不断发展和人工智能的进步,地下采煤的方法正逐渐从传统的人工方法转变为采矿机器人。但是对于采矿机器人的研究提升仍然有很大的空间。近些年来不断有学者提出将煤矿井下的作业智能化、机械化和人性化来提升井下开采的效率和安全,那么在采矿机器人工作的同时井下人员的安全成为亟需解决的问题,在机器掘进的同时避免事故的发生是非常重要的。随着深度学习的发展,对于物体识别的准确率也在一步一步的提高,尤其是近些年来提出的一系列YOLO（You Only Look Once）检测算法,使得保证检测准确度的同时,加快了检测的速度。因此,将YOLO检测算法部署到智能掘进机器人系统上充当智能掘进机器人的视觉系统具有极大的意义。视频的关键帧处理和多模态图像的融合在井下人员检测的应用中逐渐拓宽,基于多特征分析的自适应关键帧提取的技术可以将视频质量差、冗余度高的视频精简处理,加快检测速度。同时,不同模态的图像融合可以增强图像的信息,消除图像光照不均匀,可见度低的问题。结合以上研究基础和实验室的实验环境,本论文主要基于多特征分析的自适应关键帧提取和多模态图像融合的技术,结合YOLOX算法实现井下特殊环境下的人员检测,最终希望通过C++来部署到智能掘进机器人上,充当智能掘进机器人的视觉系统,使得在工作的同时可以检测到机器人前方是否有人,提高工作环境的安全性。本论文实验主要研究内容和结果如下:（1）基于多特征分析的自适应视频关键提取算法本算法结合图像的颜色特征、形状特征及质量特征求相应的相似度值并进行加权融合,通过与自适应阈值进行比较,提取视频的关键帧。其中颜色特征和形状特征的相似度值均采用量化后的直方图求解,质量特征的相似度采用结构相似性（Structural Similarity,SSIM）计算。实验结果显示,本算法能够有效的提取代表视频的帧序列,提取的关键帧准确度在85%以上,冗余度较小,基本没有漏提取的关键帧。（2）基于改进的显著性检测算法指导的多模态图像融合技术本技术采用改进的频率调谐（Frequency-Tuned,FT）显著性检测算法,将FT算法种的高斯滤波器替换成双边滤波器,能够更好的对边缘细节部分进行检测。之后通过非下采样轮廓变换（Nonsubsampled Contourlet,NSCT）方法对红外和可见光图像进行多尺度分解,用提取的显著性图指导分解后的图像融合。通过实验分析,本文改进的FT显著性检测算法指导图像融合有很好的作用。主观评价方面,融合结果显示本算法能够很好提取红外图像包含的目标信息和可见光图像的边缘细节信息,融合图像的对比度和清晰度都有更好的表现。在客观评价方面,本文融合的结果在各项指标上都有所提升,其中在对比方法中,对比各项指标最差的值,平均梯度（Average Gradient,AG）平均提升2%,信息熵（Entropy,EN）平均提升1%,空间频率（Spatial Frequency,SF）平均提升4%,互信息（Mutual Information,MI）平均提升0.5%,边缘信息保持度(Edge Based Similarity Metrics,Q^AB/F)平均提升0.25%,在客观评价指标上,本文算法均得到了很好的效果。（3）YOLOX算法实现井下人员的检测并使用C++实现工程化部署本章主要通过YOLOX算法实现人员的检测,使用两种不同数据进行训练,其中Exdark数据集与井下环境相符合,训练后测试结果显示,在井下人员检测方面取得了不错的效果,漏检问题得到一定的解决。之后通过C++完成模型的部署,C++的优点在于工程化时更加方便,目前通过Pytorch深度学习框架训练的模型很多在工程化实现时都选择C++,原因在于模型转换方便和代码的可移植性强,本章对C++实现检测的步骤进行了介绍,最终测试结果和使用Pytorch的结果相同。更多还原

【Abstract】 Owning to the constant evolution of science and technology and the development of artificial intelligence.,the method of underground coal mining is gradually changing from the traditional manual method to mining robots.However,there is still a lot of room for the research enhancement of coal mining robots.In recent years,scholars have been proposing intelligent,mechanized and humanized underground coal mining operations to improve the efficiency and safety of underground mining,so the safety of underground personnel while the coal mining robots are working has become an urgent problem to be solved,and it is very important to avoid accidents while the machines are digging.In recent years,to ensure the accuracy of detection while accelerating the speed of detection,several yolo（you look only once）detection algorithms have been proposed.Therefore,it is of great significance that yolo detection algorithms can be deployed as the vision system of intelligent digging robots.Key frame processing of video and fusion of multimodal images are gradually broadening in the application of underground personnel detection.The technology of adaptive key frame extraction based on multi-feature analysis can streamline the video with poor quality and high redundancy and accelerate the detection speed.Meanwhile,image fusion of different modalities can enhance the information of images and eliminate the problems of uneven image illumination and low visibility.Combining the above research basis and laboratory experimental environment,this thesis mainly focuses on adaptive key frame extraction based on multi-feature analysis and multi-modal image fusion technology,combined with YOLOX algorithm to realize personnel detection in special underground environment,and finally hopes to deploy it to intelligent digging robots through C++,acting as the vision system of intelligent digging robots,so that it can detect whether there is someone in front of the working environment to make sure its safety.This will enhance the security of the working environment by detecting whether there are people in front of the robot while working.The main research contents and results of this thesis experiment are as follows:（1）Adaptive video key extraction algorithm based on multi-feature analysisThe algorithm attaches the color,shape and quality characteristics of the image to find the corresponding similarity values and performs weighted fusion,and extracts the key frames of the video by comparing with the adaptive threshold.The similarity values of color features and shape features are solved by quantized histogram,and the similarity of quality features is calculated by SSIM.The results of experiment display that this algorithm is able to efficiently extract the frame sequences which are represent the video,and the accuracy of the extracted key frames is above 85%,with less redundancy and basically no missed extracted key frames.（2）Multimodal image fusion technique based on the improved saliency detection algorithm guidanceThis technique uses the improved FT saliency detection algorithm,replacing the Gaussian filter of the FT algorithm species with a bilateral filter,which can better detect the edge detail part.After that,the multi-scale decomposition of infrared and visible images is performed by NSCT method,and the extracted saliency map is used to guide the fusion of the decomposed images.Through the analysis of experiment,the advanced algorithm of FT saliency detection in this paper guides the image fusion very well.In terms of subjective evaluation,the fusion results show that the present algorithm can well extract the target information contained in the infrared image as well as the information about edge details of the visible image,and the fused image has better contrast and sharpness.In terms of objective evaluation,the results of fusion in this paper are improved in all indexes,among which in the comparison method species,comparing the worst values of each index,AG is improved by 2%on average,EN by1%on average,SF by 4%on average,MI by 0.5%on average,and Q^AB/F by 0.25%on average,in terms of objective evaluation indexes,the algorithm in this paper has got good results.（3）YOLOX algorithm to achieve detection of underground personnel and engineering deployment using C++In this chapter,the YOLOX algorithm is mainly used for personnel detection,and two different datasets are used for training,among which the Exdark dataset is compatible with the underground environment.The advantage of C++is that it is more convenient for engineering,and many models trained by Py Torch deep learning framework choose C++for engineering implementation,because of the convenience of model conversion and code portability.This chapter presents the steps for implementing detection in C++,and the final test results are the same as those using Py Torch.更多还原