【作者】 胡晨；

【导师】 吴华强；

【作者基本信息】 清华大学 ， 集成电路工程（专业学位）， 2018， 硕士

【摘要】 随着视频技术日益发展,视频拼接的技术与应用也得到了长足地发展。目前,视频拼接算法大多针对于相机角度固定的场景,而对于非固定视角则面临计算复杂度高,无法实时处理等诸多问题。视频应用和人工智能的结合也成为了近期的研究热点。但和低空无人机结合,对航拍视频的目标检测,目前还没有具代表性的工作,且已有算法对于小物体的检测也无法获得高性能,对于信息提取的维度较为单一。为获得灵活视角、高质量的实时全景视频与多维度的目标信息,本文创造性地结合无人机平台,搭建指令交互控制系统,提出了新的在线、实时视频拼接算法,智能化目标检测和提取算法,用以完成对大视场下的视频数据分析。本文通过对多相机矩阵实时视频的研究,提出了利用信息熵优化角点的特征点提取步骤,多粒度算子的特征描述步骤,形成AFD(Auto-adapted Feature Detection自适应特征提取)和反馈ROI(Region Of Interest感兴趣区域)相结合的特征提取描述方法。同时针对多相机全景视频中,各相机视角的差异,以及飞行差异而导致视频抖动问题,提出了自适应帧间约束算法和幕布固定算法,结合无人机的飞控信息来优化视频质量。通过CPU-GPU异构处理算法,提高拼接算法的计算速度。对算法流程优化,提出两种拼接算法模式FrameStitch和SimpleStitch,对于两路720p的视频最快可以达到98fps的实时拼接速度。在无人机全景视频的应用上,本文引入了基于人工智能的目标检测算法。由于低空视角下物体的像素较少,检测的准确率难以保障,针对这一问题,通过对快速目标区域产生算法的研究,提出了VRPN-VAEN(Vehicle Region Proposal NetworkVehicle Attributes Extracting Network车辆候选区域生成网络-车辆属性提取网络)流水线型的端到端深度网络,融合了浅端像素信息和深层高维特征信息,得到了更精准的候选区域。此外,还提出了“循环学习率”的概念,提高了物体识别的准确率。并利用Postfilter(滤除网络)对错误检测进行优化。针对当前无人机视角结构化数据集缺失的现状,搜集和建立了供训练和测试用的顶视视角的数据集。并且为了弥补训练数据有限的情况,对数据进行裁剪、旋转,加入随机噪声,光照随机补偿等增广处理,以扩充数据集防止过拟合。通过实验对比,最终在检测和分类测试结果上表现良好。更多还原

【Abstract】 With the development of video technology,the technology and application of video splicing have also developed rapidly.Currently,video splicing algorithms are mostly aimed at scenes where the camera angle is fixed,and for non-fixed viewing angles,they are faced with many problems such as high computational complexity and inability to process in real time.The combination of video applications and artificial intelligence has also become a recent research hotspot.However,combined with low-altitude unmanned aerial vehicles,there is currently no excellent work of detection in aerial video,and the existing algorithms cannot achieve high performance for the detection of small objects,and have a single dimension for information extraction.In order to obtain flexible perspectives,high-quality real-time panoramic video and multi-dimensional target information,this paper creatively combines drone platforms to build instruction interaction control systems,and proposes new online and real-time video splicing algorithms,intelligent target detection and extraction.The algorithm is used to analyze the video data under a large field of view.In this paper,by the research of the real-time video with multi-camera matrix,this paper proposes a feature point extraction using information entropy to optimize the corner points.Based on the feature description of multigranularity operators,AFD(auto-adaptive feature detection)and ROI(region of interest)are combined to extract feature vector.At the same time,for the multicamera panoramic video,the difference of camera angles,and the video jitter caused by the flight difference,an adaptive frame-restraining algorithm and a fixed screen algorithm are proposed,and the flight control information of the drone is used to optimize the video quality.Through the CPU-GPU heterogeneous processing algorithm,the computing speed of the splicing algorithm is improved.For the optimization of the algorithm flow,two stitching algorithms,FrameStitch and SimpleStitch,are proposed.For two channels of 720 p video,the real-time stitching speed of 98 fps can be achieved.In the application of drone panoramic video,this paper introduces a target detection algorithm based on artificial intelligence.Because there are few pixels in the low-altitude perspective,the accuracy of the detection is difficult to guarantee.To solve this problem,combined with the RCNN algorithm,an endto-end depth network based on the VRPN-VAEN pipeline is proposed,and the shallow end pixel information is merged.With deeper high-dimensional feature information,more accurate candidate regions were obtained.In addition,the concept of “cycle learning rate” was also proposed to improve the accuracy of object recognition.And use Postfilter(filtering network)to optimize false positive.Due to the lack of top-view structural datasets,datasets from top view angles for training and testing were collected and established.In order to compensate for the limited training data,the data is rotated and croped with random noise and lighting compensation to prevent overfitting.Through the experimental comparison,the test results are finally performing well in the detection and classification.更多还原