【作者】 陈浩；

【导师】 汪凯巍；

【作者基本信息】 浙江大学 ， 信息传感及仪器， 2021， 博士

【摘要】 视觉同步定位与地图构建(Visual Simultaneous Localization and Mapping),是指搭载相机的主体,在运动过程中利用视觉图像估计自身的位姿,同时建立起周围环境的模型。作为空间探测、智能机器人、自动驾驶、增强现实等应用场景中支撑定位功能的核心技术,视觉SLAM在过去的十数年里,得到了国内外研究者的广泛关注,并取得了令人瞩目的成果。目前,传统的基于透视投影的针孔相机仍然是视觉SLAM的主流传感器。然而,针孔相机一般视场角比较有限,这会导致基于针孔相机的视觉SLAM在面临一些挑战性场景时,鲁棒性和精度不足。针对这一问题,本文提出研究基于全景环带成像的视觉SLAM。全景环带成像是一种可以在单次成像中获得周遭360°全景感知的成像技术,其核心部件是全景环带镜头(Panoramic Annular Lens,PAL)。PAL所拥有的超大视场,意味着相机可以同时捕获更丰富的视觉信息,使得位姿估计和地图构建有更加充足的数据来源。本文首先对全景视觉SLAM中的数学模型进行定义和分析,包括PAL相机模型、全景SLAM的参数化方法、雅可比矩阵的形式等。此外,由于PAL的成像方式与常规的针孔相机有着显著不同,本文针对全景成像下的双视图几何关系进行了重新推导,包括基于全景相机模型的对极约束、位姿恢复和特征点三角化,为后续研究建立良好的基础。在此基础上,本文提出了 一种基于全景环带成像的稀疏直接法视觉里程计(PALVO)。基于全景相机投影关系,本文设计了基于本质矩阵的初始化模块,由粗到精的两步跟踪策略,以及沿极曲线的特征匹配搜索方法。实验表明,PALVO对于快速运动和动态场景具有很强的鲁棒性,同时达到了最先进的(state-of-the-art,SOTA)视觉里程计算法的精度水平,并且具有较高的运行速度。接着,为了进一步消除运动物体的影响,本文提出了一种全景语义视觉里程计(PASVO)。该方法在PALVO之上,结合了基于深度学习的全景环带语义分割,并将语义信息深度耦合到视觉里程计的各个模块。在位姿估计阶段,通过语义信息赋权,来减小运动物体等对位姿优化的干扰;在地图构建阶段,利用语义信息来引导关键点选择和极曲线搜索。相比于PALVO,PASVO借助语义信息,进一步提高了动态环境下的鲁棒性,并且大幅提升了精度。同时,语义分割的引入,使得在几何信息感知之上,加入了对环境的更高层次理解。最后,对于PALVO中所存在的累积误差和尺度漂移,本文提出了全景环带视觉SLAM(PA-SLAM),将基于稀疏直接法的PALVO扩展到具有闭环检测和全局优化的视觉SLAM系统。为了在保证计算效率的同时实现可靠的闭环检测,提出了关键点混合选择策略。此外,得益于PAL的360°环视成像特性,PA-SLAM能够处理不同行进方向的闭环,这与基于前视针孔相机的经典视觉SLAM算法仅可处理同向的闭环相比,具有明显的优势。实验表明,PA-SLAM显著降低了PALVO的累积误差和尺度漂移,达到了 SOTA的视觉SLAM的精度水平,并且保持了原有的鲁棒性和高效率。更多还原

【Abstract】 Visual Simultaneous Localization and Mapping(Visual SLAM)is defined as the process of estimating the camera pose(translation and rotation with respect to a reference frame)and simultaneously establishing a map of the surrounding environment,taking use of a sequence of captured images.As a fundamental technology that enables reliable positioning in applications such as space detection,intelligent robots,autonomous driving,and augmented reality,visual SLAM has received extensive attention from researchers at home and abroad in the past decade,and has achieved impressive results.At present,the traditional pinhole camera based on perspective projection is still the mainstream sensor for visual SLAM.However,the pinhole camera generally has a limited field of view(FOV),which will lead to insufficient robustness and accuracy when facing some challenging scenarios.Given this,visual SLAM based on panoramic annular imaging is proposed in this thesis.Panoramic annular imaging can produce a 3600 panoramic perception of the surroundings in a single shot,of which the core component is Panoramic Annular Lens(PAL).The larger FOV means that richer visual information can be captured by the camera at one time,and much more sufficient data can be used in pose estimation and map construction.This thesis firstly defines and analyzes mathematical models in the panoramic visual SLAM,including the PAL camera model,the parameterization method of panoramic visual SLAM,and the form of the Jacobian matrix.Besides,because the projection formula of PAL is significantly distinct from that of conventional pinhole cameras,the dual-view geometric relationship is re-derived under panoramic imaging,including the epipolar constraint,pose recovery,and feature point triangulation,to establish a good foundation for follow-up research.On this basis,a sparse direct visual odometry based on panoramic imaging(PALVO)is proposed.An initialization module based on the essential matrix,a "coarse-to-fine"tracking strategy and a feature correspondence search method along epipolar curve are specially designed.Experiments demonstrate that PALVO has excellent robustness to rapid motion and dynamic scenarios,approximate same accuracy as state-of-the-art(SOTA)visual odometry,and high efficiency.Then,a panoramic annular semantic visual odometry(PASVO)is presented to further eliminate the influence of moving objects.PASVO combines PALVO and semantic segmentation of panoramic annular images by deeply coupling semantic information to various modules of the visual odometry.In the pose estimation stage,semantic information weighting is utilized to reduce the interference of moving objects.In the mapping stage,semantic information is employed to guide the keypoint selection and feature correspondence search.Compared with PALVO,the robustness and accuracy of PASVO in practical applications are further improved.At the same time,the introduction of semantic segmentation allows a higher-level understanding of the environment besides pure geometric information.Finally,PALVO is extended to a complete visual SLAM system by integrating closure detection and global optimization to suppress the accumulated error and scale drift in PALVO.To ensure computational efficiency and achieve reliable loop detection,a hybrid keypoint selection strategy is proposed.Benefiting from the 360° imaging characteristics of PAL,PA-SLAM can handle loop closure in different travel directions,exhibiting obvious advantages compared with the classical visual SLAM based on forward-looking pinhole camera,which can only deal with loops in the same travel direction.The experimental results indicate that PA-SLAM significantly reduces the error accumulation and scale drift of PALVO,achieves equivalent accuracy of SOTA visual SLAM,and maintains the original robustness and high efficiency.更多还原