【作者】 黄国辉；

【导师】 林立雄；

【作者基本信息】 福州大学 ， 机械工程（专业学位）， 2021， 硕士

【摘要】 在室外场景下,视觉里程计能在卫星信号缺失时协助移动机器人感知外部环境,获取自身位置和速度。相比于立体相机繁琐的标定与校正过程,基于低成本单目相机的视觉解决方案更简便。卷积神经网络(Convolutional Neural Networks,CNNs)可以直接从单目图像计算移动机器人状态,不需要对摄像头进行标定与校正,并克服尺度不确定性缺点。但CNN在实际应用中仍存在困难,首先CNN运行时需要大量内存资源,而里程计通常应用于嵌入式设备,这些设备的计算能力不能满足CNN的算力需求;其次,视觉里程计在光照条件不理想或者动态物体被遮挡时,图像会产生模糊、纹理缺失等现象,导致状态估计误差大。为此,提出在扩展卡尔曼滤波器中融合CNN输出的单目相机位姿和IMU数据,弥补恶劣环境条件下单目视觉的估计误差,同时单目相机能补偿IMU传感器的累计漂移误差,有助于里程计获得高精度的导航信息。本文的主要研究内容包括:(1)提出基于CNN的单目深度及位姿预测方法。针对神经网络训练时间久、模型参数大的问题,从空间金字塔编码层、门控自适应解码层、损失函数三个方面设计神经网络模型。对比分析不同编码器对估计性能的影响,运用深度可分离卷积替代传统卷积,引入ASPP模块扩大图像空间感受野。采用子像素卷积的超分辨技术提升特征图质量,设计多尺度融合门控模块补充上下文信息,提高网络运行速度。考虑图像梯度和表面法线,在对数域构建损失函数,处理深度图边缘扭曲和模糊现象,保证模型精度。(2)提出单目图像与IMU数据互补的位姿估计方法。处理IMU传感器噪声误差,建立IMU运动学及误差传播模型,形成共同运动约束,融合惯性信息和视觉图像。不同于经典扩展卡尔曼滤波器,提出将CNN模型的预测值当作扩展卡尔曼滤波器的测量更新,利用单目相机和IMU的互补模态和误差特性,使用神经网络优化滤波器的误差协方差,实现目标物体的位姿更新和估计。(3)在公开数据集和室外环境对所提出的深度及位姿估计方法进行验证,评估算法的准确性和适用性。实验从单目深度估计评价指标、三维点云可视化、在线推理速度、平均位姿误差等方面进行验证,与其它的算法相比,本文算法能有效降低深度估计误差,大幅改进姿态预测精度,同时缩减内存需求,提高图像推理运行速度。室外场景实验表明本文算法能够满足使用需求,具有可行性。更多还原

【Abstract】 In outdoors environments,visual odometry(VO)can assist mobile robots recognize the extraneous surroundings and obtain pose when the GNSSs signal is missing.Compared with the tedious calibration process of stereo camera,the vision solution based on low-cost monocular camera is simpler.Convolutional neural networks(CNNs)can directly calculate the robot states from monocular images without camera calibration,and solve the scale ambiguity problem using a single monocular camera.However,there are some problems in CNN.Firstly,CNN consumes a lot of memory as well as computation time,and VO is usually used in embedded devices.The computing power of these devices is often limited,and cannot meet the requirements of CNN.Secondly,VO system that only relies on vision will produce blurring of detail and texture missing problem under poor light conditions or the dynamic objects are occluded,which easily engender the failure of state estimation.Therefore,applying extended Kalman filter(EKF)with IMU input to make up for the errors of monocular VO using CNN algorithms.Meanwhile,monocular camera can compensate the accumulated drift errors of IMU sensor,which is helpful for monocular visual-inertial odometry to gain high-precision navigation messages.In summary,this paper provides the following contributions:(1)A novel monocular depth and pose estimation unsupervised network is proposed.Aiming at the problems of actual latency and large amount of parameters in neural networks,the networks architecture is designed from three aspects: spatial pyramid encoder,gating adaptive decoder and loss function.The influences of different encoders on the estimation performance and efficiency are compared and analyzed.The depth separable convolution is used to substitute the common convolution,and the ASPP module is introduced to expand the spatial receptive field.To filter the feature information and improve the networks’ learning speed,the sub-pixel convolution can improve the quality of feature maps,and the context information is provided by the multi-scale fusion gate module.Considering the depth image gradient and surface normal,the loss function is calculated by addition operation logarithmic domain to deal with the edge distortion and blurring of depth maps,so as to expedite the convergence speed of the model and ensure precision.(2)A multi sensor fusion pose estimation method which combines monocular video and IMU data is proposed.The IMU kinematics and error propagation model are established to process the IMU noise error,and the inertial information and visual measurements are fused to form a common motion constraint.Different from the classical EKF,this paper proposes that the estimated data of CNN model is employed in the measurement update step of EKF.Utilizing the complementary mode and error characteristics of monocular camera and IMU,the CNN is used to optimize the error covariance of the EKF to complete the update and estimation of the motion pose.(3)The algorithm proposed in this paper is verified in datasets and outdoors environments to evaluate the accuracy and applicability of the algorithm.The experiments test the algorithm performance from monocular depth estimation evaluation indexes,3D visualization of point cloud,online inference speed,average pose error and so on.When compared with other algorithms in the literature,the proposed framework can effectively decrease the depth estimation error,notably augment the pose estimation accuracy,drastically reduce the memory requirements and enhance the processing speed.Outdoors experimentations show that the method is realistic,stable and feasible.更多还原