【作者】 王鹏飞；

【导师】 陈熙源； 陈云；

【作者基本信息】 东南大学 ， 电子信息（专业学位）， 2023， 硕士

【摘要】 近年来,随着深度学习和计算机技术的快速发展,自动驾驶已经成为一个备受关注的研究领域,在果园环境下的农机自动驾驶技术也迅速发展。果园环境与城市中结构化的道路场景有所不同,果园场景道路起伏大,没有车道线,更具有复杂性,对算法和传感器可靠性提出了挑战,研究果园场景下的农机自动驾驶感知问题具有重要意义。本文从视觉的角度出发,研究如何提升农机在果园中导航线提取的准确性和定位的精准性。本文聚焦于农机自动驾驶中的视觉感知关键技术,包括基于深度学习的语义分割技术和分别引入线特征、惯性测量单元(Inertial Measurement Unit,IMU)的视觉同时定位与构图技术(Simultaneous Localization And Mapping,SLAM)。本文主要开展以下研究:1.为了获得复杂果园环境下农机的可行驶区域以及导航线,提出一种改进U-Net网络的导航线提取方法。首先对改进的U-Net网络在真实的果园数据集上训练,根据训练好的模型对实时获取的果园图像进行语义分割,在获得可行驶区域后,通过边界点扫描并确定导航线,最后对提取的导航线进行评估,确保可以满足果园场景需求。2.为了提升农机在果园的定位精度,提高果园农机定位系统的稳定性,从视觉SLAM的角度出发,参考现有开源框架,首先在点特征的基础上,重点研究线特征的提取和匹配,然后利用线特征增加约束关系,建立线特征的重投影误差,最后建立线特征的词袋模型,实现点线综合特征的双目视觉SLAM算法。选用KITTI公开数据集进行验证,实验表明,本文的点线双目SLAM算法相较于典型的点特征算法ORB-SLAM2,可以充分利用场景的点线特征,在定位精度方面和系统稳定性方面也更有优势。3.针对纯视觉SLAM出现相机图像模糊以及提取的场景纹理特征不可靠等情况导致的位姿估计失败问题,本文在点线双目SLAM的基础上,提出一种基于点线特征的双目视觉惯性融合定位算法。首先重点研究IMU的数学模型和预积分,然后融合视觉残差和IMU残差构建代价函数进行非线性优化,最后选用Eu Ro C公开数据集进行验证,实验表明,融入IMU的视觉惯性SLAM算法相对于ORB-SLAM2鲁棒性更好,同时轨迹的精度也得到了提升。4.最后搭建农机实验平台,在实际果园中对本文所提算法进行实验验证。首先对双目视觉传感器和IMU进行标定,然后在果园梨树、樱桃树和水蜜桃树场景下分别进行实验,实验结果表明,本文搭建的硬件平台表现出良好的可靠性和稳定性,所提算法鲁棒性较好,适用于果园场景。更多还原

【Abstract】 In recent years,with the rapid development of deep learning and computer technology,autonomous driving has become a highly researched field,and autonomous driving technology for agricultural machinery in orchard environments has also developed rapidly.Orchard environments differ from structured road scenes in cities,with large road undulations,no lane markings,and greater complexity,posing challenges to algorithms and sensor reliability.Research on the perception of agricultural machinery autonomous driving in orchard environments is of great significance.How to improve the accuracy of navigation line extraction and positioning precision of agricultural machinery in orchards is studied in this thesis from the perspective of vision.The key technologies of visual perception in automatic driving of agricultural machinery are focused on in this thesis,including semantic segmentation based on deep learning,and visual SLAM(Simultaneous Localization And Mapping)techniques that incorporate line features and IMU(Inertial Measurement Unit,IMU)data.The following research is conducted:1.To obtain the drivable area and navigation line of agricultural machinery in complex orchard environments,an improved U-Net network-based navigation line extraction method is proposed.First,the improved U-Net network is trained on a real orchard dataset,and the trained model is used for semantic segmentation of real-time orchard images.After obtaining the drivable area,the navigation line is determined by scanning the boundary points.Finally,the extracted navigation line is evaluated to ensure that it meets the requirements of orchard environments.2.To improve the positioning accuracy of agricultural machinery in orchards and enhance the stability of orchard agricultural machinery positioning systems,from the perspective of visual SLAM,the existing open-source framework is referred to in this thesis.Firstly,based on point features,The extraction and matching of line features are focused on in this thesis.Then,the line features are utilized to add constraint relationships and establish the reprojection error of line features.Finally,a bag-of-words model is established for the line features to achieve a stereo visual SLAM algorithm that integrates both point and line features.The proposed stereo point-line SLAM algorithm was validated using the publicly available KITTI dataset,and the experiments showed that compared with the typical point feature-based algorithm ORBSLAM2,the proposed algorithm can fully utilize the point-line features of the scene and has advantages in terms of positioning accuracy and system stability.3.To address the issue of failed pose estimation in pure visual SLAM due to blurry camera images or unreliable scene texture features,A stereo point-line feature-based visual-inertial fusion positioning algorithm is proposed in this thesis,building on the foundation of stereo point-line SLAM.Firstly,the mathematical model and pre-integration of the IMU are focused on in the research.Then,the visual residuals and IMU residuals are fused to construct a cost function for nonlinear optimization.Finally,the algorithm is validated using the Eu Ro C public dataset.Experimental results show that the visual-inertial SLAM algorithm with IMU integration is more robust than ORB-SLAM2,and the accuracy of the trajectory is also improved.4.Finally,an experimental platform for agricultural machinery was established to verify the algorithms proposed in this thesis in actual orchard environments.Firstly,the binocular visual sensor and IMU were calibrated,and then experiments were conducted separately in multiple orchard environments,including scenes of pear orchards,cherry orchards,and peach orchards.The experimental results show that the constructed hardware platform exhibits high levels of reliability and stability,and the proposed algorithms are robust and suitable for orchard environments.更多还原