【作者】 王文杰；

【导师】 谭婷； 贡亮；

【作者基本信息】 上海交通大学 ， 机械工程（专业学位）， 2022， 硕士

【摘要】 蔬果采收费时费力,人工依赖度高,是农业自动化生产的薄弱环节。研究一种在设施农业场景中损伤率低、成功率高的蔬果采收机器人对提高农业自动化、智能化发展水平具有重要意义。自然场景复杂多变,果实鲜嫩易损,为应对果实粘连、枝叶遮挡、光线条件变化下的非结构化作业场景,准确识别姿态各异的目标果实,实现高成功率灵巧采摘作业,本文针对高准确率目标识别和高成功率采摘姿态估计问题,提出基于自然场景语义理解的蔬果采收机器人灵巧摘取方法。首先,针对在不同自然场景下单图像通道信息不充分造成粘连果实难以区分的问题,提出融合RGB图像、深度图像和红外图像的多源信息融合方法,建立基于多源图像融合扩展Mask R-CNN的图像实例分割算法模型,进行采摘机器人在线果实识别,目标分割精度提高了7.6%。其次,针对自然场景下枝叶遮挡造成基于视觉的蔬果采摘机器人定位精度和采收成功率低的问题,提出一种基于深度学习和多源图像的遮挡目标空间几何特征的高精度估计方法,构建了SPR(Shape and position restoration)模型,建立了基于UNet-like的形位估计算法,使采收机器人能够在存在遮挡的情况下对整个目标果实进行视觉感知,对果实形状和位置信息进行复原,从恢复的深度图像中提取果实半径和质心坐标。当遮挡率低于25%,照度在1-12KLux之间时,平均Io U(Intersection over Union)为0.895,质心位置误差为0.62mm。最后,针对果实柔软娇嫩、姿态多变、采摘损伤率高的问题,提出基于物景分离的姿态估计评价方法,建立基于Grasp Net的果实抓取姿态优化模型,实现最优采摘姿态的实时规划,进行果实无损灵巧采摘。基于此,搭建双臂采收机器人系统,在种植基地进行自动化采收实验,单个果实的平均采摘时间为11s,平均抓取精度8.21mm,平均采摘成功率达到73.04%。该方法赋予采收机器人在非结构化自然场景中识别感知能力,完成灵巧采摘动作,提高了采收成功率。更多还原

【Abstract】 Harvesting fruits and vegetables is time-consuming and laborintensive,with high manual dependence,and is a weak link in automated agricultural production.Researching a vegetable and fruit harvesting robot with low damage rate and high success rate in facility agriculture scenarios is important to improve the level of agricultural automation and intelligent development.In order to deal with the unstructured operation scenarios such as fruit sticking,branch and leaf shading,and changing light conditions,and to accurately identify the target fruits with different postures and achieve high success rate of dexterous picking,this paper proposes a dexterous picking method based on semantic understanding of natural scenes for vegetable and fruit harvesting robots to address the problems of high-accuracy target recognition and high-success rate of picking posture estimation.Firstly,to address the problem that it is difficult to distinguish the sticky fruits due to insufficient information of single image channel in different natural scenes,a multi-source information fusion method is proposed to fuse RGB images,depth images and infrared images,and an image instance segmentation algorithm model based on multi-source image fusion extended Mask R-CNN is established for online fruit recognition of picking robots,and the target segmentation accuracy is improved by 7.6%.Secondly,to address the problem of low localization accuracy and harvesting success rate of vision-based fruit and vegetable picking robot caused by branch and leaf occlusion in natural scenes,a high-precision estimation method of spatial geometric features of occluded targets based on deep learning and multi-source images is proposed,a SPR(Shape and position restoration)model is constructed,and a UNet-like form estimation algorithm is established,which enables the harvesting robot to locate the whole The target fruit is visually perceived,the fruit shape and position information is recovered,and the fruit radius and center-of-mass coordinates are extracted from the recovered depth image.The average Io U(Intersection over Union)is 0.895 and the center-of-mass position error is0.62 mm when the occlusion rate is less than 25% and the illumination level is between 1 and 12 KLux.Finally,to address the problems of soft and delicate fruits,variable posture and high picking damage rate,a pose estimation and evaluation method based on object-view separation is proposed,and a Grasp Netbased fruit grasping pose optimization model is established to realize realtime planning of optimal picking posture for fruit nondestructive dexterous picking.Based on this,a two-arm harvesting robot system is built and automated harvesting experiments are conducted at the planting base,with an average picking time of 11 s for a single fruit,an average grasping accuracy of 8.21 mm,and an average picking success rate of 73.04%.The method empowers the harvesting robot to recognize and perceive in unstructured natural scenes,complete dexterous picking actions,and improve the harvesting success rate.更多还原