【作者】 周军；

【导师】 刘飞；

【作者基本信息】 浙江大学 ， 农业电气化与自动化， 2023， 博士

【摘要】 水稻是全球约一半人口的主要粮食,也是中国种植粮食中的第二大作物,在我国粮食安全战略中占有重要位置。水稻产量高效、精准的估测对国家粮食政策制定、精准农业管理、水稻育种和栽培研究、作物表型分析研究都具有重要的意义。传统大田产量抽样调查法主要依靠人工破坏性采样,耗时耗力、结果容易受人主观影响;农业气象产量估测法需要长期的农业气象监测数据,作物生长模型估测法需要气象数据以及作物生物学参数,这两种基于农学机理的产量估测方法输入参数过多、数据获得难度大,导致模型估测精度不稳定、适应性差、成本高。近些年光学技术和光学传感器的飞速发展为作物产量估测奠定技术基础,人工智能算法的兴起为水稻产量估测提供新的解决手段。本文以水稻产量高效、精准估测为研究目标,以多种品种、多种施肥处理的水稻为研究对象,从水稻籽粒、稻穗、冠层多尺度出发,构建了热红外-可见光的双光成像系统,提出了图像配准算法、热红外伪影去除算法和多层嵌套粘连籽粒分割算法,实现了水稻籽粒结实率的高效、准确地检测;设计三种迁移策略的迁移学习网络Panicle Net,建立了大田稻穗检测计数模型;依据水稻籽粒结实率和稻穗计数检测模型,构建了籽粒-稻穗-冠层式水稻产量估测模型;新构建了与水稻产量密切相关的多个图谱指数,并融合多个图谱指数构建了产量指数（Yield index）YI_Fusion;提出了水稻冠层RGB图像分级建模和归一化权重的决策策略,构建了无人机图像感知的水稻产量估测模型。本文具体主要研究内容和结果如下:（1）提出了图像配准算法、热红外伪影去除算法和多层嵌套的籽粒粘连分割算法,构建了双光成像系统,实现了水稻籽粒结实率的检测。针对水稻籽粒实粒和空粒间外表特征差异小,基于人工选择的颜色和纹理特征的图像处理方法识别难的问题,选择热红外相机和RGB相机构建了具有图像采集和数据存储功能的双光成像系统,采集水稻籽粒加热处理后的热红外图像和RGB图像,研究了两种籽粒加热后的温度变化规律,在籽粒的含水量为11%-20%时,比热容为4.9-16.3kj·kg^-1·K^-1,因为实粒比空粒的质量大,故而实粒含有更多的热量。经分析发现,当籽粒温度从35℃降至22℃（室温）过程中,空粒的温度显著低于实粒的温度,两者间温差先增大后减小,30s时达到最高2.8℃,60s时超过1.7℃。基于籽粒温度变化规律提出图像配准方法、热红外去伪影算法和多层嵌套粘连籽粒分割算法,并与三种深度学习方法对比,验证了结实率检测性能。结果显示,相比于图像配准方法（86.78%）和深度学习YOLO V3方法（96.75%）,融合热红外去伪影和多层嵌套粘连籽粒分割算法构建的水稻结实率检测方法效果最优,结实率检测准确率达到97.66%。（2）提出了三种迁移学习策略,设计了稻穗计数融合算法,构建了将小麦穗源域数据集中学习的穗部特征知识迁移到水稻穗目标域的稻穗检测计数Panicle Net迁移学习模型,实现大田稻穗快速检测计数,并提出了籽粒-稻穗-冠层式水稻产量估测模型。针对基于人工选择特征的穗部检测算法抗干扰能力和稳定性差、基于深度学习方法的巨量数据标注导致成本剧增的问题,分别利用麦穗图像公开数据集和实验采集的稻穗图像数据构建了用于迁移学习的源域数据集Wheat Dataset和目标域数据集Panicle Dataset,引入残差思想设计了基于Res Net50的迁移学习网络Panicle Net,提出了基于微调策略的Panicle Net-F、基于深度相关校正策略的Panicle Net-C、基于深度域混淆策略的Panicle Net-D三种迁移学习方法,研究了将在源域数据集Wheat Dataset中学习到的穗部特征表征知识迁移到目标域数据集Panicle Dataset进行稻穗检测的可行性;并且为解决了因图像切分导致稻穗重复计数问题,设计了稻穗计数融合算法。实验数据结果分析显示,Panicle Net-F稻穗检测精确度、召回率和F1分数分别为80.13%、75.64%和0.53,显著优于未经微调的Panicle Net,基于10%小数据训练的微调模型的精确度和召回率下降幅度较小,这证明了微调策略能够将从麦穗检测上学习到的特征知识迁移到水稻检测上;Panicle Net-D稻穗检测精确度、召回率和F1分数分别为88.59%、80.37%和0.84;相比于Panicle Net-F和Panicle Net-D,Panicle Net-C检测效果最优,其稻穗检测精确度、召回率和F1分数分别为89.53%、92.85%和0.91。在完成结实率数字化检测基础上,研究融合本文第三章中基于双光成像系统的热红外去伪影去除、多层嵌套粘连分割算法和本文第四章中Panicle Net-C迁移学习网络,完成籽粒和稻穗的自动检测,构建籽粒-稻穗-冠层式水稻产量估测模型,结果表明该水稻产量估测模型的MAPE为17.12%。（3）基于建模因子逐步融合策略开发了水稻产量指数YI_Fusion,实现基于无人机多光谱图像的水稻产量估测。针对产量估测模型的精度容易受到水稻品种、施肥处理方式、环境条件差异的影响,导致鲁棒性低的问题,基于5波段多光谱图像提出了6种新植被指数（VI）、6种新颜色指数（CI）,利用多光谱图像的灰度共生矩阵构建了6种新纹理指数（TI）,研究3类新指数与水稻产量相关的相关程度,筛选出每类指数相关性排序靠前的指数作为建模因子。结果显示,新构建的VI、CI和TI的性能优于原始的特定波段计算的植被指数、颜色指数和单个纹理特征,在抽穗阶段6个VI的最佳波段组合为VI_{1（G-Red,B-G）}（R=0.5032）、VI_{2（Red,Red-RE）}（R=0.6390）、VI_3（Red,G）（R=0.6282）、VI_4（Red,B）（R=0.7036）、VI_{5（RE,Red）}（R=0.4602）和VI_6（NIR,B）（R=0.4903）,在成熟阶段6个VI的最佳波段组合为VI1_RE-NIR,B-Red（R=0.7742）、VI2Red,B</sub>（R=0.5921）、VI3NIR,Red</sub>（R=0.6741）、VI4NIR,B</sub>（R=0.6432）、VI5（Red,B）（R=0.8423）、VI6（NIR,Red）（R=0.6402）。在抽穗期、成熟期全生育期与产量相关性最高的CI分别是CI2、CI3和CI1,在这三个时期与产量相关性最高的TI分别是TI(CONNIR,CORRed)、TI(CONNIR,CONRed)与TI(CONTNIR,CORRed)。二次非线性回归模型QNR在成熟期基于VI构建的产量指数的YIVI性能优于其他阶段,并优于基于CI的产量指数YICI和基于TI的产量指数YITI。设计建模因子逐步融合策略,探究基于多元线性回归方法MLR和基于随机森林算法RF构建产量指数YI性能差异。实验结果显示,与水稻抽穗期和全生育期相比,在水稻成熟期通过随机森林算法融合新建6个VI、6个CI、6个TI,构建了产量指数YIFusion性能最佳（决定系数R²=0.84,平均绝对误差MAE=714.55 kg/ha,平均绝对百分比误差MAPE=7.86%）,证明了无人机图谱融合策略在产量估测中的可行性。（4）基于ConvNeXt提出了大田水稻冠层RGB图像分级和归一化权重的融合决策策略,构建了无人机图像感知水稻产量估测模型。针对水稻产量估测过程参数过多、效率不高和精度不稳定的问题,如何充分挖掘水稻图像信息,突破一直以来固定的回归拟合的建模方法,提出全新的建模思路和策略是提高水稻产量估测精度的关键问题。以实验水稻小区的平均产量的10%作为分级间距,依据实测产量标签数据,对子小区图像进行分级来形成产量分级检测的数据集。基于深度学习网络ConvNeXt首次提出了基于无人机水稻冠层图像分级的产量估计方法,在测试集中模型的图像分级精度为90.17%,结果表明基于ConvNeXt图像分级的产量估测性能（MAPE=3.96%）优于基于ConvNeXt的回归模型（MAPE=6.75%）,说明了通过设置平均产量10%的间隔来对地块图像分级产量估测的方法,可以有效地控制产量估测误差。为了进一步提高估测精度,充分利用产量分级模型的置信分数,在研究中引入了归一化权重决策策略,通过对置信分数降序排列和权重归一化处理提出k-权重策略。研究了2-权重至5-权重范围内不同权重策略选择对产量估计性能的影响,发现所提出的归一化权重决策策略对错误分类水稻小区图像的产量估计具有显著的校正效果,2-权重策略可以有效地将MAPE降低到3.79%。基于深度学习算法提出了大田水稻冠层RGB图像分级建模和归一化权重的融合决策策略,构建了无人机图像感知的水稻产量估测模型,经模型泛化研究,证明了水稻产量估测模型在不同地区、不同品种数据中仍能保持良好的泛化性,MAPE可以达到4.54%。</sub>更多还原

【Abstract】 Rice is the main food for about half of the world’s population and the second largest crop in China’s grain cultivation.It plays an important role in Chinese food security strategy.Efficient and accurate estimation of rice yield is of great significance to the formulation of national grain policy,precision agriculture management in the production process,rice breeding and cultivation research,and crop phenotypic analysis research.The traditional field yield sampling survey method mainly relies on artificial destructive sampling,which is time-consuming and labor-intensive,and the results are easily subject to subjective influence;The agrometeorological yield estimation method requires long-term agrometeorological monitoring data,and the crop growth model yield estimation method also requires crop biological parameters.These two yield estimation methods based on agronomic mechanism have too many input parameters,and it is difficult to obtain data,resulting in unstable model estimation accuracy,poor adaptability,and high cost.In recent years,the rapid development of optical technology and optical sensors have provided new solutions for crop yield estimation,and the rise of artificial intelligence algorithms has laid the foundation for rice yield estimation.By taking rices with multiple varieties and fertilizer treatments as the research objects,this paper aimed at building the efficient and accurate rice yield estimation system based on spectral imaging technology,RGB imaging technology,image processing method and deep learning method from the multi-scale of rice grain,panicle,and canopy.A dual light imaging system with thermal infrared and visible light was constructed,and the image registration algorithms,the thermal infrared de-ghost and multi-level nested conglutinated segmentation algorithms were proposed to achieve efficient and accurate detection of rice seed setting rate;Three strategies of transfer learning network were designed to establish a field rice panicledetection and counting model.Based on the detection models of rice seed setting rate and rice panicle count,the grain-panicle-field multiscale rice yield estimation framework was constructed.Multiple spectral indices and texture indices closely related to rice yield were newly constructed,and the yield index YI_Fusion was proposed by fusing multiple indices;A decision strategy for rice canopy RGB image classification modeling and normalized weight was proposed,and a rice yield estimation framework based on UAV image perception was constructed.The main research contents and results are as follows:（1）A dual light imaging system was established,and the image registration algorithm,thermal infrared artifact removal algorithm,and multi-layer nested grain adhesion segmentation algorithm were developed to achieve the detection of rice seed setting rate.Due to the small difference between the appearance characteristics of full and empty grains,and the difficulty of recognition through the image processing method based on manually selected color and texture characteristics,a dual-light imaging system with image acquisition and data storage functions was constructed by selecting thermal infrared camera and RGB camera to collect thermal infrared images and RGB images after the rice seeds were heated,and the temperature change laws of the two kinds of seeds after heating were studied.When the water content of grains is 11%-20%,the specific heat capacity is 4.9-16.3kj·kg^-1·K^-1.Because the weight of the full grain is larger than that of the empty grain,the full grain contains more heat.Through analysis,it was found that when the temperature of grain dropped from 35℃to 22℃（room temperature）,the temperature of empty grain was significantly lower than that of full grain,and the temperature difference between them first increased and then decreased,reaching a maximum of 2.8℃at 30s,and exceeding 1.7℃at 60s.The image registration method,thermal infrared de-ghost method and multi-layer nested adhesive segmentation algorithm were proposed to solve the problem of rice seed setting rate detection,and compared with three deep learning methods to verify the seed setting rate detection performance.The results showed that compared with the image registration method（86.78%）and the deep learning YOLO V3 method（96.75%）,our rice seed setting rate detection method based on the fusion of thermal infrared de-ghost and multi-level nested conglutinated segmentation algorithms had the best effect,and the seed setting rate detection accuracy reached 97.66%.（2）A deep learning network,Panicle Net,was designed to transfer the feature knowledge learned from the wheat panicle source domain dataset to the rice panicle target domain based on three migration learning strategies to construct a rice panicle detection and counting model,achieving rapid rice panicle detection and counting in the rice field.Aiming at the problem of poor anti-interference ability and stability of panicle detection algorithms based on manually selected features,and the sharp increase in costs caused by massive data annotation based on deep learning methods,the source domain dataset and the target domain dataset for transfer learning were constructed using the wheat panicle image public dataset and and the rice panicle image data collected through experiments,respectively.The transfer learning network,Panicle Net,was designed based on Res Net50 using the residual idea.Three transfer learning strategies,Panicle Net-F based on fine-tuning strategy,Panicle Net-C based on deep correlation alignment strategy,and PanicleNet-D based on deep domain confusion strategy,were proposed.The feasibility of transferring the panicle feature representation knowledge learned from the source domain dataset Wheat Dataset to the target domain dataset Panicle Dataset for rice panicle detection was studied.In order to solve the problem of repeated counting of rice panicle caused by image segmentation,a fusion algorithm was designed.The analysis of experimental data results showed that the precision,recall,and F1 scores of PanicleNet-F rice panicle detection were80.13%,75.64%,and 0.53,respectively,significantly superior to PanicleNet without fine tuning.The precision and recall of the fine tuning model trained based on small data in a 10%target domain dateset had a relatively small decrease,which proved that the fine tuning strategy could transfer the feature knowledge learned from wheat panicle detection to rice panicle detection.Panicle Net-D rice panicle detection precision,recall rate,and F1 score were 88.59%,80.37%,and0.84,respectively.Compared to PanicleNet-F and PanicleNet-D,Panicle Net-C had the best detection effect,with rice panicle detection precision,recall rate,and F1 score of 89.53%,92.85%,and 0.91,respectively.On the basis of completing the digital detection of seed setting rate and rice panicle,the thermal infrared de-ghost algorithm,multi-level nested conglutinated segmentation algorithm and the field rice panicle recognition and detection transfer learning network PanicleNet-C studied in the previous chapters were integrated to complete the parameter detection of seeds and rice panicles,and to build the grain-rice panicle-canopy type rice yield estimation framework based on unmanned aerial vehicle platform.The results showed that the average absolute percentage error（MAPE）of the rice yield estimation framework was 17.12%.（3）Combining the newly constructed three types of indices,the rice yield index YI_Fusion was developed to achieve rice yield estimation based on unmanned aerial vehicle multispectral images.Aiming at the problem that the accuracy of yield estimation models was often affected by the differences of rice varieties,fertilizer treatment methods and environmental conditions,resulting in low robustness,six new vegetation indices（VI）and six new color indices（CI）were proposed based on 5-band multispectral images,and six new texture indices（TI）were constructed using the gray level co-occurrence matrix of multispectral images.The correlation coefficients between three types of new indexes and rice yield was studied,and the indices with the highest correlation ranking were selected as modeling factors.The results showed that the performance of the newly constructed VI,CI and TI were better than that of the original vegetation index and single texture feature calculated at specific bands.The best band combinations of six VIs at heading stage are VI_{1（G-Red,B-G）}（R=0.5032）,VI_{2（Red,R-RE）}（R=0.6390）,VI_3（Red,G）（R=0.6282）,VI_4（Red,B）（R=0.7036）,VI_{5（RE,Red）}（R=0.4602）and VI_6（NIR,B）（R=0.4903）,The best combinations of six VI bands at maturation stage are VI_{1（RE-NIR,B-Red）}（R=0.7742）,VI_2（Red,B）（R=0.5921）,VI_{3（NIR,Red）}（R=0.6741）,IV_4（NIR,B）（R=0.6432）,VI_5（Red,B）（R=0.8423）,VI_{6（NIR,Red）}（R=0.6402）.The CI that have the highest correlation with yield at heading stage,maturity stage and all stage are CI₂,CI₃,and CI₁,respectively.The TI that have the highest correlation with yield during these three stages are TI(CON_NIR,COR_Red),TI(CON_NIR,CON_Red),and TI(CONT_NIR,COR_Red),respectively.The performance of VI-based yield index（YI）built by quadratic nonlinear regression model（QNR）in maturity stage is better than that of other stages,and better than that of CI-based and TI-based YI.The stepwise fusion strategy of modeling factors was designed to explore the performance difference between the multiple linear regression method（MLR）and the random forest method（RF）to construct the yield index YI_FUSION.The experimental results showed that,compared with the heading stage and the all stage,the yield index,YI_FUSION,was constructed by integrating 6 VIs,6 CIs and 6 TIs through the random forest algorithm at the maturity stage,with the best performance（R²=0.84,MAE=714.55 kg/ha,MAPE=7.86%）.It was proved that the feasibility of the image-spectrum fusion strategy based on UAV in yield estimation.（4）Based on a deep learning algorithm,a decision strategy for RGB image grading of rice canopy and normalizing the weight in the rice field was proposed,and the framework for estimating rice yield using UAV image sensing was constructed.Aiming at the problems of excessive parameters,low efficiency,and unstable accuracy in the rice yield estimation process,how to fully excavate rice image information,break through the traditionally fixed modeling idea of regression fitting,and propose new modeling ideas and strategies is the key issue to improve the accuracy of rice yield estimation.Using 10%of the average yield of the experimental rice plot as the grading interval,the sub plot images were graded based on the measured yield label data to form a data set for yield grading detection.Based on the deep learning network ConvNeXt,the yield estimation method based on unmanned aerial vehicle（UAV）rice canopy image classification was proposed for the first time.The image classification accuracy of the model in the test set was 90.17%.The results showed that the yield estimation performance based on ConvNeXt image classification model（MAPE=3.96%）was better than the regression model（MAPE=6.75%）.It also shows that the method of manually setting an interval of 10%of the average yield to grade the yield of the plot image can effectively control the yield estimation error.In order to further improve the estimation accuracy and make full use of the confidence scores of the yield classification model,the normalized weight decision-making strategy was introduced in the study,and a k-weight strategy was proposed through descending the confidence scores and weight normalization processing.The effects of different weight strategy choices in the range of2-weight to 5-weight on yield estimation performance were studied.It was found that the proposed normalized weight decision-making strategy had significant correction effects on yield estimation of incorrectly classified rice plot images,and could effectively reduce MAPE to 3.79%.Based on the field rice canopy RGB image classification and the decision-making strategy of normalized weight,the rice yield estimation framework based on UAV image perception was constructed.Through model generalization research,it was proved that the rice yield estimation framework can still maintain good generalization in different regions and different varieties of data,and the MAPE can reach 4.54%.更多还原