【作者】 张保华；

【导师】 赵春江； 刘成良；

【作者基本信息】 上海交通大学 ， 机械工程， 2016， 博士

【摘要】 苹果的外部品质是苹果最直观的品质特征,直接影响苹果的价格和消费者的偏爱。针对苹果外部检测的难点和关键点,基于机器视觉技术、高光谱成像技术和多光谱成像技术,综合图像处理技术、模式识别方法、化学计量学方法和光谱分析技术研究了苹果外部物理品质(形状和尺寸)和表面常见缺陷的检测方法。基于上述研究的基础上开发的检测系统和算法为我国研发基于机器视觉技术和多光谱机器视觉技术的苹果外部品质快速在线检测分级装备奠定了基础。主要研究内容和结论为:(1)基于图像处理技术的苹果形状检测中形状特征提取与果梗去除方法研究。重点研究了基于Sobel算子的边缘检测算法、形状检测中常用的特征提取算法、基于形态学特征的苹果图像中突出果梗的检测算法和基于形态学运算的突出果梗消除算法。并编写了形状检测软件。结果表明,突出果梗消除后可以提高水果形状检测的精度,为后续带有果梗的水果的形状检测提供参考。(2)基于机器视觉技术的苹果体积与重量的非接触式测量方法研究。利用机器视觉技术和近红外线阵结构光技术搭建了苹果3D信息采集系统,并基于苹果3D高度图像提取苹果投影面积、50个环状像素高度信息和50个列状像素高度信息,并利用偏最小二乘(PLS)和最小二乘支持向量机(LS-SVM)建立了苹果重量和体积的单变量和复合变量的非接触式测量模型。结果表明,基于苹果投影面积和50个列状像素高度信息为图像特征的LS-SVM复合变量模型对苹果重量和体积测量具有较高的精度。重量测量模型的Rc为0.9838,RMSEC为6.5741,Rp为0.8234,RMSEP为11.4991;体积测量模型的Rc为0.9733,RMSEC为10.0254,Rp为0.9032,RMSEP为10.1155。(3)基于机器视觉和结构光技术进行了苹果3D重建与果梗花萼识别研究。果梗花萼和缺陷在图像中很相似,它们的存在使苹果表面缺陷的自动化检测成为一种挑战。本文基于果梗花萼的凹陷性特征,基于机器视觉系统、近红外线阵结构光和3D重建技术实现了果梗花萼的检测。通过可见近红外相机、近红外线阵结构光和传送带的配合可以逐行提取苹果的3D轮廓的高度信息,经过图像拼接可以获取苹果上表面的高度图像。果梗花萼的凹陷性特征使其高度低于周围正常组织区域的高度信息。为了开发便于计算机进行识别的算法,本文提出了可根据被检测苹果的大小、形状和位置自适应建立的高度球模型。结果表明,通过3D重建模型和苹果的自适应高度球模型之间的差异便可以实现果梗花萼的识别,利用100个独立样本,200个图像进行了算法的有效性验证,总体识别精度为97.5%。(4)基于机器视觉技术进行了缺陷苹果识别方法研究。为了消除苹果表面亮度分布不均、区分真实缺陷和果梗花萼,本文提出了一种基于亮度校正算法、缺陷候选区统计和加权相关向量机算法的缺陷苹果检测算法。亮度校正算法从一定程度上消除了苹果表面亮度不均分布,尤其是边缘区域的亮度。基于果梗和花萼不能同时出现在相机视野中的常识,可以根据分割出来的缺陷候选区数目对苹果进行初步分类,对于只分割出一个缺陷候选区的苹果,提取候选区域的颜色特征、统计学特征和纹理特征,通过I-RELIEF算法计算特征的权重,通过训练加权相关向量机进行缺陷候选区的类别判断。通过对160个样本进行检测实验,总体识别精度为95.63%。结果表明,本研究提出的方法可以消除苹果表面亮度分布不均现象,缺陷候选区统计可以对苹果类别进行初步判断,利用加权相关向量机对缺陷候选区的进一步判断,效果令人满意。(5)基于高光谱成像技术进行了苹果表面隐性缺陷识别方法研究。本文基于高光谱成像技术和连续投影算法开发了苹果表面早期腐烂检测算法,提取了苹果正常表皮和早期腐烂表皮各80个矩形感兴趣区域(ROIs)的平均光谱,利用连续投影(SPA)算法进行光谱域处理,挑选出563、611、816和966 nm为识别早期腐烂的有效波长,并用偏最小二乘判别分析法(PLS-DA)分类器评估了利用SPA优选出来的四个波长。最后基于有效波长和MNF算法开发了早期腐烂识别算法。利用本研究开发的算法对测试集中20个正常苹果和80个早期腐烂苹果的最优波长处的图像进行了识别处理,检测的总体正确率为98%。利用I-RELIEF算法挑选了识别早期轻微损伤的识别算法,挑选的有效波长为590、660、720、820及960 nm,并基于有效波长处的图像和MNF开发了适合于苹果表面早期轻微损伤识别的图像处理算法,利用80个苹果样本进行了算法的验证,总体识别率为98.75%。结果表明,基于高光谱成像技术和有效波长可以实现苹果表面早期腐烂、早期轻微损伤的检测,为后续开发多光谱视觉检测系统奠定了基础。(6)基于高光谱成像技术进行了水果表面常见缺陷检测方法研究。为了消除苹果亮度分布不均和颜色的影响,实现苹果表面常见缺陷(风伤、虫伤、日灼伤、斑点、褐斑等)的精确检测,分析了正常区域和常见缺陷区域的特征光谱曲线走势,挑选了730和925 nm波长为特征波长,并基于特征波长处的图像开发了波段运算图像处理算法。由于相同位置受到的光强度的影响相似,因此像素对像素的减法和除法运算可削弱由于颜色、形状而引起的苹果表面亮度不均影响,同时基于波段运算的图像处理算法提高了缺陷和正常苹果组织之间的对比度,使缺陷提取变得容易、准确。本研究从109幅带有各种缺陷的苹果高光谱图像中提取了730和925 nm波长处的图像,并基于波段运算处理思路利用式6-10对特征波长处的图像进行了处理,总体识别率为93.6%。结果表明,波段运算方法简单有效,在一定程度上消除了颜色、形状、果梗等引起的亮度不均对缺陷识别的影响,提高了常见缺陷的识别精度。(7)基于多光谱成像技术进行了苹果表面缺陷检测方法研究。高光谱成像技术图像获取和处理需要很长的时间,不利于在线快速检测。为了继承和发展高光谱成像技术的研究成果,基于挑选出的有效波长,用SolidWorks 2011设计了转轮和封装箱,利用相机、滤光片、滤光片轮搭建了多光谱成像系统,并基于VS2010和OpenCV开发了实时检测软件。本研究是实现从高光谱视觉检测的基础性研究向多光谱视觉检测的快速应用研究转变的关键内容。本研究对68个带有各种常见缺陷的苹果进行检测,根据每个苹果的缺陷种类及分布情况,每个苹果以不同的姿态采集1-3幅(组)多光谱图像进行实时检测,本次实验共采集116组样本多光谱图像进行实时检测,总体识别率为91.38%。结果表明,利用多光谱成像系统可以继承和发展高光谱成像研究的成果,可为后续苹果表面缺陷快速在线检测装备开发提供参考。更多还原

【Abstract】 The external quality of apples is the most important and intuitional quality,which affects their point-of-sale value and consumer?s preference.In this paper,we aimed to develop algorithms to inspect the outer appearance quality and common defects of apples by using computer vision combining with hyperspectral imaging,multispectral imaging,image processing,pattern recognition,spectral analysis,and chemometrics methods.The proposed systems and methods could provide a potential support for developing fast in-line inspecting and grading equipment for apples according to their external quality.The main contents and conclusions were listed as follows:(1)Shape feature extracting and protruding stem removing algorithm were conducted by using image processing techniques.Edge detection algorithm based on Sobel Operator,the extracting methods of frequently-used shape features,protruding stem detecting method and removing algorithm were mainly studied.A hand-coded software for apple shape inspection was implemented in Visual C++ and Open Source Computer Vision(OpenCV).The results show that the protruding stem had effects on apple shape inspection,and protruding stem removing algorithm could remove the protruding stem from images and improve the inspection accuracy of apple shape.The proposed method and algorithm could be used in other fruits and vegetables? shape inspection.(2)Weight and volume of apples were non-contact measured by using computer vision and near-infrared linear-array structured light.3D height map of the upper half apple surface was obtained by using our proposed system and method.Projection area,and two different type height features would be extracted in our research.The first type height features were extracted from the 50 concentric annulus equally distributing in the height maps with an adaptive distance to the size of the inspected apples by averaging the height of all pixels in concentric annulus.The second type height features were extracted from the 50 vertical lines equally distributing in the height maps with an adaptive distance to the size of the inspected apples by averaging the height of all pixels in the vertical line.And the weight and volume evaluation models(including univariate models and composite variable models)were established by using PLS and LS-SVM methods.The results show that LS-SVM regression model based on projection area and first type height features could achieve a good prediction precision and stability.The correlation coefficient of prediction(Rp)and root mean square error of prediction(RMSEP)for the best prediction of apple weight by LS-SVM were 0.8234 and 11.4991;The Rp and RMSEP for the best prediction of apple volume by LS-SVM were 0.9032 and 10.1115.(3)Computer vision recognition of stem and calyx in apples was studied by using near-infrared linear-array structured light and 3D reconstruction in this paper.Automatic detection of common defects on apples by computer vision is still a challenge due to the similarity in appearance between true defects and stems/calyxes.Because the stem and calyx present a concave feature in apples,this paper proposes a novel stem and calyx recognition method using a computer vision system combined with near-infrared linear-array structured lighting and 3D reconstruction techniques to reveal this concavity.The 3D surface of the upper half of the inspected apples could be reconstructed by using a single multi-spectral camera and near-infrared linear-array structured light line by line on an adjustable speed conveyor belt.The height information for each pixel could be calculated by triangulation.Stems and calyxes would present a lower height than that of their neighbouring regions due to the local concave surface.In order to recognize the stems and calyxes efficiently,a standard spherical model(without stems and calyxes)is also constructed automatically,adapted to the size and boundary shape of the inspected apple.The difference between the 3D surface reconstruction and standard spherical model provides great potential for the recognition of stems and calyxes in apples.The final stem and calyx recognition algorithm was developed on the ratio images between 3D surface reconstruction images and standard spherical model construction images in gray level.The result had 97.5% overall recognition accuracy for the 100 samples(200 images),indicating that the proposed system and methods could be used for stem and calyx recognition.(4)Computer vision detection of defective apples by using automatic lightness correction and weighted RVM classifier was studied in this paper.Automatic detection of defective apples by computer vision system is still not available due to the uneven lightness distribution on the surface of apples and the similarity between the true defects,stems and calyxes.This paper presents a novel automatic defective apple detection method by using computer vision system combining with automatic lightness correction,number of the defect candidate(including true defect,stem and calyx)region counting,and weighted RVM classifier.Automatic lightness correction was used to solve the problem of the uneven lightness distribution,especially in the edge area of the apples.According to the fact that the calyx and stem cannot appear at the same view of image,some apples could be classified as sound(N=0)or defective(N≥2)apples based on the number of defect candidate region(Labeled as N)in the preliminary step without any other complex processing.For the rest uncertain apples(N=1),further discrimination was conducted.Average color,statistical,and average textural features were extracted from each candidate region,the relevant features and their weights were also analyzed by using I-RELIEF algorithm.Finally,the defect candidate regions are classified as true defect or stem/calyx by the weighted RVM classifier,and the apples would be finally classified as sound or defective class according to the category of the candidate regions.The result with 95.63% overall detection accuracy for the 160 samples indicated that the proposed algorithm was effective and suitable for the defective apples detection.(5)Recognition of unobvious defects was studied by using hyperspectral imaging techniques in this paper.Hyperspectral imaging and successive projection algorithm(SPA)were applied in the recognition of the early decay in apples.80 average spectra of the rectangle region of interests(ROIs)were extracted from sound tissues and early decay tissues.The 563,611,816,and 966 nm were selected as the potential efficient wavelengths.The classification performance of the four wavelengths was evaluated by using PLS-DA.Finally,the recognition algorithm of the early decay in apples was developed by using MNF conducted on the images at the four selected wavelengths.The detection performance of our proposed methods was evaluated by using 20 sound apples and 80 decay apples.The overall detection accuracy was 98%.Similar method was conducted to detect the early bruise in apples by using MNF and I-RELIEF methods.The 590,660,720,and 960 nm were selected by I-RELIEF as efficient for early bruise detection.80 apples were used for the detection performance evaluation,and the overall accuracy was 98.75%.Results show that hyperspectral imaging and images at the efficient wavelengths could be used for early decay and bruises detection,this could provide foundation for developing multispectral computer vision system for unobvious defect detection.(6)Detection of common defects in apples was studied by using hyperspectral imaging techniques and band math methods in this paper.In order to reduce the effects caused by uneven lightness distribution and variable color on apples surface,and improve the common defect detection accuracy,we extracted spectra from sound and defective tissues,730 and 925 nm were selected as efficient wavelengths for common defect detection by analyzing the spectral trends,band math equation was developed and applied in image processing for common defect detection.Band math methods could reduce the noise and non-uniform distribution of lightness on the fruits to some extent due to the calculation between intensity of the corresponding pixels in the same position and the influence caused by shape and color are similar in the same position.The band math method could enhance the difference between the sound and defective tissues as well,this make it easier to detect the defects.109 hyperspectral images were used to extract images at 730 and 925 nm for the common defect detection performance evaluation,and the overall accuracy was 93.6%.Results show that band math method is efficient,could reduce the uneven lightness caused by color,shape,and stem concave,and increase the detection accuracy.(7)Detection of common defects in apples was studied by using multispectral imaging techniques and band math methods in this paper.Hyperspectral imaging was verified as a useful tool for common defect detection,however,hyperspectral image acquisition and processing is time consuming,it is not suitable for inline inspection.In order to promote the development of spectral imaging in fast or inline detection of fruits,we designed a filter wheel and a box,the selected filter were installed in hole of the filter wheel,then the filter wheel and camera were enclosed in the black box to develop a multispectral imaging system.The inspection software for common defect detection in apples was implemented in Visual C++ and OpenCV.A total of 68 apple samples with various types of defects were used in our research,1 to 3 multispectral images were captured,a total of 116 multispectral images were got and used for evaluated the detection performance of our system and algorithm,the overall detection accuracy is 91.38%.Results show that the selected wavelengths and algorithms developed from hyperspectral imaging system could be implanted into multispectral imaging system for common defects inspection.And this could provide foundation for developing multispectral computer vision system and grading equipment for common defect detection or grading.更多还原