基于深度学习的绞线式弹性毫微型插针表面缺陷检测关键技术研究

【作者】 杨静；

【导师】 李少波；

【作者基本信息】 贵州大学 ， 机械电子工程， 2020， 博士

【摘要】 绞线式弹性毫微型插针由10股铜合金丝绞制而成,六七毫米长、直径小于0.2毫米,是高端电子元器件的主要连接器配件,广泛应用于载人航天、探月、北斗、大飞机和高分辨率对地观测系统等国家重大航空航天工程。在绞线式弹性毫微型插针加工过程中,由于加工误差,会出现各种质量缺陷。为保证产品的质量,应及时剔除不合格产品,但由于绞线式弹性毫微型插针产品体积小,质量要求高,质检员不能手工拿持产品进行剔除,需要用镊子在显微镜下逐只目检,效率低下,工作量大,严重影响绞线式弹性毫微型插针的生产效率。因此,如果能将绞线式弹性毫微型插针自动分拣到合格、针头散丝、压扁或鼓包、废品等接料盒中,对提高绞线式弹性毫微型插针缺陷检测效率,缩短检测时间,降低生产成本,提升产品的市场竞争力和增加用户满意度具有重要意义。视觉检测是目前工业界主流的缺陷检测方法之一,但在实际应用中会遇到许多挑战。如:传统图像特征提取算法基于经验手动设计特征,所设计特征通常处于较低水平且主观性强。绞线式弹性毫微型插针待检测样本缺陷多样、背景结构差异小、边界不清晰、透视失真、遮挡等复杂情况,使用传统依靠经验手动设计缺陷特征的方法不足以处理这些复杂情况。近年来,深度学习在图像特征提取方面具有较强的特征提取能力,能够自动的从数据样本中学习训练样本特征。因此,论文依托工信部智能制造综合标准化和新模式应用项目“精密电子元器件智能制造新模式应用”(工信部联装[2016]213号),以针头胖点偏移、尺寸错误、针头散丝、针头压扁与鼓包等常见的绞线式弹性毫微型插针表面缺陷问题为研究对象,以深度学习为技术手段,针对深度学习表面缺陷检测模型的稳定性、绞线式弹性毫微型插针表面缺陷识别、表面像素级缺陷分割和工程应用展开研究,主要研究内容包括:(1)基于深度学习的表面缺陷检测模型稳定性分析。针对工业视觉缺陷检测中,电子元器件缺陷类型多样、很难收集丰富的图像数据且各个缺陷类型之间存在缺陷类型数量不均衡问题,使深度学习模型因为输入数据类型间数量的差异而使训练过程出现梯度很难收敛,造成模型训练过程中的过学习和欠学习等问题。在不影响原有数据分布的情况下,提出基于Batch Normalization的训练数据类别平衡策略;然后,为进一步提高特征学习能力,提出特征空间数据扩增方法;最后,结合利普希茨连续函数,从理论证明采用多策略Batch Normalization平衡输入数据方法的稳定性比没有采用多策略Batch Normalization平衡输入数据方法的稳定性更好。在绞线式弹性毫微型插针数据集下的实验结果表明,模型对缺陷特征的平均分类准确率为90.27%。(2)基于卷积神经网络的端到端绞线式弹性毫微型插针表面缺陷识别方法。为了获得绞线式弹性毫微型插针的类型,提出了基于卷积神经网络的端到端绞线式弹性毫微型插针表面缺陷识别方法,首先,为了提高对相邻较小目标的识别能力,改进YOLO神经网络结构,采用一个全连接层以减少特征信息的丢失;在输入图像后,增加了一个2?2的最大池化使得在缩小图片尺寸的同时尽可能多的保存原始图片信息,并将多次卷积和池化操作后的网格由原来的7?7变为14?14以提高特征图谱的尺寸;其次,对于单一目标的数据,结合了边框回归和RPN方法,提出基于RPN的滑动窗口合并算法,保留更多的图像信息;最后,通过增加检测窗口内卷积核单元数量,移除全连接层,提高复杂工况下绞线式弹性毫微型插针缺陷的特征提取能力。在绞线式弹性毫微型插针数据集下的实验结果表明,本文方法在针头胖点偏移、缺陷尺寸错误、针头散丝、针头压扁与鼓包四类缺陷下的识别准确率、预测概率估计值均高于对比算法YOLO V3、Faster-RCNN和FPN。(3)面向绞线式弹性毫微型插针的表面像素级缺陷分割方法。绞线式弹性毫微型插针缺陷图像的背景区域与目标区域像素差异对比不明显,但在缺陷小范围内背景与目标可分且目标区域不会孤立存在,在第四章研究的基础上,将Unet网络作为基础网络,提出面向绞线式弹性毫微型插针的表面像素级缺陷分割方法。首先,针对绞线式弹性毫微型插针图像的背景区域与目标区域灰度对比不明显,考虑使用图像方差表示图像灰度值的变化幅度。结合相邻像元灰度差值大小与归属区域的关系,提出了基于K-means的缺陷簇搜索算法;其次,针对缺陷的尺寸和形态多样,在Unet的预测网络和分割网络中引入可变形卷积和可变形感兴趣池化模块提升神经网络对形变特征的学习能力;在绞线式弹性毫微型插针数据集下的实验结果表明,本文方法学习到了更多的真实标签数据信息,鲁棒性更好。(4)基于深度学习的绞线式弹性毫微型插针实例应用。设计并搭建了绞线式毫微型插针缺陷检测试验台,结合论文的研究成果,针对绞线式弹性毫微型插针缺陷检测中容易出现的漏检问题,提出基于中间变量的绞线式毫微型插针漏检判定算法;最后,开发了绞线式弹性毫微型插针表面缺陷检测原型系统,并展示了原型系统主要的功能模块。更多还原

【Abstract】 As the main connector accessories of high-end electronic components,the stranded elastic needles are twisted by 10 strands of copper alloy wires about 6 or 7 mm long and less than 0.2 mm in diameter.It is widely used in major national aerospace projects such as manned spaceflight,moon exploration,Beidou,large aircraft,and high-resolution ground observation systems.During the processing of stranded elastic needles,due to processing errors,various quality defects will occur.To ensure product quality,unqualified products should be removed in time.However,due to the small size and high-quality requirements of the stranded elastic needles,the product cannot be held manually by manual removal,and it needs to be visually inspected under the microscope with tweezers,which is inefficient and has a large workload,which seriously affects the production efficiency of stranded elastic needles.Therefore,if the relevant stranded elastic needles can be automatically sorted into the receiving boxes such as qualified,obese,lean,and scrap,the defect detection efficiency of the stranded elastic needles can be improved and the production cycles of products can be reduced.Visual inspection of surface defects of electronic components is one of the mainstream surface defect detection methods in the industry,but many problems and challenges have been encountered in practical applications.The features of traditional image feature extraction algorithms are usually at a low level,especially based on distinguishing between defective and non-defective features,which are manually designed based on experience,the samples to be detected are diversified and have small differences from the background structure,leading to complex scene changes such as unclear borders,lighting changes,perspective distortion,occlusion and so on,the features extracted by traditional methods are not enough to deal with these problems.Deep learning has strong feature extraction capabilities in image feature extraction.Therefore,this paper relies on the Ministry of Industry and Information Technology’s comprehensive standardization of intelligent manufacturing and new model application project “New Model Application of Intelligent Manufacturing of Precision Electronic Components”(Ministry of Industry and Information Technology [2016] 213 No.),taking the four common stranded elastic needles surface defects as the research object,and using deep learning as the technical method.Researching on four issues: the stability of deep learning,surface defect detection,segmentation of pixellevel defects,and engineering applications of stranded elastic needles.The main research contents and results are as follows:(1)Stability analysis of surface defect detection model of stranded elastic needles based on deep learning.Aiming at the surface defects of stranded elastic needles,a variety of defect types,difficulty in rich defect image data collection,an uneven number of defect objects among the defect types.This lack of sufficient industrial training data and the quantity imbalance between the target categories makes the deep learning model have gradient convergence instability due to different input data.Based on this,First,without affecting the original data distribution,proposing a training data category balance strategy based on Batch Normalization.Then,to further improve the feature learning ability,a feature space data augmentation method is proposed.Finally,combined with Lipschitz continuous function,it is theoretically proved that the continuity of the method using multi-strategy Batch Normalization to balance the input data is better than that without multi-strategy.The experimental results on the stranded elastic needles dataset show that the average recognition accuracy of the model for defect features is 90.27%.(2)Surface defect identification method based on the augmentation feature of stranded elastic needles.To obtain the type and approximate defect location of stranded elastic needles,a surface defect identification method for stranded elastic needles based on feature extraction is proposed.First,to improve the detection ability of small adjacent targets,and improved YOLO neural network structure is proposed.In this structure,the R-FCN method is adopted to use a fully connected layer to reduce the loss of feature information.After inputting the image,the RPN method is used to add a 2x2 maximum pooling layer to reduce the size of the picture while as much as possible save the information of the original picture,and change the grid after the multi-layer convolution and pooling operation from the original 7x7 to 14x14 to increase the size of the network feature map.Then,for the data of a single object,a combination of border regression and RPN method is proposed,and a sliding window merge algorithm based on RPN is proposed to retain more image information for the data.Finally,by increasing the number of convolution kernel units in the detection window,removing the fully connected layer,and combining border regression and sliding windows to improve feature extraction capability in complex environments.The experimental results on the stranded elastic needles dataset show that the method in this paper has higher recognition accuracy and prediction probability estimates for the four types of defects: needle tip fat offset,defect size error,needle loosening,needle flattening and bulging than the comparison algorithm.(3)Surface pixel-level defect segmentation method for stranded elastic needles.Due to the contrast between the background area and the object area of the stranded elastic needles defect image is not obvious,the background and object are separable and the object area does not exist in a small range of defects.Based on the previous research,Unet network is used as the backbone network,and a segmented method for surface defects of stranded elastic needles is proposed.First,as for the greyscale contrast between the background area and the object area of the stranded elastic needles image,the variance of the image can reflect the change of the grey value of the image,the separable background,and object,and the grey of adjacent pixels can reflect their belonging area,a K-means-based defect cluster search algorithm is proposed.Secondly,given the size and shape of the defects,Unet’s prediction network and segmentation network replaced to deformable convolution and deformable interest pooling modules to improve the neural network’s ability to learn deformation features.Experimental results on the stranded elastic needle dataset show that our method has learned more information about real label data and has good robustness.(4)Application of stranded elastic needles defect detection.Aiming at the problem of missed detection that is easy to appear in defect detection,combined with the built-in industrial online inspection platform,an algorithm for judging the missed detection of detection objects based on intermediate variables is proposed.We developed a prototype system for detecting surface defects of stranded elastic needles and demonstrated the implementation of various functional modules.The stranded elastic needles defect detection test bench and the prototype system for defect detection verify the rationality and effectiveness of the research method and theory.更多还原