【作者】 涂淑琴；

【导师】 薛月菊；

【作者基本信息】 华南农业大学 ， 农业电气化与自动化， 2016， 博士

【摘要】 RGB-D(彩色与深度)图像分类是计算机视觉领域的研究热点之一。将彩色信息与反映物体表面三维特征的深度信息融合,可极大地提高图像分类准确率。然而,对于大规模、复杂场景的RGB-D图像分类问题,传统的特征提取方法往往需要手工设计特征,且难以提取辨别性能高的稀疏性特征,影响了分类性能。稀疏表示方法具有鲁棒性好、泛化能力和抗干扰能力强特点;深度学习能自动地对输入样本图像进行本质特征抽取,这两类新方法的出现为RGB-D图像特征提取与学习注入新的方向。研究快速、有效的RGB-D图像稀疏表示方法。将其与深度学习技术融合,提出高效的RGB-D图像特征自动学习方法,应用到水果分类识别中,不仅将极大地推动图像分类研究的进程,也将促进农业计算机视觉的应用发展,具有重要的理论和现实意义。本文以提高RGB-D图像分类准确率为目标,采用稀疏表示和深度学习理论研究RGB-D图像特征提取、表达和学习方法。在分析目前特征提取与表达的基础上,提出新的特征学习方法,并将其应用到RGB-D图像分类和水果成熟度判别中。主要研究内容和创新工作包括以下几个方面:(1)提出了基于局部约束稀疏编码(Locality-constrained Linear Coding,LLC)的RGB-D图像特征表达方法。针对经典的稀疏编码速度慢以及未考虑特征的局部性问题,该方法分别对RGB图像和depth图像提取SIFT特征,利用LLC形成RGB-D图像的特征表示。与经典的稀疏表示相比,该方法同时考虑了特征的稀疏性与局部性,更加有利于图像分类和目标识别。在水果种类分类和RGB-D数据集上验证了局部约束稀疏表示的有效性。(2)提出了改进结构化稀疏表示的RGB-D图像特征学习算法。针对RGB-D形状高相似度图像难区分问题,在优化函数中引入理想类别标记项来增强字典判别性能,采用交替方向乘子(Alternating Direction Method of Multipliers,ADMM)方法推导字典学习与稀疏编码求解,获取改进的结构化稀疏表示作为特征表示,完成图像分类。该方法在保证特征学习的稀疏性基础上,同时实现类类之间距离最大和类内距离最小约束,实现相似图像有效区分。在RGB-D数据集上进行对比实验,表明了该方法比经典的组稀疏方法具有更好的性能。(3)提出了卷积神经网络(Convolutional Neural Networks,CNN)融合组稀疏表示的RGB-D图像特征学习方法。传统的特征表示方法中,其特征提取一般使用手工设计图像局部特征,这很大程度上依靠专家经验知识,缺乏特征提取的通用性,利用CNN方法,能自动从源图像中直接提取图像的边缘,纹理和颜色特征,形成图像的全局和局部融合特征。但是,CNN学习特征冗余较大,需进一步用很少的数据捕获其中的重要信息。组稀疏表示方法结合了稀疏表示中的L1-范数约束以及岭回归中的L2-范数约束,在组间尽可能稀疏,同时保持组内尽可能小的损失误差。对比稀疏表示方法,该方法融合CNN自动学习特征与组稀疏的组效应优点,在图像分类中能获取得更好的性能。(4)提出一种基于深度学习与组稀疏的RGB-D图像层次特征学习模型。首先,采用卷积递归神经网络方法分别从源RGB和深度图像中自动提取低层和中层特征;然后,采用组稀疏编码和改进的块字典学习方法对图像特征进行深入学习,获取更具区分与判别能力的高层特征表示;最后采用softmax分类器实现RGB-D图像分类。在RGB-D数据集和2D3D数据集上进行实验验证,表明该层次特征学习方法能有效完成大规模数据集分类,对比其他特征学习方法,其准确率有明显提升,同时融合彩色与深度图像特征可显著提高图像分类准确率。(5)设计与实现自然场景下百香果目标检测与成熟度分类的系统。为了实现机器人快速的百香果适时采摘,首先采用kinect2.0设备,建立果园场景下百香果RGB-D数据库;然后对RGB图像和深度图像分别采用HOG特征和Adaboost级联方法完成百香果果实检测,其检测精度达到82.9%;最后利用彩色SIFT特征融合ScSPM、彩色SIFT特征融合LLC和CNN特征学习方法对检测出的百香果果实进行成熟度判别,三种方法中,彩色SIFT特征融合LLC方法取得最好的分类准确率(91.52%),能高效地完成自然场景下百香果果实的成熟度分类作业。通过上述的研究工作,结果表明:有效的特征编码方法,即稀疏表示方法能够极大地改善RGB-D视觉图像分类和识别的性能;基于深度学习结构的特征方法能够对原始的图像数据学习出有效的特征,避免了人工设计特征的复杂性。基于稀疏表示和深度学习的特征提取方法,在图像识别上可获得较高的准确率,是一个非常前沿的研究方向且具有广泛的应用前景。更多还原

【Abstract】 RGB-D(color and depth)image classification is one of the hot spots in the field of computer vision.The depth of the image can directly reflect the 3D feature of object surface which can overcome the disadvantage of illumination changes,occlusion,shadows and changes in the complex environment,so the color and depth image fusion can improve the accuracy of image classification.However,for large-scale and complex scene RGB-D image classification problems,the traditional feature extraction methods often require hand-design features,and it is difficult to extract and identify the high performance of sparse features,affecting the classification performance.With sparse representation used in many fields rapidly,feature representation based on sparse representation has good robustness,good generalization ability and strong anti-interference ability.Deep learning is a deep-layer nonlinear neural network,which can extract the essential feature from the input data.The appearance of new approaches provides the new direction for feature extraction and classification.With research on rapid and effective RGB-D image sparse representation combining with deep learning method,the efficient features automatic learning methods of RBG-D image are developed.The features learning methods were applied in classification of fruit recognition and have important theoretical and practical significance,which not only will greatly promote the image classification process,and also promote agriculture computing machine vision application development.The dissertation dedicates to the research of feature representation and deep learning based on RGB-D image classification and recognition for the purpose of get effective vision features.By analyzing the current feature presentation and learning methods,we proposed new feature learning approaches and applied to solve the problem of RGB-D classification and fruit maturity discrimination.The main work and innovations of thisdissertation are as follow:(1)We propose a feature extraction method for RGB-D images based on locality-constrained linear coding(LLC).Aiming at the problem of slow speed and lack of locality in classical sparse coding,we extract SIFT features from the RGB image and Depth image and conduct feature coding using LLC method respectively.Compared with traditional sparse representation,the features representation method of LLC consider the sparsity and locality of the feature,which more benefits for image classification and recognition,and the experiments on RGB-D dataset and Fruit category datasets show the efficacy of LLC method.(2)We study an improved structured group sparse representation method via combining class label of image for feature representation,which enforces discriminability of dictionary by introducing an ideal regularization term to optimization problem.To solve the corresponding optimization problems,we develop efficient algorithm based on alternating direction method of multipliers method.For distinguishing the high similarity RGB-D images,the method is utilized to incorporate the label information and it promotes sparisity at the group level which realizes the maximum distance inter-class and minimum distance inner-class.Experimental results show that our approach method achieves the state-of-the-art recognition results on a household RGB-D object dataset with high shape similarity and outperforms than the present other sparse representation methods.(3)We propose a model to learn better feature representation by combing convolutional neural networks(CNN)and block group sparse coding(BGSC).Traditional feature representation method extracts the local features of image by the manual design method generally,which depend on expert experience and knowledge and the lack of feature extraction of general.The CNN method can automatically extract the image edge,texture and color features from source images directly,and can obtain the global and local feature fusion.However,it is necessary to further use very little data to capture the important information of CNN.The group sparse representation is combined with the L1-norm constraint in sparse representation and the L2-norm constraint in ridge regression,and the inter-group of sparse representation is as sparse as possible,while keeping the error of the intra-group as little as possible.Compared with sparse representation,this method can obtain better performance of image classification by fusing the features of CNN automatic learning and group effects of groups sparse.(4)We propose a model of Deep Learning-Group Sparse Coding(DL-GSC)to learn hierarchical feature representation from raw RGB-D by combing deep learning and group sparse coding.Firstly,the method of deep learning is trained to learn features from raw RGB-D images directly by unsupervised learning algorithm.Then the higher robust hierarchical feature representations can be learned adopting group sparse coding method and improved dictionary learning.Finally,we implement RGB-D classification by softmax algorithm.The proposed model is evaluated on RGB-D dataset.Experimental results demonstrate that DL-BGC approach can distinguish the high similarity RGB-D images and achieve better accuracy compared with other feature learning algorithms,and the RGB-D introduced to record high quality RGB and depth information can dramatically improve classification accuracy.(5)The system of passion fruit target detection and classification of maturity were designed and implemented under natural scenes.Firstly,we established passion fruit RGB-D database from the natural scene by the kinect2.0.Then,we developed passion fruit detection system adopting Ho G feature and Adaboost cascade method,the detection system detection accuracy reached 82.9%.Finally,we developed the color SIFT feature fusion Sc SPM,color SIFT feature fusion LLC and CNN feature learning methods to identify different growth stages of passion fruit,and color SIFT feature fusion LLC method can obtain the best classification accuracy(91.5%),which can satisfy the demand for classifying growth stages of passion fruit under natural scenes.In conclusion,through above work,it turns out that effective feature representation methods could improve the performance of image classification and recognition greatly.The feature learning method based on CNN architectures could learn feature from the raw image data avoiding the complexity of the design of the hand-crafted features.The featureextraction method based on sparse representation and deep learning can obtain high accuracy in image recognition,and the feature learning is a very frontier research direction and has wide application prospects.更多还原