【作者】 吴锐；

【导师】 唐降龙；

【作者基本信息】 哈尔滨工业大学 ， 人工智能与信息处理， 2010， 博士

【摘要】 图像中的文本信息是理解图像内容的重要线索,自然场景中存在着大量的文本信息,这些信息可以为场景理解提供极有价值的帮助。因而,研究一种自动化的文本信息识别工具,自动从自然场景图像中识别出文本内容,对场景图像的检索、分析以及场景理解均有重要意义。传统文档分析技术尽管已经取得了巨大的成果,但无法有效处理自然场景中的文本字符。自然场景中的文本图像与传统文档图像相比具有明显不同,表现在文本图像的颜色、亮度、对比度不一致;文本所在的背景复杂多变;文本图像存在变形、残缺、模糊、断裂现象;图像中可能包含强烈的噪声干扰,等等。这些因素使得自然场景文本识别技术面临诸多挑战。本文针对自然环境文本识别相关技术进行研究,重点研究自然场景下文本图像变形的矫正方法、复杂背景下文本图像的分割方法、低质量字符图像的识别方法。具体内容如下:1.自然场景的文本图像中存在大量透视变形的情况,文本图像发生变形会严重影响文本识别的整体性能。本文对场景图像的透视变形问题进行了研究,并针对自然场景图像中文本行数少,缺乏段落信息,文本边缘不完整的特点,提出了一种基于消失点的变形矫正方法。该方法分两个步骤:1)检测消失点:文本基线的交点就是消失点,对点的检测可以转化为参数空间对线的检测。首先利用数学形态学算子提取文本字符上的边缘端点,接着采用最近邻方法对端点按照文本行的位置进行分类,然后使用最小二乘法在筛选后的端点集合中拟合文本基线,最后通过RANSAC方法选择出一个最优的文本基线集合,在参数空间拟合出消失点对应的直线。2)变形恢复:文本图像的变形参数包含在单应矩阵中,利用消失点的位置信息对单应矩阵中的部分参数进行估计,然后使用单应矩阵对图像实现矫正恢复,获得正视图的文本图像。由于该方法是从文本图像的字符中获得变形参数,不受文本的边缘和段落格式限制,因此可以对场景文本进行处理。实验结果表明,使用本文方法对大量变形的场景文本图像进行矫正后,字符识别率明显提高。2.字符分割是字符识别前的关键步骤,自然场景下的字符区域常常有复杂的背景,这会给从背景中分割出字符图像造成困难,本文提出了一种基于谱聚类的字符图像分割方法。该方法与通常的谱方法最大的不同是利用颜色空间量化来构造相似矩阵,大大降低了求解特征系统时的复杂度。具体步骤是:1)构造相似矩阵:首先对图像进行HSV空间变换并量化该色彩空间,然后定义基于像素色彩信息、纹理信息、距离信息的相似度函数,并计算各像素之间的相似度,最后以色彩量化区间为元素构造图像的相似矩阵。2)求解特征系统:以相似矩阵对应的拉普拉斯矩阵为特征矩阵建立特征系统,求解该特征系统的最小特征值及其特征向量。3)图像分割:对最小特征值对应的特征向量进行两类划分,建立一个二值指示向量,按照该向量对相似矩阵进行分类,通过分类后的相似矩阵进一步实现原图像的二值分割。对大量场景文本图像进行测试,实验结果表明该方法优于传统的阈值分割方法,也优于基于谱图准则的阈值分割方法。3.自然场景中文本图像的特点决定了场景文本的质量低下,现有字符识别方法对字符图像的变形、高噪声、低分辨率等问题还不能有效处理。本文对低质量的字符识别问题进行了研究,提出了一种基于改进Gabor小波变换的特征提取方法,并在此基础上实现了一种高鲁棒的汉字识别方法。本文通过分析低质量汉字图像特点,利用Gabor函数良好的频率选择性和方向选择性,构造了一个适合于汉字特征提取的Gabor小波变换,并进一步提出了基于尺度重叠和基于方向预分类的改进小波变换。尺度重叠强化了Gabor滤波器对笔划宽度的选择,方向预分类使得滤波器对笔划方向的选择更有针对性。由于充分考虑了汉字笔划宽度和方向的多峰值分布,优化了小波变换参数,从而获得一种高鲁棒性的汉字特征。在HCL2000标准手写汉字库和低分辨率汉字库上的测试表明,该方法对笔划变形和低分辨率的汉字字符具有良好的适应性,能有效处理质量较差的汉字字符。最后,利用提出的多个方法,本文建立了一个自然场景文本信息识别系统,通过实验表明该系统在处理自然场景文本图像的有效性。由于本文建立的文本识别系统对输入图像的限制较少,可以认为是针对实际应用系统研究的一个有益探索。值得一提的是,基于颜色空间量化的谱聚类文本分割方法可以为其它图像分割问题提供思路;改进的Gabor小波特征也是对字符变换系数特征的一个补充。更多还原

【Abstract】 The texts in the image can be used as the vital clue to understand image content. There are rich texts containing important information in natural scenes, which can provide very valuable help to understand the scene. Thus, an automatic tool developed to recognize text from natural scene images is of great value to the image retrieval, analysis and scene understanding.Despite the traditional document analysis technology has made great achievements, it can not deal effectively with text in natural scene. There is obvious difference between text images of natural scene and traditional document, such as the inconsistent of color, brightness, contrast; the changeable and complex background where the texts embed; the deformation, incomplete, blur, and fracture of text image; the strong noise which contains, and so on. These make text recognition from natural scene face many challenges.This dissertation studies the related techniques for text recognition from natural scene, with the emphasis on the image distortion correction, the image segmentation in the complex background, and the low-quality character recognition. Details are as follows:1. There is much perspective distortion in the text images of natural scene, which will degrade seriously the overall performance of the text recognition. This dissertation studies the perspective distortion of the images and presents a method of distortion rectification based on the vanishing point, for that there is few text lines in scene images lack of paragraph information or the edge of document is incomplete. The method consists of two procedures:1)Detection of the vanishing point: The vanishing point is the intersection of the text baselines, which can be found by detecting the line corresponding in parameter space. Firstly, the endpoints are extracted from text characters using mathematical morphology operators, and then these tip points are classified in accordance with the location of the line using the nearest neighbor method. Secondly, the text baseline is obtained from the classified tip points based on the least-square method. Finally, the RANSAC estimation method is used to select the optimal baseline set and from which the line corresponding to the vanishing point is fitted. 2)Distortion recovery: The deformation parameters of images which contain in homography matrix can be calculated from the position of vanishing point and then the front view image achieved using homography matrix. Since the method obtained deformation parameters from characters of text out of limit of edge of the text and paragraph formatting, it can deal with the scene text image. The experiment results show that the recognition rate in deformed scene images has improved significantly after image correction using this method.2. Characters segmentation is a key step for recognition. The text images in natural scene often have a complex background, which cause difficulties separating the character from the background. A method for characters segmentation based on spectral clustering is presented. Main difference between the usual spectral method and our method is that the similarity matrix is constructed by quantifying the color space, which greatly reduces the time complexity for solving the eigenvalue system. Detailed steps are as follow: 1) Constructing the similarity matrix: Firstly, the original image is transformed into HSV space and quantified. Secondly, the similarity function is defined based on color, texture, and distance of pixels. Finally, the similarity matrix is constructed as color bins which quantified for the elements of matrix. 2) Solving the eigen-system: To establish a standard eigen-system using Laplacian matrix corresponding to the similarity matrix, and solve it to get the smallest eigenvalue and eigenvector. 3) Image segmentation: Firstly, the eigenvector corresponding to the smallest eigenvalue is divided into two classes, and then an indicator vector is built according to the division. Secondly, the similarity matrix is divided into two parts according to the indicator vector. Finally, the original image is segmented according to the matrix. A large number of scene text images are tested and the experimental results show that the method is superior to other methods in the literature.3. The features of natural scene text images determine the scene text low-quality. Existing methods can not handle the issues such as deformation, strong noise and low-resolution characters. A method for Chinese character recognition is proposed based on improved Gabor wavelet transformation in the dissertation. This method makes use the frequency selectivity of Gabor function and constructs a suitable wavelet transform to extract the feature of Chinese characters. And further the improved Gabor wavelet is proposed based on scale overlap and the direction pre-classification. The scale overlap reinforces the selection of Gabor filters to strokes width and the direction pre-classification makes the selection to the direction more accurate. As full consideration of the multi-peak distribution of character stroke width and direction, the wavelet parameters is optimized. And thus, a high robustness feature of Chinese characters is obtained. The tests on HCL2000 and low-resolution character library show that the method good performance and be able to deal effectively with low-quality Chinese characters.Finally, a text recognition system from natural scene is constructed based on several methods presented in this dissertation. Experimental results show the validity of the system. Because of less restrictive to the input image in this dissertation, it can be considered as a useful exploration towards the study of practical application system. Especially, the method of the spectral clustering based on color space quantization maybe can provide some idea for image segmentation. And the improved Gabor wavelet feature is a supplement for transform coefficients features of characters.更多还原