【作者】 刘宽；

【导师】 王继曾；

【作者基本信息】 兰州理工大学 ， 计算机应用技术， 2006， 硕士

【摘要】 自然语言处理的目标之一就是找到一种方法把由简单词序列构成的句子中加上丰富的注释符号,使句子表达的含义从有结构标注的句子中比无结构信息标注的句子中更易于抽取出来。众所周知,汉语的句子是由字串构成的,词与词之间没有空格隔开。因此汉语的自动分词就成了汉语信息处理的第一步,同时也是后续的词性标注、句法分析和语义分析的基础。其中,分词歧义排歧和未登录词识别成为汉语自动分词的两大障碍,本文集中研究了分词歧义的特征与消歧处理。首先给出了汉语分词及其歧义类型的形式化描述;然后分别对两种主要分词歧义及其消歧处理做了详细的研究;最后给出了实验结果。对于组合型歧义,通过语料库学习到歧义字段的消歧规则列表并对其进行优化,然后用来纠正这些歧义字段的分词错误。与语言学专家总结的规则相比,自动学习到的规则更客观、更全面、更节省人力,是今后计算语言学研究的发展方向。对于交集型歧义,首先通过语料库学习到了每一类交集型歧义的消歧规则,然后用于纠正交集型歧义字段。同时也采用了最大概率算法和查表的方法对交集型歧义字段进行排歧,实验取得了较好的效果。更多还原

【Abstract】 One goal of natural language processing is to discover a method for assigning a rich structural annotation to sentences that are presented as simple linear strings of words, meaning can be more readily extracted from a structurally annotated sentence than from a sentence with no structural information. Because it is well known that Chinese sentence consists of a sequence of Chinese characters, Chinese word segmentation became the first step of Chinese information processing. Moreover, it is the foundation of part of speech tagging, syntax analysis and semantic analysis. Word segmentation ambiguous and distinguish from unknown word into Chinese words are the two obstacles in the technologies of Chinese word segmentation, this dissertation focus on the research of the characteristics and disambiguation of ambiguous segmentation. Firstly, this dissertation presents the formalization description of Chinese word segmentation and its main ambiguous types;secondly, each of these two ambiguous and its disambiguation methods were thorough studied;finally, experimental results were given. To combination ambiguous segmentation, we acquire and optimize disambiguation rules list through corpus, then apply the rules to correct ambiguous segmentation. Compared with the rules created manually by language experts, the automatic acquiring rules are more objective, more comprehensive, and more saving, it is the future direction of computational linguistics research. To overlapping ambiguous segmentation, the disambiguation rules of each ambiguous class are acquired through corpus too, and correct the ambiguous segmentation. At the same time, this dissertation also used the methods of based on maximum probability arithmetic and based on search list to correct the Overlapping ambiguous segmentation and achieved good results.更多还原