结合机器学习方法的命名实体识别研究

【作者】 史永刚；

【导师】 左志宏；

【作者基本信息】 电子科技大学 ， 计算机应用技术， 2006， 硕士

【摘要】 命名实体识别是目前自然语言处理研究的热点问题。MUC(Message Understanding Conferences)对命名实体的定义是:人们感兴趣的专有名词和特定的数量词,它一般可分为:人名、地名、组织机构名、日期等类型。命名实体识别是信息抽取的一个子任务,被广泛地运用到其他计算语言学任务中,比如机器翻译等。与大多数自然语言处理技术一样,命名实体识别的方法主要分为两大类:基于规则(rule-based)的方法和基于统计(statistic-based)的方法。鉴于单独采用基于统计方法或基于规则方法的缺陷,在这篇论文中,采用了统计与规则相结合的方法来识别命名实体。为了使系统具有学习能力,我们把机器学习方法应用于中文命名实体的识别,这里我们着重研究了机器学习中的决策树方法在中文命名实体识别中的应用;设计了一种基于决策树的识别模式,该模式首先利用概率统计方法,在文本中尽量完备地识别出潜在的命名实体,然后利用潜在命名实体相关的上下文词法、语法和语义特征作为属性构建决策树,否定不正确的实体,进一步提高了命名实体识别的准确率。应用以上方法,本文主要对中文命名实体中的人名和地名识别进行了较为深入的研究。实验结果表明,与单纯基于统计和单纯基于规则的命名实体识别方法相比,本文所采用的统计和规则相结合的方法具有更加理想的识别精度;在同样的实验条件下,即同样的训练语料和测试语料的条件下,结合机器学习方法的命名实体识别系统构造简单,并且具有良好的适应性和一定的自我学习的能力。本文具体主要分为以下几个模块:1.文本预处理。2.统计和规则相结合的中文姓名和地名识别。3.结合机器学习中决策树方法的中文姓名和地名识别。4.中文姓名和地名的消歧研究。更多还原

【Abstract】 Named Entity Recognition (NER) technologies have become a hot problem of Natural Language Process recently.The definition of Named Entity by MUC(Message Understanding Conference) is the proper nouns and the quantifiers that people are Interested in.NER can be classified to person-name,location,organization,date,number and so on.NER has been applied on many compute linguistics tasts as a subtask of Information Extraction,such as machine translation.Just as most of the Natural Language Process technologies,the methods of NER have two classes, statistic-based and rule-based.Considering of the limitation of using only one of the methods,we combined both of the methods to recognize Named Entity in this thesis .we combined the maching learning with NER to make the system get the ability of self-learning.We have done research on decision tree of maching learning mainly and designed a recognize model to recognize Named Entity.This model first used the probability and statistic way to extract the potential named entities,and then some context linguistic language information are employed in the model to recognize the named entities furtherly.As the wrong entites are denied ,the recongnize effect has been improved. By using the methods above,we mainly researched on Chinese person nameand location.The result of the experiments shows that the effort of the strategy based on rules and statistics is better than use only one of them.in the same experimental condition ,the model combined on machine learning is constructed simply , has better adaptability and self-learning ability.This thesis is mainly classified to four models .1.Text preprocessing.2.Chinese name and location recognization based on statistics and rules.3.Chinese name and location recognization combined with maching learningmethods.4.Eliminating the ambiguities of Chinese name and location.更多还原