节点文献

面向真实文本的汉语词义排歧系统

Design and Implements of WSD System Based on Chinese Real Text

【作者】 李盛

【导师】 杨尔弘;

【作者基本信息】 山西大学 , 计算机应用技术, 2003, 硕士

【摘要】 在自然语言处理(NLP)中,词义排歧(Word Sense Disambiguation)一直是研究的重点和难点,对其他的语言信息处理任务具有重要的理论和实践意义。词义排歧任务是一项“中间任务”,其本身并不是一个终结,而是完成其它自然语言理解任务所必不可少的,它的研究成果可以直接应用于句法分析、机器翻译、文本处理和语音识别等自然语言处理系统。 本文的主要工作是研究获取支持词义排歧的知识的方法,并在此基础上建立一个面向真实文本中实词的汉语词义排歧系统。主要从以下几个方面进行了研究: 1.词义排歧知识的获取:词义排歧知识的获取是词义排歧的关键问题。在分析词义排歧所需的知识以及知识资源中提供的知识的基础上,本文研究了知识自动获取的方法。并以在线知识系统《知网》(2000)作为知识源,通过分析《知网》中的义原间的关系,自动建立了动态角色列表库、事物与属性关系库和动态实例库等一系列支持排歧的知识库。 2.词义排歧系统的设计和实现:通过获取的句法信息、频率信息、概念间的角色关系、搭配信息以及词的联想、上下文语义(聚类)和选择限制等信息,提出一个词义排歧模型,并抽取56,000词次的新闻文本语料,尝试对其中的实词(名词、动词、形容词)进行词义排歧。模型主要包括词性过滤器、局部分析器和搭配实例库等。在此基础上我们进行了系统的设计和实现。 3.词义排歧系统的评测:我们以新闻文本语料作为评测语料,语料涉及到政治、体育、农业、科技等几个方面。文本语料首先经过山西大学的分词与词性标注系统进行处理,然后作为我们系统的输入对象。评测结果表明排歧系统对于汉语真实文本的排歧是有效的,其准确率可达到80%左右。 本文通过知识获取建立起一个多种语言知识相结合的词义排歧模型,实验结果表明,排歧效果良好。

【Abstract】 Word Sense Disambiguation (WSD) plays an important role in Natural Language Processing (NLP). The study on WSD has great theoretical practical significance in Natural Language Understanding (NLU) and now it has become a hotspot and nodus. As an intermediate task, WSD is not an end but a prerequisite for other NLP task and it is helpful for machine translation (MT), information retrieval (IR) and syntax analysis etc. The main work in the dissertation is to study how to acquire the knowledge that is supporting WSD from different language resources and build a WSD system about Chinese real text .The research work of this dissertation is organized as follows.The process of WSD is a process of acquiring WSD knowledge in substance. Making full use of the existing dictionary resources can avoid the work of manually tagging word sense. The dissertation extracts the knowledge that can be used for WSD from HOWNET and builds some knowledge databases such as collocations, list of dynamic role, relations between substance and attribution and sememes-relations which are based on the analysis of the relation among the sememes which define the concepts.A good WSD system is a combination of different knowledge. We build a WSD model considering a diversity of information, including syntactic tags, word frequencies, collocations, semantic context, role-related expectations and selected references. There are a POS filtering, a partial filtering, and a collocation base. On the base of the model, we design and implement the system.Evaluation of WSD system is an important problem. In the dissertation, we use news corpora from China daily including politics, sports, agriculture, science. The corpora are POS tagged using the system of Shanxi University. The experiment results prove that the model is effective in tasks of WSD and the precision can reach about 80%.The dissertation attempts to disambiguate all the content words in the text and build a WSD system, which combines different language knowledge. The result is better man that the traditional system.

  • 【网络出版投稿人】 山西大学
  • 【网络出版年期】2004年 01期
  • 【分类号】TP391.12
  • 【被引频次】5
  • 【下载频次】212
节点文献中: