节点文献

消解中文三字长交集型分词歧义的算法

Algorithm for solving 3 character crossing ambiguities in Chinese word segmentation

  • 推荐 CAJ下载
  • PDF下载
  • 不支持迅雷等下载工具,请取消加速工具后下载。

【作者】 孙茂松左正平黄昌宁

【Author】 SUN Maosong, ZUO Zhengping, HUANG Changning Department of Computer Science and Technology, Tsinghua University; The State Key Laboratory of Intelligent Technology and Systems, Beijing 100084, China

【机构】 清华大学计算机科学与技术系

【摘要】 汉语自动分词在中文信息处理现实应用中占据着十分重要的位置。三字长交集型分词歧义是分词歧义的主要类型之一,在真实文本中的出现频率相当高。提出了一种针对这种分词歧义的消解算法,回避了训练代价比较高昂的词性信息而仅仅利用了词的概率信息及某些具有特定性质的常用字集合。从一个60万字的汉语语料库中抽取出全部不同的三字长交集型分词歧义共5367个作为测试样本。实验结果表明,该算法的消解正确率达到了92.07%,基本可以满足实用型中文信息处理系统的需要。

【Abstract】 The technique of Chinese word segmentation plays an important role in many applications of Chinese information processing. Being one of the major types of segmentation ambiguities, crossing ambiguities with length of 3 characters can be frequently found in Chinese running texts. An algorithm aiming at this type of ambiguities is proposed in the paper: instead of making use of part of speech statistical information which needs comparatively high training cost, the algorithm simply employs word frequency information and some common Chinese character subsets with defined properties. The preliminary experiment on 5367 examples, extracted from a Chinese corpus of 0.6 million characters, shows that the segmentation precision of the algorithm reaches 92.07%, which is satisfactory for practical Chinese information processing systems.

【基金】 国家自然科学基金
  • 【文献出处】 清华大学学报(自然科学版) ,JOURNAL OF TSINGHUA UNIVERSITY(SCIENCE AND TECHNOLOGY) , 编辑部邮箱 ,1999年05期
  • 【分类号】TP391.1
  • 【被引频次】69
  • 【下载频次】499
节点文献中: 

本文链接的文献网络图示:

本文的引文网络