【作者】 黄昌宁； 林娟； 孙承杰；

【Author】 Huang Changning Lin Juan Sun Chengjie Microsoft Research Asia, Beijing 100080 College of Chinese Language and Culture, Jinan University, Guangzhou 510610 Computer School, Harbin Institute of Technology, Harbin 150001

【机构】 微软亚洲研究院； 暨南大学华文学院预科部； 哈尔滨工业大学计算机学院；

【摘要】 近年带标语料库被业内人士誉为金本位,主要是因为它们常常被用作自然语言处理系统训练和测试的语料,起到标准答案的作用。作者指出带标语料库实际上是某种特定语言信息最原始的知识载体,是大规模词例化知识的基本表示形式,关于这种语言知识的其他表示形式都是由此派生出来的。以自动分词技术为例,从分词规范到“规范+词表”,然后到带分词标记的语料库,这条发展轨迹记录了人们在分词技术上取得的巨大进步,尤其是认识上的深化。本文通过2003年国际自动分词评测活动Bakeoffl介绍了活动用四种带分词标记的语料库来实现全自动评测的新方法。然后作者通过对BakeofflPK和AS两个测试语料库的调查发现,它们的分词出错率分别达到了1.29%和2.26%,从而对分词系统的评测结果带来显著的影响。为此作者建议用标注信息的出错率来定量地评价每个带标语料库的标注质量,以真正维护其金本位的功能。更多还原

【Abstract】 Annotated corpora are called gold standard nowadays. It is because those corpora are always used as knowledge standard of various natural language processing systems in training and testing. The authors point out that annotated corpus is a kind of initial carrier of certain specific information in fact. It is also a fundamental representation of lexicalized knowledge in large scale. Other representations of the given linguistic knowledge are all derived from the corpus. For example, from guidelines of word segmentation to "guideline + lexicon", and to word segmented corpus, the developing track records the huge improvement of the automatic word segmentation research, especially the improvement on notions. The First International Chinese Word Segmentation Bakeoff proposes a new fully automatic evaluation approach based on four annotated corpora. An investigation on Bakeoffl PK and AS test corpora has been introduced. 1.29% and 2.26% error rate of PK and AS test set caused the Bakeoff baseline F measure drops 1.36% and 1.93% respectively. Thus annotating error rate could be used as a quantitative assessment for any annotated corpus in order to insure its value of gold standard.更多还原