【作者】 张梅山； 邓知龙； 车万翔； 刘挺；

【Author】 Zhang Meishan,Deng Zhilong,Che Wanxiang,Liu Ting Center for Information Retrieval of Computer Science & Technology School,Harbin Institute of Technology,Harbin 150001

【机构】 哈尔滨工业大学信息检索研究中心；

【摘要】 基于统计的中文分词方法往往不具有良好的领域自适应性。本文通过将外部词典信息融入统计分词模型(本文使用CRF统计模型)来实现领域自适应性。实验表明,这种方法具有良好的领域自适应性。当测试领域和训练领域相同时,分词的F-measure值提升了2%;当测试领域和训练领域不同时,分词的F-measure值提升了6%。最终优化后的分诃速度也得到了很大的改善。更多还原

【Abstract】 Generally,statistical methods for Chinese Word Segmentation don’t have good domation adaption.We propose an approach which can integrate extern dictionary information into statistical models to realize domain adaption for Chinese Word Segmentation.In the paper,we use the CRF statistical model.Experimental results show that our approach have good domain adaption.When domain of test corpus is identical to the training corpus,the F-measure value has an increase of 2%;when domain of test corpus is different with the training corpus,the F-measure value has an increase of 6%.The final speed of segmentation has also been improved greatly after optimized.更多还原