【摘要】 论文介绍了一个基于词频统计的中文分词系统的设计和实现。通过这个系统,可以将输入的连续汉字串进行分词处理,输出分割后的汉语词串,一般是二字词串,并得到一个词典。词典中不重复地存储了每次处理中得到的词语,以及这些词语出现的频率。这个系统选用了三种统计原理分别进行统计:互信息,N元统计模型和t-测试。文中还对这三种原理的处理结果进行了比较,以分析各种统计原理的统计特点,以及各自所适合的应用场合。更多还原

【Abstract】 The paper introduces the design and implementation of Chinese word segmentation system,which is based on statistic the frequency of the word.Through this system,continuous character bunch input can be segmented,and then the cut apart word bunch output can be gotten,the cut apart word bunch usually is two character word bunch,and one dictionary can be gotten.The dictionary stores word and the frequency that the word appears in these disposal texts.The segmentation system selects three kinds of statistics principles to count separately:Mutual Information,N-Gram and t-test.The paper still compares the results of the three kinds of principles,analyzes the difference of statistics characteristics of the three counting principle,and finds each suitable situation.更多还原