【摘要】 基于分类的统计语言模型是解决N-gram语言模型中数据稀疏问题的有效方法之一,词的自动聚类算法一直是一个难点.如何设计一种计算速度快、收敛性好的算法是关键.提出一种根据词的上下文环境,综合考虑语言模型的困惑度和词的相似度的自动聚类算法.把词的自动聚类和提高基于分类的语言模型的性能联合起来考虑.实验结果表明,该算法执行效率高、聚类效果好.更多还原

【Abstract】 Classbased statistical language model is an effective solution to the dearth of training set. It is a tough task to automatically classify words and also improtant to design a quick algorithm with good convergence. This paper proposed a method for words clustering based on the words’ context with perplexity and similarity as a measure. The algorithm combines words classification with improving the performance of classbased language model together. The algorithm is of high executing speed and good clustering performance.更多还原