【摘要】 计算文本的相似度是许多文本信息处理技术的基础。然而,常用的基于向量空间模型(VSM)的相似度计算方法存在着高维稀疏和语义敏感度较差等问题,因此相似度计算的效果并不理想。在传统的LDA(Latent Dirichlet Allocation)模型的基础上,针对其需要人工确定主题数目的问题,提出了一种能通过模型自身迭代确定主题个数的自适应LDA(SA_LDA)模型。然后,将其引入文本的相似度计算中,在一定程度上解决了高维稀疏等问题。通过实验表明,该方法能自动确定模型主题的个数,并且利用该模型计算文本相似度时取得了比VSM模型更高的准确度。更多还原

【Abstract】 Many information processing techniques are based on computing the similarity of text.However,the traditional method of similarity calculation based on vector space model has the problems of high dimension and poor semantic sensitivity,so the performance is not very satisfactory.This paper proposed a self-adaptive LDA(SA_LDA)model based on traditional LDA model.It can manually determine the number of topic.Applying it in text similarity calculation,it can solve the high dimensional and sparse problem.Experiments show that this method improves the accuracy of similarity calculation and the effect of text clustering compared with VSM.更多还原