【摘要】 潜在语义索引被广泛应用于信息检索、文本分类、自动问答等领域中。潜在语义索引是一种降维方法,它把共现特征映射到同一维空间上,而非共现特征映射到不同的空间上。在潜在语义索引的语义空间中,共现特征通过文档内部以及文档之间的特征传递关系获得。该文认为这种特征传递关系会引入一些不存在的共现特征,从而降低潜在语义索引的性能,应该对这种特征传递关系进行一些选择,削除不存在的共现特征信息。该文采用文档频率对文档集合进行特征选择,用Complete-Link聚类算法在两个公开语料上进行三个实验,实验结果显示,保留文档频度的10%~15%时,其F1值分别提高了6.577 0%,1.992 8%和3.361 4%。更多还原

【Abstract】 Latent Semantic Indexing(LSI) has been applied to many fields,such as information retrieval,text classification,automatic question answering and so on.Basically,LSI is a dimensionality reducing method by projecting term co-occurrences into the same space.Therefore,in the semantic space of LSI,term co-occurrences are obtained by the term transfer relation both in single document and between different documents.This paper suggests that this term transfer relation causes some nonexisted term co-occurrences,which reduce the performance of the LSI.To eliminate nonexistent term co-occurrences,this paper further adopts documents frequency to select features in document sets,and experiments with Complete-Link clustering algorithm on two public corpora.The experimental results show that the F-measure of clustering increases by 6.577 0%,1.992 8% and 3.361 4% when documents frequency are reserved between 10% and 15%.更多还原