【摘要】 通过对传统Jaccard算法的研究和改进,提出了一种基于词向量的Jaccard句子相似度算法。传统的Jaccard算法以句子的字面量为特征,因而在语义层面的相似度计算方面受到了一定的限制。而随着深度学习的兴起,尤其是词向量的提出,词语在计算机中的表示有了突破性的进展。该算法首先通过训练将每个词语映射为语义层面的高维向量,然后计算各个词向量之间的相似度,高于阈值α的作为共现部分,最终计算句子的相似度。实验表明,相较于传统的Jaccard算法,该算法在短文本相似度计算的准确率上有较明显的提升。更多还原

【Abstract】 Based on the research and improvement of the traditional Jaccard algorithm,this paper proposed a Jaccard sentence similarity algorithm based on word embedding.Traditional Jaccard algorithm is characterized by literals of the sentence,so it is restricted in the respect of semantic similarity calculation.While with the rapid development of deep learning,especially the proposal of word embedding,there is a breakthrough on the expression of words in computer.This algorithm firstly maps each word into a high-dimensional vector on semantic level by training,and then calculates the similarity between the respective word vector.The results which are higher than the thresholdαare regarded as the intersection,and finally the sentence similarity is calculated.Experiments show that the algorithm significantly improves the accuracy of short text similarity calculation comparing with traditional Jaccard algorithm.更多还原