【作者】 杨永火；

【导师】 何丕廉；

【作者基本信息】 天津大学 ， 计算机应用， 2006， 硕士

【摘要】 信息检索系统的都直接或间接依赖于检索过程中的采用的检索模型,这些模型具体指明了如何比较文档和查询的特征表示来估计文档和查询相关的概率。随着各种模型的研究和开发,人们很早就注意到不同的检索模型检索出的相关文档集有很低的重叠率。因此,组合不同的文档特征表示或者不同的检索模型就成了改善信息检索系统性能的一项标准技术。信息检索的组合模型方法可以被看成是组合基于一种或多种文档特征表示的多个分类器的输出结果。在这个框架下,当每个分类器都能产生很好的相关概率估计并且相互独立时,组合模型的效果最好。向量空间模型是信息检索领域的的经典模型,自1958年提出以来,该模型在信息检索领域中一直有良好的性能表现。统计语言模型是由Ponte和Croft在1998年首次提出应用于信息检索领域的一个新颖的检索模型,它采用了完全不同于向量空间模型的方法。可以预期,组合这两种风格截然不同的模型将会使检索结果获得重要的提高。向量空间的模型的一个基本假设是检索单元之间的独立性,这样该模型就没有包含任何词序信息。但是,在几乎所有的应用中,词的相对顺序是非常有意义的,一个词的含义往往由它临近词的来解释。高阶的n-gram语言模型在一定程度上包含了词序信息。所以,组合高阶的n-gram语言模型到向量空间模型中是一个优化的组合策略。本文设计并实现了一个组合向量空间模型和Bigram语言模型的信息检索系统,在排序算法的组合上则采用了简单线性组合方式。为了兼顾计算上的简单性和性能的提高,组合模型以两阶段的方式实现,其中语言模型作为第二阶段的重排序。通过在TREC文档集上进行实验,结果表明,与向量空间模型和语言模型方法相比,该组合模型方法在不同级别的召回率上都具有更高的精度,同时平均精度也有显著的提高。更多还原

【Abstract】 Information retrieval (IR) systems are based, either directly or indirectly on models of the retrieval process. These retrieval models specify how representations of documents and query should be compared in order to estimate the relevant likelihood.As these retrieval models were being developed, quite early in the experiment it was observed that different retrieval models had surprisingly low overlap in the relevant documents that were found. So the combination of different text representations and search strategies has become a standard technique for improving the effectiveness of information retrieval.This combination approach to IR can be modeled as combining the output of classifiers. In this framework this model specifies that the best results will be achieved when the classifiers produce good probability estimates and are independent.Vector space mode (VSM) is a classic retrieval model in information retrieval area. Since it has been introduced in 1958, VSM always has a good performance in retrieval area. Statistical language model is a novel retrieval mode developed in recent years, which thinks about retrieval in a new way. Combining these two models into the same system should be expected to obtain much performance improvement..The independence of indexed terms is one of the basic assumptions in VSM, which leads to the lack of word-ordering information in this model. But the relative ordering of words is informative in almost all applications. High level n-gram language model includes the word-ordering information in some degree. So it is an optimal combination strategy to combine high level n-gram model with VSM.In this paper we design and implement an information retrieval system which combines the VSM and Bigram language model. A simple linear combination method is adopted to combine ranking algorithms. For the simplicity in computation and performance improvement, the system is realized in a two-stage way with language model as reranking used in the second stage.Through experiments in TREC document set, the result shows, compared to VSM and language model, the combination approach achieves higher precision in all levels’recall and the mean 11-AVG precision also has a significant improvement.更多还原