【作者】 许研；

【导师】 沈夏炯；

【作者基本信息】 河南大学 ， 应用数学， 2007， 硕士

【摘要】 在Internet上使用搜索引擎检索信息己成为人们获取信息的重要手段,然而,这并不表明目前的信息检索技术已经让人们满意。目前大部分中文搜索引擎的查询技术基本上都是基于关键词匹配的,在这里“关键字”仅仅是出现在网页中的符号而已,它所指代的语义并没有被使用。页面分析所依据的也是存在于页面之间的链接关系,它不能表示这些页面本身包含什么信息,这就决定了搜索引擎还不能很好地处理页面信息的语义。因此,如何表达信息需求,如何展示/浏览搜索结构,如何对个性化的信息需求建立模型等等,是未来搜索引擎应该追求的方向,基于概念的智能检索才符合信息检索的需求。形式概念分析(Formal Concept Analysis,以下简称FCA)的主要内容是研究“概念”和“概念分层”的数学化描述,其主要思想是:从被表示为形式背景(formal context)的数据中获取形式概念(formal concept)以及形式概念之间的联系,形成一种以形式概念为元素的格结构——概念格(concept lattice)。使用FCA从数据中获取概念与其它基于统计的传统数据分析方法不同,FCA用概念表示数据分析的结果,用概念格显示知识视图。如何将FCA对概念的数学描述应用在信息检索中,尤其是如何为FCA搜索引擎( Search Engine)建立基于形式背景的信息检索模型(Information Retrieval Model,以下简称IR模型),这是本文工作的中心任务。本文结合形式背景定义建立IR模型,我们定义文档集为形式背景的对象集,属性集是能够代表文档特征的关键词集合的子集。从这样的形式背景中抽取出文档集与关键词之间,以及文档与文档之间在概念层次上的关系,用概念格体现形式概念之间的关系。在搜索引擎中可以在概念格的视图上对用户进行导航。由于对象集中的文档是动态添加的,而属性集也将对应的做出调整,所以我们用Godin造格算法造格。形式背景的对象集合和属性集合的确立决定了概念格的结构,并将直接影响用户在格上浏览的效率和FCASE系统的查全率和查准率。因此,如何建立形式背景,即如何为FCASE系统建立IR模型是整个系统中最关键的一步。尤其是属性集的选择,决定了搜索引擎的性能。本文提出了基于形式背景的IR模型并进一步提出了“属性抽取算法”。属性抽取算法的基本思想为:对文档进行分词处理,统计每个词的词频,计算其tf*idf值,根据权值调整规则,计算权重weight值,最后选择合适的阈值λ限定属性个数,构造形式背景,建立IR模型。实验证明属性抽取算法提炼出形式背景,构造基于FCA的IR模型的可行性。FCA-IR模型的优越性体现在对数据源的组织方面,反映出文档之间潜在的聚类关系。结合形式背景的约简等手段,该模型为用户提供了一种有实用价值的基于概念格对文档进行聚类和浏览的方法。基于FCA的IR模型的实用价值和性能在基FCA的搜索引擎(FCA SE)系统中得到了验证。本文的主要贡献如下:(1)提出了基于FCA的检索模型,并通过实验验证了该模型的可行性和正确性。(2)提出了基于文本为对象和关键词为属性的背景关系,实现形式背景中对象集和属性集的抽取。(3)根据特征项优化选择思想,实现了形式背景的属性抽取算法。(4)FCASE系统的成功运行验证了FCA模型的的可行性,也验证了FCA模型的实用性。更多还原

【Abstract】 Nowadays, using search engines to retrieval information on the internet becomes the most important device that people obtain information.However this doesn’t means that the information retrieval technique has already satisfied people.Most query technique in Chinese information retrieval is based on keyword matching. "keyword" here is only a character presents on the pages, the semantic meanings they indicate are unused, and Page-Analyzing is based on the link relations between the pages, which doesn’t show the information involved in the page.So, how to express the information requirement, how to lay out and browse the search structure, and how to build individuated and intelligent model based on information requirement etc.are the direction that future search engine puesued.Therefore, intelligent search engine based on conceptal relations is the most way to answer for the need of information retrieval.Formal Concept Analysis (FCA) is a field of applied mathematics based on the mathematization of concept and conceptual hierarchy. Thereby, it activates mathematical thinking for conceptual data analysis and knowledge processing. The major content in FCA is to extract formal concepts and connections between them from data in form of formal context so as to form a lattice structure of formal concepts. Concept lattices have been regarded as perfect abstraction of knowledge system.the results of data analysis are conepts instead of data themselves. Differing to traditional data analysis methods based on statistics, FCA obtains knowledge view which is higher level of representation of data.How to apply FCA to information retrieval, particularly, how to build up an IR model based on Formal Context for the FCA Search Engine, is the center mission of this text. This paper establishes an IR model on basis of Formal Context. Documents set is defined as the object set and keywords set that charactize the docments is defined as the attribute set.Hiberarchy relationship which reflects the connection between the documens and keyword and the connection between documents themself can be extracted from the context.Lattice is then constructed from the context .The lattice embodies the relationship clearly, and users can browse on the lattice. Since the object set extends dynamicly and so is the attribut set, we use Godin algorithm to build lattice. The context determines the structure of the lattice as well as the users’ browsing efficiency, and finally determines the Precision and Recall of the FCASE system.Therefor, how to build up the formal context, that is, how to found the IR model for the FCASE system, is the key of the whole system.Especially the choosing of the attributes dierecterly determines the capability of the search engine.This paper proposes attribute–abstracting algorithm for the IR model. The basic thought of the attribute-abstrcting algorithm is as follows:firstly, do word-segmenting then statistics the word-frequency of each word in the documents set, secondly, calculate its tf*idf value, then reckon the weight value according to the value adjusting rules, finally, choose the suitable threshold valueλto limit the number of the attributes in order to construct formal context.Test data certificates the feasibility of the attribute- abstracting algorithm in refining the context and constructing IR model based on FCA.The superiority of the IR model based on formal context is embodied in the facet of .organising data ,that is reflecting the potential relation between the documents. Combined with the context reduction method, the model provids the customer a practical browsing manner based on lattice.The practical value and function of FCA IR model are certificated in the FCA search engine (the FCA SE) system.In this dissertation, the main four contributions are as follows:(1)Proposing the IR module based on FCA and prove its high efficiency and validity by tests.(2)Puting forward context constituted with documents as objects and keywords as attributes.And aslo achieve attributes-abstracting for the context.(3)According to the thought of Term-optimizing , realizing the algorithm of attributes-abstracting.(4)Applying the IR model into the FCASE system.The successful running of the system validates the feasibility of IR model and the practicability of the attribute-abstracting algorithm.更多还原