【作者】 张伟；

【导师】 吴中福； 廖晓峰；

【作者基本信息】 重庆大学 ， 计算机软件与理论， 2003， 博士

【摘要】 随着WWW技术在全球范围内的迅速发展与普及，网络信息资源日趋丰富，现有的万维网信息检索技术和方法已经不能满足用户对信息的快速性与有效性要求。搜索引擎是目前最主要的万维网信息检索工具，然而搜索引擎的效果还不能令人满意。因此研究新的万维网信息检索技术已成为一个很重要而又很困难的问题。数据挖掘旨在提取数据中隐含的、未知的、有用的、非一般的模式或知识，又称数据库中的知识发现。聚类是数据挖掘的基本方法之一。聚类通过比较数据的相似性和差异性，能发现数据的内在特征及分布规律，从而获得对数据更深刻的理解与认识。随着社会的信息化和各种媒体的数字化，万维网数据挖掘逐渐成了一个研究热点。信息检索技术和数据挖掘技术相结合，可使搜索引擎上升到高的理论与应用水平。把新的万维网数据挖掘技术应用到搜索引擎中去，为万维网信息的利用提出了新的解决方案，将会引起搜索引擎方面一场新的革命。因此，显而易见进行基于WWW的聚类引擎研究有着十分重要的意义。本论文的研究目的是在系统地回顾了万维网信息检索、数据挖掘、搜索引擎以及聚类的应用研究现状基础上，总结目前存在的问题，并提出相应的解决方法。特别希望通过聚类方法自动组织搜索引擎的搜索方案，便于用户发现真正需要的万维网信息。本论文主要的研究内容和创新工作体现在以下几点：（1）对万维网信息检索、数据挖掘、搜索引擎以及聚类的应用研究现状进行分析和概述，进而指出基于WWW的聚类引擎研究是一个新的具有挑战性的前沿性课题。（2）研究基于Rough集理论中的差异矩阵，并在引入扩充差异矩阵的基础上，提出了一种基于Rough集理论的不完备数据分析方法ROUSTIDA。该方法充分利用Rough集分析方法的优点，只需利用信息系统提供的信息，不需要另外附加信息，计算简单、直观。（3）分析以关键词组作为自然语言信息特征的优点，构建利用后缀数组进行关键词组提取的方法，提出了发现右完整字符串的算法find_和得到文档完整子字符串数组的算法combine__，并进一步对所提的算法进行分析，最后给出实例以验证所提算法的正确性与有效性。（4）介绍遗传算法的概念、算法结构、运算算子及主要存在问题；提出了一<WP=5>种基于遗传算法的聚类新方法MGA，该方法有两个主要优点：一是通用性强，可以对包含数值属性和符号属性的大数据集进行聚类；二是提高了数据挖掘的效率与质量。（5）构建了一个基于数据挖掘的搜索引擎原型系统，它能够以语义的、在线的、树型的方式对搜索引擎的搜索结果进行聚类（即SOTC方法），并可处理中文万维网信息。（6）论文最后除对研究工作进行总结外，还对今后的研究方向进行了展望更多还原

【Abstract】 Along with the rapid development and universal popularization of World Wide Web, information resources on Web have been expanded increasingly. This has causes current technologies of information retrieval difficult to satisfy the speediness and validity of users’ information requirement. Today, search engine is the most commonly used tool for Web information retrieval, however, its current status is still far from satisfaction. So, how to find the new information retrieval technology has been a very important and difficult question.Data mining aims at extracting hidden, unknown, useful, unusual pattern or knowledge. It is also called KDD (Knowledge Discovery in Databases). Clustering is a basic form of data mining. By contrasting the similarity and dissimilarity in data, clustering can find out the data’s inner characteristic and distribution rule, so we can obtain the further understanding. With the era of information and digital of media, Web data mining is becoming one of the hottest topics.By combining information retrieval technology with data mining technology, search engine may be up to a new high degree. It is a novel solution to apply Web data mining technologies in search engine, and it may lead to come a new revolution in search engine. So, the study of clustering engine based on WWW is very important and necessary.After systematically reviewing the development of Web information retrieval, data mining, search engine and clustering, this dissertation summarizes the existing problems in search engine, and presents the corresponding solutions. This paper focuses mainly on clustering Web search results in order to help users find relevant Web information easier and faster.The main contributions and innovations of this dissertation are as follows:(1) The current situations of application research on Web information retrieval, data mining, search engine and clustering are summarized. We pointed out the study of search engine based on WWW is a crucial research subject.(2) In this paper, the Rough set theory is deeply researched, a concept of extended discernibility matrix is introduced, and an algorithm ROUSTIDA (A Rough Set Theory based Incomplete Data Analysis Approach) for analysis with incomplete data based on Rough set theory is proposed. The advantage of this algorithm is that it uses only the<WP=7>information given by the operationalised data, and does not rely on other model assumptions.(3) The benefits of using key phrases as natural language information features are discussed. An effect method based on suffix array for key phrase extraction is presented. The algorithms of find_ and combine__ are also presented. The algorithm of find_ is to discover the right complete string, combine__ is to find the complete string of a document. We further analyze the presented algorithms and give out the example to illustrate the correctness and effectiveness of the proposed algorithm.(4) The concept of genetic algorithm, its configuration, operators and existing problems are introduced in this paper. A new algorithm for clustering analysis is presented based on genetic algorithm. There are two characteristics in our approaches. Firstly, the algorithm is the general-purposed and our clustering analyzer can cluster large data set with mixed numeric and categorical attributes. Secondly, it improves the efficiency of data mining and the quality of the knowledge.(5) A prototype system of search engine based on data mining is designed and implemented. It can group Web search results in a semantic, online and tree way, i.e. SOTC (Semantic Online Tree Clustering). It is also able to process Web information in Chinese.(6) This paper concludes by summarizing the research and indicating its future orientation更多还原