【作者】 方伟；

【导师】 李自力；

【作者基本信息】 华中科技大学 ， 软件工程， 2020， 硕士

【摘要】 在实验室工作中,需要开发一个将互联网与传统出版行业相结合的在线出版平台,平台内将产生海量的信息,比如书籍,订单,物流等信息。如果采用传统的基于数据库的SQL语句检索,一方面检索结果准确度不够高,另一方面在海量数据情况下检索性能不好。以My SQL为例,在模糊查询时会进行全表扫描,这在数据量特别大时耗时十分严重。基于以上原因,决定开发一个基于ElasticSearch的站内信息检索系统来完成网站内存在海量数据情况下的信息检索工作,来替代基于数据库SQL语句的站内信息检索。本文首先分析了国内外检索系统的研究现状,然后介绍了系统设计和实现环节运用到的相关理论技术,包括中文分词技术,倒排索引技术,ElasticSearch的文档相关性打分机制等。在用户需求分析部分,明确了站内检索系统服务于网站的普通用户和系统管理员,为他们提供包括书籍信息检索,订单信息检索,物流信息检索等功能。整个系统分为四层架构,包括数据层,分布式索引层,检索应用层和用户层。数据层负责数据的存储,分布式索引层负责索引的构建,检索应用层通过访问数据层和索引层的文件构建检索应用服务为用户提供检索功能。接着对系统的三大功能模块进行了详细的设计,包括数据模块,索引模块和检索应用模块,其中数据模块采用My SQL进行数据的存储,索引模块采用ElasticSearch进行分布式索引集群的构建,利用多节点和主副分片的方式保证索引数据的高可用性,检索应用模块采用Spring Boot构建。接着给出了系统数据库表的详细设计。最后利用黑盒测试的方法对系统的功能进行了测试,采用开源框架Jemeter对检索系统的性能做出了测试。基于ElasticSearch的站内信息检索系统实现后,完成了在线出版平台的全站信息检索工作,解决了基于数据库SQL语句检索存在的检索结果相关性不高,在海量数据情况下全站信息检索耗时严重的问题。更多还原

【Abstract】 In the laboratory work,it is necessary to develop an online publishing platform that combines the Internet with the traditional publishing industry.The platform will generate a large amount of information,such as books,orders,logistics and other information.If traditional database-based SQL statement retrieval is used,on the one hand,the accuracy of the retrieval results is not high enough,and on the other hand,the retrieval performance is not good in the case of massive data.Taking My SQL as an example,a full table scan is performed during a fuzzy query,which is very time consuming when the amount of data is particularly large.Based on the above reasons,it was decided to develop an on-site information retrieval system based on ElasticSearch to complete the information retrieval work in the case of massive data in the website,instead of simple database-based on-site information retrieval.This paper first analyzes the current research status of retrieval systems at home and abroad,and then introduces the relevant theoretical techniques used in the system design and implementation,including Chinese word segmentation technology,inverted indexing technology,and ElasticSearch’s document relevance scoring mechanism.In the analysis of user needs,it is clear that ordinary users and system administrators of the site retrieval system serve the website,and provide them with functions such as book information retrieval,order information retrieval,logistics information retrieval and so on.The whole system is divided into four layers of architecture,including data layer,distributed index layer,search application layer and user layer.The data layer is responsible for data storage,the distributed index layer is responsible for index construction,and the search application layer builds a search application service by accessing files in the data layer and index layer to provide users with retrieval functions.Then,the three major functional modules of the system are designed in detail,including data module,index module and search application module.Among them,the data module uses My SQL for data storage,and the index module uses ElasticSearch for distributed index cluster construction.The method of primary and secondary sharding ensures high availability of index data,and the search application module is built using Spring Boot.Then the detailed design of the system database table is given.Finally,the function of the system was tested by using the black box test method,and the performance of the retrieval system was tested by using the open source framework Jemeter.After the implementation of the ElasticSearch-based information retrieval system on the website,the website-wide information retrieval work of the online publishing platform was completed.The problem that the accuracy of the retrieval result based on the SQL statement is not high and the problem of time-consuming information retrieval of the whole station in the case of mass data is solved.更多还原