【作者】 王军；

【导师】 张兴忠；

【作者基本信息】 太原理工大学 ， 计算机应用技术， 2012， 硕士

【摘要】 随着企业信息化建设的推广,企业信息化系统产生海量的历史数据。海量数据造成的存储容量不够、信息检索效率低下、潜在价值难以挖掘等问题成为制约企业信息化发展的瓶颈。因此,研发一套数据归档与信息检索系统来存储和管理企业历史数据显得尤为重要。本文在对数据归档与信息检索系统进行需求分析的基础上,提出跨应用程序的归档数据存储与分类检索方案,采用LUCENE全文检索技术解决历史数据的分类检索问题,应用XML技术解决应用程序间的数据交换问题,以J2EE平台的多层架构体系为基础,将系统分为数据归档、系统管理、用户管理、重构管理及信息检索等五个功能模块,完成了数据归档与信息检索系统的总体设计论文在系统研发过程中所应用的技术主要包括信息检索技术与XML技术。信息检索技术主要介绍了索引维护与管理策略、检索查询策略与结果排序策略,详细分析了三种中文分词策略(基于词典、词频统计、言语意义的中文分词策略)和文档倒排机制。XML技术则主要分析比较了两种模型映射策略：边模型映射策略和结构模型映射策略,并总结它们在不同应用场景下的优劣特性。首先本文采用既符合节点模型映射策略,又符合边模型映射策略的XParent模型对结构化数据进行模式转换,提高数据的通信能力。对于非结构化数据则使用文档解析技术先将异构化的pdf、word、html等格式的数据进行解析,解析完成后参照结构化数据进行模式转换。其次针对LUCENE中文分词上的不足,选用中科院的ICTCLAS分词系统代替LUCENE自带的中文分词系统优化索引的生成与维护,最终按照文档倒排机制,结合文档相似度排序策略实现数据的分类检索,达到提高数据的查准率、查全率的目的。系统的运行结果表明,企业信息系统的运维成本有所降低,历史数据的查准率、查全率、通信能力都符合企业指标,因此,论文所提出的方案是可行的,具有良好的应用前景。更多还原

【Abstract】 With the development of information construction, the information system will produce huge volumes of historical data. Then, problems, such as lack of storage capacity, low efficiency of information retrieval, difficult to mine the potential value will restrict the information development. Therefore, it is so important to research and develop a system about data archiving and information retrieval to store and manage historical data.According to the needs of the data archiving and information retrieval system, this thesis presents a cross-application archiving data storage and classification retrieval program, take the LUCENE full text retrieval technology to solve classification retrieval of historical data, use the XML technology to solve the data exchange between applications, and base on J2EE multi-layer Technique, this thesis divides the system into five functional modules to complete the overall design of the data archiving and information retrieval system; The five parts are data archiving, system management, user management, reconstruction management and information retrieval. During the system development process, the technology used by the thesis can divided into information retrieval technology and XML technology. The information retrieval technology mainly introduce the index maintenance and management strategies, retrieval query strategy and results sorting strategy, and deeply analyses the document inverted mechanism and three kinds of Chinese word segmentation strategies (based on the dictionary, word frequency statistics, the verbal meaning of Chinese word segmentation strategy). The XML technology mainly gives the analysis of the two model mapping strategies:edge model mapping strategy and node model mapping strategy, and summarizes pros and cons of features in different application scenarios.Firstly, the thesis use the Xparent model for structured data mode conversion to improve the data communication capabilities, which in line with the node model mapping strategy and the edge model mapping strategy. For these unstructured data, the thesis use the document parsing technology to parse the unstructured data into structured data. Secondly, in order to make up the inadequacy of the Chinese word segmentation strategy, this paper use the ICTCLAS Chinese sub-word system instead of LUCENE’S Chinese sub-word system to optimize the generation and maintenance of the index. finally, according to the document invert mechanism and combined with the document similarity sorting strategy to achieve improve the precision of the data, the purpose of recall rate.Operation of the system running results demonstrate that the operation and maintenance costs were lower, and the recall ratio and precision ratio communication ability of the historical data are accord with the enterprise indicators. Therefore, the solution proposed by the paper is feasible and has a good application prospect.更多还原