ETL在ISBN中心业务系统中的研究与应用

【作者】 王雪；

【导师】 吴洁明；

【作者基本信息】 北方工业大学 ， 计算机应用技术， 2005， 硕士

【摘要】 本文介绍了ETL原理、国内外数据转换技术以及ETLI具发展等内容。在详细分析了数据转换理论和现有的数据转换技术后,指出了现有工具的不足之处。本文结合ISBN中心业务系统的实际需求,提出了以XML技术为基础的ETLI具解决方案,在一定程度上解决了传统ETLI具对数据库服务器造成的负载问题,均衡了各服务器的利用率。 本文根据实际项目的具体需求,采用XML为主要技术手段,建立了如下的模型:异构数据库元数据的获取模型、数据转换和清洗任务流定义模型、数据上载任务调度模型等。作者研究了数据转换过程中需要解决的技术问题,并以前述模型为基础,开发了对应ETL四个阶段的数据转换工具,通过该工具能处理JDBC所支持的数据库、EXCEL文件,以及基于多种异构数据库提取的数据之间的清洗与转换。使用者可以自己设计转换流程、定义转换规则、定时调度转换任务,该软件具有较好的通用性和灵活性。作者用此工具进行过多种数据转换的测试,效果较好。 作者在研究、开发与应用的实践中,总结出了进一步改进和完善该软件的方法,这是作者下一步要做的工作。更多还原

【Abstract】 This paper introduces ETL principle, data transformation technology, the development status of transformation tools. After analyzing the data integration theory and technology, this paper has pointed out the weak point of the existing tool. This paper proposes ETL tool solution based on XML technology, which according to the actual demand of ISBN center’s system. The solution partly transfers the large loading pressure of existing database server, solves the loading problem of the database server, and balances utilization ratio of every server to a certain extent.Accounting to the new demand that the actual project puts forward, this paper adopts XML as the main technological means, and sets up the model such as the model to obtain the metadata of heterogeneous database, the model based on XML language which used to transform and clean data and the task scheduling model to execute the working flow. The author has studied the technological problem to be solved in the transforming course. Based on the model told in the past, the author develops the integrated tools for data which corresponding the four stages of ETL tools, this tool can support RDBMS, Excel file and data from multi-source. User can design transform flow and transform rules himself. Author had test this tool with multiple data source and give a example how the data to be transformed. At the end of the paper, the author thinks what needs to do in the further.更多还原