【作者】 刘勇；

【导师】 吴海；

【作者基本信息】 华中科技大学 ， 计算机技术， 2017， 硕士

【摘要】 随着数据库技术的不断发展,企业环境下不同系统使用的数据库可能不同,基本上都是关系数据库,如何解决不同系统访问的数据库间数据的一致在实际环境中尤为重要。因此,建立一个通用的数据同步中间件系统已成为解决异构数据库系统间的数据同步问题主要技术途径。目前的一些数据同步方法虽然很多,但是这些方法各有自己的优缺点,很难具体适应企业环境下的需求,在实时性要求较高的环境下很难应用,同时这些方法大多都忽略了异构数据库中事务的概念,当问题发生时容易造成数据不一致的情况。针对这些问题,本文设计并实现了一种基于Zookeeper的异构关系数据库同步中间件系统,主要工作概括如下:为了解决异构数据库同步中变化捕获方法系统资源占用高,通用性差的问题,提出了一种基于事件通知与日志的变化捕获方法相结合的捕获方式,避免日志捕获程序轮询。该方法在本文环境适用于Oracle数据库的变化捕获,系统资源开销小。为了实现同步功能,在日志捕获基础上采用基于Zookeeper的分布式服务组件以及消息队列,保证消息的顺序消费,不会出现同步语句在执行时顺序上的错乱。在异构数据库同步过程中,由于各个数据库产品数据结构以及数据类型异构,提出了一种基于SQL还原法的方法,将捕获的SQL语句转化为对应数据库平台的SQL语句。使用此方法保证了系统的开放性和扩展性。为了检验本文提出的解决方案,实现了一个可配置,性能高的异构数据库同步中间件系统。使用程序模拟数据更改,并同步这些更改到其它数据库,测试并分析结果,发现影响系统性能的地方以及不足之处。实验结果证明,本系统在能够充分保证数据库系统的自治性以及完整性,对数据库系统性能影响低,能稳定可靠的完成异构数据库之间的同步。更多还原

【Abstract】 With the continuous development of database technology,enterprises in different environments using the database may be different,basically are relational database,how to solve the different systems to access the data between the database agreement is particularly important in the actual environment.Therefore,the establishment of a common data synchronization middleware system has become the main technical way to solve the problem of data synchronization between heterogeneous database systems.Although some of the current data synchronization methods are many,but these methods have their own advantages and disadvantages,it is difficult to specifically meet the needs of the enterprise environment,real-time requirements in the environment is difficult to apply,while most of these methods are ignored The concept of transaction in the heterogeneous database,when the problem occurs easily lead to data inconsistencies.In view of these problems,this paper designs and implements a heterogeneous relational database synchronization middleware system based on Zookeeper.The main work is summarized as follows:In order to solve the problem of high resource occupancy and poor universality in the process of changing the acquisition method in heterogeneous database synchronization,a method based on event notification and the method of changing capture uses log is proposed to avoid the log training program.The method is suitable for Oracle database changing capture in this paper’s environment,the system resource overhead is small.In order to achieve the synchronization function,log capture based on Zookeeper distributed service components and message queues ensures the order of the message consumption,there will be no synchronization statement disorder at the time of execution.In the process of heterogeneous database synchronization,due to the heterogeneity of each database product data structure and data type,this paper proposes a method based on SQL restoring method to transform the captured SQL statement into SQL statement corresponding to the database platform.This method ensures the openness and scalability of the system.In order to test the solution proposed in this paper,a heterogeneous database synchronization middleware system with configurable and high performance is implemented.Use the program to simulate data changes,and synchronize these changes to other databases,test and analyze the results,find the factors that affect the system performance and shortcomings.The experimental results show that the system can fully guarantee the autonomy and integrity of the database system,the impact on the performance of the database system is low,it can be stable and reliable to complete the synchronization between heterogeneous databases.更多还原