【作者】 李爽；

【导师】 强晓永； 康玉琢；

【作者基本信息】 天津科技大学 ， 电子信息（专业学位）， 2024， 硕士

【摘要】 近些年来,随着科技的进步和发展,满足人类生活的计算和存储需求在不断增加,分布式海量数据存储系统技术的应用也日益普及。与传统的中心化服务器存储的方式相比,分布式海量数据存储系统在存储效率、访问速度、安全性以及存储量级等方面都具有显著的优势,但是会引入复杂的数据一致性、系统高可用性以及稳定性等相关问题。在此背景下,本文基于ZooKeeper分布式协调服务框架,提出了分布式海量数据存储系统,主要研究内容及创新点如下:(1)设计了基于ZooKeeper的高可用作业架构。对ZooKeeper原生的功能进行了改进,设计出将作业分割成多个独立阶段的接口,每个阶段即为高可用任务支持的最小执行单位;在每个阶段执行之前,将任务信息持久化记录到ZooKeeper中,以便于任务的恢复;支持对ZooKeeper操作的事务机制,确保对ZooKeeper的操作是原子性的;支持新的高可用服务接管任务时的恢复和继续执行任务的能力。(2)完成了选举功能的设计。本论文基于ZooKeeper的临时序列节点机制定制了选举模块,支持根据访问区隔离,以避免跨越不同访问区的业务操作;确保整个生命周期中,在任何访问区内只存在一个主节点提供服务;节点发生故障时,能够主动退出选举,以免不健康的节点成为主节点,从而影响整个集群的业务运行;选举过程需要迅速恢复,以确保集群在选举期间不会长时间处于不可用状态。(3)完成了内部代理(负载均衡)的设计。为了让请求在系统中负载更加均衡和发挥整个系统的性能,本论文构造了内部代理模块:Name Node Internal Rpc Server(简称执行节点)、Name Node Master Rpc Server(简称master节点)和Name Node Ap Rpc Server(简称ap节点)。通过内部代理的设计将系统请求的具体内容进行无损转发,还对本地调用的上下文信息进行透传以便转发链路的各个节点能够感知到请求的完整信息来完成业务,提高了系统的高可用性。(4)完成了缓存功能的设计。考虑到分布式海量数据存储系统获取元数据信息会有指数级的成本消耗,本论文设计出一个敏感的缓存机制来提升系统获取元数据信息的性能:以路径关系和文件名作为基础结构按照树形进行缓存;系统需要高效的缓存失效策略来避免脏数据,采用一个主节点来进行缓存的维护,而不是使用分布式缓存,因此降低了系统的维护成本,保障了数据的一致性。最后完成了分布式海量数据存储系统的搭建和测试。根据测试用例编写自动化脚本,通过最小集测试和长稳测试的结果表明系统能够稳定运行,没有出现过数据不一致以及节点宕机系统长时间不可用的情况,验证了基于ZooKeeper的分布式海量数据存储系统方案的可行性。更多还原

【Abstract】 In recent years,with the progress and development of science and technology,the computing and storage needs to meet human life are increasing,and the application of distributed massive data storage system technology is becoming increasingly popular.Compared with the traditional centralized server storage method,the distributed massive data storage system has significant advantages in terms of storage efficiency,access speed,security,and storage volume,but it will introduce complex data consistency,system high availability,and stability and other related issues.In this context,based on the ZooKeeper distributed coordination service framework,this thesis proposes a distributed massive data storage system,and the main research contents and innovations are as follows:(1)A high-availability operation architecture based on ZooKeeper is designed.The native functions of ZooKeeper are improved,and an interface is designed to divide the job into multiple independent stages,each stage is the minimum execution unit supported by the high-availability task,the task information is persistently recorded to ZooKeeper before each stage is executed,so as to facilitate the recovery of the task,the transaction mechanism of the ZooKeeper operation is supported to ensure that the operation on the ZooKeeper is atomic,and the ability to recover and continue to execute the task when the new high-availability service takes over the task。(2)Completed the design of the election function.Based on the temporary sequence node mechanism of Zookeeper,this thesis customizes the election module to support isolation according to the access area to avoid business operations across different access areas,ensure that only one master node provides services in any access area during the whole life cycle,actively withdraw from the election when the node fails,so as to prevent unhealthy nodes from becoming the master node,which will affect the business operation of the whole cluster,and the election process needs to be quickly restored to ensure that the cluster will not be unavailable for a long time during the election period.(3)Completed the design of the internal proxy(load balancing).In order to balance the load of requests in the system and give full play to the performance of the whole system,this thesis constructs internal proxy modules:Name Node Internal Rpc Server(referred to as the execution node),Name Node Master Rpc Server(referred to as the master node),and Name Node Ap Rpc Server(referred to as the ap node).Through the design of the internal agent,the specific content of the system request is forwarded losslessly,and the context information of the local call is transparently transmitted so that each node of the forwarding link can perceive the complete information of the request to complete the business,which improves the high availability of the system.(4)The design of the caching function is completed.Considering that the distributed massive data storage system will have exponential cost consumption in obtaining metadata information,this thesis designs a sensitive caching mechanism to improve the performance of the system in obtaining metadata information:the path relationship and file name are used as the infrastructure to cache according to the tree shape,and the system needs an efficient cache invalidation strategy to avoid dirty data,and a master node is used to maintain the cache instead of using the distributed cache,so the maintenance cost of the system is reduced and the consistency of data is ensured.Finally,the construction and testing of the distributed massive data storage system were completed.The results of the minimum set test and the long-term stability test show that the system can run stably,and there is no data inconsistency,node downtime,and the system is unavailable for a long time,which verifies the feasibility of the distributed massive data storage system scheme based on ZooKeeper.更多还原