【作者】 孙青；

【导师】 曹忠升；

【作者基本信息】 华中科技大学 ， 计算机技术， 2020， 硕士

【摘要】 在基于计算与存储分离架构的分布式数据库中,计算节点和存储节点的本地缓存容量对数据库系统的整体性能均有着极大的影响。为了应对因海量数据引起的缓存容量不足的问题,有必要为计算节点和存储节点提供容量大、成本低的多级缓存系统,以此来支持数据信息的快速获取,从而提高数据库系统应对海量数据处理的能力。针对计算节点和存储节点中对数据页高速获取的要求,设计了集缓存分区管理机制、多级缓存机制和监控调节机制于一体的多级缓存系统架构,并给出了相关策略。多级缓存系统主要包含5个模块,分别是一级缓存模块,二级缓存模块,分区管理模块,过滤层模块和监控调整模块,其中二级缓存模块建立在NVMe固态硬盘上。给出了计算与存储分离架构的分布式数据库多级缓存系统的分区管理策略,获取数据页策略,添加缓存页策略和监控调整策略。其中,多级缓存系统的分区管理策略采用对缓存进行分区的方式来减少操作之间对锁资源的争抢;获取数据页策略和添加缓存页策略共同规定了多级缓存在工作过程中对数据页的处理方式;监控调整策略则是根据数据库系统的真实负载情况对多级缓存系统的二级缓存容量进行适应性调整。为了在充分利用NVMe固态硬盘高速读写优点的同时尽可能规避其写放大的问题及耐擦写性较差的问题,多级缓存系统采用了定长存储以及限定写入二级缓存数据页数量的方式。采取了多种方案对所实现的多级缓存系统的性能进行了测试。测试结果表明,计算节点和存储节点多级缓存系统均能提升系统整体性能,但存储节点的多级缓存系统对系统整体处理数据的性能提升较小,计算节点的多级缓存系统对系统整体处理数据的性能提升较大;在计算节点与存储节点之间的网络传输速率越低,二级缓存数据存储介质读写速率越快的情况下,计算节点的多级缓存系统的能效越大。更多还原

【Abstract】 In the architecture of computing and storage separation,the local cache of both the computing and storage nodes has a significant impact on the overall performance of the distributed database system.In order to cope with the shortage of cache caused by dealing with the massive data,it is necessary to provide a multi-level cache system with large capacity and low cost for computing and storage nodes to support the rapid acquisition of data information.Aiming at requiring of accessing data pages in a high speed of computing and storage nodes,a multi-level cache system architecture is designed,which integrates cache partition management,multi-level cache and monitor and adjustment,and the relevant strategies are given.The multi-level cache system consists of 5 modules,namely,the first level cache module,the second level cache module,the partition management module,the filter layer module and the monitor and adjustment module.This paper presents the partition management strategy,data page fetching strategy,cache page adding strategy and monitoring and adjusting strategy of the distributed database based on the architecture of computing and storage separation.The Partition Management Strategy of multi-level cache system uses the partition method to reduce the contention of lock resources between operations,the policies of getting data pages and adding cache pages jointly prescribe the handling of data pages in the working process of multi-level cache,the strategy of monitoring and adjusting is to adjust the capacity of two-level cache in multi-level cache system according to the real load of database system.In order to make full use of the advantages of the NVMe solid-state drives and avoid the problems of write amplification and poor write resistance,the multi-level cache system adopts the method of fixed-length storage and limited number of data pages written to two-level cache respectively.Several schemes are adopted to test the performance of the implemented multi-level cache system.The test results show that the performance of the system can be improved by the multi-level cache system both on computing node and the storage node,but the performance of the multi-level cache system on storage node is improved little,when the rate of the network transmission between the computing node and the storage node is lower or the read-write rate of the NVMe solid-state drives is higher,the more greatly performance of the multi-level cache system of the computing node can be improved,the more energyefficient of the multi-level caching system of computing nodes is.更多还原