【作者】 李亮；

【导师】 王芳；

【作者基本信息】 华中科技大学 ， 计算机科学与技术， 2016， 硕士

【摘要】 随着数据量的爆发式增长以及Infiniband等高速网络技术的普及,使得高性能分布式文件系统为大数据应用提供了有效支撑。传统的TCP/IP网络架构很难充分利用网络带宽,已成为整个系统的瓶颈。远程内存直接访问(Remote Direct Memory Access,RDMA)技术采用直接读写远端内存的方式,避免了多次内存拷贝的开销,满足了分布式应用中的高速通信需求。在分布式文件系统中,为了保证较高的读写性能,客户端通常在内核态实现,而后端存储服务器出于简化设计的目的,在用户态实现,于是出现跨态对接通信的问题。若基于TCP/IP协议栈应用远程过程调用(Remote Procedure Call,RPC)模型,即使硬件上采用Mellanox公司提供的40Gbps Infiniband高速互联设备,因软件栈通信架构的限制,仍然无法获得高网络带宽。因此设计开发了一套基于RDMA的网络通信架构,实现并提供跨态的RPC over RDMA通信接口。在该跨态通信架构实现中,客户端在内核态基于NFSoRDMA状态机模型设计实现,服务器端在用户态利用RPC over RDMA设计实现。按照传统RPC逻辑,结合RDMA原语操作,完成了客户端和服务器的上下半程调用,编解码等,实现了一种分布式跨态通信加速方案,适用于用户态和内核态高效通信的分布式应用场景。对比测试结果表明,RPC over RDMA通信架构的设计使得网络带宽利用率明显提升,大约可以提升将近50%的峰值带宽,充分发挥了Infiniband网络的优势。更多还原

【Abstract】 With the explosive growth of data and the popularity of Infiniband and other high speed network technology, the high performance distributed file system provides an effective support for large data storage.The traditional TCP/IP network architecture has become a bottleneck,which is difficult to make full use of network bandwidth,. RDMA(Remote Direct Memory Access) technique is adopted to directly read and write to the remote memory, avoids the overhead of duplicated memory copy, to meet the high-speed communication needs in the distributed application.In the distributed file system, in order to ensure higher read and write performance, the client is usually implemented in the kernel space, while the back-end storage server is implemented in the user space for the purpose of simplifying the design, and this will arise the problem of communication across user and kernel space.It uses Remote Procedure Call(RPC) model based on TCP/IP protocol stack, even if the hardware using 40 Gbps high-speed Internet provided by Mellanox Infiniband company, due to the limitation of the software stack communication architecture, network bandwidth can not be fully utilized.Therefore design a set based on RDMA application in network communication architecture, which achieves RPC over RDMA communication interface across user and kernel space.In the implementation of communication architecture across user and kernel space, the client in kernel space is based on the NFSoRDMA state machine model, and the server is accomplished by RPC over RDMA in user space. In accordance with the logic of the traditional RPC achieve the the second half of the call, coding and decoding process on client and server combined with RDMA primitive operation,which implements a scheme of accelerated distributed systems communication across user and kernel space, which is suitable for the distributed application scenarios of efficient communication between the user space and the kernel space.Comparative test results show that the RPC over RDMA communication architecture design makes network bandwidth utilization rate improving significantly, nearly 50% increase of the peak bandwidth,which makes full advantages of Infiniband network.更多还原