节点文献
Readdir++:一种并行网络文件系统中海量小文件读优化技术
Readdir + + : A Performance Optimization Technology for Massive Small-file Read-access in Parallel Network File System
【摘要】 高效管理海量小文件,提供低延时的小文件访问服务,是并行网络文件系统的研究热点.p NFS采用数据带外访问模式,元数据的访问频率高,对海量小文件的访问性能造成了很大影响.本文扩展了p NFS中的读目录授权机制,并在其框架下提出了一种Readdir++技术,用于批量预取layout等元数据,并确保预取元数据的有效性.Readdir++技术避免了小文件读取过程中同步的元数据获取开销,有效降低了访问延迟,提升了海量小文件的读取访问性能.典型应用环境实测表明,对比p NFS系统,Readdir++技术可将海量小文件读取访问过程中元数据性能提升到14.27倍,总体性能提升到1.78倍.
【Abstract】 Howto manage massive small-file efficiently and provide low-latency service is a hot topic in p NFS. Because of the Out-Ofn Band data model in p NFS,metadata access frequency is too high,which have a negative impact on the performance of massive smallfile storage system. This paper extends the read-only delegation mechanism in p NFS,presents a technology called Readdir + + in order to prefetch metadata in batch,and is able to ensure the effectiveness of these metadata. This technology effectively reduces the accesslatency and greatly improves the massive small-file read access performance. The effectiveness of readdir + + is proved with intensive experiments: when accessing massive small-file,compared with p NFS,the performance of metadata is 1427% higher,and the overall performance is 178% higher.
【Key words】 massive small-file; parallel network file system; meta data; prefetching technique;
- 【文献出处】 小型微型计算机系统 ,Journal of Chinese Computer Systems , 编辑部邮箱 ,2014年11期
- 【分类号】TP333
- 【被引频次】2
- 【下载频次】72