【作者】 王波；

【导师】 王翠荣；

【作者基本信息】 东北大学 ， 计算机系统结构， 2013， 硕士

【摘要】 随着互联网的迅猛发展,信息呈现出爆炸式的增长,传统的文件存储方式已经不能满足系统对于存储空间和处理时间的要求。高性能的分布式文件系统成为相关领域的研究热点。FastDFS是一个开源的分布式文件系统,FastDFS的文件命名方式和分组方式等机制使得它具有轻量级、高扩展性等优点。在国内外学者已有研究工作的基础上,基于开源分布式文件系统FastDFS,本文针对如今互联网应用的特点,为要求高实时性的互联网应用量身定制了一款分布式文件系统,相比于FastDFS更适合大量文件的存储,特别是短时间内需要响应大量请求的应用。本文主要对FastDFS在以下方面进行了改进：一是同组storage server间同步机制的优化,由原来的根据binlog定时同步更新优化为上传文件的同时进行文件的同步；二是加入缓存层,因为对于互联网应用,本次访问的图片、视频等资源很有可能在短时间内继续被访问。三是对大文件进行分片存储,分片存储方式不仅使系统可以存储更大的文件,而且如果在某台机器宕机或者新增加存储服务器的情况下,只需要同组其他机器同步过来一个文件块的情况下,本机就可对外提供该文件的下载服务,相比于同步整个文件可以更快的对外提供服务,提高系统的可用性和吞吐量。基于VMware虚拟机和Ubuntu11.04操作系统,对本文提出的方案进行了实验测试,与现在FastDFS的最新版本V4.06相比,本文设计的一系列机制能够更快的响应客户端的请求,具有更高的实时性,更好的满足了如今互联网应用对文件操作的需求。更多还原

【Abstract】 With the rapid development of Internet, information explosion. Traditional way of file storage has been unable to meet the demand for storage space and processing ability. High-performance distributed file system have become a hot research field.FastDFS is an open source distributed file system.Given FastDFS’naming methods and grouping strategies, it is lightweight and has high scalability. Based on existing research work of scholars at home and abroad and open source distributed file system FastDFS, this paper, for the characteristics of Internet applications today and the actual situation of Internet start-ups for demanding real-time Internet applications, tailored a distributed file system. It is more suitable for applications needs store large number of files than FastDFS. This paper focuses on the following aspects which can improve the performance of FastDFS: First,optimization of synchronization strategy among storage server in the same group. Originally timing synchronization based Binlog is optimized to synchronization at the same time of uploading files;Second, add a cache layer, because for Internet applications, the visit of pictures, videos and other resources is likely to continue to be accessed in a short time. Third, use of fragmentation strategy for large file storage. Fragmentation strategy not only can make the entire system store larger files, and in case of a machine downtime,other machines within the same group only need to synchronize with over a block, the unit can provide the file download service. Compared to synchronize the entire file it can be quicker to provide services, and also improves system availability and throughput.Based on VMware virtual machines and Ubuntu11.04operating system, strategies in this paper are tested. Compare with the latest version V4.06of FastDFS, mechanisms proposed in this paper response to client requests faster, and meet the Internet applications’need for file operations better.更多还原