【作者】 华清；

【导师】 黄林鹏；

【作者基本信息】 上海交通大学 ， 计算机软件与理论， 2007， 硕士

【摘要】 今时今日,网格技术已经不再把自己局限在高性能计算的范畴中,而是通过向网络服务靠拢,建立起一套面向服务的体系架构。相应于此,网格中的数据模块也不再是一个提供资源的底层支持模块,而逐渐向一个功能独立的,相对自治的分布式文件系统发展。本文的目的在于构造一种网格环境下的分布式文件系统。在结构上它相当于中国教育科研网格底层支撑平台的数据模块部分,为CGSP其他功能模块提供持久的数据存储功能以及稳定而高效的数据传输服务。另一方面,这个系统也可以独立运行,网格终端用户可以通过它建立自己的数据空间。传统的分布式文件系统往往是紧耦合的、基于文件级别应用副本策略的。这些系统虽然性能出众、但往往是应对专门系统设计,通用性不强。而过去基于Web的分布式文件系统往往性能低下、存储能力弱、传输效率低。为了满足网格环境下海量数据密集型应用的需求以及面向服务的框架,我们设计并实现了一个基于分片的、松耦合的分布式文件系统,它包括底层存储资源集合、存储资源管理模块、元数据管理模块、副本管理模块、数据传输管理模块、信息监控模块、策略分析模块以及虚拟文件视图终端等部件。本文设计并实现的系统具有以下的特点和优势:稳定性:通过副本管理模块调整系统中文件合理的冗余度,对”零副本危机”进行预测并予以避免。策略分析模块会在存储资源选择时挑选健壮性最佳的数据节点进行存储;高效性:在传输中使用GridFTP带状并行传输,同时提高服务器端和客户端的带宽利用率。策略分析模块会在构造传输计划时挑选最近、当前可用带宽最大的节点进行传输;通用性:通过使用网络服务资源框架实现远程调用,存储资源只需向存储资源管理模块汇报就可以加入资源集,用户也可以在任何终端登入自己的用户空间;更多还原

【Abstract】 Nowadays, grid technology would no longer limit itself in the field of high performancecomputing, instead it is moving along to the web service technology and has been construct-ing an SOA grid architecture. Accordingly, the data module of grid will become an individualautonomic distributed file system instead of just a support module.This thesis aims to construct a distributed file system in grid for China Grid SupportPlatform and provides other submodule in CGSP with permanent data access and stableefficient data transfer service. In the other side, this system can be run independently and letendpoint grid users create their own data space via the network.Traditional distributed file systems are tight-coupled and have outstanding performance.However, they are always designed for a specific system and lost the generality. Whilethe performance of some web-based distributed file systems are at poor performance. Tomeet the requirement of large distributed data-intensive applications, we designed and im-plemented a slice-based, loose-coupled distributed file system, which includes data storageresources, data storage management module, meta data management module, data transfermanagement module, replica management module, information monitoring module, strategyanalysis module and all kinds of portals.Comparing with other distributed file system, our works have the following features andimprovements:Stability: Replica management module will adjust the redundancy degree of file slicein the system and thus avoids“zero replica crisis”. Besides strategy analysis modulewill choose the most robust data node to store data;Efficiency: The system use GridFTP stripped parallelled transfer to increase the usageof the bandwidth both on client and on server. Besides strategy analysis module willchoose a best data node to start a transfer;Generality: In this system, we use web service resource framework to implement re-mote call. Thus the only thing storage resource must do to enter into resource set is toreport its state to management module. Besides, users can log in their own space viaany terminal;更多还原