【作者】 杨光；

【导师】 刘靖宇；

【作者基本信息】 河北工业大学 ， 计算机技术（专业学位）， 2021， 硕士

【摘要】 随着大数据时代的到来,人们对数据处理的要求不断提高,同时数据信息系统也呈现出爆发性的扩大趋势,由于存储系统规模的持续增长,对硬盘在容量、节能等方面都提出了更高的需求,对海量数据信息进行高效益的保存、管理与利用等成为互联网及相关行业的热门研究领域。连续数据存放方式,如数据备份,存档,录像监控等,虽然具备了连续读写、偶尔读取、几乎没有被改动等优点,但对各类型数据的存放负载有不平衡性,且信息量较大,对数据的传递效率要求并不高,这些都对存储系统的性能和能耗提出了更高的要求。面对连续数据存储系统的这一特性,以及存储系统动态负荷变化和双校验盘性能瓶颈问题,本文提出以SSD(固态硬盘)为缓存架构,双校验盘循环校验的节能存储数据布局:即DS-RAID。DS-RAID由SSD和双校验盘磁盘阵列构成,由硬盘构成的RAID为主存分区,而以SSD为系统缓存分区包括两个部分:数据缓存分区和双校验数据缓存。系统首先将数据写入数据缓存区,待系统空闲或达到设定阈值时再写回到主存储区。磁盘通常处于待机状态,以节约系统的能耗。仅当数据由SSD写回到主存储区时,磁盘才由待机状态调度到运行状态。实验表明,相同配置下DS-RAID在写性能、响应时间方面得到明显优化,同时取得了较好的节能效果。提出一种基于NILFS2文件系统的DS-RAID优化数据布局:LDS-RAID。LDS-RAID在DS-RAID基础上调整了数据布局和校验值生成方法,提升了存储系统的写性能。LDS-RAID首先针对NILFS2的数据特征分离随机I/O访问和顺序I/O访问,缓存超级块的读写操作,有效降低了超级块读写操作对磁盘性能和能耗的影响。针对分区后的LDS-RAID,校验值生成方法采用数据增量校验算法:使用局部数据校验,只计算同条带内有数据写入的数据块的校验值,避免了新数据写入时对旧数据的读操作,降低了因“读-改-写”带来的写惩罚。实验表明,与DS-RAID相比,LDS-RAID取得了56%～486%的写性能提升。在数据恢复过程中,由于减少了需要启动磁盘的数量和读取的数据量,缩短了修复时间,降低了在修复过程中因第二块磁盘故障导致整个存储系统数据失效的概率,提高了系统可靠性。更多还原

【Abstract】 In the new era of big data processing,a large number of data information systems have shown an explosive expansion trend.Due to the continuous growth of the scale of storage systems,higher demands have been placed on hard disks in terms of capacity and energy saving.High-efficiency preservation,management and utilization have become a hot research field in the Internet and related industries.Continuous data storage methods,such as data backup,archiving,video monitoring,etc.,although it has the advantages of continuous reading and writing,occasional reading,and almost no changes,but at the same time,there is an imbalance in the storage load of various types of data,and the amount of information Larger,the requirement for data transmission efficiency is not high.All these put forward higher requirements on the performance and energy consumption of the storage system.Faced with this characteristic of continuous data storage systems,as well as storage system dynamic load changes and dual-check disk performance bottlenecks,this article proposes an energy-saving storage data layout with SSD as the cache architecture and dual-check disk cyclic verification: namely DS-RAID.DS-RAID is composed of SSD and dual parity disk array.The RAID composed of hard disks is the main memory partition,and the SSD is the system cache partition.It includes two parts: data cache partition and dual parity data cache.The system first writes the data into the data cache,and then writes it back to the main storage area when the system is idle or reaches the set threshold.The disk is usually in a standby state to save the energy consumption of the system.Only when the data is written back to the main storage area by the SSD,the disk is scheduled from the standby state to the running state.Experiments show that under the same configuration,DS-RAID has been significantly optimized in terms of write performance and response time,while achieving better energy-saving effects.LDS-RAID is proposed,which is a DS-RAID optimized data layout based on NILFS2 file system.On the basis of DS-RAID,LDS-RAID adjusts the data layout and the generation method of check value,which improves the write performance of the storage system.LDS-RAID first separates random I/O access from sequential I/O access according to the data characteristics of NILFS2.This method caches the read and write operations of super block,and effectively reduces the impact of super block read and write operations on disk performance and energy consumption.For partitioned LDS-RAID,the verification value generation method adopts data incremental verification algorithm: using local data verification,only the verification value of data blocks written to the same strip is calculated,which avoids the read operation of old data and reduces the write penalty caused by "read overwrite" when writing new data.Experiments show that compared with DS-RAID,LDS-RAID achieves 56% ～ 86% write performance improvement.In the process of data recovery,the number of boot disks and the amount of data read are reduced,the repair time is shortened,and the probability of data failure of the whole storage system caused by the failure of the second disk is reduced.Reduce maintenance procedures and improve system reliability.更多还原