【作者】 黄健；

【导师】 林穗；

【作者基本信息】 广东工业大学 ， 计算机科学与技术， 2016， 硕士

【摘要】 由于互联网技术和云服务的发展,网民数量不断增长,使得各种信息也呈现爆炸式的增长。在这样的趋势下,传统的存储服务已经满足不了现在用户的需求。不管是企业还是个人都需要一种新型的存储服务。云存储是云计算概念上发展而来。它是利用网络,分布式文件系统等技术通过软硬件结合的方式将网络中的物理存储连接在一起,并为用户提供标准存储接口,方便用户使用。云存储的发展,促使越来越多的用户将自己的数据迁移到云端存储上,同时关于云存储安全也成为必不可少的关注焦点。当我们的数据迁移到云端,云存储提供商就拥有我们数据的绝对控制权,而我们也失去对数据绝对的控制。HDFS作为Hadoop底层分布式文件系统,是当前云存储构建的主流方案之一。由于利用HDFS部署的云存储平台对数据拥有绝对控制问题,本文对团队提出的自主可控安全云存储模型ASOM (All Self-Organization Model)和HDFS内部实现进行研究。针对HDFS的实现主要分析NameNode管理元数据和管理整个集群方式。通过分析DataNode与NameNode之间的有关数据块交互过程,和NameNode中元数据组织流程,在ASOM中设计并实现了元数据子服务和设备管理服务器。在成功提取分离元数据后,设备管理服务器负责管理整个集群的正常运行,元数据子服务器负责命名空间和文件定位。在数据服务器向设备管理服务器上报数据时,设备管理服务器会根据需要转发信息给元数据子服务器,完成元数据组织。而在整个集群运行过程中元数据子服务器和设备管理服务器又不断进行信息交互,为了在交互过程中防止可能泄露敏感数据,在实现时引入了随机表机制。在这个交互的过程里,本文实现了Co-work算法保证这两个服务器在数据交互时的一致性。Co-Work算法是在锁的基础上通过分析实际场景流程使得两个服务器能保证数据的一致性。同时针对ASOM中小文件对应一个数据块内容的安全问题,进行研究和解决。通过对小文件中数据内容部分细粒度数据加密,不仅保证数据的安全,还可以改进因全部数据加密而导致的效率低下。最后本文对该模型的安全性进行实验,对性能,内存使用进行对比分析,证明用户可以对元数据的进行有效的控制与管理,也证明了云服务提供商平台管理员很难对元数据进行可能的恶意操作而带来的安全隐患。更多还原

【Abstract】 Due to the development of Internet technology and cloud services, the number of Internet users is growing, making all kinds of information also showed explosive growth.In this trend, traditional storage services have failed to meet the current needs of the user. Whether business or personal needs a new storage service.Cloud storage is developed from the concept of cloud computing. It is using the Internet, distributed file system technology to connect physical storage together in the network by a combination of hardware and software and to provide users with a standard,user-friendly storage interface.The development of cloud storage, prompting more and more users to migrate users’data to the cloud storage,while the cloud storage security has become an essential focus.When our data migration to the cloud, cloud storage provider will have absolute control over our data, and we have lost absolute control of the data.As the underlying Hadoop Distributed File System, HDFS is one of the mainstream to build cloud storage. Due to the use of HDFS deployment of cloud storage platform data have absolute control problem, the research of this paper is the self-controlled secure cloud storage model ASOM (All Self-Organization Model) and the internal implementation of HDFS.For the realization of HDFS,the main the analysis of the approach of managing metadata for NameNode and management of the entire cluster to the realization of HDFS.By analyzing the related data-blocks interaction betweent DataNode and NameNode, and the metadata organizational processes in NameNode,we design and implement a metadata subservices and device management server for ASOM.After a successful extraction and separation of metadata, the device management server is responsible for managing the entire cluster normal operation, the metadata server is responsible for the namespace and file positioning. When reporting data from the data server to the device management server,the device management server will forward the necessary information to the metadata sub-server to complete metadata organization. In the whole process of running a cluster,the metadata sub-server and device management server constantly exchange information.In order to prevent possible leakage of sensitive data in the interactive process, in the realization we introduce a random table mechanism. In this interactive process, this paper also implements Co-work algorithm to ensure consistency in both server data interaction.Co-Work algorithm is based on the lock through the analysis of the actual scene process to make two servers ensure data consistency.While for ASOM security issues that small file corresponds to a data block content make research and resolve. By encrypting section fine-grained data for the small data file contents, not only to ensure the security of data, but also can improve efficient of encrypting all the data.At the end of this paper, through the model experiment of security and comparative analysis on the performance and memory usage, we not only prove that user can make effective control and manage for metadata,but also prove the platform administrator of cloud service provider is difficult of possible to malicious operation for metadata,and bring safety hazards.更多还原