【摘要】 MapReduce集群系统的单一控制节点结构使其存在着性能瓶颈问题,限制了集群的规模.为较好地解决系统性能瓶颈问题,提出了MapReduce架构的多控制点改进.新系统的实现基于Hadoop MapReduce,在对架构进行改进的同时,重新设计了信令系统,并增加了控制节点间的热备份机制.因此,新架构除了能解决瓶颈问题外,还可以通过控制节点间的热备份策略有效地应对控制节点故障,降低故障对集群性能的影响,保证应用的正常执行.更多还原

【Abstract】 In the existing MapReduce framework,the single master is the performance bottleneck.As a result,the size of the cluster is limited.In order to break the bottleneck,a multi-master framework is proposed in this paper.The implementation is based on Apache Hadoop MapReduce.The main framework and signaling system are redesigned to sustain more than one master.Meanwhile a double-master hot backup scheme is designed to solve single master failure,so that the new framework is able to reduce the impact of master failure.更多还原