【摘要】 为了能有效处理海量数据,进行关联分析、商业预测等,Hadoop分布式云计算平台应运而生。但随着Hadoop的广泛应用,其作业调度方面的不足也显现出来,现有的多种作业调度器存在参数设置复杂、启动时间长等缺陷。借助于人工蜂群算法的自组织性强、收敛速度快的优势,设计并实现了能实时检测Hadoop内部资源使用情况的资源感知调度器。相比于原有的作业调度器,该调度器具有参数设置少、启动速度快等优势。基准测试结果表明,该调度器在异构集群上,调度资源密集型作业比原有调度器快10%~20%左右。更多还原

【Abstract】 In order to effectively deal with massive data,correlation analysis,business forecast and so on,the distributed cloud computing platform Hadoop has emerged.But with the wide application of Hadoop,the job scheduling problems become prominent.Many existing job schedulers have defects such as complexity of parameters setting,long start time and so on.Taking full advantages of the artificial bee colony algorithm in terms of strong self-organizing ability and fast convergence,we design and implement a resource-aware scheduler of Hadoop which can detect real-time internal resources utilization.Compared to those classical job schedulers,the proposed scheduler has advantages of few parameters and fast starting speed.Experiments on benchmark programs show that the proposal can schedule resource intensive work faster than the original one by 10% ~ 20% on heterogeneous clusters.更多还原