【摘要】 为了提高BP神经网络算法并行化速率,利用神经网络并行化思想,提出了一种基于Hadoop平台的改进Map Reduce编程模型及并行化的实现。采用Map Reduce编程模型,用神经网络训练集的一组样本的键/值替代单一键/值,通过分组标记将同一value值对应的reduce工作方式分散为多个reduce进行工作,实现各个任务节点并行处理大数据,从而减少了处理大规模数据集的运行时间。选用不同大小数据集进行测试,通过与传统的神经网络并行化进行对比,发现改进后的Map Reduce并行编程模型提高了神经网络的并行速率,在处理大数据集时具有一定的优越性。更多还原

【Abstract】 In order to improve the parallelization rate of BP neural network algorithm, using neural network parallelization idea, an improved Map Reduce programming model based on Hadoop platform and its parallelization are proposed. By using Map Reduce programming model, the keysalues of a set of samples of a neural network training set are used to replace a single keyalue, and the reduce work mode corresponding to the same value is dispersed into multiple reduce tasks by grouping markup to implement each task node to process large data in parallel, thereby reducing time for processing large-scale data sets. Different sizes of data sets are selected for testing and compared with the traditional neural network parallelization. It was found that the improved Map Reduce parallel programming model improves the parallel speed of neural networks and has certain advantages when dealing with large data sets.更多还原