【摘要】 平方公里阵列(Square Kilometre Array, SKA)射电望远镜的数据处理是通过管线方式进行的,管线的执行效率是SKA区域中心考虑的重要因素.连续谱成像观测是SKA的主要观测模式之一,也是许多科学工作的基础.本文以SKA低频先导设备(Murchison Widefield Array, MWA)的成像管线为例,在中国SKA区域中心原型机(China SKA Regional Centre Prototype, CSRC-P)上进行并行处理管线优化.以往的优化方案都集中在少数性能热点,缺乏对整体管线的系统优化,导致整体加速比相对较低.针对这一问题,本文提出一种全局优化方案,针对管线使用多种编程语言和图像数据可独立处理的特点,综合使用C++多线程、Python多进程和Shell多任务并行等优化方法,并验证了优化结果的准确性.实验表明,优化后的代码在CSRC-P的x86节点和ARM (Advanced RISC Machine)节点上分别获得了2.7和2.4倍加速,运行时间分别从7479和9666 s,降低为2759和4061 s. ARM计算节点展现出对SKA应用良好的适应性.本文的优化策略和方法也适用于其他SKA科学应用,对SKA先导望远镜的科学运行和未来的运行也有帮助.更多还原

【Abstract】 Data processing of the square kilometer array(SKA) is performed in pipeline mode, and the execution efficiency of pipeline mode is an important factor in SKA data processing. Continuum imaging is a primary observation mode of SKA and a prerequisite for many other scientific works. In this paper, we take the imaging pipeline of the SKA lowfrequency precursor Murchison widefield array as an example and optimize the parallel processing pipeline on the China SKA regional centre prototype(CSRC-P). Previous optimization schemes have focused on a few performance hotspots and lacked systematic optimization of the overall pipeline, resulting in a relatively poor overall speedup ratio. In this paper, we propose a global optimization scheme that combines C++ multi-threading, Python multi-processing, and Shell multi-tasking parallelism for pipelines using multiple programming languages and image datasets that can be processed independently and verify the accuracy of the optimization results. Experiments show that the optimized pipeline achieves an overall speedup of 2.7-and 2.4-fold on the x86 and advanced RISC machine(ARM) nodes of CSRC-P, respectively,and the ARM compute nodes show good adaptability to SKA applications. The optimization strategies and methods in this paper also apply to other SKA applications and will be useful for the scientific operation and future operation of the SKA precursor telescope.更多还原