【作者】 张延松； 张宇； 黄伟； 王珊； 陈红；

【Author】 Zhang Yan-Song~(1,2+),Zhang Yu~3,Huang Wei~(1,2),Wang Shan~(1,2),Chen Hong~(1,2) 1(Key Laboratory of Data Engineering and Knowledge Engineering(Renmin University of China),MOE,Beijing 100872,China) 2(School of Information,Renmin University of China,Beijing 100872,China) 3(Department of Computer Science,Harbin Financial College,Harbin 150030,Heilongjiang)

【机构】 数据工程与知识工程教育部重点实验室(中国人民大学)； 中国人民大学信息学院； 哈尔滨金融高等专科学校计算机系；

【摘要】 根据OLAP查询的特点和内存数据库的性能特征提出了由多个内存数据库组成的并行OLAP查询处理系统,将OLAP应用中的多维聚集查询分布到各个计算节点并行进行聚集计算并将聚集计算的结果进行合并输出.与其他并行处理方法相比,该算法充分利用OLAPDB结构中维表远小于事实表的特性,根据数据库中事实表的数据量和节点的数据处理能力进行水平数据库分片并根据聚集函数的可分布计算特性,提高查询处理的并行度,延迟并行查询处理中的合并过程,充分利用节点的并行处理能力,减少并行查询处理过程中的数据通讯量,提高系统并行查询处理性能.该算法易于实现,具有较好的可扩展性和性能,适用于企业级海量数据处理领域的需求.更多还原

【Abstract】 A multi-node parallel main-memory OLAP system is proposed in this paper which is considered by the character of OLAP queries and the performance of main-memory database system.In this system, multi-dimensional OLAP queries with aggregate functions are distributed to each computing node to get aggregate results and final result can be available by merging all the aggregate results from multiple computing nodes. Comparing with other solutions,this system uses horizontal distribution policy to distribute massive data in multi-node with only consideration of capacity of computation main-memory node.According to feature of distributed aggregate function,system can improve parallel processing capacity by lazy results merging which can reduce the volume of message between nodes,the overall performance of parallel query processing can be improved.This system is easy to deploy,it is also practical with good scalability and performance for the requirements of enterprise massive data processing.更多还原