【作者】 江伟；

【导师】 李国徽；

【作者基本信息】 华中科技大学 ， 计算机技术， 2015， 硕士

【摘要】 在大数据时代,企业信息化过程中由于各种业务的需要每天会产生很多数据,各种业务操作人员随时都需要从这些数据中快速查询出对自己有用的信息,并且这些数据信息一般都是多维度甚至多层次的,针对这些特点选取多层次高维数据立方体模型来对这些数据进行预计算处理并存储,以便后面的查询。国内外相关研究者针对相关问题提出了很多关于数据立方体的计算方法。对已有的这些计算方法进行分析可知,这些算法都还存在一些缺陷,有待进一步改进。比如使用BUC方法计算时不能进行数据的增量更新,而且受维度次序的影响;立方体外壳片段的计算方法在维度的基数比较大时计算量仍然很大,而且由于人们的查询集中在少数维上,所以很多的计算是多余的,没有价值的。针对以上这些问题,在立方体外壳片段的基础上提出IMC方法。首先,根据企业信息化过程中具体的需要,在涉及到查询的少量维度上对立方体进行分割,形成少数子立方体外壳片段,没有涉及到的只保持维度成员属性值的倒排索引列表即可,这样减少了要物化的子立方体的数量;其次,针对多层次的维度,使用维层次树以节省空间,并且方便数据增量更新;最后,在给具有多层次的维度建立倒排索引的时候,使用层次掩码和维层次编码的位与运算进行维层次的攀升,建立高层次维度成员的倒排索引列表。并且在最后通过对比实验证明,使用IMC方法能有效地提高计算数据立方体的速度、减少存储空间以及实现数据立方体的增量更新维护,该改进的方法对于企业信息化中数据的处理、决策支持都有重要的意义。更多还原

【Abstract】 In the era of big data, in the process of enterprise information, it will generate a great deal of data every day, and the staff need to query useful information from these data dues at any moment, and these data are generally multi-dimensional hierarchical. For these characteristics we use multi-dimensional hierarchical data cube model to pre-compute data and facilitate subsequent inquiries.Some related researchers have proposed many computing methods on data cube, including the calculation of high-dimensional data cube. The analysis of these proposed computing methods shows that these algorithms are still to be improved in some aspects. For example, the BUC can not be adaptable to incremental updates and it will be affected by the order of dimension. As for the shell fragment, if the cardinality of dimension is very big, the calculation amount will increase. And people always focus on few important dimensions, so a lot of calculations are superfluous and worthless.To solve these problems, it proposes a method called IMC based on shell fragment and dimension hierarchical encoding. Firstly, according to the needs of enterprise information, we choose a few dimensions related to the query to form sub-cube shell fragments and keep the inverted index list of other dimensions, thus reducing the number of materialized cuboids and the storage space. Secondly, for these high-level dimensions, we use dimension hierarchy tree instead of dimension hierarchy table to save storage space, and it will be adaptable to incremental updates. At last, for the construction of inverted index of different hierarchies, we take advantage of bitwise AND of dimension hierarchy encoding and hierarchy mask to construct the inverted index of higher hierarchies. The experiments show that the IMC can effectively improve the speed of calculation, and reduce the storage space and be adaptable to incremental updates. Additionally, the improved method is good for the data processing of enterprise information and decision support.更多还原