【作者】 王丹；

【导师】 孙广中；

【作者基本信息】 中国科学技术大学 ， 计算机软件与理论， 2015， 硕士

【摘要】 随着信息技术的快速发展,在人们的日常生活中出现了各种信息系统,产生了海量的时间序列数据。如何有效地利用这些历史时序数据,从中挖掘出用户需要的信息已经成为当今数据处理领域广泛关注的热点问题。首先分析了当前分布式计算的关键技术,然后对时序数据处理进行了相关研究,最后提出了一种基于Hadoop平台的时间序列数据增量计算方法及系统。主要工作内容如下：1.分析分布式系统关键技术,对Hadoop整体框架进行分析。文中首先对分布式系统中关键技术进行探讨,然后通过搭建Hadoop平台,结合MapReduce计算模型给出了实际数据处理的实验案例,最后对实验结果进行了详细的分析。2.对时间序列预测算法、时间序列相似性度量算法进行了分析和改进。文中通过对真实数据的理解及建模,实现了时序数据的预测过程。最后,在分析时间序列相似性度量算法的基础之上,提出了支持增量数据的时间序列相似性度量算法—Inc-DTW,并通过实验及理论对所提出的方法进行验证。3.在进行数据处理时,数据是随时间的推进在不断的增加,如何才能进行高效的增量计算是目前数据处理领域的一个研究热点。本文中结合Hadoop平台以及时间序列数据相关特点提出了一种Hadoop平台时序数据增量计算方法及系统-TSI-Hadoop。TSI-Hadoop具有以下特点：(1)提供了常用时序数据处理算法的支持；(2)提出基于MapReduce计算模型的分段时序数据增量计算模型；(3)针对时序数据特点,提出带有状态的滑动窗口增量计算方法。最后对我们提出的方法及系统进行了相应的实验验证。更多还原

【Abstract】 With the rapid development of information technology, a variety of information systems have been widely used in people’s daily life. These systems produce a massive time series data. Today how to use these time series data efficiently and mine useful information is a hot topic in data processing.Firstly, we analyze the key technologies of existing distributed systems. Then, do some research on time series data analysis. Finally, we propose a method and system for incremental processing time series data on Hadoop platform. The main works of this dissertation are as follows:1. We analyze the key technologies in distributed systems and Hadoop platform. In this paper, firstly, we analyze the key technologies of distributed systems. And then we construct a Hadoop platform and give a practical data processing exper-iment based on MapReduce computation model. Finally, we analyze the experi-mental result in detail.2. We analyze and improve time series prediction and time series similarity measure algorithms. In this paper, firstly, we have a good understand of the real data and model the data and then predict the time series data. Finally, on the basis of analyzing the time series similarity measure algorithm, we propose an algorithm Inc-DTW to support incremental data time series similarity measure. Moreover, we show the efficiency of the algorithm from experiment and theory.3. During data processing, data is increasing with time passing, how to efficiently compute the incremental data is currently a hot research topic in the field of data processing. In this paper, combining with the characteristics of Hadoop plat-form and time series data, we propose a time series data incremental processing method and system TSI-Hadoop based on Hadoop platform. TSI-Hadoop has the following features:(1) Provides a common support for temporal data processing algorithm;(2) Proposes segment time series data incremental calculation mode based on MapReduce computation model;(3) Proposes a method of sliding win-dow incremental calculation with state according to the characteristics of time series data. At last, we do some experiments to verify the effectiveness of the method and system.更多还原