【作者】 关大伟；

【导师】 李雄飞；

【作者基本信息】 吉林大学 ， 软件工程， 2006， 硕士

【摘要】 随着社会的发展和数据库的应用,各领域的应用数据库中都积累了大量的历史数据。如何利用这些有潜在价值的数据,从中提取出有用的信息和知识,是应用者日益关注的问题,也是数据挖掘技术的关键所在。要进行数据挖掘,首先要保证数据质量,良好的数据能提高数据挖掘效果和效率,数据预处理逐渐成为数据挖掘不可缺少的重要前提。在数据挖掘的过程中如果只着眼于数据挖掘算法的探讨,而忽视了对数据预处理的研究,在一定程度上往往会失去数据挖掘的某些重要意义。因为实际系统中的数据一般都具有不完整性、冗余性和模糊性,很少能直接满足数据挖掘算法的要求。另外,海量的数据中无意义的成分很多,严重影响了数据挖掘算法的执行效率,而且由于其中的噪音干扰还会造成挖掘结果的偏差。因此,对不理想的原始数据进行有效的归纳和预处理,已经成为数据挖掘系统实现过程中的关键问题。本文通过对数据挖掘、数据预处理技术和理论的学习,以及对国内外数据挖掘与数据预处理系统的发展情况的研究,归纳总结了国内、外数据挖掘系统中数据预处理的特点,根据当今数据挖掘技术和数据挖掘系统的发展趋势,设计了一个数据预处理系统,该软件设计实现的预处理系统主要包括数据预处理过程中最常用、最直接、最有效的和有一定通用价值的维规约、聚集、过滤异常值、去掉重复记录处理,软件在一定程度上实现了对大量数据的清洗工作,为进一步数据挖掘提供了可靠的数据保障。更多还原

【Abstract】 The scale is being expanded and analysed today when the demand increases day by day constantly in the data, various kinds of data analysis tool and data are excavating the system and is being developed, developed emerging constantly. A lot of successful data excavate the system, such as SAS Enterprise Miner, SPSS Clementine , IBM Intelligent Miner ,etc., after more than ten years of development, the application succeeding in the fields of much commerce , scientific research, have not merely guide the management and development of enterprises, has brought the enormous economic benefits to enterprises, and has made enormous contribution to the research that the data excavated technology for the scientific research institution. It is that the data excavate the bridge studied and used that the data excavate the system, play a very big role to the popularization that the data excavate technology.Very high guarantee that can be carried on under the enviroment of high-quality data that the data pretreatment in the data are excavated is the data to excavate algorithms, the data excavate a lot of algorithms at present, but the algorithm and software which are used in the pretreatment have not appeared in a large amount , its main reason is pretreatment of the data, to different field and trade, there are different needs, the method of the pretreatment of the data is various at present, how to guarantee the quality of the data, raise data can guarantee commonability of software while excavating efficiency of algorithm and good embedding can be in existing data it excavates to be the place in advance of data at present of the system Pay attention to the software and remain to solve the problem that and improve.This text passes correctly the data excavate the study of treatment technology in advance of data in the system, and consult the domestic and international data and excavate the systematic study on pretreatment respect of the data, have summarized data the problem faced in process system development and development trend in advance, have designed a process system in advance of datum that can meet the need to a certain extent . It is unusually worth , remove the repeated record algorithm to compare interchangeably that what this text explains links the stipulations, assembles, filters, the data that can need planning before excavating the data carry on a certain degree and essential washing , conversion, have improved quality and efficiency that the data excavate to a certain extent.First, two chapters have recommended the data to excavate technology and data and更多还原