【摘要】 研究了在数据模式与匹配规则不变的前提下 ,数据集动态增加时近似重复记录的识别问题 ,提出了一种基于聚类树的增量式数据清洗算法IACT .该算法通过构建聚类树先对记录进行分区 ,然后在划分的区域内进行相似度的计算识别出近似重复记录 ,从而完成了增量式相似重复记录的检测 .实验结果证明了IACT算法在无损精度的情况下 ,在效率上优于多趟邻近排序 (MPN)算法 .更多还原

【Abstract】 This paper studied the problem of detecting approximately duplicate records while receiving increments of data with no changes in data schema and matching rule set, and presented an incremental algorithm IACT (Incremental Algorithms based on Clustering Trees for data cleansing). IACT divided the data records into a few areas and computed their similarity to identify the approximately duplicate records to accomplish the data cleansing task in the partitioned areas through creating clustering tree. Compared with the algorithm MPN, the experimental result proves that IACT algorithm is more effective while possessed of the same precision.更多还原