【摘要】 针对数据清洗时数据的标准化问题提出采用基于特征的马尔可夫模型来解决这一问题。在学习模型的过程中,通过最大熵方法提高样本学习的泛化能力。这种方法能够充分利用数据的重叠特征来辨识数据项对应的状态,结合了统计模型和规则模型的优点。理论分析和实验表明,该方法可以有效地实现数据清洗时的数据规格化。更多还原

【Abstract】 This paper proposed a feature-based Markov model for data standardization during data cleansing. This approach makes use of overlapping features to identify the corresponding state of data items and every state is the result of state-state transition and observation-state transition probability.Thus,this model may combine both advantages of statistical model and rule-based model.Theory and experiment shows that our approach has a good performance for data standardization during data cleansing.更多还原