节点文献
基于元数据映射关系的结构化数据血缘分析方法
Method of structured data kinship analysis based on metadata mapping relationship
【摘要】 现有的结构化数据的数据血缘分析方法以改造数据库查询语言的执行过程为主,应用局限性较强,对此文中提出一种基于元数据映射关系的结构化数据血缘分析方法。该方法对业务表和字段的元数据信息进行统一维护,支持通过SQL解析、手动配置等手段构造并存储表和字段的元数据映射关系,以此记录表和字段的数据流向。然后,给出针对特定表和特定字段的血缘分析算法,算法分别基于深度优先搜索和广度优先搜索的思想,对存储的元数据映射关系进行分析,得到对输入表和输入字段有贡献的上游表和上游字段。再以有向图形式展示的分析结果直观地表达结构化数据的起源信息。该方法可以帮助数据分析人员快速地定位到问题数据的来源和加工过程,减少数据问题排查分析的时间和难度。
【Abstract】 Since the existing data kinship analysis methods for structured data are mainly used to transform the execution process of the database query language,which has strong application limitations,a structured data kinship analysis method based on metadata mapping relationship is proposed. In this method,the metadata information of business tables and fields is uniformly maintained,and the means of SQL analysis and manual configuration are supported to construct and store the metadata mapping relationship between tables and fields,so as to record the data flow direction of tables and fields. The kinship analysis algorithm for specific tables and specific fields is given. Based on the idea of depth first search and breadth first search respectively,the algorithm is used to analyze the mapping relationship of the stored metadata to obtain the upstream tables and upstream fields that contribute to the input tables and fields. The analysis results displayed in the form of directed graphs can intuitively express the origin information of the structured data. This method can help data analysts quickly locate the source and processing process of problem data,and reduce the time and difficulty of data problem investigation and its analysis.
【Key words】 structured data; data kinship; metadata mapping relationship; data flow direction; kinship analysis; data analysis;
- 【文献出处】 现代电子技术 ,Modern Electronics Technique , 编辑部邮箱 ,2022年16期
- 【分类号】TP311.13
- 【下载频次】208