【作者】 王妍；

【导师】 于明远；

【作者基本信息】 浙江工业大学 ， 计算机技术（专业学位）， 2017， 硕士

【摘要】 近些年频频发生的食源性疾病事件,使得人们越来越关注食源性疾病问题。如今,食源性疾病已经成为世界各国重点关注的事件之一,人们在日常生活中通过饮食的各方面极有可能导致食源性疾病。且从研究现状来看,食源性疾病的安全意识和相关的食源性监测系统还没有形成规范化的管理与监督,食源性疾病预测分析中仍然存在数据挖掘技术效率不高和可视化技术效果不明显等问题。针对上述问题,本文对食源性疾病预测分析方法的研究,主要从以下三个方面展开:首先本文提出了Random Forests预测分析算法(Spark-IRF,Improved Random Forests algorithm based on Spark),该算法主要是从降维和权重投票方法两个部分来改进DRF(Dynamic Random Forests)算法。接着从准确率和召回率两个方面对实验数据分析,结果显示Spark-IRF算法在准确率和召回率上相比DRF算法、Spark-MLRF(Spark Machine Learning Random Forests)算法有明显的提高。其次提出了聚类可视分析算法WIK-means(the Weighted Intelligent K-means Algorithm),解决IK-means(Intelligent K-means)算法中存在的问题,即IK-means算法在计算数据点到质心之间的距离时,同等地对待每一个特征变量属性,可能会使得距离较远或者同等重要的两个相关属性被分配到不同的簇中,而导致数据结果失真。WIK-means算法在计算距离的过程中,对每个特征属性赋予相应的权重值来计算误差平方和的最小值。最后通过不同的聚类中心点不同的迭代次数,对K-means算法IK-means算法WK-means(Weighted K-means)算法DBScan算法、K-SCP算法和WIK-means算法进行分析,结果表明我们提出的WIK-means算法在时间效率和准确率方面都具有相对明显的优势。最后本文设计并实现了基于Spark的食源性疾病预测分析系统FASBSFD(Forecasting and Analysis System Based on Spark for Foodborne Diseases)。在该系统中,食源性疾病预测分析方法采用了Spark-IRF预测分析算法及WIK-means算法进行聚类可视化展示,最后通过食源性疾病数据验证表明该系统是可行有效的。更多还原

【Abstract】 In recent years,foodborne disease occurred frequently,which make people pay more and more attention on it.Nowadays,foodborne disease has become one of the most important events in the world.It is likely to cause foodborne diseases when we have some unhealthy food in our daily life.In addition,for the view of current status,the food safety awareness and the related foodborne diseases monitoring system have not yet formed a standardized management and supervision.In the forecasting and analysis of foodborne disease,there still exist some problems in the inefficient data mining technology and unobvious visualization technology.For this case,the research of forecasting and analysis prediction method on foodborne diseases in this paper is mainly studying from the following three aspects.In the first place,a predictive analysis algorithm(Spark-IRF,Improved Random Forests algorithm based on Spark)has been proposed,it improved the original algorithm mainly from two parts on dimension reduction and weighted voting method.Then,we have an analysis of experiment data on accuracy,precision and recall.The results analyses on the experiment show that the Spark-IRF algorithm has the advantage over DRF(Dynamic Random Forests)algorithm and Spark-MLRF(Spark Machine Learning Random Forests)algorithm on accuracy and recall rate.In the next place,we proposed a clustering algorithm(WIK-means,The Weighted Intelligent K-means Algorithm),it solved the existing problems in IK-means(Intelligent K-means)algorithm,that is,Euclidean distance is used to calculate the distance between data points and the center of a cluster.And each feature variable is treated equally,so it may make two related attributes which are too far away or has the same importance be assigned to different clusters,thus result in data distortion.In the course of calculating the distance between data points and the center of a cluster,WIK-means algorithm assigns the corresponding weights to each feature variable to minimize the sum of the squared errors between entities and their respective centroids.In the end,we make a detailed analysis of K-means algorithm,IK-means algorithm,WK-means algorithm WK-means(Weighted K-means)and WIK-means algorithm by different centroids and different number of iterations.It turns out that WIK-means algorithm has relatively obvious advantages in time,space efficiency and accuracy.Finally,a forecasting and analysis system based on Spark for foodborne diseases(FASBSFD)has been designed and implemented,which uses the Random Forests algorithm as predictive analysis algorithm and WIK-means algorithm as clustering visualization.Ultimately,the results show that the prototype is feasible and effective for us.更多还原