【作者】 梁柱；

【导师】 李军怀；

【作者基本信息】 西安理工大学 ， 计算机应用技术， 2017， 硕士

【摘要】 校园一卡通系统在高校中的普遍部署,不仅推进了数字化校园的建设,极大地方便了学生的日常生活,也为我们分析学生行为和挖掘学生规律提供了数据基础。然而,许多学校的管理还在沿用传统的学生管理和服务方式,通过专业、年级对学生划分,对学生采用单一化的管理方式,缺乏针对学生行为特征的个性化管理和服务。针对以上问题,本文通过分析高校校园大数据的特点,研究并开发了一个基于Spark平台的学生行为分析与预测系统,主要工作:(1)针对学生行为细分问题,根据可获得的校园数据类型,首先设计了学生行为描述指标体系;然后,设计了基于聚类分析的学生行为细分模型,并对传统的K-means聚类算法从初始聚类中心的选择和聚类数量的确定两个方面进行了改进,提出了一个基于密度优化的K-means改进方法;最后,在Spark平台上对改进方法进行了并行化,并应用于学生行为细分中,对学生进行类别划分,通过实验验证了该划分结果的可靠性。(2)针对学生行为提醒的后置性和时效性问题,本文提出了基于学生行为分层的K近邻非参数回归预测模型,有效解决了K近邻学生预测模型存在的预测误差较大的问题。其次,采用决策树对学生行为的预测结果给予预警分析,实现了学生行为后置性应急到前置性预警引导的转变。最后,针对串行化对海量数据处理的时间复杂度过高问题,设计及实现了上述学生行为预测与预警算法在Spark平台下的并行化,提高了数据处理效率及时间性能。(3)在上述理论研究基础上,设计并实现了基于Spark的学生行为分析与预测应用平台,并详细介绍了Spark大数据平台的构建、系统设计与开发方法。本文所开发的系统可以为学生、学院、学校及后勤管理部门提供学生消费、学习等多方位的行为分析与预测功能,具有较好的实际意义。更多还原

【Abstract】 Widespread deployment of campus one-card system in colleges and universities not only promote the construction of digital campus and facilitate the students’ daily life greatly,but provide data to analyze student conduct and mining laws.Whereas,the management of numerous school is still using the traditional student management and service method which subdivides students through professional and grade,and adopting a simple management style,which cannot be carried out students personalized management and service in accordance with the characteristic behavior.In view of the above problems,this thesis studied and developed a student behavior analysis and prediction system based on the Spark by analyzing the characteristics of university campus big data,the following work is done:(1)Aiming at the problem of student behavior subdivision,the thesis,first designed the student behavior description index system,then,devised a student behavior segmentation model based on clustering analysis,and improved the traditional K-means clustering algorithms from two aspects that include the choice of initial clustering center and the number of clustering,proposed a K-means improved method based on density of optimization.In the Spark platform,parallelizing the improved method,applying it to student behavior Finally,the reliability of the results is verified by experiment.(2)For the rear and timeliness of student behavior reminded,the article has proposed K neighbor nonparametric regression forecasting model based on student behavior layered and solved the problems of the prediction larger error about K neighbor students forecasting model.Secondly,Using decision tree to give early warning analysis to the predictive results of the student behavior,has realized shift of the student behavior of rear emergency to front warning.Finally,in view of the serialized high time complexity problem of mass data processing,designing and implementing parallelization of the student behavior prediction and early warning algorithm under the Spark platform,at the same time,improving the efficiency of data processing and time performance.(3)On the basis of the above theoretical research,the thesis designed and implemented the student behavior analysis and prediction application platform based on Spark.Meanwhile,it introduces the construction,system design and development method of Spark big data platform in detail.The system,which has practical implications,developed in this article can offer a series of all-round behavior analysis and forecast functions,such as the students’ consumption and learning,for students,colleges and the administrative department.更多还原