【作者】 陈敏；

【导师】 郑骏；

【作者基本信息】 华东师范大学 ， 计算机应用技术， 2015， 硕士

【摘要】 在互联网技术迅猛发展的今天,人们在获取信息的同时也在不断的创造新的信息。在这样一个充斥着海量数据的时代,用户很容易在信息的海洋中迷失。个性化推荐系统因为这个实际需求而产生,它通过分析用户在系统中产生的历史行为数据,预测用户的兴趣偏好,然后把合适的物品推送给用户。协同过滤算法是推荐系统中常用的技术,它可以分为两类算法：侧重于挖掘数据的局部特性或侧重于挖掘数据的全局特性,混合推荐算法可以将这两类算法加以结合来保持它们各自的优点。但是,协同过滤算法都面临着可扩展性差和数据稀疏性问题,尤其是侧重挖掘局部数据特性的协同过滤算法对数据稀疏性问题更加敏感。本文针对协同过滤算法及其混合算法面临的可扩展性问题和数据稀疏性问题,做了一些研究工作：1.研究设计了Weight-Slope One和RSVD模型等协同过滤算法在Hadoop分布式平台中并行化的求解方法,以解决用户行为数据中用户或物品规模庞大而导致算法面临可扩展性差的问题,该求解方法可用于本文提出的改进Weight-Slope One算法以及混合推荐算法中离线计算部分。然后,提出了利用相似性和时间因素来改进Weight-Slope One算法,提升了算法挖掘局部数据特性的能力。同时针对改进后算法面临的数据稀疏性问题,利用用户相关性信息和补全矩阵的方式,在一定程度上缓解了改进算法在面对稀疏数据时预测精度的损失。本部分研究内容提升了算法挖掘局部数据特性的能力和缓解了数据稀疏性造成的精度损失。2.在前面第一点的研究基础上,提出了基于改进后Weight-Slope One算法和RSVD模型的混合推荐算法,混合推荐算法的离线计算部分可使用前文研究设计的Hadoop平台下并行化的实现方案,实验结果显示,该混合算法能够更好的挖掘局部和全局的数据特性从而提升算法预测精度,同时在一定程度上更能适应数据的稀疏性。更多还原

【Abstract】 In the rapid development of Internet technology today, people were creating new information while getting information. Being around with huge amounts of data, users might not find the correct information. Then Personalized Recommendation System was created. It could predict a user’s interest preferences by analyzing the historical user-generated data in the system and showed the right items to the user.Collaborative Filtering Algorithm was commonly used in the recommended system technology, which could be divided into two categories algorithms:data mining focus on local properties and data mining focuses on global characteristics. Hybrid Recommendation System contained these two algorithms and could keep their respective advantages. However, Collaborative Filtering Algorithms was faced with the problems of poor scalability and data sparsity, and especially Collaborative Filtering Algorithm focusing on the characteristics of local data mining was more sensitive to data sparsity issues. To solve the problems of poor scalability and data sparsity faced by Collaborative Filtering Algorithm and Hybrid Algorithm, this paper had some research works below:1. Research and design of the Weight-Slope One and the RSVD model solving methods in Hadoop parallel distributed platform, in order to solve the problem of user behavior data in large scale and the algorithm to face the problem of scalability, and this method can be used for improved the Weight-Slope One algorithm and the hybrid recommendation algorithm off-line calculation. Then, proposed the use of similarity and the time factor to the improved Weight-Slope One algorithm, enhance the ability of the algorithm to mining the local characteristics of the data. At the same time, the improved algorithm with the data sparseness problem, the use of user information and the complement of the correlation matrix, to a certain extent, ease the improved algorithm prediction accuracy’ s loss in the face of sparse data. The contents of this part of research to enhance the ability of local characteristics of the data mining algorithm and alleviate the loss of accuracy caused by data sparseness.2. In the basis of the above study on the first point, proposes an improved hybrid algorithm based on Weight-Slope One and RSVD model, the off-line’s Calculate in hybrid recommendation algorithm can be parallel implementation scheme of using the above research and design on Hadoop platform, experimental results show that the hybrid algorithm can better so as to enhance the precision of prediction algorithm, at the same time, the algorithm can adapt to the sparsity of data to some extent.更多还原