Please wait a minute...
New Technology of Library and Information Service  2013, Vol. 29 Issue (1): 83-89    DOI: 10.11925/infotech.1003-3513.2013.01.13
Current Issue | Archive | Adv Search |
Design and Implementation of Distributed Collaborative Filtering Algorithm on Hadoop
Xiao Qiang1, Zhu Qinghua1, Zheng Hua2, Wu Kewen1
1. School of Information Management, Nanjing University, Nanjing 210093, China;
2. School of Engineering Management, Nanjing University, Nanjing 210093, China
Download: PDF(598 KB)   HTML  
Export: BibTeX | EndNote (RIS)      
Abstract  Based on Hadoop, this paper demonstrates that traditional collaborative filtering algorithm cannot adjust to cloud computing platform, then improves traditional collaborative filtering algorithm to adapt to the Hadoop platform from similarity and prediction,and also achieves sequential modular MapReduce collaborative filtering computing tasks.
Key wordsHadoop      Collaborative filtering      Big data      Distributed      Cloud computing     
Received: 27 December 2012      Published: 29 March 2013
:  TP393  

Cite this article:

Xiao Qiang, Zhu Qinghua, Zheng Hua, Wu Kewen. Design and Implementation of Distributed Collaborative Filtering Algorithm on Hadoop. New Technology of Library and Information Service, 2013, 29(1): 83-89.

URL:

http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.1003-3513.2013.01.13     OR     http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2013/V29/I1/83

[1] 李树青.个性化信息检索技术综述[J]. 情报理论与实践,2009,32(5):107-113.(Li Shuqin. Review of Personalized Information Retrieval Technology[J].Information Studies:Theory & Application, 2009,32(5):107-113.)
[2] Liu Z B,Qu W Y,Li H T,et al.A Hybrid Collaborative Filtering Recommendation Mechanism for P2P Networks[J].Future Generation Computer Systems,2010,26(8):1409-1417.
[3] Pan R, Scholz M. Mind the Gaps: Weighting the Unknown in Large-Scale One-Class Collaborative Filtering[C].In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France. New York: ACM, 2009:667-676.
[4] Pan R, Zhou Y H, Cao B, et al. One-Class Collaborative Filtering[C].In: Proceedings of the 8th IEEE International Conference on Data Mining, Pisa. Washington, DC, USA: IEEE Computer Society,2008:502-511.
[5] Salakhutdinov R, Mnih A. Probabilistic Matrix Factorization[C].In: Proceedings of the 25th International Conference on Machine Learning. New York: ACM,2008:880-887.
[6] 侯经川,方静怡.数据引证研究:进展与展望[J]. 中国图书馆学报, 2013,39(1):112-118.(Hou Jingchuan, Fang Jingyi. Review on Data Citation in the Context of Big Data[J].Journal of Library Science in China, 2013,39(1):112-118.)
[7] 韩翠峰.大数据带给图书馆的影响与挑战[J]. 图书与情报,2012(5):37-40.(Han Cuifeng. The Impact and Challenges of the Library Based on Big Data[J]. Library & Information, 2012(5):37-40.)
[8] Sarwar B, Karypis G, Konstan J,et al.Item-based Collaborative Filtering Recommendation Algorithms[C].In: Proceedings of the 10th International Conference on World Wide Web. New York, NY, USA: ACM,2001:285-295.
[9] Resnick P,Iacovou N,Suchak M et al.GroupLens:An Open Architecture for Collaborative Filtering of Netnews[C]. In:Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work. New York, NY, USA:ACM, 1994:175-186.
[10] Sarwar B,Karypis G,Konstan J,et al.Item-based Collaborative Filtering Recommendation Algorithms[C].In:Proceedings of the 10th International Conference on World Wide Web. New York, NY, USA:ACM,2001:285-295.
[11] White T. Hadoop: The Definitive Guide[M].The 3rd Edition. USA: O'Reilly Media, 2012.
[12] Dean J, Ghemawat S. MapReduce: Simplified Data Processing on Large Clusters[J].Communications of the ACM, 2008,51(1):107-113.
[13] Hadoop.HDFS Users Guide[EB/OL].[2012-12-02]. http://hadoop.apache.org/docs/stable/hdfs_user_guide.html.
[14] Bahga A, Madisetti V K.Analyzing Massive Machine Maintenance Data in a Computing Cloud[J].IEEE Transactions on Parallel and Distributed Systems,2012,23(10):1831-1843.
[15] 冯璐,冷伏海.共词分析方法理论进展[J]. 中国图书馆学报,2006,32(2):88-92.(Feng Lu, Leng Fuhai. Development of Theoretical Studies of Co-word Analysis[J].Journal of Library Science in China,2006,32(2):88-92.)
[16] Sarwar B, Karypis G, Konstan J,et al. Analysis of Recommendation Algorithms for E-commerce[C].In: Proceedings of the 2nd ACM Conference on Electronic Commerce. New York: ACM, 2000:158-167.
[1] Beibei Kong,Jing Xie,Li Qian,Zhijun Chang,Zhenxin Wu. Methodology and Tools to Enrich Sci-Tech Big Data[J]. 数据分析与知识发现, 2019, 3(7): 113-122.
[2] Xiaozhou Dong,Xinkang Chen. E-Coupon and Economic Performance of E-commerce[J]. 数据分析与知识发现, 2019, 3(6): 42-49.
[3] Quan Lu,Anqi Zhu,Jiyue Zhang,Jing Chen. Research on User Information Requirement in Chinese Network Health Community: Taking Tumor-forum Data of Qiuyi as an Example[J]. 数据分析与知识发现, 2019, 3(4): 22-32.
[4] Ying Wang,Li Qian,Jing Xie,Zhijun Chang,Beibei Kong. Building Knowledge Graph with Sci-Tech Big Data[J]. 数据分析与知识发现, 2019, 3(1): 15-26.
[5] Li Qian,Jing Xie,Zhijun Chang,Zhenxin Wu,Dongrong Zhang. Designing Smart Knowledge Services with Sci-Tech Big Data[J]. 数据分析与知识发现, 2019, 3(1): 4-14.
[6] Jiying Hu,Jing Xie,Li Qian,Changlei Fu. Constructing Big Data Platform for Sci-Tech Knowledge Discovery with Knowledge Graph[J]. 数据分析与知识发现, 2019, 3(1): 55-62.
[7] Jing Xie,Li Qian,Hongbo Shi,Beibei Kong,Jiying Hu. Designing Framework for Precise Service of Scholarly Big Data[J]. 数据分析与知识发现, 2019, 3(1): 63-71.
[8] Jie Li,Fang Yang,Chenxi Xu. A Personalized Recommendation Algorithm with Temporal Dynamics and Sequential Patterns[J]. 数据分析与知识发现, 2018, 2(7): 72-80.
[9] Daoping Wang,Zhongyang Jiang,Boqing Zhang. Collaborative Filtering Algorithm Based on Gray Correlation Analysis and Time Factor[J]. 数据分析与知识发现, 2018, 2(6): 102-109.
[10] Yong Wang,Yongdong Wang,Huifang Guo,Yumin Zhou. Measuring Item Similarity Based on Increment of Diversity[J]. 数据分析与知识发现, 2018, 2(5): 70-76.
[11] Lingfeng Hua,Gaoming Yang,Xiujun Wang. Recommending Diversified News Based on User’s Locations[J]. 数据分析与知识发现, 2018, 2(5): 94-104.
[12] Zhihong Shen,Chang Yao,Yanfei Hou,Linhuan Wu,Yuepeng Li. Big Linked Data Management: Challenges, Solutions and Practices[J]. 数据分析与知识发现, 2018, 2(1): 9-20.
[13] Cao Yang, Fan Wenfei, Yuan Tengfei. Is Big Data Analytics Beyond the Reach of Small Companies?[J]. 数据分析与知识发现, 2017, 1(9): 1-7.
[14] Fuliang Xue,Junling Liu. Improving Collaborative Filtering Recommendation Based on Trust Relationship Among Users[J]. 数据分析与知识发现, 2017, 1(7): 90-99.
[15] Lemen Chao,Canjun Yang,Shengjie Wang,Junpeng Zhao,Mengtian Xu. Data Science Curriculums Around the World: An Empirical Study[J]. 数据分析与知识发现, 2017, 1(6): 12-21.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn