Please wait a minute...
New Technology of Library and Information Service  2013, Vol. 29 Issue (1): 83-89    DOI: 10.11925/infotech.1003-3513.2013.01.13
Current Issue | Archive | Adv Search |
Design and Implementation of Distributed Collaborative Filtering Algorithm on Hadoop
Xiao Qiang1, Zhu Qinghua1, Zheng Hua2, Wu Kewen1
1. School of Information Management, Nanjing University, Nanjing 210093, China;
2. School of Engineering Management, Nanjing University, Nanjing 210093, China
Download:
Export: BibTeX | EndNote (RIS)      
Abstract  Based on Hadoop, this paper demonstrates that traditional collaborative filtering algorithm cannot adjust to cloud computing platform, then improves traditional collaborative filtering algorithm to adapt to the Hadoop platform from similarity and prediction,and also achieves sequential modular MapReduce collaborative filtering computing tasks.
Key wordsHadoop      Collaborative filtering      Big data      Distributed      Cloud computing     
Received: 27 December 2012      Published: 29 March 2013
:  TP393  

Cite this article:

Xiao Qiang, Zhu Qinghua, Zheng Hua, Wu Kewen. Design and Implementation of Distributed Collaborative Filtering Algorithm on Hadoop. New Technology of Library and Information Service, 2013, 29(1): 83-89.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.1003-3513.2013.01.13     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2013/V29/I1/83

[1] 李树青.个性化信息检索技术综述[J]. 情报理论与实践,2009,32(5):107-113.(Li Shuqin. Review of Personalized Information Retrieval Technology[J].Information Studies:Theory & Application, 2009,32(5):107-113.)
[2] Liu Z B,Qu W Y,Li H T,et al.A Hybrid Collaborative Filtering Recommendation Mechanism for P2P Networks[J].Future Generation Computer Systems,2010,26(8):1409-1417.
[3] Pan R, Scholz M. Mind the Gaps: Weighting the Unknown in Large-Scale One-Class Collaborative Filtering[C].In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France. New York: ACM, 2009:667-676.
[4] Pan R, Zhou Y H, Cao B, et al. One-Class Collaborative Filtering[C].In: Proceedings of the 8th IEEE International Conference on Data Mining, Pisa. Washington, DC, USA: IEEE Computer Society,2008:502-511.
[5] Salakhutdinov R, Mnih A. Probabilistic Matrix Factorization[C].In: Proceedings of the 25th International Conference on Machine Learning. New York: ACM,2008:880-887.
[6] 侯经川,方静怡.数据引证研究:进展与展望[J]. 中国图书馆学报, 2013,39(1):112-118.(Hou Jingchuan, Fang Jingyi. Review on Data Citation in the Context of Big Data[J].Journal of Library Science in China, 2013,39(1):112-118.)
[7] 韩翠峰.大数据带给图书馆的影响与挑战[J]. 图书与情报,2012(5):37-40.(Han Cuifeng. The Impact and Challenges of the Library Based on Big Data[J]. Library & Information, 2012(5):37-40.)
[8] Sarwar B, Karypis G, Konstan J,et al.Item-based Collaborative Filtering Recommendation Algorithms[C].In: Proceedings of the 10th International Conference on World Wide Web. New York, NY, USA: ACM,2001:285-295.
[9] Resnick P,Iacovou N,Suchak M et al.GroupLens:An Open Architecture for Collaborative Filtering of Netnews[C]. In:Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work. New York, NY, USA:ACM, 1994:175-186.
[10] Sarwar B,Karypis G,Konstan J,et al.Item-based Collaborative Filtering Recommendation Algorithms[C].In:Proceedings of the 10th International Conference on World Wide Web. New York, NY, USA:ACM,2001:285-295.
[11] White T. Hadoop: The Definitive Guide[M].The 3rd Edition. USA: O'Reilly Media, 2012.
[12] Dean J, Ghemawat S. MapReduce: Simplified Data Processing on Large Clusters[J].Communications of the ACM, 2008,51(1):107-113.
[13] Hadoop.HDFS Users Guide[EB/OL].[2012-12-02]. http://hadoop.apache.org/docs/stable/hdfs_user_guide.html.
[14] Bahga A, Madisetti V K.Analyzing Massive Machine Maintenance Data in a Computing Cloud[J].IEEE Transactions on Parallel and Distributed Systems,2012,23(10):1831-1843.
[15] 冯璐,冷伏海.共词分析方法理论进展[J]. 中国图书馆学报,2006,32(2):88-92.(Feng Lu, Leng Fuhai. Development of Theoretical Studies of Co-word Analysis[J].Journal of Library Science in China,2006,32(2):88-92.)
[16] Sarwar B, Karypis G, Konstan J,et al. Analysis of Recommendation Algorithms for E-commerce[C].In: Proceedings of the 2nd ACM Conference on Electronic Commerce. New York: ACM, 2000:158-167.
[1] Chang Zhijun,Qian Li,Xie Jing,Wu Zhenxin,Zhang Hu,Yu Qianqian,Wang Ying,Wang Yongji. Big Data Platform for Sci-Tech Literature Based on Distributed Technology[J]. 数据分析与知识发现, 2021, 5(3): 69-77.
[2] Li Zhenyu, Li Shuqing. Deep Collaborative Filtering Algorithm with Embedding Implicit Similarity Groups[J]. 数据分析与知识发现, 2021, 5(11): 124-134.
[3] Chen Shiji, Qiu Junping, Yu Bo. Topic Analysis of LIS Big Data Research with Overlay Mapping[J]. 数据分析与知识发现, 2021, 5(10): 51-59.
[4] Yang Chen, Chen Xiaohong, Wang Chuhan, Liu Tingting. Recommendation Strategy Based on Users’ Preferences for Fine-Grained Attributes[J]. 数据分析与知识发现, 2021, 5(10): 94-102.
[5] Zhao Yuxiang,Lian Jingwen. Review of Cultural Heritage Crowdsourcing in the Domain of Digital Humanities[J]. 数据分析与知识发现, 2021, 5(1): 36-55.
[6] Wang Jiandong,Yu Shiyang. Principles on Constructing National Economic Brain[J]. 数据分析与知识发现, 2020, 4(7): 2-17.
[7] Qiu Erli,He Hongwei,Yi Chengqi,Li Huiying. Research on Public Policy Support Based on Character-level CNN Technology[J]. 数据分析与知识发现, 2020, 4(7): 28-37.
[8] Yang Heng,Wang Sili,Zhu Zhongming,Liu Wei,Wang Nan. Recommending Domain Knowledge Based on Parallel Collaborative Filtering Algorithm[J]. 数据分析与知识发现, 2020, 4(6): 15-21.
[9] Su Qing,Chen Sizhao,Wu Weimin,Li Xiaomei,Huang Tiankuan. Personalized Recommendation Model Based on Collaborative Filtering Algorithm of Learning Situation[J]. 数据分析与知识发现, 2020, 4(5): 105-117.
[10] Zheng Songyin,Tan Guoxin,Shi Zhongchao. Recommending Tourism Attractions Based on Segmented User Groups and Time Contexts[J]. 数据分析与知识发现, 2020, 4(5): 92-104.
[11] Shi Hongbo,Guo Hongmei,Yue Ting,Qian Li,Huang Dingyu,Chang Zhijun. Developing Modularity Scientometrics System with Distributed Technology[J]. 数据分析与知识发现, 2020, 4(2/3): 231-238.
[12] Ding Yong,Chen Xi,Jiang Cuiqing,Wang Zhao. Predicting Online Ratings with Network Representation Learning and XGBoost[J]. 数据分析与知识发现, 2020, 4(11): 52-62.
[13] Jiandong Wang. Monitoring and Forecasting Economic Performance with Big Data[J]. 数据分析与知识发现, 2020, 4(1): 12-26.
[14] Fusen Jiao,Shuqing Li. Collaborative Filtering Recommendation Based on Item Quality and User Ratings[J]. 数据分析与知识发现, 2019, 3(8): 62-67.
[15] Shan Li,Yehui Yao,Hao Li,Jie Liu,Karmapemo. ISA Biclustering Algorithm for Group Recommendation[J]. 数据分析与知识发现, 2019, 3(8): 77-87.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn