Please wait a minute...
New Technology of Library and Information Service  2011, Vol. 27 Issue (4): 17-23    DOI: 10.11925/infotech.1003-3513.2011.04.03
Current Issue | Archive | Adv Search |
Research and Initial Implementation of Large-scale Data Processing Based on Cloud Computing
Zhang Xingwang1, Li Chenhui2, Qin Xiaozhu1
1. Guilin University of Technology Library, Guilin 541004, China;
2. Modern Education Technology Center, Guilin University of Technology, Guilin 541004, China
Download:
Export: BibTeX | EndNote (RIS)      
Abstract  This paper introduces a large-scale data processing method based on cloud computing, builds a dynamic, scalable, cost-effective, easy to use and high-performance computing platform on a large of centralized or distributed inexpensive computer cluster, and creates a cloud computing-based framework for large-scale data processing model. It also discusses the methods and applications in this large-scale data processing environment. The computing platform is set up to verify the computing cluster and the feasibility of this model.
Key words： Cloud computing      Large-scale data      Low-cost computing platform      Hadoop      MapReduce     
Received: 17 January 2011      Published: 11 June 2011
: 

TP393

 		 
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Zhang Xingwang
Li Chenhui
Qin Xiaozhu

Cite this article:

Zhang Xingwang, Li Chenhui, Qin Xiaozhu. Research and Initial Implementation of Large-scale Data Processing Based on Cloud Computing. New Technology of Library and Information Service, 2011, 27(4): 17-23.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.1003-3513.2011.04.03     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2011/V27/I4/17

[1] 陈全,邓倩妮.云计算及其关键技术[J].计算机应用,2009,29(9):2562-2567.

[2] 孙兆玉,袁志平,黄宇光.面向数据密集型计算Hadoop及其应用初探[C].见: 2008年全国高性能计算学术年会.2008:441-443.

[3] Armbrust M,Fox A, Griffith R, et al. Above the Clouds: A Berkeley View of Cloud Computing[EB/OL].[2011-01-10]. http://www.EECS.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf.

[4] 刘真,刘峰,张宝鹏,等. 云计算模型在铁路大规模数据处理中的应用[J]. 北京交通大学学报,2010,34(5):14-19.

[5] Davis D. Server Virtualization, Network Virtualization & Storage Virtualization Explained[EB/OL].[2011-01-10]. http://www.petri.co.il/server-virtualization-network-stroage-virtualization.html.

[6] 虚拟化与云计算小组.虚拟化与云计算[M].北京:电子工业出版社,2009:56-81.

[7] Menken I, Blokdijk G. Cloud Computing Virtualization Specialist Complete Certification Kit[M].2009: 26-166.

[8] Pavlo A, Paulson E, Rasin A, et al. A Comparison of Approaches to Large Scale Data Analysis[C].In: Proceedings of the ACM SIGMOD International Conference. New York: ACM Press, 2009: 165-178.

[9] Kozuch M A, Ryan M P, Gass R, et al. Tashi: Location Aware Cluster Management [C].In:Proceedings of the ACM. Barcelona: ACDC,2009: 43-48.

[10] White T. Hadoop: The Definitive Guide[M]. 2nd Edition. O’Reilly Publications,2010:167-188.
[1] Yang Heng,Wang Sili,Zhu Zhongming,Liu Wei,Wang Nan. Recommending Domain Knowledge Based on Parallel Collaborative Filtering Algorithm[J]. 数据分析与知识发现, 2020, 4(6): 15-21.
[2] Gao Changyuan,Yu Jianping,He Xiaoyan. Knowledge Search for Cloud Computing Industry Alliance: An Algorithm Based on Improved Particle Swarm Optimization[J]. 数据分析与知识发现, 2017, 1(3): 81-89.
[3] Yang Aidong,Liu Dongsu. Hadoop Based Public Opinion Monitoring System for Micro-blogs[J]. 现代图书情报技术, 2016, 32(5): 56-63.
[4] Fan Yunman, Hong Na, Qian Qing, Fang An. The Research Practices of DataBase Cloud Storage Using Hadoop/HBase for the Pharmacogenomics Data[J]. 现代图书情报技术, 2015, 31(5): 73-79.
[5] Zhuo Keqiu, Yu Wei, Su Xinning. Parallel Implementing Bursty Events Detection Using MapReduce[J]. 现代图书情报技术, 2015, 31(2): 46-54.
[6] Ma Bin, Yin Lifeng. A Parallel Naive Bayesian Network Public Opinion Fast Classification Algorithm Based on Hadoop Platform[J]. 现代图书情报技术, 2015, 31(2): 78-84.
[7] Zhao Huaming. Research and Implementation of Textual Clustering in Distributed Environment[J]. 现代图书情报技术, 2015, 31(1): 82-88.
[8] Yan Shiyan, Wang Shengqing, Luo Yunchuan, Huang Haojun. An Ontology Collaborative Construction Model Based on FCA in Cloud Computing Environment[J]. 现代图书情报技术, 2014, 30(3): 49-56.
[9] Yu Wei, Chen Junpeng. Linking and Mapping of Library Catalogue Data Based on MapReduce[J]. 现代图书情报技术, 2013, 29(9): 15-22.
[10] Xiao Qiang, Zhu Qinghua, Zheng Hua, Wu Kewen. Design and Implementation of Distributed Collaborative Filtering Algorithm on Hadoop[J]. 现代图书情报技术, 2013, 29(1): 83-89.
[11] Kang Liyun, Wang Xiaoyue, Bai Rujiang. Analysis of MapReduce Principle and Its Main Implementation Platforms[J]. 现代图书情报技术, 2012, 28(2): 60-67.
[12] Wang Weijun, Jiang Yi, Liu Rui, Kari Smolander. Research Progress in Software Testing on Cloud Computing[J]. 现代图书情报技术, 2012, (11): 3-9.
[13] Jiang Yi, Cao Li, Wang Weijun, Ossi Taipale. Research on the Concept Model of Testing as a Service[J]. 现代图书情报技术, 2012, (11): 10-15.
[14] Zhang Yichi, Xiong Xiangwen, Huang Yawen, Wang Shixiong. Definition and Management of Test Data on Cloud Computing[J]. 现代图书情报技术, 2012, (11): 16-21.
[15] Udhyan Timilsina, Leah Riungu-Kalliosaari, Ossi Taipale, Kari Smolander, Wang Weijun. Security Issues on Testing of Public Cloud Applications[J]. 现代图书情报技术, 2012, (11): 22-33.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn