Research and Initial Implementation of Large-scale Data Processing Based on Cloud Computing
Zhang Xingwang1, Li Chenhui2, Qin Xiaozhu1
1. Guilin University of Technology Library, Guilin 541004, China;
2. Modern Education Technology Center, Guilin University of Technology, Guilin 541004, China
Abstract：This paper introduces a large-scale data processing method based on cloud computing, builds a dynamic, scalable, cost-effective, easy to use and high-performance computing platform on a large of centralized or distributed inexpensive computer cluster, and creates a cloud computing-based framework for large-scale data processing model. It also discusses the methods and applications in this large-scale data processing environment. The computing platform is set up to verify the computing cluster and the feasibility of this model.
张兴旺, 李晨晖, 秦晓珠. 云计算环境下大规模数据处理的研究与初步实现[J]. 现代图书情报技术, 2011, 27(4): 17-23.
Zhang Xingwang, Li Chenhui, Qin Xiaozhu. Research and Initial Implementation of Large-scale Data Processing Based on Cloud Computing. New Technology of Library and Information Service, 2011, 27(4): 17-23.
[1] 陈全,邓倩妮.云计算及其关键技术[J].计算机应用,2009,29(9):2562-2567.[2] 孙兆玉,袁志平,黄宇光.面向数据密集型计算Hadoop及其应用初探[C].见: 2008年全国高性能计算学术年会.2008:441-443.[3] Armbrust M,Fox A, Griffith R, et al. Above the Clouds: A Berkeley View of Cloud Computing[EB/OL].[2011-01-10]. http://www.EECS.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf.[4] 刘真,刘峰,张宝鹏,等. 云计算模型在铁路大规模数据处理中的应用[J]. 北京交通大学学报,2010,34(5):14-19.[5] Davis D. Server Virtualization, Network Virtualization & Storage Virtualization Explained[EB/OL].[2011-01-10]. http://www.petri.co.il/server-virtualization-network-stroage-virtualization.html.[6] 虚拟化与云计算小组.虚拟化与云计算[M].北京:电子工业出版社,2009:56-81.[7] Menken I, Blokdijk G. Cloud Computing Virtualization Specialist Complete Certification Kit[M].2009: 26-166.[8] Pavlo A, Paulson E, Rasin A, et al. A Comparison of Approaches to Large Scale Data Analysis[C].In: Proceedings of the ACM SIGMOD International Conference. New York: ACM Press, 2009: 165-178.[9] Kozuch M A, Ryan M P, Gass R, et al. Tashi: Location Aware Cluster Management [C].In:Proceedings of the ACM. Barcelona: ACDC,2009: 43-48.[10] White T. Hadoop: The Definitive Guide[M]. 2nd Edition. O’Reilly Publications,2010:167-188.
