Please wait a minute...
New Technology of Library and Information Service  2016, Vol. 32 Issue (5): 56-63    DOI: 10.11925/infotech.1003-3513.2016.05.07
Orginal Article Current Issue | Archive | Adv Search |
Hadoop Based Public Opinion Monitoring System for Micro-blogs
Yang Aidong(),Liu Dongsu
School of Economics and Management, Xidian University, Xi’an 710126, China
Download: PDF(1165 KB)   HTML ( 58
Export: BibTeX | EndNote (RIS)      

[Objective] This paper presents a new model for public opinion monitoring system based on Hadoop to retrieve and analyze information from the micro-blog platforms. [Methods] We first surveyed the existing technology of the public opinion monitoring systems and proposed a new model with modified algorithm. Then, we built a big data analysis platform with Hadoop to examine the model’s feasibility through experimental simulations. [Results] The proposed model can detect and retrieve public opinion data effectively. [Limitations] The Hadoop cluster was relatively small. We did not compare our model with other clustering algorithms to discuss their advantages and disadvantages. [Conclusions] The proposed model can conduct public opinion analysis with micro-blog data and provide scientific information for the policy makers to improve crisis management.

Key wordsMonitoring public opinion      Hadoop      Micro-blog      Big data     
Received: 11 December 2015      Published: 24 June 2016

Cite this article:

Yang Aidong,Liu Dongsu. Hadoop Based Public Opinion Monitoring System for Micro-blogs. New Technology of Library and Information Service, 2016, 32(5): 56-63.

URL:     OR

[1] 张克生. 国家决策:机制与舆情[M]. 天津: 天津社会科学院出版社, 2004: 17.
[1] (Zhang Kesheng.National Decision- making: Mechanism and Public Opinion [M]. Tianjin: Tianjin Academy of Social Sciences Press, 2014: 17.)
[2] 中国互联网络信息中心. 第36次中国互联网络发展状况统计报告[R/OL]. [2015-07-23]. .
[2] (CNNIC.The 36th China Internet Development Statistics Report [R/OL]. [2015-07-23].
[3] Wasserman S, Faust K.Social Network Analysis: Methods and Applications [M]. Cambridge, NY: Cambridge University Press, 1994.
[4] 兰月新, 董希琳, 苏国强, 等. 大数据背景下微博舆情信息交互模型研究[J]. 现代图书情报技术, 2015(5): 24-33.
[4] (Lan Yuexin, Dong Xilin, Su Guoqiang, et al.Research on Micro-blog Public Opinion Information Interaction Model Under the Background of Big Data[J]. New Technology of Library and Information Service, 2015(5): 24-33.)
[5] 田占伟, 隋玚. 基于复杂网络理论的微博信息传播实证分析[J]. 图书情报工作, 2012, 56(8): 42-46.
[5] (Tian Zhanwei, Sui Yang.The Empirical Analysis of Micro-blog Information Flow Based on Complex Network Theory[J]. Library and Information Service, 2012, 56(8): 42-46.)
[6] 刘志明, 刘鲁. 微博网络舆情中的意见领袖识别及分析[J]. 系统工程, 2011, 29(6): 8-16.
[6] (Liu Zhiming, Liu Lu.Recognition and Analysis of Opinion Leaders in Microblog Public Opinions[J]. Systems Engineering, 2011, 29(6): 8-16.)
[7] 高承实, 荣星, 陈越. 微博舆情监测指标体系研究[J]. 情报杂志, 2011, 30(9): 66-70.
[7] (Gao Chengshi, Rong Xing, Chen Yue.Research on Public Opinion Monitoring Index- system in Micro-blogging[J]. Journal of Information, 2011, 30(9): 66-70.)
[8] 马彦. 大数据环境下微博舆情热点话题挖掘方法研究[J]. 现代情报, 2014, 34(11): 29-33.
[8] (Ma Yan.Study on the Method of Micro-blogging Public Opinion Hotspots Mining in Big Data[J]. Modern Information, 2014, 34(11): 29-33.)
[9] 潘芳, 张霞, 仲伟俊. 基于BP神经网络的微博网络社群突发舆情的预警监控[J]. 情报杂志, 2014, 33(5): 125-128.
[9] (Pan Fang, Zhang Xia, Zhong Weijun.Precautionary Monitoring of the Sudden Burst of Public Opinion in Weibo Community on Internet Based on BP Neural Network[J]. Journal of Information, 2014, 33(5): 125-128.)
[10] Hadoop [EB/OL]. [2016-01-12]. .
[11] HDFS User Guide [EB/OL]. [2016-01-12]. .
[12] Dean J, Ghemawat S.MapReduce: Simplified Data Processing on Large Clusters[J]. Communications of the ACM, 2004, 51(1): 107-113.
[13] George L.HBase: The Definitive Guide[M]. O’Reilly Media, 2011.
[14] Song Y, Cai D F, Zhang G P, et al.Approach to Chinese Word Segmentation Based on Character-Word Joint Decoding[J]. Journal of Software, 2009, 20(9): 2366-2375.
[15] TF-IDF [EB/OL]. [2016-01-12]. .
[16] 知网[EB/OL]. [2016-01-12]. . (HowNet Knowledge Database [EB/OL]. [2016-01-12].
[17] 李应安. 基于MapReduce的聚类算法的并行化研究[D]. 广州:中山大学, 2010.
[17] (Li Ying’an.Research on Parallelization of Clustering Algorithm Based on MapReduce [D]. Guangzhou: Sun Yat-Sen University, 2010.)
[18] 冯希莹, 王来华. 舆情概念辨析[J]. 社会工作, 2011(10): 83-87.
[18] (Feng Xiying, Wang Laihua.Discussion of the Concept of Public Opinion and Sentiments[J]. Journal of Social Work, 2011(10): 83-87.)
[19] 张伟舒, 吕云翔. 微博情感倾向算法的改进与实现[J]. 知识管理论坛, 2013(9): 21-27.
[19] (Zhang Weishu, Lv Yunxiang.The Improvement and Implementation of the Micro-blog Sentiment Orientation Algorithm[J]. Knowledge Management Forum, 2013(9): 21-27.)
[20] Shen Y.Emotion Mining Research on Micro-blog [C]. In: Proceedings of the 1st IEEE Symposium on Web Society. Lanzhou: Lanzhou University, 2009.
[1] Beibei Kong,Jing Xie,Li Qian,Zhijun Chang,Zhenxin Wu. Methodology and Tools to Enrich Sci-Tech Big Data[J]. 数据分析与知识发现, 2019, 3(7): 113-122.
[2] Xiaozhou Dong,Xinkang Chen. E-Coupon and Economic Performance of E-commerce[J]. 数据分析与知识发现, 2019, 3(6): 42-49.
[3] Quan Lu,Anqi Zhu,Jiyue Zhang,Jing Chen. Research on User Information Requirement in Chinese Network Health Community: Taking Tumor-forum Data of Qiuyi as an Example[J]. 数据分析与知识发现, 2019, 3(4): 22-32.
[4] Mingqing Zhao,Shengqiang Wu. Research on Stock Market Weighted Prediction Method Based on Micro-blog Sentiment Analysis[J]. 数据分析与知识发现, 2019, 3(2): 43-51.
[5] Ying Wang,Li Qian,Jing Xie,Zhijun Chang,Beibei Kong. Building Knowledge Graph with Sci-Tech Big Data[J]. 数据分析与知识发现, 2019, 3(1): 15-26.
[6] Li Qian,Jing Xie,Zhijun Chang,Zhenxin Wu,Dongrong Zhang. Designing Smart Knowledge Services with Sci-Tech Big Data[J]. 数据分析与知识发现, 2019, 3(1): 4-14.
[7] Jiying Hu,Jing Xie,Li Qian,Changlei Fu. Constructing Big Data Platform for Sci-Tech Knowledge Discovery with Knowledge Graph[J]. 数据分析与知识发现, 2019, 3(1): 55-62.
[8] Jing Xie,Li Qian,Hongbo Shi,Beibei Kong,Jiying Hu. Designing Framework for Precise Service of Scholarly Big Data[J]. 数据分析与知识发现, 2019, 3(1): 63-71.
[9] Ziming Zeng,Qianwen Yang. Sentiment Analysis for Micro-blogs with LDA and AdaBoost[J]. 数据分析与知识发现, 2018, 2(8): 51-59.
[10] Zhihong Shen,Chang Yao,Yanfei Hou,Linhuan Wu,Yuepeng Li. Big Linked Data Management: Challenges, Solutions and Practices[J]. 数据分析与知识发现, 2018, 2(1): 9-20.
[11] Yongbing Gao,Guipeng Yang,Di Zhang,Zhanfei Ma. Detecting Events from Official Weibo Profiles Based on Post Clustering with Burst Words[J]. 数据分析与知识发现, 2017, 1(9): 57-64.
[12] Yue He,Can Zhu. Sentiment Analysis of Weibo Opinion Leaders——Case Study of “Illegal Vaccine” Event[J]. 数据分析与知识发现, 2017, 1(9): 65-73.
[13] Cao Yang, Fan Wenfei, Yuan Tengfei. Is Big Data Analytics Beyond the Reach of Small Companies?[J]. 数据分析与知识发现, 2017, 1(9): 1-7.
[14] Lemen Chao,Canjun Yang,Shengjie Wang,Junpeng Zhao,Mengtian Xu. Data Science Curriculums Around the World: An Empirical Study[J]. 数据分析与知识发现, 2017, 1(6): 12-21.
[15] Xuefeng Shen, Yongzhen Ke, Nan Yao. Visualization of Coalition Data Based on Multi View Cooperation[J]. 数据分析与知识发现, 2017, 1(3): 21-28.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938