|
|
|
| Hadoop Based Public Opinion Monitoring System for Micro-blogs |
Yang Aidong( ),Liu Dongsu |
| School of Economics and Management, Xidian University, Xi’an 710126, China |
|
|
|
|
Abstract [Objective] This paper presents a new model for public opinion monitoring system based on Hadoop to retrieve and analyze information from the micro-blog platforms. [Methods] We first surveyed the existing technology of the public opinion monitoring systems and proposed a new model with modified algorithm. Then, we built a big data analysis platform with Hadoop to examine the model’s feasibility through experimental simulations. [Results] The proposed model can detect and retrieve public opinion data effectively. [Limitations] The Hadoop cluster was relatively small. We did not compare our model with other clustering algorithms to discuss their advantages and disadvantages. [Conclusions] The proposed model can conduct public opinion analysis with micro-blog data and provide scientific information for the policy makers to improve crisis management.
|
|
Received: 11 December 2015
Published: 24 June 2016
|
| [1] | 张克生. 国家决策:机制与舆情[M]. 天津: 天津社会科学院出版社, 2004: 17. | | [1] | (Zhang Kesheng.National Decision- making: Mechanism and Public Opinion [M]. Tianjin: Tianjin Academy of Social Sciences Press, 2014: 17.) | | [2] | 中国互联网络信息中心. 第36次中国互联网络发展状况统计报告[R/OL]. [2015-07-23]. . | | [2] | (CNNIC.The 36th China Internet Development Statistics Report [R/OL]. [2015-07-23]. | | [3] | Wasserman S, Faust K.Social Network Analysis: Methods and Applications [M]. Cambridge, NY: Cambridge University Press, 1994. | | [4] | 兰月新, 董希琳, 苏国强, 等. 大数据背景下微博舆情信息交互模型研究[J]. 现代图书情报技术, 2015(5): 24-33. | | [4] | (Lan Yuexin, Dong Xilin, Su Guoqiang, et al.Research on Micro-blog Public Opinion Information Interaction Model Under the Background of Big Data[J]. New Technology of Library and Information Service, 2015(5): 24-33.) | | [5] | 田占伟, 隋玚. 基于复杂网络理论的微博信息传播实证分析[J]. 图书情报工作, 2012, 56(8): 42-46. | | [5] | (Tian Zhanwei, Sui Yang.The Empirical Analysis of Micro-blog Information Flow Based on Complex Network Theory[J]. Library and Information Service, 2012, 56(8): 42-46.) | | [6] | 刘志明, 刘鲁. 微博网络舆情中的意见领袖识别及分析[J]. 系统工程, 2011, 29(6): 8-16. | | [6] | (Liu Zhiming, Liu Lu.Recognition and Analysis of Opinion Leaders in Microblog Public Opinions[J]. Systems Engineering, 2011, 29(6): 8-16.) | | [7] | 高承实, 荣星, 陈越. 微博舆情监测指标体系研究[J]. 情报杂志, 2011, 30(9): 66-70. | | [7] | (Gao Chengshi, Rong Xing, Chen Yue.Research on Public Opinion Monitoring Index- system in Micro-blogging[J]. Journal of Information, 2011, 30(9): 66-70.) | | [8] | 马彦. 大数据环境下微博舆情热点话题挖掘方法研究[J]. 现代情报, 2014, 34(11): 29-33. | | [8] | (Ma Yan.Study on the Method of Micro-blogging Public Opinion Hotspots Mining in Big Data[J]. Modern Information, 2014, 34(11): 29-33.) | | [9] | 潘芳, 张霞, 仲伟俊. 基于BP神经网络的微博网络社群突发舆情的预警监控[J]. 情报杂志, 2014, 33(5): 125-128. | | [9] | (Pan Fang, Zhang Xia, Zhong Weijun.Precautionary Monitoring of the Sudden Burst of Public Opinion in Weibo Community on Internet Based on BP Neural Network[J]. Journal of Information, 2014, 33(5): 125-128.) | | [10] | Hadoop [EB/OL]. [2016-01-12]. . | | [11] | HDFS User Guide [EB/OL]. [2016-01-12]. . | | [12] | Dean J, Ghemawat S.MapReduce: Simplified Data Processing on Large Clusters[J]. Communications of the ACM, 2004, 51(1): 107-113. | | [13] | George L.HBase: The Definitive Guide[M]. O’Reilly Media, 2011. | | [14] | Song Y, Cai D F, Zhang G P, et al.Approach to Chinese Word Segmentation Based on Character-Word Joint Decoding[J]. Journal of Software, 2009, 20(9): 2366-2375. | | [15] | TF-IDF [EB/OL]. [2016-01-12]. . | | [16] | 知网[EB/OL]. [2016-01-12]. . (HowNet Knowledge Database [EB/OL]. [2016-01-12]. | | [17] | 李应安. 基于MapReduce的聚类算法的并行化研究[D]. 广州:中山大学, 2010. | | [17] | (Li Ying’an.Research on Parallelization of Clustering Algorithm Based on MapReduce [D]. Guangzhou: Sun Yat-Sen University, 2010.) | | [18] | 冯希莹, 王来华. 舆情概念辨析[J]. 社会工作, 2011(10): 83-87. | | [18] | (Feng Xiying, Wang Laihua.Discussion of the Concept of Public Opinion and Sentiments[J]. Journal of Social Work, 2011(10): 83-87.) | | [19] | 张伟舒, 吕云翔. 微博情感倾向算法的改进与实现[J]. 知识管理论坛, 2013(9): 21-27. | | [19] | (Zhang Weishu, Lv Yunxiang.The Improvement and Implementation of the Micro-blog Sentiment Orientation Algorithm[J]. Knowledge Management Forum, 2013(9): 21-27.) | | [20] | Shen Y.Emotion Mining Research on Micro-blog [C]. In: Proceedings of the 1st IEEE Symposium on Web Society. Lanzhou: Lanzhou University, 2009. |
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
