Please wait a minute...
Data Analysis and Knowledge Discovery  2018, Vol. 2 Issue (3): 70-78    DOI: 10.11925/infotech.2096-3467.2017.0997
Current Issue | Archive | Adv Search |
Mining News on Competitors with Sentiment Classification
Wang Shuyi(), Liao Huatao, Wu Chake
School of Management, Tianjin Normal University, Tianjin 300387, China
Download: PDF (4337 KB)   HTML ( 7
Export: BibTeX | EndNote (RIS)      

[Objective] This paper aims to improve the efficiency of topic modeling from news reports, and reduce the cost of competitive intelligence analysis. [Context] The proposed method could help competitive intelligence analysts accomplish environmental scanning tasks with the help of news reports. [Methods] First, we retrieved news stories with the help of a web crawler. Then, we categorized these articles based on a sentiment analysis API. Third, we identified and visualized news topics with the help of Latent Dirichlet Allocation method. We used Python to finish the data collection, cleansing, analyzing and visualizing jobs. [Results] We identified positive and negative sentiments as well as related keywords from news reports on the bike-sharing industry. [Conclusions] The proposed topic mining method based on sentiment analysis helps enterprises identify competitive advantages. It also improves the effectiveness of environmental scanning for competitive intelligence.

Key wordsSentiment Classification      Topic Mining      Competitive Intelligence     
Received: 29 September 2017      Published: 03 April 2018
ZTFLH:  TP393  

Cite this article:

Wang Shuyi,Liao Huatao,Wu Chake. Mining News on Competitors with Sentiment Classification. Data Analysis and Knowledge Discovery, 2018, 2(3): 70-78.

URL:     OR

分类 ofo 摩拜
[1] Sjöblom S.Competitive Intelligence-Conducting an Analysis of a Business Environment[D]. Tampere University of Technology, 2015. .
[2] Tuan L T.Organizational Social Capital as a Moderator for the Effect of Entrepreneurial Orientation on Competitive Intelligence[J]. Journal of Strategic Marketing, 2015, 25(4): 301-315.
doi: 10.1080/0965254X.2015.1076884
[3] 肖璐, 陈果, 刘继云. 基于情感分析的企业产品级竞争对手识别研究——以用户评论为数据源[J]. 图书情报工作, 2016, 60(1): 83-90, 97.
doi: 10.13266/j.issn.0252-3116.2016.01.012
[3] (Xiao Lu, Chen Guo, Liu Jiyun.Study on Identification of Enterprise Product Level Competitor Based on Sentiment Analysis: Taking User Reviews for Data Resources[J]. Library and Information Service,2016, 60(1): 83-90, 97.)
doi: 10.13266/j.issn.0252-3116.2016.01.012
[4] 王伟, 王洪伟. 面向竞争力的特征比较网络: 情感分析方法[J]. 管理科学学报, 2016, 19(9): 109-126.
[4] (Wang Wei, Wang Hongwei.Comparative Network for Product Competition in Feature-levels Through Sentiment Analysis[J]. Journal of Management Sciences in China, 2016, 19(9): 109-126.)
[5] 唐晓波, 刘广超. 细粒度情感分析研究综述[J]. 图书情报工作, 2017, 61(5): 132-140.
[5] (Tang Xiaobo, Liu Guangchao.Research Review on Fine-grained Sentiment Analysis[J]. Library and Information Service, 2017, 61(5): 132-140.)
[6] 吴应良, 黄媛, 王选飞. 在线中文用户评论研究综述: 基于情感计算的视角[J]. 情报科学, 2017, 35(6): 159-163.
[6] (Wu Yingliang, Huang Yuan, Wang Xuanfei.Research on Online Users’ Reviews in Chinese: Basing on the Perspective of Affective Computing[J]. Information Science, 2017, 35(6): 159-163.)
[7] He W, Zha S, Li L.Social Media Competitive Analysis and Text Mining: A Case Study in the Pizza Industry[J]. International Journal of Information Management, 2013, 33(3): 464-472.
doi: 10.1016/j.ijinfomgt.2013.01.001
[8] He W, Wu H, Yan G, et al.A Novel Social Media Competitive Analytics Framework with Sentiment Benchmarks[J]. Information & Management, 2015, 52(7): 801-812.
doi: 10.1016/
[9] He W, Shen J, Tian X, et al.Gaining Competitive Intelligence from Social Media Data: Evidence from Two Largest Retail Chains in the World[J]. Industrial Management & Data Systems, 2015, 115(9): 1622-1636.
[10] Fan W, Gordon M D.The Power of Social Media Analytics[J]. Communications of the ACM, 2014, 57(6): 74-81.
doi: 10.1145/2602574
[11] Papadopoulos S, Bontcheva K, Jaho E, et al. Overview of the Special Issue on Trust and Veracity of Information in Social Media[J]. ACM Transactions on Information Systems (TOIS), 2016, 34(3): Article No.14.
doi: 10.1145/2870630
[12] Allcott H, Gentzkow M.Social Media and Fake News in the 2016 Election[J]. Journal of Economic Perspectives, 2017, 31(2): 211-236.
doi: 10.1257/jep.31.2.211
[13] Luca M, Zervas G.Fake It till You Make It: Reputation, Competition, and Yelp Review Fraud[J]. Management Science, 2016, 62(12): 3412-3427.
doi: 10.2139/ssrn.2293164
[14] Filieri R, McLeay F. E-WOM and Accommodation: An Analysis of the Factors That Influence Travelers’ Adoption of Information from Online Reviews[J]. Journal of Travel Research, 2014, 53(1): 44-57.
doi: 10.1177/0047287513481274
[15] Fearn-Banks K.Crisis Communications: A Casebook Approach[M]. Routledge, 2016.
[16] Kleinnijenhuis J, Schultz F, Utz S, et al.The Mediating Role of the News in the BP Oil Spill Crisis 2010: How US News is Influenced by Public Relations and in Turn Influences Public Awareness, Foreign News, and the Share Price[J]. Communication Research, 2015, 42(3): 408-428.
doi: 10.1177/0093650213510940
[17] Du Toit A S. Using Environmental Scanning to Collect Strategic Information: A South African Survey[J]. International Journal of Information Management, 2016, 36(1): 16-24.
doi: 10.1016/j.ijinfomgt.2015.08.005
[18] Cheng X Y, Zhu L L, Zhu Q, et al.The Framework of Network Public Opinion Monitoring and Analyzing System Based on Semantic Content Identification[J]. Journal of Convergence Information Technology, 2010, 5(10): 1-5.
doi: 10.4156/jcit.vol5.issue10.1
[19] Chung W.BizPro: Extracting and Categorizing Business Intelligence Factors from Textual News Articles[J]. International Journal of Information Management, 2014, 34(2): 272-284.
doi: 10.1016/j.ijinfomgt.2014.01.001
[20] Yang C S, Ye H C.Mining Company Competitor/Collaborator Network from Online News for Competitive Intelligence[C]// Proceedings of the 2nd International Conference on Intelligent Technologies and Engineering Systems (ICITES2013). Springer, 2014: 627-634.
[21] Ma Z, Pant G, Sheng O R.Mining Competitor Relationships from Online News: A Network-Based Approach[J]. Electronic Commerce Research and Applications, 2011, 10(4): 418-427.
doi: 10.1016/j.elerap.2010.11.006
[22] Pang B, Lee L, Vaiythyanathan S.Thumbs up?: Sentiment Classification Using Machine Learning Techniques[C]// Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - Volume 10. Association for Computational Linguistics, 2002: 79-86.
[23] Blei D M, Ng A Y, Jordan M I.Latent Dirichlet Allocation[J]. Journal of Machine Learning Research, 2003, 3: 993-1022.
[24] 刘启华. 基于LDA 和领域本体的竞争情报采集研究[J]. 情报科学, 2013, 31(4): 51-55.
[24] (Liu Qihua.A Study of Competitive Intelligence Acquisition System Based on LDA and Domain Ontology[J]. Information Science, 2013, 31(4): 51-55.)
[25] Shi Z M, Lee G, Whinston A B.Toward a Better Measure of Business Proximity: Topic Modeling for Industry Intelligence[J]. MIS Quarterly, 2016, 40(4): 1035-1056.
doi: 10.25300/MISQ/2016/40.4.11
[26] Wang B, Liu S, Ding K, et al.Identifying Technological Topics and Institution-Topic Distribution Probability for Patent Competitive Intelligence Analysis: A Case Study in LTE Technology[J]. Scientometrics, 2014, 101(1): 685-704.
doi: 10.1007/s11192-014-1342-3
[27] 潘云仙, 袁方. 基于JST 模型的新闻文本的情感分类研究[J]. 郑州大学学报: 理学版, 2015, 47(1):64-68.
doi: 10.3969/j.issn.1671-6841.2015.01.014
[27] (Pan Yunxian, Yuan Fang.News-text Sentiment Classification Research Based on JST Model[J]. Journal of Zhengzhou University: Natural Science Edition, 2015, 47(1): 64-68.)
doi: 10.3969/j.issn.1671-6841.2015.01.014
[28] Calheiros A C, Moro S, Rita P.Sentiment Classification of Consumer-Generated Online Reviews Using Topic Modeling[J]. Journal of Hospitality Marketing & Management, 2017(13): 1-19.
doi: 10.1080/19368623.2017.1310075
[29] Papanikolaou Y, Foulds J R, Rubin T N, et al.Dense Distributions from Sparse Samples: Improved Gibbs Sampling Parameter Estimators for LDA[J]. Journal of Machine Learning Research, 2017, 18: 1-58.
[30] Sievert C, Shirley K E.LDAvis: A Method for Visualizing and Interpreting Topics[C]//Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces. 2014: 63-70.
[31] Underwood T.Topic Modeling Made Just Simple Enough [EB/OL]. [2017-10-25]..
[1] Manyu Huang,Qi Yun,Hufeng Peng,Xuemeng Dou. Analyzing Textual Features of Excess-funded Agricultural Products——Case Study of Crowdfunding Website[J]. 数据分析与知识发现, 2019, 3(9): 124-134.
[2] Qingqing Zhang,Xingshi He,Huimin Wang,Shengjun Meng. Text Sentiment Classification Based on Deep Belief Network[J]. 数据分析与知识发现, 2019, 3(4): 71-79.
[3] Lei Yang,Zirun Wang,Guisheng Hou. Discovering Topics of Online Health Community with Q-LDA Model[J]. 数据分析与知识发现, 2019, 3(11): 52-59.
[4] Qiang Lu,Zhenfang Zhu,Fuyong Xu,Qiangqiang Guo. Chinese Sentiment Classification Method with Bi-LSTM and Grammar Rules[J]. 数据分析与知识发现, 2019, 3(11): 99-107.
[5] Hui Li,Yaqing Chai. Fine-Grained Sentiment Analysis Based on Convolutional Neural Network[J]. 数据分析与知识发现, 2019, 3(1): 95-103.
[6] Lin Sun,Yanzhang Wang. Identifying Competitive Intelligence Based on Knowledge Element[J]. 数据分析与知识发现, 2018, 2(6): 25-36.
[7] Zhang Qingqing,Liu Xilin. Classifying Sentiments Based on BPSO Random Subspace[J]. 数据分析与知识发现, 2017, 1(5): 71-81.
[8] Wang Xiaoyun,Yuan Yuan,Shi Lingling. Predicting Opening Weekend Box Office Prediction Based on Microblog[J]. 现代图书情报技术, 2016, 32(4): 31-39.
[9] Guo Shunli,Zhang Xiangxian. Building Sentiment Analysis Dictionary for Chinese Book Reviews[J]. 现代图书情报技术, 2016, 32(2): 67-74.
[10] Yang Haixia,Gao Baojun,Sun Hanlin. Extracting Topics of Computer Science Literature with LDA Model[J]. 现代图书情报技术, 2016, 32(11): 20-26.
[11] Shao Jian, Zhang Chengzhi, Li Lei. Survey on Hashtag Mining and Its Application[J]. 现代图书情报技术, 2015, 31(10): 40-49.
[12] Bi Qiumin, Li Ming, Zeng Zhiyong. Semi-supervised Micro-blog Sentiment Classification Method Combining Active Learning and Co-training[J]. 现代图书情报技术, 2015, 31(1): 38-44.
[13] Wang Ping, Zhi Fengwen, Wang Yi, Shen Tao. Analyzing Competitive Intelligence of Enterprises with Concept Lattice[J]. 现代图书情报技术, 2013, 29(10): 66-72.
[14] Zhu Hengmin, Zhu Weiwei. Study on Web Topic Online Clustering Approach Based on Single-Pass Algorithm[J]. 现代图书情报技术, 2011, 27(12): 52-57.
[15] Xu Xin, Yu Fei, Zhang Li. A Method and Its Application of Text Semantic Orientation[J]. 现代图书情报技术, 2011, 27(10): 54-62.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938