[Objective] This paper tries to efficiently and accurately extract sentiment information from Weibo texts and improve sentiment analysis performance. [Methods] First, we used WoBERT Plus and ALBERT to dynamically encode the character and word-level texts. Then, we extracted key local features with convolution operation. Next, we utilized cross-channel feature fusion and multi-head self-attention pooling operation to extract global semantic information and filter out critical data. Finally, we fused character-level and word-level semantic information using a multi-granularity feature interaction fusion operation and generated the classification results with the Softmax function. [Results] This model’s accuracy and F1 value were 98.51% and 98.53% on the weibo_senti_100k dataset and 80.11% and 75.62% on the SMP2020-EWECT dataset, respectively. Its performance was better than the advanced sentiment analysis models on each dataset. [Limitations] Our model does not include multimodal information such as video, image, and audio for sentiment classification. [Conclusions] The proposed model could effectively accomplish sentiment analysis of Weibo texts.
闫尚义, 王靖亚, 刘晓文, 崔雨萌, 陶知众, 张晓帆. 基于多头自注意力池化与多粒度特征交互融合的微博情感分析*[J]. 数据分析与知识发现, 2023, 7(4): 32-45.
Yan Shangyi, Wang Jingya, Liu Xiaowen, Cui Yumeng, Tao Zhizhong, Zhang Xiaofan. Microblog Sentiment Analysis with Multi-Head Self-Attention Pooling and Multi-Granularity Feature Interaction Fusion. Data Analysis and Knowledge Discovery, 2023, 7(4): 32-45.
(He Yue, Zhu Can. Sentiment Analysis of Weibo Opinion Leaders——Case Study of “Illegal Vaccine” Event[J]. Data Analysis and Knowledge Discovery, 2017, 1(9): 65-73.)
(Luo Shuangling, Xia Haoxiang, Wang Yanzhang. Review on Research of Social Networks of Micro-Blogging and Its Propagation Dynamics[J]. Journal of the China Society for Scientific and Technical Information, 2015, 34(12): 1304-1313.)
(Shen Bin, Yan Xin, Zhou Lihua, et al. Microblog Sentiment Analysis with ERNIE and Dual Attention Mechanism[J]. Journal of Yunnan University: Natural Sciences Edition, 2022, 44(3): 480-489.)
[4]
Alharbi N M, Alghamdi N S, Alkhammash E H, et al. Evaluation of Sentiment Analysis via Word Embedding and RNN Variants for Amazon Online Reviews[J]. Mathematical Problems in Engineering, 2021. https://doi.org/10.1155/2021/5536560.
[5]
Lee G T, Kim C O, Song M. Semisupervised Sentiment Analysis Method for Online Text Reviews[J]. Journal of Information Science, 2021, 47(3): 387-403.
doi: 10.1177/0165551520910032
(Zeng Ziming, Yang Qianwen. Sentiment Analysis for Micro-blogs with LDA and AdaBoost[J]. Data Analysis and Knowledge Discovery, 2018, 2(8): 51-59.)
[7]
Jin W Z, Zhu H, Yang G C. An Efficient Character-Level and Word-Level Feature Fusion Method for Chinese Text Classification[J]. Journal of Physics: Conference Series, 2019, 1229(1): 012057.
doi: 10.1088/1742-6596/1229/1/012057
(Chen Xin, Yang Xiaobing, Yao Yuhong. Two-Channel Mixed Neural Network Sentiment Analysis Model Based on Character and Word Fusion[J]. Journal of Chinese Computer Systems, 2021, 42(2): 279-284.)
(Han Pu, Zhang Wei, Zhang Zhanpeng, et al. Sentiment Analysis of Weibo Posts on Public Health Emergency with Feature Fusion and Multi-channel[J]. Data Analysis and Knowledge Discovery, 2021, 5(11): 68-79.)
[10]
Yan W, Zhou L F, Qian Z Y, et al. Sentiment Analysis of Student Texts Using the CNN-BiGRU-AT Model[J]. Scientific Programming, 2021, 2021: 8405623.
[11]
You H F, Yu L, Tian S W, et al. MC-Net: Multiple Max-pooling Integration Module and Cross Multi-scale Deconvolution Network[J]. Knowledge-Based Systems, 2021, 231: 107456.
doi: 10.1016/j.knosys.2021.107456
(Yang Bing, Xu Dan, Zhang Haoyuan, et al. Minority Clothing Recognition Based on Improved DenseNet-BC[J]. Journal of Zhejiang University (Science Edition), 2021, 48(6): 676-683.)
[13]
Tong X, Wang J Y, Jiao K N, et al. Robustness Detection Method of Chinese Spam Based on the Features of Joint Characters-Words[C]// Proceedings of International Conference on Computer Engineering and Networks. 2020: 845-851.
[14]
Chen W T, Fan C X, Wu Y X, et al. A Chinese Character-Level and Word-Level Complementary Text Classification Method[C]// Proceedings of 2020 International Conference on Technologies and Applications of Artificial Intelligence. 2020: 187-192.
[15]
Mikolov T, Sutskever I, Chen K, et al. Distributed Representations of Words and Phrases and Their Compositionality[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2. 2013: 3111-3119.
[16]
Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1 (Long and Short Papers). 2019: 4171-4186,
[17]
Peters M E, Neumann M, Iyyer M, et al. Deep Contextualized Word Representations[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1 (Long Papers). 2018: 2227-2237.
[18]
Radford A, Narasimhan K, Salimans T, et al. Improving Language Understanding by Generative Pre-training[EB/OL]. [2022-04-20]. https://www.cs.ubc.ca/-amuham01/LI-NG530/papers/radford2018improving.pdf.
[19]
Lan Z Z, Chen M D, Goodman S, et al. ALBERT: A Lite BERT for Self-Supervised Learning of Language Representations[OL]. arXiv Preprint, arXiv: 1909.11942.
[20]
Liu Y H, Ott M, Goyal N, et al. RoBERTa: A Robustly Optimized BERT Pretraining Approach[OL]. arXiv Preprint, arXiv: 1907.11692.
[21]
Cui Y M, Che W X, Liu T, et al. Pre-training with Whole Word Masking for Chinese BERT[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 3504-3514.
doi: 10.1109/TASLP.2021.3124365
[22]
Joshi M, Chen D Q, Liu Y H, et al. SpanBERT: Improving Pre-Training by Representing and Predicting Spans[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 64-77.
doi: 10.1162/tacl_a_00300
(Zhang Yangsen, Wang Sheng, Wei Wenjie, et al. An Answer Selection Model Based on Multi-Stage Attention Mechanism with Combination of Semantic Information and Key Information of the Question[J]. Chinese Journal of Computers, 2021, 44(3): 491-507.)
[26]
Sangeetha K, Prabha D. Sentiment Analysis of Student Feedback Using Multi-head Attention Fusion Model of Word and Context Embedding for LSTM[J]. Journal of Ambient Intelligence and Humanized Computing, 2021, 12(3): 4117-4126.
doi: 10.1007/s12652-020-01791-9
[27]
India M, Safari P, Hernando J. Self Multi-head Attention for Speaker Recognition[OL]. arXiv Preprint, arXiv: 1906.09890.
[28]
Fang Y, Gao J, Huang C, et al. Self Multi-head Attention-Based Convolutional Neural Networks for Fake News Detection[J]. PLoS One, 2019, 14(9): e0222713.
[29]
Yao L, Mao C S, Luo Y. Graph Convolutional Networks for Text Classification[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 33: 7370-7377.
(Guo Fanrong, Huang Xiaoxi, Wang Rongbo, et al. Identifying Metaphor with Transformer and Graph Convolutional Network[J]. Data Analysis and Knowledge Discovery, 2022, 6(4): 120-129.)
(Jiang Haoquan, Zhang Ruqing, Guo Jiafeng, et al. A Comparative Study of Graph Convolutional Networks and Self-Attention Mechanism on Text Classification[J]. Journal of Chinese Information Processing, 2021, 35(12): 84-93.)
Kim Y. Convolutional Neural Networks for Sentence Classification[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1746-1751.
[35]
Cho K, van Merrienboer B, Gulcehre C, et al. Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1724-1734.
[36]
Zhou P, Shi W, Tian J, et al. Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2:Short Papers). 2016: 207-212.
[37]
Lai S W, Xu L H, Liu K, et al. Recurrent Convolutional Neural Networks for Text Classification[C]// Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015: 2267-2273.
(Du Lin, Cao Dong, Lin Shuyuan, et al. Extraction and Automatic Classification of TCM Medical Records Based on Attention Mechanism of BERT and Bi-LSTM[J]. Computer Science, 2020, 47(S2): 416-420.)
(Zhao Hong, Fu Zhaoyang, Zhao Fan. Microblog Sentiment Analysis Based on BERT and Hierarchical Attention[J]. Computer Engineering and Applications, 2022, 58(5): 156-162.)
doi: 10.3778/j.issn.1002-8331.2107-0448
[40]
Wang B K, Shan D H, Fan A W, et al. A Sentiment Classification Method of Web Social Media Based on Multidimensional and Multilevel Modeling[J]. IEEE Transactions on Industrial Informatics, 2022, 18(2): 1240-1249.
doi: 10.1109/TII.2021.3085663
[41]
Mu Z B, Zheng S, Wang Q M. ACL-RoBERTa-CNN Text Classification Model Combined with Contrastive Learning[C]// Proceedings of 2021 International Conference on Big Data Engineering and Education. 2021: 193-197.
[42]
Qiu H, Fan C D, Yao J, et al. Chinese Microblog Sentiment Detection Based on CNN-BiGRU and Multihead Attention Mechanism[J]. Scientific Programming, 2020, 2020: 8865983.
[43]
Tang F H, Nongpong K. Chinese Sentiment Analysis Based on Lightweight Character-Level BERT[C]// Proceedings of 2021 13th International Conference on Knowledge and Smart Technology. 2021: 27-32.