Sentiment Analysis of Micro-blog on Public Health Emergency with Prompt Embedding
Lai Yubin1,Chen Yan1(),Hu Xiaochun2,Huang Xin3
1School of Computer, Electronics and Information, Guangxi University, Nanning 530004, China 2School of Big Data and Artificial Intelligence, Guangxi University of Finance and Economics, Nanning 530007, China 3College of Information Engineering, Guangxi Vocational University of Agriculture, Nanning 530007, China
[Objective] At the early stage of public health emergencies, limited Weibo posts and informal expressions lead to ineffective sentiment analysis. We propose a sentiment analysis model for Weibo posts based on prompt embedding and emotion feature fusion to address this issue. [Methods] First, we extracted the sentiment information from Weibo posts based on the emotional dictionary. Then, we used the pre-trained RoBERTa model to establish semantic and sentiment vectors. We also embedded prompts as prefixes for the semantic vectors. Third, we utilized the Transformer encoder and attention mechanism to extract semantic and emotional features. We also computed the sample feature weights using the focal loss function. Finally, we combined the semantic and emotional features to conduct sentiment analysis. [Results] We examined the new model with Weibo comments on the outbreak of COVID-19 in Shenzhen. The accuracy and F1 score of the model reached 93.46% and 93.49%, which were 6.78% and 6.97% higher than the baseline BERT model. [Limitations] Weibo data contains a large amount of images and videos. However, our model did not include multi-modal fusion for sentiment analysis. [Conclusions] The proposed model could improve the effectiveness of sentiment classification with a small sample data size.
赖宇斌, 陈燕, 胡小春, 黄欣. 基于提示嵌入的突发公共卫生事件微博文本情感分析*[J]. 数据分析与知识发现, 2023, 7(11): 46-55.
Lai Yubin, Chen Yan, Hu Xiaochun, Huang Xin. Sentiment Analysis of Micro-blog on Public Health Emergency with Prompt Embedding. Data Analysis and Knowledge Discovery, 2023, 7(11): 46-55.
(Zhao Hong. Contraction and Boundary of the Individual’s Right under the Epidemic Prevention and Control[J]. Journal of Comparative Law, 2022(2): 11-24.)
(Liu Zhongbao, Qin Quan, Zhao Wenjuan. Research on the Influence of COVID-19 Event on the Netizen Emotion under the Microblog Environment[J]. Journal of Intelligence, 2021, 40(2): 138-145.)
(Zhou Ning, Zhong Na, Jin Gaoya, et al. Chinese Text Sentiment Analysis Based on Dual Channel Attention Network with Hybrid Word Embedding[J]. Data Analysis and Knowledge Discovery, 2023, 7(3): 58-68.)
(Han Pu, Zhang Wei, Zhang Zhanpeng, et al. Sentiment Analysis of Weibo Posts on Public Health Emergency with Feature Fusion and Multi-Channel[J]. Data Analysis and Knowledge Discovery, 2021, 5(11): 68-79.)
[5]
Liu P F, Yuan W Z, Fu J L, et al. Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing[OL]. arXiv Preprint, arXiv: 2107.13586.
[6]
Brown T B, Mann B, Ryder N, et al. Language Models are Few-Shot Learners[C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020: 1877-1901.
[7]
Liu Y H, Ott M, Goyal N, et al. RoBERTa: A Robustly Optimized BERT Pretraining Approach[OL]. arXiv Preprint, arXiv: 1907.11692.
Shen Bin, Yan Xin, Zhou Lihua, et al. Microblog Sentiment Analysis Based on ERNIE and Dual Attention Mechanism[J]. Journal of Yunnan University(Natural Sciences Edition), 2022, 44(3): 480-489.)
(Zhong Jiawa, Liu Wei, Wang Sili, et al. Review of Methods and Applications of Text Sentiment Analysis[J]. Data Analysis and Knowledge Discovery, 2021, 5(6): 1-13.)
(Wang Ting, Yang Wenzhong. Review of Text Sentiment Analysis Methods[J]. Computer Engineering and Applications, 2021, 57(12): 11-24.)
doi: 10.3778/j.issn.1002-8331.2101-0022
[11]
Kim Y. Convolutional Neural Networks for Sentence Classification[OL]. arXiv Preprint, arXiv:1408.5882.
(Yue Zengying, Ye Xia, Liu Ruiheng. A Survey of Language Model Based Pre-Training Technology[J]. Journal of Chinese Information Processing, 2021, 35(9): 15-29.)
[13]
Munikar M, Shakya S, Shrestha A. Fine-Grained Sentiment Classification Using BERT[OL]. arXiv Preprint, arXiv: 1910.03474.
(Qi Mengna, Zhu Liping, Li Ning. Sentiment Analysis Model of Commodity Reviews Based on ERNIE and CNN[J]. Journal of Computer Applications, 2022, 42(S1): 7-11.)
(Wang Shuyan, Yuan Ke. Sentiment Analysis Model of College Student Forum Based on RoBERTa-WWM[J]. Computer Engineering, 2022, 48(8): 292-298.)
doi: 10.19678/j.issn.1000-3428.0062008
[16]
Gu Y, Han X, Liu Z, et al. PPT: Pre-trained Prompt Tuning for Few-shot Learning[OL]. arXiv Preprint, arXiv: 2109.04332.
(Zhang Boxu, Pu Zhi, Cheng Xi. Research on Uyghur Text Classification Based on Prompt Learning[J]. Computer Engineering, 2023, 49(6): 292-299.)
doi: 10.19678/j.issn.1000-3428.0064892
(Su Hang, Hu Yahao, Xie Yifei, et al. Model-Reused Method of Two-Stage Relations and Entities Extraction with Prompt Tuning[J]. Application Research of Computers, 2022, 39(12): 3598-3604.)
[20]
Liu X, Zheng Y N, Du Z X, et al. GPT Understands, too[OL]. arXiv Preprint, arXiv: 2103.10385.
[21]
Lester B, Al-Rfou R, Constant N. The Power of Scale for Parameter-Efficient Prompt Tuning[OL]. arXiv Preprint, arXiv: 2104.08691.
[22]
Liu X, Ji K X, Fu Y C, et al. P-Tuning v2: Prompt Tuning can be Comparable to Fine-Tuning Universally Across Scales and Tasks[OL]. arXiv Preprint, arXiv: 2110.07602.
[23]
Li X L, Liang P. Prefix-Tuning: Optimizing Continuous Prompts for Generation[OL]. arXiv Preprint, arXiv: 2101.00190.
[24]
Lei Z Y, Yang Y J, Yang M, et al. A Multi-Sentiment-Resource Enhanced Attention Network for Sentiment Classification[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics(Volume 2:Short Papers). 2018: 758-763.
(Zhang Yangsen, Zheng Jia, Huang Gaijuan, et al. Microblog Sentiment Analysis Method Based on a Double Attention Model[J]. Journal of Tsinghua University (Science and Technology), 2018, 58(2): 122-130.)
[26]
Lin T Y, Goyal P, Girshick R, et al. Focal Loss for Dense Object Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327.
doi: 10.1109/TPAMI.34
[27]
Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1(Long and Short Papers). 2019: 4171-4186.
[28]
Sun Y, Wang S H, Li Y K, et al. ERNIE: Enhanced Representation Through Knowledge Integration[OL]. arXiv Preprint,arXiv:1904.09223.
[29]
Cui Y, Che W, Liu T, et al. Revisiting Pre-Trained Models for Chinese Natural Language Processing[OL]. arXiv Preprint,arXiv: 2004.13922.
