|
|
|
| Research on Recognition Model and Recognition Effect of Network Public Opinion Visual Emotion under Multi-Dimensional Attention Mechanism |
Wang Xiwei1,2,3,Wang Qiuyue4( ),Cai Hongtian5 |
1School of Business and Management, Jilin University, Changchun 130022, China
2Research Center for Big Data Management, Jilin University, Changchun 130022, China
3Cyberspace Governance Research Center, Jilin University, Changchun 130022, China
4School of Economics, Jilin University, Changchun 130022, China
5School of Software, Jilin University, Changchun 130022, China |
|
|
|
|
Abstract [Objective] To fill the current deficiency in visual emotional analysis research, a ResNet34-based improved emotion analysis model was constructed to analyze and improve the accuracy of image emotion classification. [Methods] Firstly, a visual emotion recognition model was established based on the ResNet34. Then, by integrating the CBAM module and Non-Local module, emotion features were learned and represented. Finally, the above model was used to classify and recognize emotional features, and compared with VGG16 and ResNet50 models. [Results] The recognition effect of the constructed model was verified through experiments, and the research results showed that the accuracy, precision, recall, and F1 score of the model reached 84.42%, 84.10%, 83.70%, and 83.80% respectively. Compared with the baseline models of the VGG16 and ResNet50, the accuracy of the proposed model was improved by 4.17% and 3.44%, and the F1 score was improved by 4.20% and 3.30%. [Limitations] The scale of the test dataset is relatively small, the effectiveness of annotation was not calculated using metrics such as the Pearson correlation coefficient, and a comparison was not made with visual-based emotion classification algorithms. [Conclusions] From the perspective of visual emotional analysis, optimizing the emotion recognition model supplements the analysis mode of emotional computation, providing support for the extraction and analysis of emotional features in public opinion information.
|
|
Received: 26 September 2023
Published: 12 April 2024
|
|
|
| Fund:National Social Science Fund of China(18ZDA310) |
|
Corresponding Authors:
Wang Qiuyue，ORCID：0009-0008-2149-515X，E-mail： wangqiuyue2022@163.com。
|
| [1] |
吴鹏, 强韶华, 高庆宁. 基于SOAR模型的网民群体负面情感建模研究[J]. 中国管理科学, 2018, 26(3): 126-138.
|
| [1] |
(Wu Peng, Qiang Shaohua, Gao Qingning. Modelling Internet Users’ Negative Emotion Based on SOAR Model[J]. Chinese Journal of Management Science, 2018, 26(3): 126-138.)
|
| [2] |
孔婧媛, 滕广青, 王思茗, 等. 舆情当事人回应对网民情感的影响研究[J]. 图书情报工作, 2020, 64(18): 89-96.
doi: 10.13266/j.issn.0252-3116.2020.18.010
|
| [2] |
(Kong Jingyuan, Teng Guangqing, Wang Siming, et al. Impact of Party’s Responses on Netizens’ Emotions in Public Opinion[J]. Library and Information Service, 2020, 64(18): 89-96.)
doi: 10.13266/j.issn.0252-3116.2020.18.010
|
| [3] |
Huang F R, Zhang X M, Zhao Z H, et al. Image-Text Sentiment Analysis via Deep Multimodal Attentive Fusion[J]. Knowledge-Based Systems, 2019, 167: 26-37.
doi: 10.1016/j.knosys.2019.01.019
|
| [4] |
You Q Z, Luo J B, Jin H L, et al. Cross-Modality Consistent Regression for Joint Visual-Textual Sentiment Analysis of Social Multimedia[C]// Proceedings of the 9th ACM International Conference on Web Search and Data Mining. 2016: 13-22.
|
| [5] |
Picard R W. Affective Computing[M]. Cambridge: MIT Press, 2000.
|
| [6] |
Tan X W, Cai Y, Xu J Y, et al. Improving Aspect-Based Sentiment Analysis via Aligning Aspect Embedding[J]. Neurocomputing, 2020, 383: 336-347.
doi: 10.1016/j.neucom.2019.12.035
|
| [7] |
Hu M Q, Liu B. Mining and Summarizing Customer Reviews[C]// Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2004: 168-177.
|
| [8] |
Plutchik R. The Nature of Emotions: Human Emotions Have Deep Evolutionary Roots, a Fact That May Explain Their Complexity and Provide Tools for Clinical Practice[J]. American Scientist, 2001, 89(4): 344-350.
doi: 10.1511/2001.28.344
|
| [9] |
张亚洲, 戎璐, 宋大为, 等. 多模态情感分析研究综述[J]. 模式识别与人工智能, 2020, 33(5): 426-438.
doi: 10.16451/j.cnki.issn1003-6059.202005005
|
| [9] |
(Zhang Yazhou, Rong Lu, Song Dawei, et al. A Survey on Multimodal Sentiment Analysis[J]. Pattern Recognition and Artificial Intelligence, 2020, 33(5): 426-438.)
doi: 10.16451/j.cnki.issn1003-6059.202005005
|
| [10] |
Lazarus R S. Emotion and Adaptation[J]. Basic Books, 1991.DOI:10.1097/00005053-199303000-00014.
|
| [11] |
Mikolov T, Sutskever I, Chen K, et al. Distributed Representations of Words and Phrases and Their Compositionality[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems-Volume 2. 2013: 3111-3119.
|
| [12] |
朱晓霞, 宋嘉欣, 孟建芳. 基于动态主题-情感演化模型的网络舆情信息分析[J]. 情报科学, 2019, 37(7): 72-78.
|
| [12] |
(Zhu Xiaoxia, Song Jiaxin, Meng Jianfang. Analysis of Online Public Opinion Information Based on the Dynamic Theme-Emotion Evolution Model[J]. Information Science, 2019, 37(7): 72-78.)
|
| [13] |
李婷婷, 姬东鸿. 基于SVM和CRF多特征组合的微博情感分析[J]. 计算机应用研究, 2015, 32 (4): 978-981.
|
| [13] |
(Li Tingting, Ji Donghong. Sentiment Analysis of Micro-blog Based on SVM and CRF Using Various Combinations of Features[J]. Application Research of Computers, 2015, 32 (4): 978-981.)
|
| [14] |
Kim Y. Convolutional Neural Networks for Sentence Classification[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1746-1751.
|
| [15] |
Ma Y K, Peng H Y, Cambria E. Targeted Aspect-Based Sentiment Analysis via Embedding Commonsense Knowledge into an Attentive LSTM[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 5876-5883.
|
| [16] |
Lu X, Suryanarayan P, Adams R B, et al. On Shape and the Computability of Emotions[C]// Proceedings of the 20th ACM International Conference on Multimedia. 2012: 229-238.
|
| [17] |
范涛, 吴鹏, 王昊, 等. 基于多模态联合注意力机制的网民情感分析研究[J]. 情报学报, 2021, 40(6): 656-665.
|
| [17] |
(Fan Tao, Wu Peng, Wang Hao, et al. Sentiment Analysis of Online Users Based on Multimodal Co-Attention[J]. Journal of the China Society for Scientific and Technical Information, 2021, 40(6): 656-665.)
|
| [18] |
Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1800-1807.
|
| [19] |
胡雅敏, 吴晓燕, 陈方. 基于机器学习的技术术语识别研究综述[J]. 数据分析与知识发现, 2022, 6(2/3): 7-17.
|
| [19] |
(Hu Yamin, Wu Xiaoyan, Chen Fang. Review of Technology Term Recognition Studies Based on Machine Learning[J]. Data Analysis and Knowledge Discovery, 2022, 6(2/3): 7-17.)
|
| [20] |
倪亮, 吴鹏, 周雪晴. 基于深度学习的多模态新闻数据主题发现研究[J]. 数据分析与知识发现, DOI: 10.11925/infotech.2096-3467.2023.0021.
|
| [20] |
(Ni Liang, Wu Peng, Zhou Xueqing. Topic Detecting on Multimodal News Data Based on Deep Learning[J]. Data Analysis and Knowledge Discovery, DOI: 10.11925/infotech.2096-3467.2023.0021.)
|
| [21] |
魏建香, 陆谦, 韩普, 等. 基于多语义信息融合的事件检测模型[J]. 数据分析与知识发现, 2023, 7(12): 64-74.
|
| [21] |
(Wei Jianxiang, Lu Qian, Han Pu, et al. Event Detection Model Based on Semantic Information Fusion[J]. Data Analysis and Knowledge Discovery, 2023, 7(12): 64-74.)
|
| [22] |
Pang B, Lee L. Opinion Mining and Sentiment Analysis[J]. Foundations and Trends in Information Retrieval, 2008, 2(1-2): 1-135.
doi: 10.1561/1500000011
|
| [23] |
Nasukawa T, Yi J. Sentiment Analysis: Capturing Favorability Using Natural Language Processing[C]// Proceedings of the 2nd International Conference on Knowledge Capture. 2003: 70-77.
|
| [24] |
Sasikala D, Sukumaran S. A Survey on Lexicon and Machine Learning Based Classification Methods for Sentimental Analysis[J]. International Journal of Research and Analytical Reviews, 2019, 6(2): 256-259.
|
| [25] |
Prabha M I, Srikanth G U. Survey of Sentiment Analysis Using Deep Learning Techniques[C]// Proceedings of the 2019 1st International Conference on Innovations in Information and Communication Technology. 2019: 1-9.
|
| [26] |
Tedmori S, Awajan A. Sentiment Analysis Main Tasks and Applications: A Survey[J]. Journal of Information Processing Systems, 2019, 15(3): 500-519.
|
| [27] |
Yu Y H, Lin H F, Meng J N, et al. Visual and Textual Sentiment Analysis of a Microblog Using Deep Convolutional Neural Networks[J]. Algorithms, 2016, 9(2): 41.
doi: 10.3390/a9020041
|
| [28] |
范涛, 王昊, 林克柔, 等. 基于视觉的网络舆情事件中网民情感分析研究[J]. 情报资料工作, 2022, 43(4): 83-91.
|
| [28] |
(Fan Tao, Wang Hao, Lin Kerou, et al. A Probein to Netizen Sentiment Analysis in Vision-based Internet Public Opinion Events[J]. Information and Documentation Services, 2022, 43(4): 83-91.)
|
| [29] |
黄兆培, 张峰源, 赵金明, 等. 情感识别中的迁移学习问题综述[J]. 信号处理, 2023, 39(4): 588-615.
|
| [29] |
(Huang Zhaopei, Zhang Fengyuan, Zhao Jinming, et al. A Survey of Transfer Learning Problems in Emotion Recognition[J]. Journal of Signal Processing, 2023, 39(4): 588-615.)
|
| [30] |
Yang J F, Sun M, Sun X X. Learning Visual Sentiment Distributions via Augmented Conditional Probability Neural Network[C]// Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017: 224-230.
|
| [31] |
Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large- Scale Image Recognition[OL]. arXiv Preprint, arXiv: 1409.1556.
|
| [32] |
Yanulevskaya V, Uijlings J, Bruni E, et al. In the Eye of the Beholder: Employing Statistical Analysis and Eye Tracking for Analyzing Abstract Paintings[C]// Proceedings of the 20th ACM International Conference on Multimedia. 2012: 349-358.
|
| [33] |
Yan Q M, Sun Y B, Fan S J, et al. Polarity-Aware Attention Network for Image Sentiment Analysis[J]. Multimedia Systems, 2023, 29(1): 389-399.
doi: 10.1007/s00530-022-00935-5
|
| [34] |
滕磊, 严馨, 徐广义, 等. 使用胶囊网络的细粒度情感分析方法[J]. 小型微型计算机系统, 2020, 41(12): 2550-2556.
|
| [34] |
(Teng Lei, Yan Xin, Xu Guangyi, et al. Fine-Grained Sentiment Analysis Based on Capsule Network[J]. Journal of Chinese Computer Systems, 2020, 41(12): 2550-2556.)
|
| [35] |
Farag H H, Said L A A, Rizk M R M, et al. Hyperparameters Optimization for ResNet and Xception in the Purpose of Diagnosing COVID-19[J]. Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology, 2021(2):41.
|
| [36] |
Katsurai M, Satoh S. Image Sentiment Analysis Using Latent Correlations among Visual, Textual, and Sentiment Views[C]//Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing. 2016: 2837-2841.
|
| [37] |
Woo S, Park J, Lee J Y, et al. CBAM: Convolutional Block Attention Module[C] // Proceedings of the European Conference on Computer Vision. 2018: 3-19.
|
| [38] |
He K M, Zhang X Y, Ren S Q, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
|
| [39] |
Meena G, Mohbey K K, Kumar S, et al. Image-Based Sentiment Analysis Using InceptionV3 Transfer Learning Approach[J]. SN Computer Science, 2023, 4: 242.
doi: 10.1007/s42979-023-01695-3
|
| [40] |
Liu P F, Chang S C, Huang X H, et al. Contextualized Non-local Neural Networks for Sequence Learning[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Intelligence Conference and 9th AAAI Symposium on Educational Advances in Artificial Intelligence. 2019: 6762-6769.
|
| [41] |
Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization[C]// Proceedings of 2017 IEEE International Conference on Computer Vision. 2017: 618-626.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
