[Objective] This paper proposes an adversarial neural network model combining the text and image data, aiming to improve the effectiveness of rumor detection. [Methods] First, we integrated the self-attention mechanism with the Bi-directional Long Short-Term Memory network (BiLSTM) model to represent the text features. Then, we used the pre-trained VGG19 network model to represent the image features. Finally, we used the adversarial neural network to study the events’ common features. [Results] It is superior to the existing baseline models in terms of accuracy, precision, recall and F1 scores. The accuracy on Weibo and Twitter data sets is 3.6% and 3.5%, higher than the best result compared with the baseline models respectively. [Limitations] More research is needed to examine the feature association between the modal information, and bridge the semantic gap of cross-modal data. [Conclusions] The proposed model could more effectively learn feature representation and detect rumors.
(Tan Ying, Zhang Jin, Xia Lixin. A Survey of Sentiment Analysis on Social Media[J]. Data Analysis and Knowledge Discovery, 2020, 4(1): 1-11.)
[2]
Castillo C, Mendoza M, Poblete B. Information Credibility on Twitter[C]// Proceedings of the 20th International Conference on World Wide Web. 2011: 675-684.
[3]
Wu K, Yang S, Zhu K Q. False Rumors Detection on Sina Weibo by Propagation Structures[C]// Proceedings of the 31st International Conference on Data Engineering. 2015: 651-662.
[4]
Rashkin H, Choi E, Jang J Y, et al. Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 2931-2937.
[5]
Gupta A, Lamba H, Kumaraguru P, et al. Faking Sandy: Characterizing and Identifying Fake Images on Twitter During Hurricane Sandy[C]// Proceedings of the 22nd International Conference on World Wide Web. 2013: 729-736.
[6]
Guo H, Cao J, Zhang Y Z, et al. Rumor Detection with Hierarchical Social Attention Network[C]// Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2018: 943-951.
(Jiang Yuxiao, Ding Shengchun, Wu Peng. A Study on the Classification of Features of Multi-Modal Information Based on BiLSTM-VGG16[J]. Information Studies: Theory & Application, 2021, 44(11): 180-186, 179.)
(Li Sha, Zhang Huaiwen, Qian Shengsheng, et al. Multi-Modal Multi-Level Event Network for Rumor Detection[J]. Journal of Image and Graphics, 2021, 26(7): 1648-1657.)
[9]
Jin Z W, Cao J, Guo H, et al. Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs[C]// Proceedings of the 25th ACM International Conference on Multimedia. 2017: 795-816.
(Song Zhijie, Wang Jian, Jia Jie. Research on Rumor Spreading in Emergency Based on SIHR1R2 Model[J]. Journal of Intelligence, 2016, 35(3):118-124, 136.)
[11]
Kwon S, Cha M, Jung K, et al. Prominent Features of Rumor Propagation in Online Social Media[C]// Proceedings of the 13th International Conference on Data Mining. 2013: 1103-1108.
[12]
Tong S, Koller D. Support Vector Machine Active Learning with Applications to Text Classification[J]. Journal of Machine Learning Research. 2002, 2:45-66.
[13]
Ma J, Gao W, Mitra P, et al. Detecting Rumors from Microblogs with Recurrent Neural Networks[C]// Proceedings of the 25th International Joint Conference on Artificial Intelligence. 2016: 3818-3824.
(Liu Zheng, Wei Zhihua, Zhang Renxian. Rumor Detection Based on Convolutional Neural Network[J]. Journal of Computer Applications, 2017, 37(11): 3053-3056.)
doi: 10.11772/j.issn.1001-9081.2017.11.3053
[15]
Chen T, Li X, Yin H Z, et al. Call Attention to Rumors: Deep Attention Based Recurrent Neural Networks for Early Rumor Detection[C]// Proceedings of Pacific-Asia Conference on Knowledge Discovery and Data Mining. 2018: 40-52.
[16]
Popat K, Mukherjee S, Yates A, et al. DeClarE: Debunking Fake News and False Claims Using Evidence-Aware Deep Learning[OL]. arXiv Preprint, arXiv: 1809.06416.
[17]
Wang Y Q, Ma F L, Jin Z W, et al. EANN: Event Adversarial Neural Networks for Multi-Modal Fake News Detection[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018: 849-857.
[18]
Yang X Y, Lyu Y F, Tian T, et al. Rumor Detection on Social Media with Graph Structured Adversarial Learning[C]// Proceedings of the 29th International Conference on Artificial Intelligence. 2021: 1417-1423.
[19]
Ni S W, Li J W, Kao H Y. Rumor Detection on Social Media with Hierarchical Adversarial Training[OL]. arXiv Preprint, arXiv: 2110.00425.
(Li Ao, Dan Zhiping, Dong Fangmin, et al. An Improved Generative Adversarial Network for Rumor Detection[J]. Journal of Chinese Information Processing, 2020, 34(9): 78-88.)
[21]
Ganin Y, Lempitsky V. Unsupervised Domain Adaptation by Backpropagation[C]// Proceedings of the 32nd International Conference on Machine Learning. 2015: 1180-1189.
[22]
Ganin Y, Ustinova E, Ajakan H, et al. Domain-Adversarial Training of Neural Networks[J]. The Journal of Machine Learning Research, 2016, 17(1): 2096-2030.
[23]
Maigrot C, Claveau V, Kijak E, et al. MediaEval 2016:A Multimodal System for the Verifying Multimedia Use Task[C]// Proceedings of the 2016 MediaEval Workshop. 2016.
[24]
Chen Y H. Convolutional Neural Network for Sentence Classification[D]. Waterloo, ON: University of Waterloo, 2015.
[25]
Schuster M, Paliwal K K. Bidirectional Recurrent Neural Networks[J]. IEEE Transactions on Signal Processing, 1997, 45(11): 2673-2681.
doi: 10.1109/78.650093
[26]
Antol S, Agrawal A, Lu J S, et al. VQA: Visual Question Answering[C]// Proceedings of 2015 IEEE International Conference on Computer Vision. 2015: 2425-2433.
[27]
刘金硕, 冯阔, Jeff Z. Pan, 等. MSRD: 多模态网络谣言检测方法[J]. 计算机研究与发展, 2020, 57(11): 2328-2336.
[27]
(Liu Jinshuo, Feng Kuo, Pan J Z, et al. MSRD: Multi-Modal Web Rumor Detection Method[J]. Journal of Computer Research and Development, 2020, 57(11): 2328-2336.)
