|
|
|
| Detecting Social Media Rumors Based on Multimodal Heterogeneous Graph |
Qiang Zishan,Gu Yijun( ) |
| College of Information and Cyber Security, People’s Public Security University of China, Beijing 100038, China |
|
|
|
|
Abstract [Objective] This paper proposes a social media rumor detection model based on the multimodal heterogeneous graph, aiming to verify the correlation between different rumor modalities and improve the accuracy of rumor detection. [Methods] First, we retrieved multimodal posts from social platforms. Then, we extracted feature representations of texts, pictures, and user attributes through preprocessing. Third, we constructed a heterogeneous graph based on the correlation between texts, pictures, and users. Fourth, we extracted the embeddings of text-type nodes according to their specified meta path. Finally, we input the embedding into the classifier to determine whether or not it is a rumor. [Results] We examined the proposed model with two open data sets. The accuracy of our model reached 91.3% and 93.8%, which were also higher than the baseline models. [Limitations] The three types of nodes from the sharing multimodal rumors will make the heterogeneous graph sparse. The proposed model is more suitable for small topic communities. [Conclusions] There is a correlation between different modalities of rumors, which helps the proposed model effectively detect multimodal rumors.
|
|
Received: 28 August 2022
Published: 22 March 2023
|
|
|
| Fund:Ministry of Public Security Science and Technology to Strengthen the Basic Work of the Police Project(2020GABJC02);Fundamental Research Funds for the Central Universities(2022JKF02039) |
|
Corresponding Authors:
Gu Yijun，E-mail：guyijun@ppsuc.edu.cn。
|
| [1] |
Castillo C, Mendoza M, Poblete B. Information Credibility on Twitter[C]// Proceedings of the 20th International Conference on World Wide Web. ACM, 2011: 675-684.
|
| [2] |
Wu K, Yang S, Zhu K Q. False Rumors Detection on Sina Weibo by Propagation Structures[C]// Proceedings of 2015 IEEE 31st International Conference on Data Engineering. 2015: 651-662.
|
| [3] |
Qazvinian V, Rosengren E, Radev D R, et al. Rumor Has It: Identifying Misinformation in Microblogs[C]// Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. 2011: 1589-1599.
|
| [4] |
祖坤琳, 赵铭伟, 郭凯, 等. 新浪微博谣言检测研究[J]. 中文信息学报, 2017, 31(3): 198-204.
|
| [4] |
(Zu Kunlin, Zhao Mingwei, Guo Kai, et al. Research on the Detection of Rumor on Sina Weibo[J]. Journal of Chinese Information Processing, 2017, 31(3): 198-204.)
|
| [5] |
李文政, 顾益军, 闫红丽. 基于网络贝叶斯信息准则算法的社区数量预测研究[J]. 数据分析与知识发现, 2020, 4(4): 72-82.
|
| [5] |
(Li Wenzheng, Gu Yijun, Yan Hongli. Predicting Community Numbers with Network Bayesian Information Criterion[J]. Data Analysis and Knowledge Discovery, 2020, 4(4): 72-82.)
|
| [6] |
王本钰, 顾益军, 彭舒凡. 基于粒子竞争机制的半监督社区发现算法[J]. 计算机科学与探索, 2023, 17(3): 608-619.
doi: 10.3778/j.issn.1673-9418.2106040
|
| [6] |
(Wang Benyu, Gu Yijun, Peng Shufan. Semi-supervised Community Detection Algorithm Based on Particle Competition[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(3): 608-619.)
doi: 10.3778/j.issn.1673-9418.2106040
|
| [7] |
Ma J, Gao W, Mitra P, et al. Detecting Rumors from Microblogs with Recurrent Neural Networks[C]// Proceedings of the 25th International Joint Conference on Artificial Intelligence. 2016: 3818-3824.
|
| [8] |
刘政, 卫志华, 张韧弦. 基于卷积神经网络的谣言检测[J]. 计算机应用, 2017, 37(11): 3053-3056, 3100.
doi: 10.11772/j.issn.1001-9081.2017.11.3053
|
| [8] |
(Liu Zheng, Wei Zhihua, Zhang Renxian. Rumor Detection Based on Convolutional Neural Network[J]. Journal of Computer Applications, 2017, 37(11): 3053-3056, 3100.)
doi: 10.11772/j.issn.1001-9081.2017.11.3053
|
| [9] |
Ma J, Gao W, Wong K F. Rumor Detection on Twitter with Tree-Structured Recursive Neural Networks[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers). 2018: 1980-1989.
|
| [10] |
Bian T, Xiao X, Xu T Y, et al. Rumor Detection on Social Media with Bi-directional Graph Convolutional Networks[C]// Proceedings of the AAAI Conference on Artificial Intelligence. 2020: 549-556.
|
| [11] |
王雷全. 基于图模型的多模态社会媒体分析[D]. 北京: 北京邮电大学, 2016.
|
| [11] |
(Wang Leiquan. Analysis of Multi-modal Social Media Based on Graph Model[D]. Beijing: Beijing University of Posts and Telecommunications, 2016.)
|
| [12] |
吴友政, 李浩然, 姚霆, 等. 多模态信息处理前沿综述:应用、融合和预训练[J]. 中文信息学报, 2022, 36(5): 1-20.
|
| [12] |
(Wu Youzheng, Li Haoran, Yao Ting, et al. A Survey of Multimodal Information Processing Frontiers: Application, Fusion and Pre-training[J]. Journal of Chinese Information Processing, 2022, 36(5): 1-20.)
|
| [13] |
Ngiam J, Khosla A, Kim M, et al. Multimodal Deep Learning[C]// Proceedings of the 28th International Conference on International Conference on Machine Learning. 2011: 689-696.
|
| [14] |
Lu J S, Yang J W, Batra D, et al. Hierarchical Question-Image Co-attention for Visual Question Answering[C]// Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016: 289-297.
|
| [15] |
Nam H, Ha J W, Kim J. Dual Attention Networks for Multimodal Reasoning and Matching[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017: 2156-2164.
|
| [16] |
Wöllmer M, Metallinou A, Eyben F, et al. Context-Sensitive Multimodal Emotion Recognition from Speech and Facial Expression Using Bidirectional LSTM Modeling[C]// Proceedings of the 11th Annual Conference of the International Speech Communication Association. 2010: 2362-2365.
|
| [17] |
Hu J W, Liu Y C, Zhao J M, et al. MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1:Long Papers). 2021: 5666-5675.
|
| [18] |
Jin Z W, Cao J, Guo H, et al. Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs[C]// Proceedings of the 25th ACM International Conference on Multimedia. 2017: 795-816.
|
| [19] |
刘金硕, 冯阔, Jeff Z. Pan, 等. MSRD: 多模态网络谣言检测方法[J]. 计算机研究与发展, 2020, 57(11): 2328-2336.
|
| [19] |
(Liu Jinshuo, Feng Kuo, Jeff Z. Pan, et al. MSRD: Multi-modal Web Rumor Detection Method[J]. Journal of Computer Research and Development, 2020, 57(11): 2328-2336.)
|
| [20] |
陈志毅, 隋杰. 基于DeepFM和卷积神经网络的集成式多模态谣言检测方法[J]. 计算机科学, 2022, 49(1): 101-107.
doi: 10.11896/jsjkx.201200007
|
| [20] |
(Chen Zhiyi, Sui Jie. DeepFM and Convolutional Neural Networks Ensembles for Multimodal Rumor Detection[J]. Computer Science, 2022, 49(1): 101-107.)
doi: 10.11896/jsjkx.201200007
|
| [21] |
孟佳娜, 王晓培, 李婷, 等. 基于对抗神经网络的跨模态谣言检测[J]. 数据分析与知识发现, 2022, 6(12): 32-42.
|
| [21] |
(Meng Jiana, Wang Xiaopei, Li Ting, et al. Cross-Modal Rumor Detection Based on Adversarial Neural Networks[J]. Data Analysis and Knowledge Discovery, 2022, 6(12): 32-42.)
|
| [22] |
戚力鑫, 万书振, 唐斌, 等. 基于注意力机制的多模态融合谣言检测方法[J]. 计算机工程与应用, 2022, 58(19): 209-217.
doi: 10.3778/j.issn.1002-8331.2102-0229
|
| [22] |
(Qi Lixin, Wan Shuzhen, Tang Bin, et al. Multimodal Fusion Rumor Detection Method Based on Attention Mechanism[J]. Computer Engineering and Applications, 2022, 58(19): 209-217.)
doi: 10.3778/j.issn.1002-8331.2102-0229
|
| [23] |
张少钦, 杜圣东, 张晓博, 等. 融合多模态信息的社交网络谣言检测方法[J]. 计算机科学, 2021, 48(5): 117-123.
doi: 10.11896/jsjkx.200400057
|
| [23] |
(Zhang Shaoqin, Du Shengdong, Zhang Xiaobo, et al. Social Rumor Detection Method Based on Multimodal Fusion[J]. Computer Science, 2021, 48(5): 117-123.)
doi: 10.11896/jsjkx.200400057
|
| [24] |
唐樾, 马静. 基于增强对抗网络和多模态融合的谣言检测方法[J]. 情报科学, 2022, 40(6): 108-114.
|
| [24] |
(Tang Yue, Ma Jing. A Rumor Detection Method Based on Enhance Adversarial Network and Multimodal Fusion[J]. Information Science, 2022, 40(6): 108-114.)
|
| [25] |
陶霄, 朱焱, 李春平. 基于注意力与多模态混合融合的谣言检测方法[J]. 计算机工程, 2021, 47(12): 71-77.
doi: 10.19678/j.issn.1000-3428.0059683
|
| [25] |
(Tao Xiao, Zhu Yan, Li Chunping. Rumor Detection Method Based on Attention and Multi-modal Hybrid Fusion[J]. Computer Engineering, 2021, 47(12): 71-77.)
doi: 10.19678/j.issn.1000-3428.0059683
|
| [26] |
何俊, 张彩庆, 李小珍, 等. 面向深度学习的多模态融合技术研究综述[J]. 计算机工程, 2020, 46(5): 1-11.
doi: 10.19678/j.issn.1000-3428.0057370
|
| [26] |
(He Jun, Zhang Caiqing, Li Xiaozhen, et al. Survey of Research on Multimodal Fusion Technology for Deep Learning[J]. Computer Engineering, 2020, 46(5): 1-11.)
doi: 10.19678/j.issn.1000-3428.0057370
|
| [27] |
Wang X, Ji H Y, Shi C, et al. Heterogeneous Graph Attention Network[C]// Proceedings of WWW’19. 2019: 2022-2032.
|
| [28] |
Zhang C X, Song D J, Huang C, et al. Heterogeneous Graph Neural Network[C]// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019: 793-803.
|
| [29] |
Fu X Y, Zhang J N, Meng Z Q, et al. MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding[C]// Proceedings of the WWW 2020. 2020: 2331-2341.
|
| [30] |
李昊杰. 异质图神经网络研究及其在推荐系统中的应用[D]. 成都: 电子科技大学, 2022.
|
| [30] |
(Li Haojie. Research on Heterogeneous Graph Neural Network and Its Application in Recommendation System[D]. Chengdu: University of Electronic Science and Technology of China, 2022.)
|
| [31] |
李良训. 基于异质图嵌入的Android恶意软件检测的研究与实现[D]. 北京: 北京邮电大学, 2021.
|
| [31] |
(Li Liangxun. Research and Implementation of Heterogeneous Graph Embedding for Android Malware Detection[D]. Beijing: Beijing University of Posts and Telecommunications, 2021.)
|
| [32] |
Huang Q, Yu J S, Wu J, et al. Heterogeneous Graph Attention Networks for Early Detection of Rumors on Twitter[C]// Proceedings of 2020 International Joint Conference on Neural Networks. 2020: 1-8.
|
| [33] |
毕蓓, 潘慧瑶, 陈峰, 等. 基于异构图注意力网络的微博谣言监测模型[J]. 计算机应用, 2021, 41(12): 3546-3550.
doi: 10.11772/j.issn.1001-9081.2021060981
|
| [33] |
(Bi Bei, Pan Huiyao, Chen Feng, et al. Microblog Rumor Detection Model Based on Heterogeneous Graph Attention Network[J]. Journal of Computer Applications, 2021, 41(12): 3546-3550.)
doi: 10.11772/j.issn.1001-9081.2021060981
|
| [34] |
Sun Z, Deng Z H, Nie J Y, et al. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space[OL]. arXiv Preprint, arXiv:1902.10197.
|
| [35] |
Boididou C, Andreadou K, Papadopoulos S, et al. Verifying Multimedia Use at MediaEval 2015[C]// Proceedings of MediaEval 2015 Workshop. 2015.
|
| [36] |
Maigrot C, Claveau V, Kijak E, et al. MediaEval 2016: A Multimodal System for the Verifying Multimedia Use Task[C]// Proceedings of MediaEval 2016 Workshop. 2016.
|
| [37] |
王壮, 隋杰. 基于多级融合的多模态谣言检测模型[J]. 计算机工程与设计, 2022, 43(6): 1756-1761.
|
| [37] |
(Wang Zhuang, Sui Jie. Multimodal Rumor Detection Model Based on Multilevel Fusion[J]. Computer Engineering and Design, 2022, 43(6): 1756-1761.)
|
| [38] |
Godbole S, Sarawagi S. Discriminative Methods for Multi-labeled Classification[C]// Proceedings of Pacific-Asia Conference on Knowledge Discovery and Data Mining. 2004: 22-30.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
