An Overview of Research on Knowledge Graph Completion Based on Graph Neural Network
Wu Yue1,Sun Haichun1,2()
1School of Information and Cyber Security, People’s Public Security University of China, Beijing 100038, China 2Key Laboratory of Security Technology & Risk Assessment, Beijing 100026, China
【目的】通过调研和梳理文献，总结基于图神经网络的知识图谱补全方法。【文献范围】以“Knowledge Graph Completion”、“知识图谱补全”作为检索词在Web of Science、DBLP和CNKI数据库中进行检索，共筛选出79篇文献。【方法】分别归纳总结图卷积神经网络、图注意力网络、图自动编码网络三种基于图神经网络的知识图谱补全方法类别，并对每种类别的技术脉络、典型方法、模型框架优缺点等进行对比论述。【结果】运用知识图谱补全任务的常用数据集和评价指标，从MRR、MR、Hit@k等性能评价角度对各类模型的效果进行对比分析，并对未来研究提出展望。【局限】在实验结果对比中，只讨论了FB15K-237和WN18RR数据集上部分应用较广的模型的评估结果，缺乏全部模型在同一数据集上的对比。【结论】相比基于表示学习模型和基于神经网络模型，基于图神经网络模型具有更好的图谱补全性能，但模型关系复杂性高、过平滑、可扩展性通用性差，这也是未来研究要解决的问题。
[Objective] This paper summarizes the knowledge graph completion methods based on graph neural network through research and literature review. [Coverage] With “knowledge graph completion” as search terms to retrieve literature from the Web of Science, DBLP and CNKI, a total of 79 representative literature were screened out for review. [Methods] Based on the model structure, three knowledge graph completion methods based on graph neural networks were summarized, including graph convolutional neural networks, graph attention networks, and graph auto encoder. [Results] Using common data sets and evaluation indicators for knowledge graph completion tasks, the effects of various models were comparatively analyzed in terms of MRR, MR, Hit@k and other performance evaluations, and prospects for future research were suggested. [Limitations] In the comparison of experimental results, only the evaluation results of some widely used models on the FB15K-237 and WN18RR datasets are discussed, the comparison of all models on the same dataset is lacking. [Conclusions] Compared with the representation learning model and the neural network model, the graph neural network model has better performance, but it still faces difficulties such as high complexity of model relationships, over-smoothness, and poor scalability and universality.
吴越, 孙海春. 基于图神经网络的知识图谱补全研究综述*[J]. 数据分析与知识发现, 2024, 8(3): 10-28.
Wu Yue, Sun Haichun. An Overview of Research on Knowledge Graph Completion Based on Graph Neural Network. Data Analysis and Knowledge Discovery, 2024, 8(3): 10-28.
Singhal A. Introducing the Knowledge Graph: Things, Not Strings[EB/OL]. [2023-07-02]. https://www.blog.google/products/search/introducing-knowledge-graph-things-not/.
[2]
Auer S, Bizer C, Kobilarov G, et al. DBpedia: A Nucleus for a Web of Open Data[C]// Proceedings of the 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference. 2007: 722-735.
[3]
Bollacker K, Evans C, Paritosh P, et al. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge[C]// Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. 2008: 1247-1250.
[4]
Miller G A. WordNet: A Lexical Database for English[J]. Communications of the ACM, 1995, 38(11): 39-41.
[5]
Noy N, Gao Y Q, Jain A, et al. Industry-Scale Knowledge Graphs: Lessons and Challenges[J]. Communications of the ACM, 2019, 62(8): 36-43.
doi: 10.1145/3331166
[6]
Shrivastava S. Bring Rich Knowledge of People, Places, Things and Local Businesses to Your Apps[EB/OL].[2023-07-02]. https://blogs.bing.com/search-quality-insights/2017-07/bring-rich-knowledge-of-people-places-things-and-local-businesses-to-your-apps.
(Yang Zhe, Xu Tian, Jin Zhe, et al. Construction and Application of Knowledge Graph of Sheep & Goat Disease[J]. Journal of Huazhong Agricultural University, 2023, 42(3): 63-70.)
(Zhao Bo, Wang Yujia, Ni Ji. E-TUP: A Joint Knowledge Graph Learning Recommendation Method Incorporating E-CP and TUP[J/OL]. Computer Engineering and Applications. https://kns.cnki.net/kcms/detail/11.2127.tp.20230321.0843.002.htm.)
[9]
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 6000-6010.
(Jin Dewu, Li Peng. Framework on Intelligent Decision Support System for Coal Seam Floor Water Hazard Prevention and Control[J]. Coal Geology & Exploration, 2021, 49(1): 161-169.)
[12]
Yuan J B, Jin Z W, Guo H, et al. Constructing Biomedical Domain-Specific Knowledge Graph with Minimum Supervision[J]. Knowledge and Information Systems, 2020, 62(1): 317-336.
doi: 10.1007/s10115-019-01351-4
(Zhang Zhijian, Ni Zhenni, Liu Zhenghao, et al. Predicting Dynamic Relationship for Financial Knowledge Graph[J]. Data Analysis and Knowledge Discovery, 2023, 7(9): 39-50.)
[14]
You X D, Yang M J, Han J M, et al. Attribute Extraction Study in the Field of Military Equipment Based on Distant Supervision[J]. Wireless Communications and Mobile Computing, 2021, 2021: 2549488.
(Chen Kean. Ontology Repository Construction Research of Anti-terrorism Knowledge Graph Based on Open Source Data[D]. Beijing: People’s Public Security University of China, 2022.)
[16]
Ji S X, Pan S R, Cambria E, et al. A Survey on Knowledge Graphs: Representation, Acquisition, and Applications[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(2): 494-514.
doi: 10.1109/TNNLS.2021.3070843
(Wu Guodong, Liu Hanwei, He Zhangwei, et al. Review of Knowledge Graph Completion Technology[J]. Journal of Chinese Computer Systems, 2023, 44(3): 471-482.)
[18]
Zamini M, Reza H, Rabiei M. A Review of Knowledge Graph Completion[J]. Information, 2022, 13(8): 396.
doi: 10.3390/info13080396
(Yang Dawei, Zhou Gang, Lu Jicang, et al. Review of Knowledge Graph Completion Based on Knowledge Representation Learning[J]. Journal of Information Engineering University, 2021, 22(5): 558-565.)
[20]
Shen T, Zhang F, Cheng J W. A Comprehensive Overview of Knowledge Graph Completion[J]. Knowledge-Based Systems, 2022, 255: 109597.
doi: 10.1016/j.knosys.2022.109597
[21]
Arora S. A Survey on Graph Neural Networks for Knowledge Graph Completion[OL]. arXiv Preprint, arXiv: 2007.12374.
[22]
Li Z F, Zhao Y, Zhang Y, et al. Multi-relational Graph Attention Networks for Knowledge Graph Completion[J]. Knowledge-Based Systems, 2022, 251: 109262.
doi: 10.1016/j.knosys.2022.109262
[23]
Dai G Q, Wang X Z, Zou X Y, et al. MRGAT: Multi-Relational Graph Attention Network for Knowledge Graph Completion[J]. Neural Networks, 2022, 154: 234-245.
doi: 10.1016/j.neunet.2022.07.014
pmid: 35908373
[24]
Yin H, Zhong J, Wang C, et al. GS-InGAT: An Interaction Graph Attention Network with Global Semantic for Knowledge Graph Completion[J]. Expert Systems with Applications, 2023, 228: 120380.
doi: 10.1016/j.eswa.2023.120380
[25]
Xu H C, Bao J P, Liu W B. Double-Branch Multi-Attention Based Graph Neural Network for Knowledge Graph Completion[C]// Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers). 2023: 15257-15271.
[26]
Hogan A, Blomqvist E, Cochez M, et al. Knowledge Graphs[J]. ACM Computing Surveys, 2021, 54(4): 71.
[27]
Neelakantan A, Chang M W. Inferring Missing Entity Type Instances for Knowledge Base Completion: New Dataset and Methods[C]// Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2015: 515-525.
[28]
Schlichtkrull M, Kipf T N, Bloem P, et al. Modeling Relational Data with Graph Convolutional Networks[C]// Proceedings of European Semantic Web Conference 2018:The Semantic Web. 2018: 593-607.
[29]
Bordes A, Usunier N, Garcia-Durán A, et al. Translating Embeddings for Modeling Multi-relational Data[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems-Volume 2. 2013: 2787-2795.
[30]
Cai H Y, Zheng W C, Chang C C. A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications[J]. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(9): 1616-1637.
doi: 10.1109/TKDE.69
[31]
Pan S R, Luo L H, Wang Y F, et al. Unifying Large Language Models and Knowledge Graphs: A Roadmap[J]. IEEE Transactions on Knowledge and Data Engineering, DOI: 10.1109/TKDE.2024.3352100.
[32]
Lovelace J, Rosé C. A Framework for Adapting Pre-trained Language Models to Knowledge Graph Completion[C]// Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022: 5937-5955.
[33]
Wang Z, Zhang J W, Feng J L, et al. Knowledge Graph Embedding by Translating on Hyperplanes[C]// Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014: 1112-1119.
[34]
Lin Y K, Liu Z Y, Sun M S, et al. Learning Entity and Relation Embeddings for Knowledge Graph Completion[C]// Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015: 2181-2187.
[35]
Sun Z Q, Deng Z H, Nie J Y, et al. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space[OL]. arXiv Preprint, arXiv: 1902.10197.
[36]
Nickel M, Tresp V, Kriegel H P. A Three-Way Model for Collective Learning on Multi-Relational Data[C] //Proceedings of the 28th International Conference on Machine Learning. 2011: 809-816.
[37]
Yang B S, Yih W T, He X D, et al. Embedding Entities and Relations for Learning and Inference in Knowledge Bases[OL]. arXiv Preprint, arXiv: 1412.6575.
[38]
Trouillon T, Welbl J, Riedel S, et al. Complex Embeddings for Simple Link Prediction[C]// Proceedings of the 33rd International Conference on Machine Learning - Volume 48. 2016: 2071-2080.
[39]
Balažević I, Allen C, Hospedales T. TuckER: Tensor Factorization for Knowledge Graph Completion[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019: 5185-5194.
[40]
Dettmers T, Minervini P, Stenetorp P, et al. Convolutional 2D Knowledge Graph Embeddings[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 1811-1818.
[41]
Vashishth S, Sanyal S, Nitin V, et al. InteractE: Improving Convolution-Based Knowledge Graph Embeddings by Increasing Feature Interactions[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020: 3009-3016.
[42]
Nguyen D Q, Nguyen D Q, Nguyen T D, et al. A Convolutional Neural Network-Based Model for Knowledge Base Completion and Its Application to Search Personalization[J]. Semantic Web, 2019, 10(5): 947-960.
doi: 10.3233/SW-180318
[43]
Guo L B, Sun Z Q, Hu W. Learning to Exploit Long-Term Relational Dependencies in Knowledge Graphs[C]// Proceedings of the 36th International Conference on Machine Learning. 2019: 2505-2514.
[44]
Xiong W H, Hoang T, Wang W Y. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 564-573.
[45]
Yao L, Mao C S, Luo Y. KG-BERT: BERT for Knowledge Graph Completion[OL]. arXiv Preprint, arXiv: 1909.03193.
[46]
Tian A Q, Zhang C H, Rang M, et al. RA-GCN: Relational Aggregation Graph Convolutional Network for Knowledge Graph Completion[C]// Proceedings of the 12th International Conference on Machine Learning and Computing. ACM, 2020: 580-586.
[47]
Shang C, Tang Y, Huang J, et al. End-to-End Structure-Aware Convolutional Networks for Knowledge Base Completion[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019: 3060-3067.
(Luo Meiqiu, Zhang Chunxia, Peng Cheng, et al. Knowledge Graph Completion Based on Parsing Graph Embedding and a Weighted Graph Convolutional Network[J]. SCIENTIA SINICA Informationis, 2022, 52(11): 2037-2057.)
doi: 10.1360/SSI-2021-0217
[49]
Cai L, Yan B, Mai G C, et al. TransGCN: Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction[C]// Proceedings of the 10th International Conference on Knowledge Capture. 2019: 131-138.
[50]
Ye R, Li X, Fang Y J, et al. A Vectorized Relational Graph Convolutional Network for Multi-relational Network Alignment[C]// Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019: 4135-4141.
Yu D H, Yang Y M, Zhang R H, et al. Knowledge Embedding Based Graph Convolutional Network[C]// Proceedings of the Web Conference 2021. 2021: 1619-1628.
[53]
He H T, Niu H R, Feng J Z, et al. An Embedding Model for Knowledge Graph Completion Based on Graph Sub-Hop Convolutional Network[J]. Big Data Research, 2022, 30: 100351.
doi: 10.1016/j.bdr.2022.100351
[54]
Nathani D, Chauhan J, Sharma C, et al. Learning Attention-Based Embeddings for Relation Prediction in Knowledge Graphs[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 4710-4723.
(Lu Yiqin, Pan Zhoushuang, Zhang Yang, et al. Knowledge Graph Completion Method Based on Interactively Connected Graph Attention Network[J]. Journal of South China University of Technology (Natural Science Edition), 2022, 50(12): 13-19.)
doi: 10.12141/j.issn.1000-565X.220384
[56]
Zhang Z, Zhuang F Z, Zhu H S, et al. Relational Graph Neural Network with Hierarchical Attention for Knowledge Graph Completion[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020: 9612-9619.
[57]
Ji KX, Hui B, Luo G C. Graph Attention Networks with Local Structure Awareness for Knowledge Graph Completion[J]. IEEE Access, 2020, 8: 224860-224870.
doi: 10.1109/ACCESS.2020.3044343
[58]
Ma J T, Li D Y, Zhu H D, et al. GAFM: A Knowledge Graph Completion Method Based on Graph Attention Faded Mechanism[J]. Information Processing & Management, 2022, 59(5): 103004.
doi: 10.1016/j.ipm.2022.103004
[59]
Wu J K, Shi W T, Cao X Z, et al. DisenKGAT: Knowledge Graph Embedding with Disentangled Graph Attention Network[C]// Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021: 2140-2149.
[60]
Zhang X, Zhang C X, Guo J T, et al. Graph Attention Network with Dynamic Representation of Relations for Knowledge Graph Completion[J]. Expert Systems with Applications, 2023, 219: 119616.
doi: 10.1016/j.eswa.2023.119616
(Liu Danyang, Fang Quan, Zhang Xiaowei, et al. Knowledge Graph Completion Based on Graph Contrastive Attention Network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(8): 1428-1435.)
[62]
Liang S, Zhu A J, Zhang J S, et al. Hyper-node Relational Graph Attention Network for Multi-modal Knowledge Graph Completion[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2023, 19(2): Article No. 62.
[63]
Kifp T N, Welling M. Variational Graph Auto-Encoders[OL]. arXiv Preprint, arXiv: 1611.07308.
[64]
Han Y F, Fang Q, Hu J, et al. GAEAT: Graph Auto-Encoder Attention Networks for Knowledge Graph Completion[C]// Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020: 2053-2056.
[65]
Zhang S Y, Zhang Z, Zhuang F Z, et al. Compressing Knowledge Graph Embedding with Relational Graph Auto-Encoder[C]// Proceedings of 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication. 2020: 366-370.
[66]
Wang S, Wei X K, dos Santos C N N, et al. Mixed-Curvature Multi-relational Graph Neural Network for Knowledge Graph Completion[C]// Proceedings of the Web Conference 2021. 2021: 1761-1771.
[67]
Salehi A, Davulcu H. Graph Attention Auto-Encoders[C]// Proceedings of 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence. 2020: 989-996.
[68]
Liu W, Wang P J, Zhang Z H, et al. Multi-scale Convolutional Neural Network for Temporal Knowledge Graph Completion[J]. Cognitive Computation, 2023, 15(3): 1016-1022.
doi: 10.1007/s12559-023-10134-7
[69]
Kipf T N, Welling M. Semi-supervised Classification with Graph Convolutional Networks[OL]. arXiv Preprint, arXiv: 1609.02907.
[70]
Veličković P, Cucurull G, Casanova A, et al. Graph Attention Networks[C]// Proceedings of the International Conference on Learning Representations (ICLR). 2018.
(Sun Shuifa, Li Xiaolong, Li Weisheng, et al. Review of Graph Neural Networks Applied to Knowledge Graph Reasoning[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(1): 27-52.)
doi: 10.3778/j.issn.1673-9418.2207060
[72]
Salha G, Limnios S, Hennequin R, et al. Gravity-Inspired Graph Autoencoders for Directed Link Prediction[C]// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019: 589-598.
[73]
Jang E, Gu S X, Poole B. Categorical Reparameterization with Gumbel-Softmax[OL]. arXiv Preprint, arXiv: 1611.01144.
[74]
Suchanek F, Kasneci G M, Weikum G M. Yago: A Core of Semantic Knowledge Unifying WordNet and Wikipedia[C]// Proceedings of the 16th International Conference on World Wide Web. 2007.
[75]
Ristoski P, de Vries G K D, Paulheim H. A Collection of Benchmark Datasets for Systematic Evaluations of Machine Learning on the Semantic Web[C]// Proceedings of International Semantic Web Conference. 2016: 186-194.
(Wang Chunlei, Wang Xiao, Liu Kai. Multimodal Knowledge Graph Representation Learning: A Review[J]. Journal of Computer Applications, 2024, 44(1): 1-15.)
doi: 10.11772/j.issn.1001-9081.2023050583
[77]
Wang X Z, Gao T Y, Zhu Z C, et al. KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation[J]. Transactions of the Association for Computational Linguistics, 2021, 9: 176-194.
doi: 10.1162/tacl_a_00360
[78]
Yan S R, Lin K J, Zheng X L, et al. LkeRec: Toward Lightweight End-to-End Joint Representation Learning for Building Accurate and Effective Recommendation[J]. ACM Transactions on Information Systems, 2021, 40(3): Article No. 54.
(Hou Jinglin, Qiu Runhe, Xue Jiai, et al. Elevator Failure Prediction Based on Embedding and Completion of Knowledge Graph[J]. Computer Engineering and Design, 2022, 43(1): 224-230.)
