Research on the Semantic and Structure Fusion-Based Knowledge Graph Completion Model
Ma Zhiyuan1,Gao Ying2,Zhang Qiang2,Zhou Hong3,Li Bing4,Tao Wan1()
1School of Computer and Information, Anhui Polytechnic University, Wuhu 241000, China 2School of Information Management, Central China Normal University, Wuhan 430079, China 3National Science Library (WuHan), Chinese Academy of Science, Wuhan 430071, China 4Exchange, Development & Service Center for Science & Technology Talents, the Ministry of Science & Technology, Beijing 100045, China
[Objective] This paper proposes a knowledge graph completion model with semantic and structural information. It improves the completion, reliability, and quality of the knowledge graph. [Methods] First, we used a pre-trained language model to enhance the knowledge graph’s embedded text and context data. Then, we captured the semantic information of entities and relationships. Third, we constructed an entity-relationship matrix to map the network structure of the knowledge graph and obtain each entity’s neighborhood information and relationship constraints. Finally, we integrated the potential data to train the model and predict the missing entity of the knowledge graph. [Results] Compared to the baseline method, the proposed model’s Hits@3 metric improved by 0.5%, 0.6% and 0.6% on the FB15k-237, WN18RR and UMLS data sets, respectively. [Limitations] Due to the language models’ information representation ability limits, we cannot further improve the knowledge graph by completing tasks with the help of multimodal data. [Conclusions] The proposed method can perform better for the knowledge graph completion task, promoting the knowledge graph’s development and its downstream application.
马志远, 高颖, 张强, 周洪, 李兵, 陶皖. 融合语义与结构信息的知识图谱补全模型研究*[J]. 数据分析与知识发现, 2024, 8(4): 39-49.
Ma Zhiyuan, Gao Ying, Zhang Qiang, Zhou Hong, Li Bing, Tao Wan. Research on the Semantic and Structure Fusion-Based Knowledge Graph Completion Model. Data Analysis and Knowledge Discovery, 2024, 8(4): 39-49.
(Sun Tan, Liu Zheng, Cui Yunpeng, et al. Analysis and Design of a New Generation of Open Knowledge Service System Integrating Knowledge Organization and Cognitive Computing[J]. Journal of Library Science in China, 2019, 45(3): 38-48.)
(Zhao Wei, Zhang Lan, Wang Juncheng. The Construction and Realization of Knowledge Graph of Standard Literature in Financial Field[J]. Technology Intelligence Engineering, 2022, 8(6): 103-113.)
(Zheng Wei, Liu Chengyuan, Wu Xiaoxue, et al. Cross-Project Prediction Method of Security Bug Reports Based on Knowledge Graph[J]. Journal of Software, 2023: 1-23.(2023-07-06)[2023-07-27]. DOI:10.13328/j.cnki.jos.006812.
(Xu Xinran, Wang Tengyu, Lu Cai. Research Progress of Graph Neural Network in Knowledge Graph Construction and Application[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(10): 2278-2299.)
doi: 10.3778/j.issn.1673-9418.2302059
(Luo Ling, Yang Zhihao, Song Yawen, et al. Chinese Clinical Named Entity Recognition Based on Stroke ELMo and Multi-Task Learning[J]. Chinese Journal of Computers, 2020, 43(10): 1943-1957.)
(Liu Zhenghao, Zeng Xi, Zhang Zhijian. Research on Construction and Analysis of Financial Emergencies Knowledge Graph with Event Logical for Emergency Management[J]. Journal of Information Resources Management, 2022, 12(3): 137-151.)
doi: 10.13365/j.jirm.2022.03.137
(Zhang Jianhua, Liu Yilin, Guo Qidi, et al. Knowledge Supply and Demand Matching Based on Improved FCM and PFS[J]. Computer Engineering and Design, 2023, 44(1): 99-107.)
(Li He, Liu Jiayu, Li Shiyu, et al. Optimizing Automatic Question Answering System Based on Disease Knowledge Graph[J]. Data Analysis and Knowledge Discovery, 2021, 5(5): 115-126.)
(Ma Zili, Wang Shuying, Zhang Haizhu, et al. Joint Model of Intelligent Q&A Intent Recognition Based on Knowledge Graph[J]. Computer Engineering and Applications, 2023, 59(6): 171-178.)
doi: 10.3778/j.issn.1002-8331.2206-0419
(Liu Huan, Liu Liu, Wang Dongbo. Research on Automatic Question Answering of Domain Knowledge Graph from the Perspective of Digital Humanities[J]. Scientific Information Research, 2022, 4(1): 46-59.)
[14]
Bordes A, Usunier N, Garcia-Durán A, et al. Translating Embeddings for Modeling Multi-Relational Data[C]// Proceedings of the 27th Annual Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2013: 2787-2795.
[15]
Wang Z, Zhang J W, Feng J L, et al. Knowledge Graph Embedding by Translating on Hyperplanes[C]// Proceedings of the 28th AAAI Conference on Artificial Intelligence. Menlo Park: AAAI Press, 2014: 1112-1119.
[16]
Lin Y K, Liu Z Y, Sun M S, et al. Learning Entity and Relation Embeddings for Knowledge Graph Completion[C]// Proceedings of 29th the AAAI Conference on Artificial Intelligence. Menlo Park: AAAI Press, 2015: 2181-2187.
[17]
Ji G L, Liu K, He S Z, et al. Knowledge Graph Completion with Adaptive Sparse Transfer Matrix[C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence. Menlo Park: AAAI Press, 2016: 985-991.
[18]
Zhang W, Paudel B, Wang L, et al. Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning[C]// WWW'19:The World Wide Web Conference. New York: ACM, 2019: 2366-2377.
[19]
Sun Z, Deng Z H, Nie J Y, et al. Rotate: Knowledge Graph Embedding by Relational Rotation in Complex Space[OL]. arXiv Preprint, arXiv:1902.10197.
[20]
Nickel M, Tresp V, Kriegel H P. A Three-Way Model for Collective Learning on Multi-Relational Data[C]// Proceedings of the 28th International Conference on Machine Learning (ICML 2011). New York: ACM, 2011: 809-816.
[21]
Yang B, Yih S W, He X, et al. Embedding Entities and Relations for Learning and Inference in Knowledge Bases[C]// Proceedings of the 3rd International Conference on Learning Representations (ICLR). Washington: ICLR, 2015.
[22]
Nickel M, Rosasco L, Poggio T. Holographic Embeddings of Knowledge Graphs[C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence. Menlo Park: AAAI Press, 2016: 1955-1961.
[23]
Trouillon T, Welbl J, Riedel S, et al. Complex Embeddings for Simple Link Prediction[C]// Proceedings of the 33rd International Conference on Machine Learning. New York: ACM, 2016: 2071-2080.
[24]
Luo Y, Yang C M, Li B, et al. CP Tensor Factorization for Knowledge Graph Completion[C]// Proceedings of the 15th International Conference on Knowledge Science, Engineering and Management. Cham: Springer, 2022: 240-254.
[25]
Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need[C]// Proceedings of the 31st Conference on Neural Information Processing Systems. New York: ACM, 2017: 5998-6008.
[26]
Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding[C]// Proceedings of the 2019 Conference on the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2019: 4171-4186.
[27]
Liu Y, Ott M, Goyal N, et al. RoBERTa: A Robustly Optimized BERT Pretraining Approach[OL]. arXiv Preprint, arXiv:1907.11692.
[28]
Lan Z Z, Chen M D, Goodman S, et al. ALBERT: A Lite BERT for Self-Supervised Learning of Language Representations[C]// Proceedings of the 8th International Conference on Learning Representations. Washington: ICLR, 2020: 1-14.
[29]
Sun Y, Wang S H, Li Y K, et al. Ernie: Enhanced Representation Through Knowledge Integration[OL]. arXiv Preprint, arXiv:1904.09223.
[30]
Radford A, Wu J, Child R, et al. Language Models are Unsupervised Multitask Learners[J]. OpenAI Blog, 2019, 1(8): 9-33.
(Liu Yang, Ma Lili, Zhang Wen, et al. Detecting Sarcasm from Travel Reviews Based on Cross-Modal Deep Learning[J]. Data Analysis and Knowledge Discovery, 2022, 6(12): 23-31.)
(Bai Simeng, Niu Zhendong, He Hui, et al. Biomedical Text Classification Method Based on Hypergraph Attention Network[J]. Data Analysis and Knowledge Discovery, 2022, 6(11): 13-24.)
(Liu Yang, Zhang Wen, Hu Yi, et al. Hotel Stock Prediction Based on Multimodal Deep Learning[J]. Data Analysis and Knowledge Discovery, 2023, 7(5): 21-32.)
[34]
Balažević I, Allen C, Hospedales T M. Hypernetwork Knowledge Graph Embeddings[C]// Proceedings of the 28th International Conference on Artificial Neural Networks. Cham: Springer, 2019: 553-565.
[35]
Schlichtkrull M, Kipf T N, Bloem P, et al. Modeling Relational Data with Graph Convolutional Networks[C]// Proceedings of the 15th Extended Semantic Web Conference. Cham: Springer, 2018: 593-607.
[36]
Yao L, Mao C S, Luo Y. KG-BERT: BERT for Knowledge Graph Completion[J]. arXiv Preprint, arXiv:1909.03193.
[37]
Nathani D, Chauhan J, Sharma C, et al. Learning Attention-Based Embeddings for Relation Prediction in Knowledge Graphs[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 4710-4723.
[38]
Xie Z W, Zhou G Y, Liu J, et al. ReInceptionE: Relation-Aware Inception Network with Joint Local-Global Structural Information for Knowledge Graph Embedding[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 5929-5939.
[39]
Dettmers T, Minervini P, Stenetorp P, et al. Convolutional 2D Knowledge Graph Embeddings[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park: AAAI Press, 2018: 1811-1818.
[40]
Zeb A, Saif S, Chen J D, et al. Complex Graph Convolutional Network for Link Prediction in Knowledge Graphs[J]. Expert Systems with Applications, 2022, 200: 116796.
[41]
Vashishth S, Sanyal S, Nitin V, et al. Composition-Based Multi-Relational Graph Convolutional Networks[C]// Proceedings of the 8th International Conference on Learning Representations. Washington: ICLR, 2020: 1-15.
[42]
Shen J H, Wang C G, Gong L Y, et al. Joint Language Semantic and Structure Embedding for Knowledge Graph Completion[C]// Proceedings of the 29th International Conference on Computational Linguistics. New York: ACM, 2022: 1965-1978.
[43]
Bollacker K, Evans C, Paritosh P, et al. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge[C]// Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York: ACM, 2008: 1247-1250.
[44]
Toutanova K, Chen D Q. Observed Versus Latent Features for Knowledge Base and Text Inference[C]// Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality. Stroudsburg: Association for Computational Linguistics, 2015: 57-66.
[45]
Miller G A. WordNet: A Lexical Database for English[J]. Communications of the ACM, 1995, 38(11): 39-41.
[46]
Kok S, Domingos P. Statistical Predicate Invention[C]// Proceedings of the 24th International Conference on Machine Learning. New York: ACM, 2007: 433-440.
[47]
Loshchilov I, Hutter F. Decoupled Weight Decay Regularization[C]// Proceedings of the 7th European International Conference on Learning Representations. Washington: ICLR, 2017: 1-19.
