Few-Shot Knowledge Graph Completion Combined with Type-Aware Attention
Pu Xianghe,Wang Hongbin(),Xian Yantuan
Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China Key Laboratory of Artificial Intelligence in Yunnan Province, Kunming University of Science and Technology, Kunming 650500, China Yunnan Key Laboratory of Computer Technology Application, Kunming University of Science and Technology, Kunming 650500, China
[Objective] The existing few-shot knowledge graph completion methods could not distinguish the importance of neighbors when dealing with complex relations, which resulting in low performance of entity prediction. The few-shot knowledge graph completion methods could be improved by considering entity neighbor information. [Methods] First, we use a type-aware neighbor coder to learn the implicit type information in the entity neighbor such that the type-aware attention can be obtained and the entity representation can be enhanced. Then, a Transformer encoder is used to capture different meanings of a task relation. Finally, the reference set representation is obtained by jointly matching the prototype network aggregation and predicting the entity. [Results] The proposed method is verified on NELL and Wiki datasets through entity prediction tasks. The results show that the MRR is 1.6% and 1.2% higher than the baseline methods on the two datasets, respectively. [Limitations] Neighbors with lower physical relevance were not filtered, and the noise would affect the distribution of type-aware attention weights. [Conclusions] Experimental results show that the proposed method improves the few-shot knowledge graph completion performance by learning more abundant entity neighbor information.
(Li Juanzi, Hou Lei. Reviews on Knowledge Graph Research[J]. Journal of Shanxi University(Natural Science Edition), 2017, 40(3): 454-459.)
[2]
Bollacker K, Evans C, Paritosh P, et al. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge[C]// Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. 2008: 1247-1250.
[3]
Lehmann J, Isele R, Jakob M, et al. DBpedia - A Large-Scale, Multilingual Knowledge Base Extracted from Wikipedia[J]. Semantic Web, 2015, 6(2): 167-195.
doi: 10.3233/SW-140134
[4]
Carlson A, Betteridge J, Kisiel B, et al. Toward an Architecture for Never-Ending Language Learning[C]// Proceedings of the 24th AAAI Conference on Artificial Intelligence. 2010: 1306-1313.
[5]
Vrandečić D, Krötzsch M. Wikidata: A Free Collaborative Knowledgebase[J]. Communications of the ACM, 2014, 57(10): 78-85.
[6]
Xiong W H, Yu M, Chang S Y, et al. One-Shot Relational Learning for Knowledge Graphs[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 1980-1990.
[7]
Chen M Y, Zhang W, Zhang W, et al. Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019: 4216-4225.
[8]
Zhang C X, Yao H X, Huang C, et al. Few-Shot Knowledge Graph Completion[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020: 3041-3048.
[9]
Sheng J W, Guo S, Chen Z Y, et al. Adaptive Attentional Network for Few-Shot Knowledge Graph Completion[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020: 1681-1691.
[10]
Wang F, Xie Y Q, Zhang K, et al. Bert-Based Knowledge Graph Completion Algorithm for Few-Shot[C]// Proceedings of the 2nd International Conference on Big Data Economy and Information Management. 2021: 217-224.
[11]
王桢. 基于嵌入模型的知识图谱补全[D]. 广州: 中山大学, 2017.
[11]
(Wang Zhen. Complement of Knowledge Map Based on Embedded Model[D]. Guangzhou: Sun Yat-Sen University, 2017.)
[12]
Bordes A, Usunier N, Garcia-Durán A, et al. Translating Embeddings for Modeling Multi-Relational Data[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2. 2013: 2787-2795.
[13]
Wang Z, Zhang J W, Feng J L, et al. Knowledge Graph Embedding by Translating on Hyperplanes[C]// Proceedings of the 28th AAAI Conference on Artificial Intelligence and the 26th Innovative Applications of Artificial Intelligence Conference. 2014: 1112-1119.
[14]
Lin Y K, Liu Z Y, Sun M S, et al. Learning Entity and Relation Embeddings for Knowledge Graph Completion[C]// Proceedings of the 29th AAAI Conference on Artificial Intelligence and the 27th Innovative Applications of Artificial Intelligence Conference. 2015: 2181-2187.
[15]
Ji G L, He S Z, Xu L H, et al, Knowledge Graph Embedding via Dynamic Mapping Matrix[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Languages Processing(Volume 1:Long Papers). 2015: 687-696.
[16]
Sun Z Q, Deng Z H, Nie J Y, et al. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space[OL]. arXiv Preprint, arXiv:1902.10197.
[17]
Zhang Z Q, Cai J Y, Zhang Y D, et al. Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020: 3065-3072.
[18]
Kazemi S M, Poole D. SimplE Embedding for Link Prediction in Knowledge Graphs[C]// Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018: 4289-4300.
[19]
Ji S X, Pan S R, Cambria E, et al. A Survey on Knowledge Graphs: Representation, Acquisition, and Applications[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(2): 494-514.
doi: 10.1109/TNNLS.2021.3070843
[20]
Nickel M, Tresp V, Kriegel H P. A Three-Way Model for Collective Learning on Multi-Relational Data[C]// Proceedings of the 28th International Conference on Machine Learning. 2011: 809-816.
[21]
Yang B, Yih W T, He X, et al. Embedding Entities and Relations for Learning and Inference in Knowledge Bases[OL]. arXiv Preprint, arXiv:1412.6575.
[22]
Nickel M, Rosasco L, Poggio T. Holographic Embeddings of Knowledge Graphs[C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence and the 28th Innovative Applications of Artificial Intelligence Conference. 2016: 1955-1961.
[23]
Trouillon T, Welbl J, Riedel S, et al. Complex Embeddings for Simple Link Prediction[C]// Proceedings of the 33rd International Conference on Machine Learning- Volume 48. 2016: 2071-2080.
[24]
Dettmers T, Minervini P, Stenetorp P, et al. Convolutional 2D Knowledge Graph Embeddings[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 1811-1818.
[25]
Cai L, Yan B, Mai G C, et al. TransGCN: Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction[C]// Proceedings of the 10th International Conference on Knowledge Capture. 2019: 131-138.
[26]
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 6000-6010.
[27]
Wang Q, Huang P, Wang H, et al. CoKE: Contextualized Knowledge Graph Embedding[OL]. arXiv Preprint, arXiv:1911.02168.
[28]
Snell J, Swersky K, Zemel R S. Prototypical Networks for Few-Shot Learning[OL]. arXiv Preprint, arXiv:1703.05175.
[29]
Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[OL]. arXiv Preprint, arXiv:1412.6980.
