Please wait a minute...
Data Analysis and Knowledge Discovery  2023, Vol. 7 Issue (9): 51-63    DOI: 10.11925/infotech.2096-3467.2022.0799
Current Issue | Archive | Adv Search |
Few-Shot Knowledge Graph Completion Combined with Type-Aware Attention
Pu Xianghe,Wang Hongbin(),Xian Yantuan
Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
Key Laboratory of Artificial Intelligence in Yunnan Province, Kunming University of Science and Technology, Kunming 650500, China
Yunnan Key Laboratory of Computer Technology Application, Kunming University of Science and Technology, Kunming 650500, China
Download: PDF (1516 KB)   HTML ( 16
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] The existing few-shot knowledge graph completion methods could not distinguish the importance of neighbors when dealing with complex relations, which resulting in low performance of entity prediction. The few-shot knowledge graph completion methods could be improved by considering entity neighbor information. [Methods] First, we use a type-aware neighbor coder to learn the implicit type information in the entity neighbor such that the type-aware attention can be obtained and the entity representation can be enhanced. Then, a Transformer encoder is used to capture different meanings of a task relation. Finally, the reference set representation is obtained by jointly matching the prototype network aggregation and predicting the entity. [Results] The proposed method is verified on NELL and Wiki datasets through entity prediction tasks. The results show that the MRR is 1.6% and 1.2% higher than the baseline methods on the two datasets, respectively. [Limitations] Neighbors with lower physical relevance were not filtered, and the noise would affect the distribution of type-aware attention weights. [Conclusions] Experimental results show that the proposed method improves the few-shot knowledge graph completion performance by learning more abundant entity neighbor information.

Key wordsFew-Shot Knowledge Graph Completion      Entity Prediction      Type-Aware Attention     
Received: 01 August 2022      Published: 24 October 2023
ZTFLH:  TP391  
  G350  
Fund:The National Key Research and Development Plans Project of Yunnan Province(202202AD080003);The National Natural Science Foundation of China(61966020)
Corresponding Authors: Wang Hongbin,ORCID:0000-0003-2176-2998,E-mail:whbin2007@126.com。   

Cite this article:

Pu Xianghe, Wang Hongbin, Xian Yantuan. Few-Shot Knowledge Graph Completion Combined with Type-Aware Attention. Data Analysis and Knowledge Discovery, 2023, 7(9): 51-63.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022.0799     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2023/V7/I9/51

Entity and the Neighbors
m 实体数(NELL) 实体数(Wiki)
2 10 173 111 012
3 3 671 21 233
4 1 886 5 879
10+ 2 204 14 047
Scale of Complex Entity in NELL and Wiki
数据集 实体数 关系数 三元组数 任务数
NELL 68 545 358 181 109 67
Wiki 4 838 244 822 5 859 240 183
Datasets Size
Overall Framework of the Method
方法 MRR Hits@10 Hits@5 Hits@1
TransE 0.174 0.313 0.231 0.101
DistMult 0.200 0.311 0.251 0.137
ComplEx 0.184 0.297 0.229 0.118
SimplE 0.158 0.285 0.226 0.097
RotatE 0.176 0.329 0.247 0.101
GMatching (MaxP) 0.176 0.294 0.233 0.113
GMatching (MeanP) 0.141 0.272 0.201 0.080
GMatching(Max) 0.147 0.244 0.197 0.090
FSRL 0.153 0.319 0.212 0.073
MetaR 0.209 0.355 0.280 0.141
FAAN 0.279 0.428 0.364 0.200
B-GMatching 0.198 0.276 0.280 0.134
本文方法(Multi-head 1) 0.295 0.430 0.372 0.221
本文方法(Multi-head 2) 0.296 0.426 0.362 0.227
本文方法(Multi-head 3) 0.291 0.418 0.362 0.218
Experimental Results of Different Methods on NELL When K = 5
方法 MRR Hits@10 Hits@5 Hits@1
TransE 0.133 0.187 0.157 0.100
DistMult 0.071 0.151 0.099 0.024
ComplEx 0.080 0.181 0.122 0.032
SimplE 0.093 0.180 0.128 0.043
RotatE 0.049 0.090 0.064 0.026
GMatching (MaxP) 0.263 0.387 0.337 0.197
GMatching (MeanP) 0.254 0.374 0.314 0.193
GMatching(Max) 0.245 0.372 0.295 0.185
FSRL 0.158 0.287 0.206 0.097
MetaR 0.323 0.418 0.385 0.270
FAAN 0.309/
0.341
0.451/
0.463
0.380/
0.395
0.239/
0.281
B-GMatching 0.220 0.336 0.275 0.172
本文方法(Multi-head 1) 0.321 0.468 0.389 0.251
本文方法(Multi-head 2) 0.322 0.463 0.397 0.251
本文方法(Multi-head 3) 0.321 0.451 0.386 0.257
Experimental Results of Different Methods on Wiki When K = 5
Experimental Results of the Effects of Few-Shot Size
关系 方法 Hits@10 Hits@5 Hits@1 MRR
1 本文方法 1.000 0.971 0.971 0.976
FAAN 1.000 0.971 0.971 0.974
2 本文方法 0.179 0.104 0.029 0.077
FAAN 0.168 0.133 0.058 0.101
3 本文方法 0.781 0.781 0.234 0.467
FAAN 0.766 0.719 0.406 0.533
4 本文方法 0.007 0.007 0.007 0.009
FAAN 0.014 0.014 0.007 0.012
5 本文方法 0.341 0.307 0.157 0.231
FAAN 0.346 0.286 0.152 0.220
6 本文方法 0.617 0.527 0.346 0.442
FAAN 0.602 0.505 0.220 0.352
7 本文方法 0.745 0.694 0.531 0.602
FAAN 0.724 0.643 0.480 0.558
8 本文方法 0.296 0.163 0.059 0.135
FAAN 0.281 0.163 0.022 0.112
9 本文方法 0.857 0.813 0.407 0.572
FAAN 0.835 0.758 0.505 0.615
10 本文方法 0.128 0.074 0.033 0.061
FAAN 0.134 0.092 0.042 0.073
11 本文方法 0.659 0.601 0.322 0.435
FAAN 0.649 0.596 0.303 0.432
Experimental Results of Different Relations
消融实验 MRR Hits@10 Hits@5 Hits@1
本文方法(- a 0.289 0.423 0.371 0.215
本文方法(-EP) 0.289 0.427 0.373 0.214
本文方法(-RP) 0.224 0.330 0.270 0.165
本文方法(-TAA) 0.278 0.420 0.351 0.208
本文方法(FAAN+) 0.289 0.426 0.368 0.218
FAAN 0.279 0.428 0.364 0.200
本文方法 0.295 0.430 0.372 0.221
Ablation Experiment Results
Visualization of NELL Dataset(The Number of Nodes is 1,000)
Attention Distribution
[1] 李涓子, 侯磊. 知识图谱研究综述[J]. 山西大学学报(自然科学版), 2017, 40(3): 454-459.
[1] (Li Juanzi, Hou Lei. Reviews on Knowledge Graph Research[J]. Journal of Shanxi University(Natural Science Edition), 2017, 40(3): 454-459.)
[2] Bollacker K, Evans C, Paritosh P, et al. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge[C]// Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. 2008: 1247-1250.
[3] Lehmann J, Isele R, Jakob M, et al. DBpedia - A Large-Scale, Multilingual Knowledge Base Extracted from Wikipedia[J]. Semantic Web, 2015, 6(2): 167-195.
doi: 10.3233/SW-140134
[4] Carlson A, Betteridge J, Kisiel B, et al. Toward an Architecture for Never-Ending Language Learning[C]// Proceedings of the 24th AAAI Conference on Artificial Intelligence. 2010: 1306-1313.
[5] Vrandečić D, Krötzsch M. Wikidata: A Free Collaborative Knowledgebase[J]. Communications of the ACM, 2014, 57(10): 78-85.
[6] Xiong W H, Yu M, Chang S Y, et al. One-Shot Relational Learning for Knowledge Graphs[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 1980-1990.
[7] Chen M Y, Zhang W, Zhang W, et al. Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019: 4216-4225.
[8] Zhang C X, Yao H X, Huang C, et al. Few-Shot Knowledge Graph Completion[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020: 3041-3048.
[9] Sheng J W, Guo S, Chen Z Y, et al. Adaptive Attentional Network for Few-Shot Knowledge Graph Completion[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020: 1681-1691.
[10] Wang F, Xie Y Q, Zhang K, et al. Bert-Based Knowledge Graph Completion Algorithm for Few-Shot[C]// Proceedings of the 2nd International Conference on Big Data Economy and Information Management. 2021: 217-224.
[11] 王桢. 基于嵌入模型的知识图谱补全[D]. 广州: 中山大学, 2017.
[11] (Wang Zhen. Complement of Knowledge Map Based on Embedded Model[D]. Guangzhou: Sun Yat-Sen University, 2017.)
[12] Bordes A, Usunier N, Garcia-Durán A, et al. Translating Embeddings for Modeling Multi-Relational Data[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2. 2013: 2787-2795.
[13] Wang Z, Zhang J W, Feng J L, et al. Knowledge Graph Embedding by Translating on Hyperplanes[C]// Proceedings of the 28th AAAI Conference on Artificial Intelligence and the 26th Innovative Applications of Artificial Intelligence Conference. 2014: 1112-1119.
[14] Lin Y K, Liu Z Y, Sun M S, et al. Learning Entity and Relation Embeddings for Knowledge Graph Completion[C]// Proceedings of the 29th AAAI Conference on Artificial Intelligence and the 27th Innovative Applications of Artificial Intelligence Conference. 2015: 2181-2187.
[15] Ji G L, He S Z, Xu L H, et al, Knowledge Graph Embedding via Dynamic Mapping Matrix[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Languages Processing(Volume 1:Long Papers). 2015: 687-696.
[16] Sun Z Q, Deng Z H, Nie J Y, et al. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space[OL]. arXiv Preprint, arXiv:1902.10197.
[17] Zhang Z Q, Cai J Y, Zhang Y D, et al. Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020: 3065-3072.
[18] Kazemi S M, Poole D. SimplE Embedding for Link Prediction in Knowledge Graphs[C]// Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018: 4289-4300.
[19] Ji S X, Pan S R, Cambria E, et al. A Survey on Knowledge Graphs: Representation, Acquisition, and Applications[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(2): 494-514.
doi: 10.1109/TNNLS.2021.3070843
[20] Nickel M, Tresp V, Kriegel H P. A Three-Way Model for Collective Learning on Multi-Relational Data[C]// Proceedings of the 28th International Conference on Machine Learning. 2011: 809-816.
[21] Yang B, Yih W T, He X, et al. Embedding Entities and Relations for Learning and Inference in Knowledge Bases[OL]. arXiv Preprint, arXiv:1412.6575.
[22] Nickel M, Rosasco L, Poggio T. Holographic Embeddings of Knowledge Graphs[C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence and the 28th Innovative Applications of Artificial Intelligence Conference. 2016: 1955-1961.
[23] Trouillon T, Welbl J, Riedel S, et al. Complex Embeddings for Simple Link Prediction[C]// Proceedings of the 33rd International Conference on Machine Learning- Volume 48. 2016: 2071-2080.
[24] Dettmers T, Minervini P, Stenetorp P, et al. Convolutional 2D Knowledge Graph Embeddings[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 1811-1818.
[25] Cai L, Yan B, Mai G C, et al. TransGCN: Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction[C]// Proceedings of the 10th International Conference on Knowledge Capture. 2019: 131-138.
[26] Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 6000-6010.
[27] Wang Q, Huang P, Wang H, et al. CoKE: Contextualized Knowledge Graph Embedding[OL]. arXiv Preprint, arXiv:1911.02168.
[28] Snell J, Swersky K, Zemel R S. Prototypical Networks for Few-Shot Learning[OL]. arXiv Preprint, arXiv:1703.05175.
[29] Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[OL]. arXiv Preprint, arXiv:1412.6980.
[1] He Li,Yang Meihua,Liu Luyao. Detecting Events with SPO Semantic and Syntactic Information[J]. 数据分析与知识发现, 2023, 7(9): 114-124.
[2] Zhai Dongsheng, Lou Ying, Kan Huimin, He Xijun, Liang Guoqiang, Ma Zifei. Constructing TCM Knowledge Graph with Multi-Source Heterogeneous Data[J]. 数据分析与知识发现, 2023, 7(9): 146-158.
[3] Han Pu, Gu Liang, Ye Dongyu, Chen Wenqi. Recognizing Chinese Medical Literature Entities Based on Multi-Task and Transfer Learning[J]. 数据分析与知识发现, 2023, 7(9): 136-145.
[4] He Chaocheng, Huang Qian, Li Xinru, Wang Chunying, Wu Jiang. Trending Topics on Metaverse: A Microblog Text Analysis with BERT and DTM[J]. 数据分析与知识发现, 2023, 7(9): 25-38.
[5] Bao Tong, Zhang Chengzhi. Extracting Chinese Information with ChatGPT:An Empirical Study by Three Typical Tasks[J]. 数据分析与知识发现, 2023, 7(9): 1-11.
[6] Zhang Yingyi, Zhang Chengzhi, Zhou Yi, Chen Bikun. ChatGPT-Based Scientific Paper Entity Recognition: Performance Measurement and Availability Research[J]. 数据分析与知识发现, 2023, 7(9): 12-24.
[7] Xu Chen, Zhang Wei. Detecting Crowdfunding Frauds Based on Textual and Imbalanced Data[J]. 数据分析与知识发现, 2023, 7(9): 125-135.
[8] Xiang Zhuoyuan, Chen Hao, Wang Qian, Li Na. Few-Shot Language Understanding Model for Task-Oriented Dialogues[J]. 数据分析与知识发现, 2023, 7(9): 64-77.
[9] Qian Cong, Qi Jianglei, Ding Hao. Online Publication Recommendation Based on Weighted Features of User Multiple Interest Drift[J]. 数据分析与知识发现, 2023, 7(8): 119-127.
[10] Nie Hui, Cai Ruisheng. Online Doctor Recommendation System with Attention Mechanism[J]. 数据分析与知识发现, 2023, 7(8): 138-148.
[11] Li Yang, Zhao Jichang. CEO Facial Expression Analysis Based on Neural Networks and Its Impacts on Media Attention at Press Conferences[J]. 数据分析与知识发现, 2023, 7(8): 46-61.
[12] Liu Yang, Ding Xingchen, Ma Lili, Wang Chunyang, Zhu Lifang. Usefulness Detection of Travel Reviews Based on Multi-dimensional Graph Convolutional Networks[J]. 数据分析与知识发现, 2023, 7(8): 95-104.
[13] Liu Zhenghao, Zhang Zhijian, Chen Shuaipu, Zeng Xi. Modelling and Representation of Risk Event Evolution in Financial Field[J]. 数据分析与知识发现, 2023, 7(8): 78-94.
[14] Di Luyang, Zhong Han, Shi Shuicai. Review of Early Warning for Online Public Opinion[J]. 数据分析与知识发现, 2023, 7(8): 17-29.
[15] Chen Liu, Guo Yuhong. Literature Recommendation Algorithm Integrating High-Order Similarity of Motif Structure[J]. 数据分析与知识发现, 2023, 7(7): 146-155.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn