Please wait a minute...
Data Analysis and Knowledge Discovery  2023, Vol. 7 Issue (1): 138-149    DOI: 10.11925/infotech.2096-3467.2022.0225
Current Issue | Archive | Adv Search |
Reasoning Model for Temporal Knowledge Graph Based on Entity Multiple Unit Coding
Peng Cheng,Zhang Chunxia(),Zhang Xin,Guo Jingtao,Niu Zhendong
School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China
Download: PDF (1568 KB)   HTML ( 22
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper tries to address the issues of incomplete entity information extraction and importance measurement of different timestamps for the events to be reasoned in temporal knowledge graph. [Methods] We proposed a new model based on entity multiple unit coding(EMUC). The EMUC introduces the entity slice feature encodings for the current timestamps, the entity dynamic feature encodings fusing timestamp embedding and entity static features, as well as entity segment feature encodings over historical steps. We also utilized a temporal attention mechanism to learn the importance weights of local structural information at different timestamps to the inference target. [Results] The experimental results of the proposed model on the ICEWS14 test set were MRR: 0.470 4, Hits@1: 40.31%, Hits@3: 50.02%, Hits@10: 59.98%, while on the ICEWS18 test set were MRR: 0.438 5, Hits@1: 37.55%, Hits@3: 46.92%, Hits@10: 56.85%, and on the YAGO test set are MRR: 0.656 4, Hits@1: 63.07%, Hits@3 : 65.87%, Hits@10: 68.37%. Our model outperforms the existing methods on these evaluating metrics. [Limitations] EMUC has slow inference speed for large-scale datasets. [Conclusions] The newly temporal attention mechanism measures the importance of historical local structure information for reasoning, which effectively improves the reasoning performance of the temporal knowledge graph.

Key wordsTemporal Knowledge Graph      Temporal Knowledge Graph Reasoning      Entity Multiple Unit Coding      Temporal Attention Mechanism      Knowledge Graph     
Received: 17 March 2022      Published: 16 February 2023
ZTFLH:  TP391  
Fund:National Key R&D Program of China(2020AAA0104903);National Natural Science Foundation of China(62072039)
Corresponding Authors: Zhang Chunxia,ORCID:0000-0003-0897-7986,E-mail: cxzhang@bit.edu.cn。   

Cite this article:

Peng Cheng, Zhang Chunxia, Zhang Xin, Guo Jingtao, Niu Zhendong. Reasoning Model for Temporal Knowledge Graph Based on Entity Multiple Unit Coding. Data Analysis and Knowledge Discovery, 2023, 7(1): 138-149.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022.0225     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2023/V7/I1/138

Temporal Knowledge Graph Reasoning Model Based on Entity Multiple Unit Coding
名称 实体数 关系数 训练集 验证集 测试集 间隔
ICEWS14 7 128 230 63 685 13 823 13 222 24 Hour
ICEWS18 23 033 256 373 018 45 995 49 545 24 Hour
YAGO 10 623 10 161 540 19 523 20 026 1 Year
Dataset Statistics
模型 MRR Hits@1/% Hits@3/% Hits@10/%
TTransE 0.130 0 2.78 16.78 34.28
TA-DistMult 0.264 7 17.09 30.22 45.41
DE-SimplE 0.326 7 24.43 35.69 49.11
TNTComplEx 0.321 2 23.35 36.03 49.13
CyGNet 0.327 3 23.69 36.31 50.67
RE-Net 0.382 8 28.68 41.34 54.52
xERTE 0.407 9 32.70 45.67 57.30
TLogic 0.430 4 33.56 48.27 61.23
EMUC w/o attention & w/o time-related 0.465 8 39.91 49.23 59.53
EMUC w/o attention & w/o slice 0.472 6 40.71 49.93 59.90
EMUC w/o time-related & w/o slice 0.463 3 39.59 49.03 59.42
EMUC w/o time-related 0.462 0 39.41 48.73 59.62
EMUC w/o attention 0.471 4 40.56 49.96 59.67
EMUC w/o dynamic 0.461 1 39.20 48.93 59.41
EMUC w/o fragment 0.461 5 39.29 48.78 59.34
EMUC w/o slice 0.467 1 39.98 49.35 59.66
EMUC 0.470 4 40.31 50.02 59.98
Results of Temporal Knowledge Graph Reasoning on Dataset ICEWS14
模型 MRR Hits@1/% Hits@3/% Hits@10/%
TTransE 0.084 3 2.00 8.71 22.06
TA-DistMult 0.167 5 8.61 18.41 33.59
DE-SimplE 0.193 0 11.53 21.86 34.80
TNTComplEx 0.212 3 13.28 24.02 36.91
CyGNet 0.249 3 15.90 28.28 42.61
RE-Net 0.288 1 19.05 32.44 47.51
xERTE 0.293 1 21.03 33.51 46.48
TLogic 0.298 2 20.54 33.95 48.53
TPath 0.421 9 37.61 45.74 50.82
EMUC w/o attention & w/o time-related 0.428 9 36.10 45.36 55.93
EMUC w/o attention & w/o slice 0.423 3 35.52 44.88 55.51
EMUC w/o slice & w/o time-related 0.428 3 36.04 45.44 55.89
EMUC w/o time-related 0.422 7 35.45 44.81 55.50
EMUC w/o attention 0.430 2 35.17 47.66 56.22
EMUC w/o dynamic 0.425 3 35.71 45.04 55.77
EMUC w/o fragment 0.422 4 35.35 44.90 55.39
EMUC w/o slice 0.429 3 36.25 45.49 55.92
EMUC 0.438 5 37.55 46.92 56.85
Results of Temporal Knowledge Graph Reasoning on Dataset ICEWS18
模型 MRR Hits@1/% Hits@3/% Hits@10/%
TTransE 0.321 1 19.04 42.00 52.21
TA-DistMult 0.617 2 63.32 65.19
EvolveRGCN 0.597 4 61.03 61.69
R-GCRN+MLP 0.538 9 56.06 61.19
RE-Net 0.651 6 65.63 68.08
EMUC w/o attention & w/o time-related 0.648 1 63.27 65.52 67.90
EMUC w/o attention & w/o slice 0.647 1 63.14 64.86 67.95
EMUC w/o slice & w/o time-related 0.642 5 62.51 64.61 67.24
EMUC w/o time-related 0.639 3 62.25 64.29 66.73
EMUC w/o attention 0.640 7 62.31 64.44 67.12
EMUC w/o dynamic 0.641 5 62.49 64.46 67.04
EMUC w/o fragment 0.643 0 62.62 64.68 67.30
EMUC w/o slice 0.650 0 63.31 65.43 67.87
EMUC 0.656 4 63.07 65.87 68.37
Results of Temporal Knowledge Graph Reasoning on Dataset YAGO
Parameter Experimental Results of EMUC on the ICEWS18 Dataset
[1] Bordes A, Usunier N, García-Durán A, et al. Translating Embeddings for Modeling Multi-Relational Data[C]// Proceedings of the 2013 Conference and Workshop on Neural Information Processing Systems. 2013: 2787-2795.
[2] Yang B S, Yih S W, He X D, et al. Learning Multi-Relational Semantics Using Neural-Embedding Models[C]// Proceedings of the 2014 Conference and Workshop on Neural Information Processing Systems. 2014.
[3] Trouillon T, Welbl J, Riedel S, et al. Complex Embeddings for Simple Link Prediction[C]// Proceedings of the 2016 International Conference on Machine Learning. 2016: 2071-2080.
[4] Schlichtkrull M, Kipf T N, Bloem P, et al. Modeling Relational Data with Graph Convolutional Networks[C]// Proceedings of the 2018 European Semantic Web Symposium. 2018: 593-607.
[5] Zhang W, Paudel B, Wang L, et al. Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning[C]// Proceedings of the 2019 World Wide Web Conference. 2019: 2366-2377.
[6] 阮小芸, 廖健斌, 李祥, 等. 基于人才知识图谱推理的强化学习可解释推荐研究[J]. 数据分析与知识发现, 2021, 5(6):36-50.
[6] ( Ruan Xiaoyun, Liao Jianbin, Li Xiang, et al. Interpretable Recommendation of Reinforcement Learning Based on Talent Knowledge Graph Reasoning[J]. Data Analysis and Knowledge Discovery, 2021, 5(6): 36-50.)
[7] 朱超宇, 刘雷. 基于知识图谱的医学决策支持应用综述[J]. 数据分析与知识发现, 2020, 4(12): 26-32.
[7] ( Zhu Chaoyu, Liu Lei. A Review of Medical Decision Supports Based on Knowledge Graph[J]. Data Analysis and Knowledge Discovery, 2020, 4(12): 26-32.)
[8] Leblay J, Chekol M W. Deriving Validity Time in Knowledge Graph[C]// Proceedings of the 2018 World Wide Web Conference. 2018: 1771-1776.
[9] García-Durán A, Dumančić S, Niepert M. Learning Sequence Encoders for Temporal Knowledge Graph Completion[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 4816-4821.
[10] Goel R, Kazemi S M, Brubaker M, et al. Diachronic Embedding for Temporal Knowledge Graph Completion[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020: 3988-3995.
[11] Lacroix T, Obozinski G, Usunier N. Tensor Decompositions for Temporal Knowledge Base Completion[C]// Proceedings of the 8th International Conference on Learning Representations. 2020.
[12] Jin W, Qu M, Jin X S, et al. Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020: 6669-6683.
[13] 陈浩, 李永强, 冯远静. 基于多关系循环事件的动态知识图谱推理[J]. 模式识别与人工智能, 2020, 33(4): 337-343.
doi: 10.16451/j.cnki.issn1003-6059.202004006
[13] ( Chen Hao, Li Yongqiang, Feng Yuanjing, Dynamic Knowledge Graph Inference Based on Multiple Relational Cyclic Events[J]. Pattern Recognition and Artificial Intelligence, 2020, 33(4): 337-343.)
doi: 10.16451/j.cnki.issn1003-6059.202004006
[14] Wu J P, Cao M, Cheung J C K, et al. TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020: 5730-5746.
[15] Han Z, Chen P, Ma Y, et al. Explainable Subgraph Reasoning for Forecasting on Temporal Knowledge Graphs[C]// Proceedings of the 2021 International Conference on Learning Representations. 2021.
[16] Li Z X, Jin X L, Li W, et al. Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning[C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021: 408-417.
[17] Liu Y S, Ma Y P, Hildebrandt M, et al. TLogic: Temporal Logical Rules for Explainable Link Forecasting on Temporal Knowledge Graphs[C]// Proceedings of the 36th AAAI Conference on Artificial Intelligence. 2022: 4120-4127.
[18] Boschee E, Lautenschlager J, O’Brien S, et al. ICEWS Coded Event Data[OL]. Harvard Dataverse, 2015. https://doi.org/10.7910/DVN/28075.
[19] Mahdisoltani F, Biega J A, Suchanek F M.YAGO3: A Knowledge Base from Multilingual Wikipedias[C]// Proceedings of the 2014 Conference on Innovative Data Systems Research. 2014.
[20] Zhu C C, Chen M H, Fan C J, et al. Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks[C]// Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021: 4732-4740.
[21] Bai L Y, Yu W T, Chen M Z, et al. Multi-Hop Reasoning over Paths in Temporal Knowledge Graphs Using Reinforcement Learning[J]. Applied Soft Computing, 2021, 103: 107144.
doi: 10.1016/j.asoc.2021.107144
[22] Pareja A, Domeniconi G, Chen J, et al. EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020.
[1] Liu Chunjiang, Li Shuying, Hu Hanlin, Fang Shu. Graph Databases for Complex Network Analysis[J]. 数据分析与知识发现, 2022, 6(7): 1-11.
[2] Zhang Han, An Xinyu, Liu Chunhe. Building Multi-Source Semantic Knowledge Graph for Drug Repositioning[J]. 数据分析与知识发现, 2022, 6(7): 87-98.
[3] Liu Kan, Xu Qinya, Yu Lu. Constructing Knowledge Graph for Business Environment[J]. 数据分析与知识发现, 2022, 6(4): 82-96.
[4] Zhang Wei, Wang Hao, Chen Yuetong, Fan Tao, Deng Sanhong. Identifying Metaphors and Association of Chinese Idioms with Transfer Learning and Text Augmentation[J]. 数据分析与知识发现, 2022, 6(2/3): 167-183.
[5] Liu Zhenghao, Qian Yuxing, Yi Tianlong, Lv Huakui. Constructing Knowledge Graph for Financial Securities and Discovering Related Stocks with Knowledge Association[J]. 数据分析与知识发现, 2022, 6(2/3): 184-201.
[6] Cheng Zijia, Chen Chong. Question Comprehension and Answer Organization for Scientific Education of Epidemics[J]. 数据分析与知识发现, 2022, 6(2/3): 202-211.
[7] Hou Dang, Fu Xiangling, Gao Songfeng, Peng Lei, Wang Youjun, Song Meiqi. Mining Enterprise Associations with Knowledge Graph[J]. 数据分析与知识发现, 2022, 6(2/3): 212-221.
[8] Li Zhijie, Wang Rui, Li Changhua, Zhang Jie. Embedding Knowledge Graph with Negative Sampling and Joint Relational Contexts[J]. 数据分析与知识发现, 2022, 6(12): 90-98.
[9] Hua Bin,Kang Yue,Fan Linhao. Knowledge Modeling and Association Q&A for Policy Texts[J]. 数据分析与知识发现, 2022, 6(11): 79-92.
[10] Zhou Yang,Li Xuejun,Wang Donglei,Chen Fang,Peng Lijuan. Visualizing Knowledge Graph for Explosive Formula Design[J]. 数据分析与知识发现, 2021, 5(9): 42-53.
[11] Shen Kejie, Huang Huanting, Hua Bolin. Constructing Knowledge Graph with Public Resumes[J]. 数据分析与知识发现, 2021, 5(7): 81-90.
[12] Ruan Xiaoyun,Liao Jianbin,Li Xiang,Yang Yang,Li Daifeng. Interpretable Recommendation of Reinforcement Learning Based on Talent Knowledge Graph Reasoning[J]. 数据分析与知识发现, 2021, 5(6): 36-50.
[13] Li He,Liu Jiayu,Li Shiyu,Wu Di,Jin Shuaiqi. Optimizing Automatic Question Answering System Based on Disease Knowledge Graph[J]. 数据分析与知识发现, 2021, 5(5): 115-126.
[14] Dai Bing,Hu Zhengyin. Review of Studies on Literature-Based Discovery[J]. 数据分析与知识发现, 2021, 5(4): 1-12.
[15] Zhu Dongliang, Wen Yi, Wan Zichen. Review of Recommendation Systems Based on Knowledge Graph[J]. 数据分析与知识发现, 2021, 5(12): 1-13.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn