Please wait a minute...
Data Analysis and Knowledge Discovery  2022, Vol. 6 Issue (12): 90-98    DOI: 10.11925/infotech.2096-3467.2022.0214
Current Issue | Archive | Adv Search |
Embedding Knowledge Graph with Negative Sampling and Joint Relational Contexts
Li Zhijie(),Wang Rui,Li Changhua,Zhang Jie
School of Information and Control Engineering, Xi’an University of Architectural Science and Technology, Xi’an 710055, China
Download: PDF (1063 KB)   HTML ( 9
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper proposes a knowledge graph model based on negative sampling and joint relational contexts, aiming to improve the quality of current translation-based knowledge graph embedding models. [Methods] Firstly, we extracted the neighbors of the target instances from the original knowledge graph to generate the context vector. Then, we decided the properties of adjacent relations, which also provided information on the nature or type of a given entity. Third, we used the Concat function to aggregate contexts of the given entities of negative sampling and determined the entity attributes to be replaced. Finally, we adopted the triple embedding of the TransE model to generate negative triples, and improved the similarities of positive and negative triples. [Results] We examined the proposed model with data sets of FB15K-237 and WN18RR. The entity link was 18.3% and 29.2% higher than those of the benchmark model. Meantime, the relationship link was 0.7% better than the optimal result of the benchmark model. [Limitations] Our model only included the semantics of the relational contexts, which is very hard to determine their relative positions. [Conclusions] The proposed sampling strategy effectively improves the quality of negative triples, as well as the accuracy of knowledge graph.

Key wordsKnowledge Graph      Negative Sampling Strategy      Entity Link      Relation Link     
Received: 14 March 2022      Published: 03 February 2023
ZTFLH:  TP391  
Fund:National Natural Science Foundation of China(51878536);National Natural Science Foundation of Shaanxi Province(2020JQ-687);Science and Technology Plan Project of Shaanxi Province Housing and Urban-Rural Construction Department(2020-K09)
Corresponding Authors: Li Zhijie,ORCID:0000-0003-4362-5652     E-mail: lizhijie@xauat.edu.cn

Cite this article:

Li Zhijie, Wang Rui, Li Changhua, Zhang Jie. Embedding Knowledge Graph with Negative Sampling and Joint Relational Contexts. Data Analysis and Knowledge Discovery, 2022, 6(12): 90-98.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022.0214     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2022/V6/I12/90

Adjacency Relationship Provides Attributes of a Given Entity
Relationship Subgraph
rcTransE Architecture
Negative Sampling Process
数据集 关系数量 实体数量 训练集 测试集 验证集
FB15K-237 237 14 541 272 115 20 466 17 535
WN18RR 11 40 943 86 835 3 134 3 034
Information of Dataset
数据集 上下文跳数m 学习率λ 嵌入维度n
FB15K-237 2 0.001 200
WN18RR 3 0.005 200
Optimal Parameter Value
实验
模型
FB15K-237 WN18RR
MRR Hits@1 Hits@3 MRR Hits@1 Hits@3
TransE 0.966 0.946 0.984
0.927
0.841
0.844
0.936
0.835
0.988
0.784 0.669 0.870
TransD 0.845 0.851 0.811 0.863 0.923
TransH 0.824 0.813 0.630 0.602 0.753
TransR 0.837 0.826 0.685 0.663 0.726
DistMult 0.875 0.806 0.847 0.787 0.891
PTransE 0.795 0.817 0.632 0.693 0.743
rcTransE 0.978 0.961 0.849 0.894 0.993
Evaluation Results of Entity Prediction
Performance of the Experimental Dataset on Different Aggregate Functions
Performance of Different Relational Context Hop Counts on Dataset
实验模型 FB15K-237 WN18RR
Hits@1 Hits@1
TransE 84.3

33.3
93.6
94.0
87.5
66.1
TransR 87.6
RTtransE 28.9
PTransE(2-step) 67.3
PTransE(3-step) 34.6
rcTransE 88.3
Evaluation Results of Relation Prediction
The Effect of Negative Sampling Method on the Model
[1] Amit S. Introducing the Knowledge Graph[R]. America: Official Blog of Google, 2012.
[2] Bollacker K, Cook R, Tufts P. Freebase: A Shared Database of Structured General Human Knowledge[C]// Proceedings of the 22nd AAAI Conference on Artificial Intelligence. 2007: 1962-1963.
[3] WMF. Wikidata[EB/OL]. [2019-11-11]. https://www.wikidata.org/wiki/Wikidata:Main_Page.
[4] Suchanek F M, Kasneci G, Weikum G. YAGO: A Large Ontology from Wikipedia and WordNet[J]. Journal of Web Semantics, 2008, 6(3): 203-217.
doi: 10.1016/j.websem.2008.06.001
[5] Bizer C, Lehmann J, Kobilarov G, et al. DBpedia - A Crystallization Point for the Web of Data[J]. Journal of Web Semantics, 2009, 7(3): 154-165.
doi: 10.1016/j.websem.2009.07.002
[6] Li M D, Sun Z Y, Zhang S H, et al. Enhancing Knowledge Graph Embedding with Relational Constraints[J]. Neurocomputing, 2021, 429: 77-88.
doi: 10.1016/j.neucom.2020.12.012
[7] Li Z F, Liu H, Zhang Z L, et al. Recalibration Convolutional Networks for Learning Interaction Knowledge Graph Embedding[J]. Neurocomputing, 2021, 427: 118-130.
doi: 10.1016/j.neucom.2020.07.137
[8] Gong F, Wang M, Wang H F, et al. SMR: Medical Knowledge Graph Embedding for Safe Medicine Recommendation[J]. Big Data Research, 2021, 23: 100174.
doi: 10.1016/j.bdr.2020.100174
[9] 徐增林, 盛泳潘, 贺丽荣, 等. 知识图谱技术综述[J]. 电子科技大学学报, 2016, 45(4): 589-606.
[9] (Xu Zenglin, Sheng Yongpan, He Lirong, et al. Review on Knowledge Graph Techniques[J]. Journal of University of Electronic Science and Technology of China, 2016, 45(4): 589-606.)
[10] 舒世泰, 李松, 郝晓红, 等. 知识图谱嵌入技术研究进展[J]. 计算机科学与探索, 2021, 15(11): 2048-2062.
doi: 10.3778/j.issn.1673-9418.2103086
[10] (Shu Shitai, Li Song, Hao Xiaohong, et al. Knowledge Graph Embedding Technology: A Review[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(11): 2048-2062.)
doi: 10.3778/j.issn.1673-9418.2103086
[11] Bengio Y, Senecal J S. Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model[J]. IEEE Transactions on Neural Networks, 2008, 19(4): 713-722.
doi: 10.1109/TNN.2007.912312 pmid: 18390314
[12] Zhen Y, Ming D, Chang Z, et al. Understanding Negative Sampling in Graph Representation Learning[C]// Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2020: 1666-1676.
[13] Wang Q, Mao Z D, Wang B, et al. Knowledge Graph Embedding: A Survey of Approaches and Applications[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(12): 2724-2743.
doi: 10.1109/TKDE.2017.2754499
[14] Bordes A, Usunier N, Garcia-Duran A, et al. Translating Embeddings for Modeling Multi-Relational Data[C]// Proceedings of the 27th Annual Conference on Neural Information Processing Systems. 2013: 2787-2795.
[15] Socher R, Chen D, Manning C D, et al. Reasoning with Neural Tensor Networks for Knowledge Base Completion[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 26: 926-934.
[16] Wang Z, Zhang J W, Feng J L, et al. Knowledge Graph Embedding by Translating on Hyperplanes[C]// Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014: 1112-1119.
[17] Wang P, Li S, Pan R. Incorporating GAN for Negative Sampling in Knowledge Representation Learning[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 2005-2012.
[18] Sun Z, Deng Z H, Nie J Y, et al. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space[OL]. arXiv Preprint, arXiv: 1902.10197.
[19] 郭智, 郑彦斌, 夏志超, 等. 融合属性信息的知识表示方法[J]. 科学技术与工程, 2019, 19(33): 259-265.
[19] (Guo Zhi, Zheng Yanbin, Xia Zhichao, et al. Knowledge Representation Learning Method with Attribute Information[J]. Science Technology and Engineering, 2019, 19(33): 259-265.)
[20] Duchi J, Hazan E, Singer Y. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization[J]. Journal of Machine Learning Research, 2011, 12(7): 257-269.
[21] Toutanova K, Chen D Q. Observed Versus Latent Features for Knowledge Base and Text Inference[C]// Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality. 2015: 57-66.
[22] Dettmers T, Minervini P, Stenetorp P, et al. Convolutional 2D Knowledge Graph Embeddings[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 1811-1818.
[23] Lin Y K, Liu Z Y, Luan H B, et al. Modeling Relation Paths for Representation Learning of Knowledge Bases[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015: 705-714.
[24] Garcia-Duran A, Bordes A, Usunier N. Composing Relationships with Translations[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015: 286-290.
[1] Liu Chunjiang, Li Shuying, Hu Hanlin, Fang Shu. Graph Databases for Complex Network Analysis[J]. 数据分析与知识发现, 2022, 6(7): 1-11.
[2] Zhang Han, An Xinyu, Liu Chunhe. Building Multi-Source Semantic Knowledge Graph for Drug Repositioning[J]. 数据分析与知识发现, 2022, 6(7): 87-98.
[3] Liu Kan, Xu Qinya, Yu Lu. Constructing Knowledge Graph for Business Environment[J]. 数据分析与知识发现, 2022, 6(4): 82-96.
[4] Zhang Wei, Wang Hao, Chen Yuetong, Fan Tao, Deng Sanhong. Identifying Metaphors and Association of Chinese Idioms with Transfer Learning and Text Augmentation[J]. 数据分析与知识发现, 2022, 6(2/3): 167-183.
[5] Liu Zhenghao, Qian Yuxing, Yi Tianlong, Lv Huakui. Constructing Knowledge Graph for Financial Securities and Discovering Related Stocks with Knowledge Association[J]. 数据分析与知识发现, 2022, 6(2/3): 184-201.
[6] Cheng Zijia, Chen Chong. Question Comprehension and Answer Organization for Scientific Education of Epidemics[J]. 数据分析与知识发现, 2022, 6(2/3): 202-211.
[7] Hou Dang, Fu Xiangling, Gao Songfeng, Peng Lei, Wang Youjun, Song Meiqi. Mining Enterprise Associations with Knowledge Graph[J]. 数据分析与知识发现, 2022, 6(2/3): 212-221.
[8] Hua Bin,Kang Yue,Fan Linhao. Knowledge Modeling and Association Q&A for Policy Texts[J]. 数据分析与知识发现, 2022, 6(11): 79-92.
[9] Zhou Yang,Li Xuejun,Wang Donglei,Chen Fang,Peng Lijuan. Visualizing Knowledge Graph for Explosive Formula Design[J]. 数据分析与知识发现, 2021, 5(9): 42-53.
[10] Shen Kejie, Huang Huanting, Hua Bolin. Constructing Knowledge Graph with Public Resumes[J]. 数据分析与知识发现, 2021, 5(7): 81-90.
[11] Ruan Xiaoyun,Liao Jianbin,Li Xiang,Yang Yang,Li Daifeng. Interpretable Recommendation of Reinforcement Learning Based on Talent Knowledge Graph Reasoning[J]. 数据分析与知识发现, 2021, 5(6): 36-50.
[12] Li He,Liu Jiayu,Li Shiyu,Wu Di,Jin Shuaiqi. Optimizing Automatic Question Answering System Based on Disease Knowledge Graph[J]. 数据分析与知识发现, 2021, 5(5): 115-126.
[13] Dai Bing,Hu Zhengyin. Review of Studies on Literature-Based Discovery[J]. 数据分析与知识发现, 2021, 5(4): 1-12.
[14] Zhu Dongliang, Wen Yi, Wan Zichen. Review of Recommendation Systems Based on Knowledge Graph[J]. 数据分析与知识发现, 2021, 5(12): 1-13.
[15] Yu Chuanming, Zhang Zhengang, Kong Lingge. Comparing Knowledge Graph Representation Models for Link Prediction[J]. 数据分析与知识发现, 2021, 5(11): 29-44.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn