Interpretable Recommendation of Reinforcement Learning Based on Talent Knowledge Graph Reasoning

doi:10.11925/infotech.2096-3467.2020.1218

Data Analysis and Knowledge Discovery

2021, Vol. 5

Issue (6): 36-50 DOI: 10.11925/infotech.2096-3467.2020.1218

Current Issue | Archive | Adv Search

Interpretable Recommendation of Reinforcement Learning Based on Talent Knowledge Graph Reasoning

Ruan Xiaoyun,Liao Jianbin,Li Xiang,Yang Yang,Li Daifeng(

)

School of Information Management, Sun Yat-Sen University, Guangzhou 510006, China

Download: PDF (2560 KB) HTML ( 42 )
Export: BibTeX | EndNote (RIS)

Abstract

[Objective] This paper proposes an interpretable reinforcement learning method for job recommendation based on talent knowledge graph reasoning, which addresses the issues of difficulties in large-scale application, cold start, and lack of novelty. [Methods] First, we constructed a knowledge graph for the social experience of the job applicants based on their resume data. Then, we trained a strategic agent with the knowledge graph and the theory of reinforcement learning. This algorithm, which divided the reasoning process into choosing directions and nodes, could identify potential high-quality recommendation targets from the knowledge graph. [Results] The MRR@20 (81.7%), Hit@1 (74.8%), Hit@5 (92.2%) and Hit@10 (97.0%) of the proposed model were higher than those of the LR, BPR, JRL-int, JRL-rep and PGPR models. [Limitations] The size of the experimental datasets and the task-types needs to be further expanded. [Conclusions] Our model could effectively recommend jobs for applicants based on their previous experience or other successful recommendations. It also provides reasoning paths with the help of knowledge graph.

Key words： Work Recommendation Knowledge Graph Reasoning Reinforcement Learning Interpretable Recommendation

Received: 06 December 2020 Published: 19 March 2021

ZTFLH:

TP393

Fund:National Natural Science Foundation of China(61702564);National Natural Science Foundation of China(72074231);Soft Science General Program of Guangdong Province, China(2019A101002020)

Corresponding Authors: Li Daifeng E-mail: lidaifeng@mail.sysu.edu.cn

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Xiaoyun Ruan
	Jianbin Liao
	Xiang Li
	Yang Yang
	Daifeng Li

Cite this article:

Ruan Xiaoyun,Liao Jianbin,Li Xiang,Yang Yang,Li Daifeng. Interpretable Recommendation of Reinforcement Learning Based on Talent Knowledge Graph Reasoning. Data Analysis and Knowledge Discovery, 2021, 5(6): 36-50.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2020.1218 OR https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2021/V5/I6/36

Examples of Job Recommendations Based on the Talent Social Experience Knowledge Graph

Main Workflow

Hierarchical Reasoning Work Recommendation Model Based on the Talent Social Experience Knowledge Graph

Examples of Position Categories

Visualization of Talent’s Social Experience Knowledge Graph

Entity Statistics of Talent Social Experience Knowledge Graph

关系类型	描述	数量
Employer	$Corporation ← → Employer User$	97 003
Work_as	$User ← → Work_as Position$	97 003
Work	$User ← → Work Work$	97 003
Provide	$Corporation ← → Provide Position$	97 003
Describe_position	$Position ← → Describe_position Key_word$	897 510
Field	$User ← → Field Industry$	97 003
Involve	$Position ← → Involve Industry$	97 003
Graduation	$User ← → Graduation School$	46 178
Major	$User ← → Major Discipline$	46 178
Attr_Position	$Work ← → Attr_Position Position$	97 003
Attr_Corporation	$Work ← → Attr_Corporation Corporation$	97 003
Attr_Key_word	$Work ← → Attr_Key_word Key_word$	897 510
Attr_Industry	$Work ← → Attr_Industry Industry$	97 003

Relationship Statistics of Talent Social Experience Knowledge Graph

Comparison of Effects for Ranking-only Model

Comparison of Effects for Recall and Ranking Model

Effect of Graph Embedding Dimension on Model Performance

Number and Effectiveness of Path Patterns with Different Lengths

Effective Path Patterns of Different Lengths Discovered by Agents

Case Analysis of Real Job Recommendation

[1]	腾讯网. 2020年1-7月全国就业情况分析全国城镇新增就业674万人[EB/OL]. (2020-09-16). [2020-09-30]. https://new.qq.com/rain/a/20200916A05NLM00.
[1]	(Tecent. Analysis of National Employment from January to July 2020: 6.74 Million New Urban Jobs were Created[EB/OL]. (2020-09-16). [2020-09-30]. https://new.qq.com/rain/a/20200916A05NLM00.)
[2]	Zhu H, Li X, Zhang P Y, et al. Learning Tree Based Deep Model for Recommender Systems[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018: 1079-1088.
[3]	Kanakia A, Shen Z H, Eide D, et al. A Scalable Hybrid Research Paper Recommender System for Microsoft Academic[C]// Proceedings of the 2019 World Wide Web Conference. 2019: 2893-2899.
[4]	Kumar B, Sharma N. Approaches, Issues and Challenges in Recommender Systems: A Systematic Review[J]. Indian Journal of Science and Technology, 2016,9(47):1-12.
[5]	Schein A I, Popescul A, Ungar L H, et al. Pennock Methods and Metrics for Cold Start Recommendations[C]// Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2002: 253-260.
[6]	Gugnani A, Misra H. Implicit Skills Extraction Using Document Embedding and Its Use in Job Recommendation[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020,34(8):13286-13293.
[7]	Xian Y K, Fu Z H, Muthukrishnan S, et al. Reinforcement Knowledge Graph Reasoning for Explainable Recommendation[C]// Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019: 285-294.
[8]	Lin X V, Socher R, Xiong C M. Multi-Hop Knowledge Graph Reasoning with Reward Shaping[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 3243-3253.
[9]	Domeniconi G, Moro G, Pagliarani A, et al. Job Recommendation from Semantic Similarity of LinkedIn Users’ Skills[C]// Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods. 2016: 270-277.
[10]	Qin C, Zhu H S, Xu T, et al. Enhancing Person-Job Fit for Talent Recruitment: An Ability-aware Neural Network Approach[C]// Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. 2018: 25-34.
[11]	Meng Q X, Zhu H S, Xiao K L, et al. A Hierarchical Career-Path-Aware Neural Network for Job Mobility Prediction[C]// Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2019: 14-24.
[12]	Bordes A, Usunier N, García-Durán A, et al. Translating Embeddings for Modeling Multi-Relational Data[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013: 2787-2795.
[13]	Nickel M, Tresp V, Kriegel H. A Three-Way Model for Collective Learning on Multi-Relational Data[C]// Proceedings of the 28th International Conference on Machine Learning. 2011: 809-816.
[14]	Zhang F Z, Yuan J N, Lian D F, et al. Collaborative Knowledge Base Embedding for Recommender Systems[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016: 353-362.
[15]	Huang J, Zhao W X, Dou H J, et al. Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks[C]// Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 2018: 505-514.
[16]	Wang H W, Zhang F Z, Wang J L, et al. RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems[C]// Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2018: 417-426.
[17]	Ai Q Y, Azizi V, Chen X, et al. Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation[J]. Algorithms, 2018,11(9):137. doi: 10.3390/a11090137
[18]	Wang X, Wang D X, Xu C R, et al. Explainable Reasoning over Knowledge Graphs for Recommendation[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019: 5329-5336.
[19]	Silver D, Huang A, Maddison C J, et al. Mastering the Game of Go with Deep Neural Networks and Tree Search[J]. Nature, 2016,529(7587):484-489. doi: 10.1038/nature16961 pmid: 26819042
[20]	Theocharous G, Thomas P S, Ghavamzadeh M, et al. Personalized Ad Recommendation Systems for Life-Time Value Optimization with Guarantees[C]// Proceedings of the 24th International Conference on Artificial Intelligence. 2015: 1806-1812.
[21]	Zheng G J, Zhang F Z, Zheng Z H, et al. DRN: A Deep Reinforcement Learning Framework for News Recommendation[C]// Proceedings of the 2018 World Wide Web Conference. 2018: 167-176.
[22]	Wang X T, Chen Y R, Yang J, et al. A Reinforcement Learning Framework for Explainable Recommendation[C]// Proceedings of the 2018 IEEE International Conference on Data Mining. 2018: 587-596.
[23]	Xiong W H, Hoang T, Wang W Y. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 564-573.
[24]	Das R, Dhuliawala S, Zaheer M, et al. Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases Using Reinforcement Learning[C]// Proceedings of the 31st Conference on Neural Information Processing Systems. 2017.
[25]	国家统计局. 2017年国民经济行业分类(GB/T 4754-2017)[EB/OL]. (2017-09-29). [2020-09-30]. http://www.stats.gov.cn/tjsj/tjbz/hyflbz/201710/t20171012_1541679.html.
[25]	(National Bureau of Statistics. 2017 National Economic Industry Classification (GB/T 4754-2017) [EB/OL]. (2017-09-29). [2020-09-30]. http://www.stats.gov.cn/tjsj/tjbz/hyflbz/201710/t20171012_1541679.html.)
[26]	中国学位与研究生教育信息网. 学科、专业目录[EB/OL]. [2020-09-30]. http://www.cdgdc.edu.cn/xwyyjsjyxx/sy/glmd/264462.shtml.
[26]	(China Academic Degrees & Education Information. Disciplines and Specialties Directory [EB/OL]. [2020-09-30]. http://www.cdgdc.edu.cn/xwyyjsjyxx/sy/glmd/264462.shtml.)
[27]	中华人民共和国教育部. 1990年以来高校合并情况(截止到2006年5月15日)[EB/OL]. (2006-05-15). [2020-09-30]. http://www.moe.gov.cn/srcsite/A03/moe_634/200605/t20060515_88440.html.
[27]	(Ministry of Education of the People’s Republic of China. Mergers of Colleges and Universities Since 1990 (as of May 15, 2006) [EB/OL]. (2006-05-15). [2020-09-30]. http://www.moe.gov.cn/srcsite/A03/moe_634/200605/t20060515_88440.html.)
[28]	中华人民共和国教育部. 教育部发展规划司院校设置[EB/OL]. (2020-08-28). [2020-09-30]. http://www.moe.gov.cn/s78/A03/ghs_left/s181/.
[28]	(Ministry of Education of the People’s Republic of China. Colleges and Universities Setting, The Ministry of Education Development Department [EB/OL]. (2020-08-28). [2020-09-30]. http://www.moe.gov.cn/s78/A03/ghs_left/s181/.)
[29]	Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv:1810.04805.
[30]	Lerer A, Wu L, Shen J J, et al. PyTorch-BigGraph: A Large-scale Graph Embedding System[C]// Proceedings of the 2nd Conference on Systems and Machine Learning. 2019.
[31]	Yu X, Gu Q Q, Zhou M W, et al. Citation Prediction in Heterogeneous Bibliographic Networks[C]// Proceedings of the 2012 SIAM International Conference on Data Mining. 2012: 1119-1130.
[32]	Frans K, Ho J, Chen X, et al. Meta Learning Shared Hierarchies[C]// Proceedings of the 6th International Conference on Learning Representations. 2018.
[33]	Mnih V, Badia A P, Mirza M, et al. Asynchronous Methods for Deep Reinforcement Learning[C]// Proceedings of the 33rd International Conference on Machine Learning. 2016: 1928-1937.
[34]	e成科技. “e成科技简历脱敏数据集”[EB/OL]. [2020-06-10]. http://hdl.handle.net/20.500.12291/10226 V1 [Version].
[34]	(Ifchange. “Resume Desensitization Dataset of Ifchange”[EB/OL]. [2020-06-10]. http://hdl.handle.net/20.500.12291/10226 V1 [Version].)
[35]	Wright R E. Logistic Regression[J]. Reading and Understanding Multivariate Statistics, 1995: 217-244.
[36]	Rendle S, Freudenthaler C, Gantner Z, et al. BPR: Bayesian Personalized Ranking from Implicit Feedback[C]// Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2009: 452-461.
[37]	Zhang Y F, Ai Q Y, Chen X, et al. Joint Representation Learning for Top-N Recommendation with Heterogeneous Information Sources[C]// Proceedings of the 2017 ACM Conference on Information and Knowledge Management. 2017: 1449-1458.
[38]	Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780. pmid: 9377276
[39]	Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[OL]. arXiv Preprint, arXiv: 1412.6980.

[1]	Ma Jiangwei, Lv Xueqiang, You Xindong, Xiao Gang, Han Junmei. Extracting Relationship Among Military Domains with BERT and Relation Position Features[J]. 数据分析与知识发现, 2021, 5(8): 1-12.
[2]	Zhang Le, Leng Jidong, Lv Xueqiang, Cui Zhuo, Wang Lei, You Xindong. RLCPAR: A Rewriting Model for Chinese Patent Abstracts Based on Reinforcement Learning[J]. 数据分析与知识发现, 2021, 5(7): 59-69.
[3]	Zhu Chaoyu, Liu Lei. A Review of Medical Decision Supports Based on Knowledge Graph[J]. 数据分析与知识发现, 2020, 4(12): 26-32.
[4]	Bai Chen,Gan Liren. Study on the Learning Behavior of Science and Technology User for Academic Database Website Based on Reinforcement Models[J]. 现代图书情报技术, 2009, 25(7-8): 85-92.

Viewed

Full text

Abstract

Cited

Shared

Discussed