Please wait a minute...
Data Analysis and Knowledge Discovery  2021, Vol. 5 Issue (6): 36-50    DOI: 10.11925/infotech.2096-3467.2020.1218
Current Issue | Archive | Adv Search |
Interpretable Recommendation of Reinforcement Learning Based on Talent Knowledge Graph Reasoning
Ruan Xiaoyun,Liao Jianbin,Li Xiang,Yang Yang,Li Daifeng()
School of Information Management, Sun Yat-Sen University, Guangzhou 510006, China
Download: PDF (2560 KB)   HTML ( 24
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper proposes an interpretable reinforcement learning method for job recommendation based on talent knowledge graph reasoning, which addresses the issues of difficulties in large-scale application, cold start, and lack of novelty. [Methods] First, we constructed a knowledge graph for the social experience of the job applicants based on their resume data. Then, we trained a strategic agent with the knowledge graph and the theory of reinforcement learning. This algorithm, which divided the reasoning process into choosing directions and nodes, could identify potential high-quality recommendation targets from the knowledge graph. [Results] The MRR@20 (81.7%), Hit@1 (74.8%), Hit@5 (92.2%) and Hit@10 (97.0%) of the proposed model were higher than those of the LR, BPR, JRL-int, JRL-rep and PGPR models. [Limitations] The size of the experimental datasets and the task-types needs to be further expanded. [Conclusions] Our model could effectively recommend jobs for applicants based on their previous experience or other successful recommendations. It also provides reasoning paths with the help of knowledge graph.

Key wordsWork Recommendation      Knowledge Graph Reasoning      Reinforcement Learning      Interpretable Recommendation     
Received: 06 December 2020      Published: 19 March 2021
ZTFLH:  TP393  
Fund:National Natural Science Foundation of China(61702564);National Natural Science Foundation of China(72074231);Soft Science General Program of Guangdong Province, China(2019A101002020)
Corresponding Authors: Li Daifeng     E-mail: lidaifeng@mail.sysu.edu.cn

Cite this article:

Ruan Xiaoyun,Liao Jianbin,Li Xiang,Yang Yang,Li Daifeng. Interpretable Recommendation of Reinforcement Learning Based on Talent Knowledge Graph Reasoning. Data Analysis and Knowledge Discovery, 2021, 5(6): 36-50.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2020.1218     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2021/V5/I6/36

Examples of Job Recommendations Based on the Talent Social Experience Knowledge Graph
Main Workflow
Hierarchical Reasoning Work Recommendation Model Based on the Talent Social Experience Knowledge Graph
大类 中类 原始岗位
计算机技术 测试 软件测试工程师,测试工程师,软件测试,云产品测试开发,集成测试工程师……
后端开发 后端工程师,web后端工程师,python后端开发工程师,高级后端工程师,后端组长,广告后端架构师……
前端开发 前端工程师助理,js工程师,Web前端开发工程师,html工程师,前端工程师,移动前端负责人……
信息安全 网络信息安全工程师,信息安全经理,信息安全保障中心网络工程师,安全咨询顾问,密码算法工程师……
大数据&数据分析 应用分析专员,数据支持专员,数据主管,大数据工程师,数据支撑专员,舆情分析,数据科学家……
算法 语音算法工程师,推荐算法工程师,图像算法工程师,融合算法工程师,人工智能工程师,数据算法工程师……
数据库开发 dba专员,oracle erp应用工程师,etl+oracle数据仓库,oracle高级工程师,数据库设计员,SQL技术支持……
…… ……
Examples of Position Categories
Visualization of Talent’s Social Experience Knowledge Graph
实体类型 描述 数量
User 用户 39 081
Work 工作 96 990
Corporation 公司/单位名称 61 125
Position 岗位名称 249
Industry 行业名称 38
Keyword 关键词 90 149
School 毕业院校 1 149
Discipline 专业 84
Entity Statistics of Talent Social Experience Knowledge Graph
关系类型 描述 数量
Employer CorporationEmployerUser 97 003
Work_as UserWork_asPosition 97 003
Work UserWorkWork 97 003
Provide CorporationProvidePosition 97 003
Describe_position PositionDescribe_positionKey_word 897 510
Field UserFieldIndustry 97 003
Involve PositionInvolveIndustry 97 003
Graduation UserGraduationSchool 46 178
Major UserMajorDiscipline 46 178
Attr_Position WorkAttr_PositionPosition 97 003
Attr_Corporation WorkAttr_CorporationCorporation 97 003
Attr_Key_word WorkAttr_Key_wordKey_word 897 510
Attr_Industry WorkAttr_IndustryIndustry 97 003
Relationship Statistics of Talent Social Experience Knowledge Graph
模型 MRR@20(%) Hit@1(%) Hit@5(%) Hit@10(%)
LR 28.7 18.0 50.8 73.7
BPR 30.6 20.2 51.7 74.1
JRL-int 34.9 25.3 55.6 77.4
JRL-rep 34.7 25.0 61.8 80.6
Our 81.7 74.8 92.2 97.0
Comparison of Effects for Ranking-only Model
模型 P@10(%) R@10(%) P@20(%) R@20(%) MRR@20(%) 候选集平均大小(个)
PGPR 0.2 1.9 0.3 6.1 0.8 69.6
Our 1.6 16.1 0.9 17.1 11.8 58.1
Comparison of Effects for Recall and Ranking Model
嵌入维度 P@10(%) R@10(%) P@20(%) R@20(%) MRR@20(%) 候选集平均大小(个)
200 0.7 7.4 0.4 8.3 5.7 61.8
250 1.6 16.1 0.9 17.1 11.8 59.1
300 0.7 7.2 0.4 7.6 5.5 58.1
Effect of Graph Embedding Dimension on Model Performance
路径长度 2-hop 3-hop 4-hop 5-hop 6-hop
路径模式数量 3 5 19 6 25
推理路径占比 16.0% 16.2% 63.4% 1.9% 2.5%
有效路径占比 82.3% 4.5% 12.5% 0.2% 0.6%
Number and Effectiveness of Path Patterns with Different Lengths
Effective Path Patterns of Different Lengths Discovered by Agents
Case Analysis of Real Job Recommendation
[1] 腾讯网. 2020年1-7月全国就业情况分析全国城镇新增就业674万人[EB/OL]. (2020-09-16). [2020-09-30]. https://new.qq.com/rain/a/20200916A05NLM00.
[1] (Tecent. Analysis of National Employment from January to July 2020: 6.74 Million New Urban Jobs were Created[EB/OL]. (2020-09-16). [2020-09-30]. https://new.qq.com/rain/a/20200916A05NLM00.)
[2] Zhu H, Li X, Zhang P Y, et al. Learning Tree Based Deep Model for Recommender Systems[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018: 1079-1088.
[3] Kanakia A, Shen Z H, Eide D, et al. A Scalable Hybrid Research Paper Recommender System for Microsoft Academic[C]// Proceedings of the 2019 World Wide Web Conference. 2019: 2893-2899.
[4] Kumar B, Sharma N. Approaches, Issues and Challenges in Recommender Systems: A Systematic Review[J]. Indian Journal of Science and Technology, 2016,9(47):1-12.
[5] Schein A I, Popescul A, Ungar L H, et al. Pennock Methods and Metrics for Cold Start Recommendations[C]// Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2002: 253-260.
[6] Gugnani A, Misra H. Implicit Skills Extraction Using Document Embedding and Its Use in Job Recommendation[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020,34(8):13286-13293.
[7] Xian Y K, Fu Z H, Muthukrishnan S, et al. Reinforcement Knowledge Graph Reasoning for Explainable Recommendation[C]// Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019: 285-294.
[8] Lin X V, Socher R, Xiong C M. Multi-Hop Knowledge Graph Reasoning with Reward Shaping[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 3243-3253.
[9] Domeniconi G, Moro G, Pagliarani A, et al. Job Recommendation from Semantic Similarity of LinkedIn Users’ Skills[C]// Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods. 2016: 270-277.
[10] Qin C, Zhu H S, Xu T, et al. Enhancing Person-Job Fit for Talent Recruitment: An Ability-aware Neural Network Approach[C]// Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. 2018: 25-34.
[11] Meng Q X, Zhu H S, Xiao K L, et al. A Hierarchical Career-Path-Aware Neural Network for Job Mobility Prediction[C]// Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2019: 14-24.
[12] Bordes A, Usunier N, García-Durán A, et al. Translating Embeddings for Modeling Multi-Relational Data[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013: 2787-2795.
[13] Nickel M, Tresp V, Kriegel H. A Three-Way Model for Collective Learning on Multi-Relational Data[C]// Proceedings of the 28th International Conference on Machine Learning. 2011: 809-816.
[14] Zhang F Z, Yuan J N, Lian D F, et al. Collaborative Knowledge Base Embedding for Recommender Systems[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016: 353-362.
[15] Huang J, Zhao W X, Dou H J, et al. Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks[C]// Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 2018: 505-514.
[16] Wang H W, Zhang F Z, Wang J L, et al. RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems[C]// Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2018: 417-426.
[17] Ai Q Y, Azizi V, Chen X, et al. Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation[J]. Algorithms, 2018,11(9):137.
doi: 10.3390/a11090137
[18] Wang X, Wang D X, Xu C R, et al. Explainable Reasoning over Knowledge Graphs for Recommendation[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019: 5329-5336.
[19] Silver D, Huang A, Maddison C J, et al. Mastering the Game of Go with Deep Neural Networks and Tree Search[J]. Nature, 2016,529(7587):484-489.
doi: 10.1038/nature16961 pmid: 26819042
[20] Theocharous G, Thomas P S, Ghavamzadeh M, et al. Personalized Ad Recommendation Systems for Life-Time Value Optimization with Guarantees[C]// Proceedings of the 24th International Conference on Artificial Intelligence. 2015: 1806-1812.
[21] Zheng G J, Zhang F Z, Zheng Z H, et al. DRN: A Deep Reinforcement Learning Framework for News Recommendation[C]// Proceedings of the 2018 World Wide Web Conference. 2018: 167-176.
[22] Wang X T, Chen Y R, Yang J, et al. A Reinforcement Learning Framework for Explainable Recommendation[C]// Proceedings of the 2018 IEEE International Conference on Data Mining. 2018: 587-596.
[23] Xiong W H, Hoang T, Wang W Y. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 564-573.
[24] Das R, Dhuliawala S, Zaheer M, et al. Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases Using Reinforcement Learning[C]// Proceedings of the 31st Conference on Neural Information Processing Systems. 2017.
[25] 国家统计局. 2017年国民经济行业分类(GB/T 4754-2017)[EB/OL]. (2017-09-29). [2020-09-30]. http://www.stats.gov.cn/tjsj/tjbz/hyflbz/201710/t20171012_1541679.html.
[25] (National Bureau of Statistics. 2017 National Economic Industry Classification (GB/T 4754-2017) [EB/OL]. (2017-09-29). [2020-09-30]. http://www.stats.gov.cn/tjsj/tjbz/hyflbz/201710/t20171012_1541679.html.)
[26] 中国学位与研究生教育信息网. 学科、专业目录[EB/OL]. [2020-09-30]. http://www.cdgdc.edu.cn/xwyyjsjyxx/sy/glmd/264462.shtml.
[26] (China Academic Degrees & Education Information. Disciplines and Specialties Directory [EB/OL]. [2020-09-30]. http://www.cdgdc.edu.cn/xwyyjsjyxx/sy/glmd/264462.shtml.)
[27] 中华人民共和国教育部. 1990年以来高校合并情况(截止到2006年5月15日)[EB/OL]. (2006-05-15). [2020-09-30]. http://www.moe.gov.cn/srcsite/A03/moe_634/200605/t20060515_88440.html.
[27] (Ministry of Education of the People’s Republic of China. Mergers of Colleges and Universities Since 1990 (as of May 15, 2006) [EB/OL]. (2006-05-15). [2020-09-30]. http://www.moe.gov.cn/srcsite/A03/moe_634/200605/t20060515_88440.html.)
[28] 中华人民共和国教育部. 教育部发展规划司院校设置[EB/OL]. (2020-08-28). [2020-09-30]. http://www.moe.gov.cn/s78/A03/ghs_left/s181/.
[28] (Ministry of Education of the People’s Republic of China. Colleges and Universities Setting, The Ministry of Education Development Department [EB/OL]. (2020-08-28). [2020-09-30]. http://www.moe.gov.cn/s78/A03/ghs_left/s181/.)
[29] Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv:1810.04805.
[30] Lerer A, Wu L, Shen J J, et al. PyTorch-BigGraph: A Large-scale Graph Embedding System[C]// Proceedings of the 2nd Conference on Systems and Machine Learning. 2019.
[31] Yu X, Gu Q Q, Zhou M W, et al. Citation Prediction in Heterogeneous Bibliographic Networks[C]// Proceedings of the 2012 SIAM International Conference on Data Mining. 2012: 1119-1130.
[32] Frans K, Ho J, Chen X, et al. Meta Learning Shared Hierarchies[C]// Proceedings of the 6th International Conference on Learning Representations. 2018.
[33] Mnih V, Badia A P, Mirza M, et al. Asynchronous Methods for Deep Reinforcement Learning[C]// Proceedings of the 33rd International Conference on Machine Learning. 2016: 1928-1937.
[34] e成科技. “e成科技简历脱敏数据集”[EB/OL]. [2020-06-10]. http://hdl.handle.net/20.500.12291/10226 V1 [Version].
[34] (Ifchange. “Resume Desensitization Dataset of Ifchange”[EB/OL]. [2020-06-10]. http://hdl.handle.net/20.500.12291/10226 V1 [Version].)
[35] Wright R E. Logistic Regression[J]. Reading and Understanding Multivariate Statistics, 1995: 217-244.
[36] Rendle S, Freudenthaler C, Gantner Z, et al. BPR: Bayesian Personalized Ranking from Implicit Feedback[C]// Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2009: 452-461.
[37] Zhang Y F, Ai Q Y, Chen X, et al. Joint Representation Learning for Top-N Recommendation with Heterogeneous Information Sources[C]// Proceedings of the 2017 ACM Conference on Information and Knowledge Management. 2017: 1449-1458.
[38] Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780.
pmid: 9377276
[39] Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[OL]. arXiv Preprint, arXiv: 1412.6980.
[1] Ma Jiangwei, Lv Xueqiang, You Xindong, Xiao Gang, Han Junmei. Extracting Relationship Among Military Domains with BERT and Relation Position Features[J]. 数据分析与知识发现, 2021, 5(8): 1-12.
[2] Zhang Le, Leng Jidong, Lv Xueqiang, Cui Zhuo, Wang Lei, You Xindong. RLCPAR: A Rewriting Model for Chinese Patent Abstracts Based on Reinforcement Learning[J]. 数据分析与知识发现, 2021, 5(7): 59-69.
[3] Zhu Chaoyu, Liu Lei. A Review of Medical Decision Supports Based on Knowledge Graph[J]. 数据分析与知识发现, 2020, 4(12): 26-32.
[4] Bai Chen,Gan Liren. Study on the Learning Behavior of Science and Technology User for Academic Database Website Based on Reinforcement Models[J]. 现代图书情报技术, 2009, 25(7-8): 85-92.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn