[Objective] This paper proposes an interpretable reinforcement learning method for job recommendation based on talent knowledge graph reasoning, which addresses the issues of difficulties in large-scale application, cold start, and lack of novelty. [Methods] First, we constructed a knowledge graph for the social experience of the job applicants based on their resume data. Then, we trained a strategic agent with the knowledge graph and the theory of reinforcement learning. This algorithm, which divided the reasoning process into choosing directions and nodes, could identify potential high-quality recommendation targets from the knowledge graph. [Results] The MRR@20 (81.7%), Hit@1 (74.8%), Hit@5 (92.2%) and Hit@10 (97.0%) of the proposed model were higher than those of the LR, BPR, JRL-int, JRL-rep and PGPR models. [Limitations] The size of the experimental datasets and the task-types needs to be further expanded. [Conclusions] Our model could effectively recommend jobs for applicants based on their previous experience or other successful recommendations. It also provides reasoning paths with the help of knowledge graph.
(Tecent. Analysis of National Employment from January to July 2020: 6.74 Million New Urban Jobs were Created[EB/OL]. (2020-09-16). [2020-09-30]. https://new.qq.com/rain/a/20200916A05NLM00.)
[2]
Zhu H, Li X, Zhang P Y, et al. Learning Tree Based Deep Model for Recommender Systems[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018: 1079-1088.
[3]
Kanakia A, Shen Z H, Eide D, et al. A Scalable Hybrid Research Paper Recommender System for Microsoft Academic[C]// Proceedings of the 2019 World Wide Web Conference. 2019: 2893-2899.
[4]
Kumar B, Sharma N. Approaches, Issues and Challenges in Recommender Systems: A Systematic Review[J]. Indian Journal of Science and Technology, 2016,9(47):1-12.
[5]
Schein A I, Popescul A, Ungar L H, et al. Pennock Methods and Metrics for Cold Start Recommendations[C]// Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2002: 253-260.
[6]
Gugnani A, Misra H. Implicit Skills Extraction Using Document Embedding and Its Use in Job Recommendation[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020,34(8):13286-13293.
[7]
Xian Y K, Fu Z H, Muthukrishnan S, et al. Reinforcement Knowledge Graph Reasoning for Explainable Recommendation[C]// Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019: 285-294.
[8]
Lin X V, Socher R, Xiong C M. Multi-Hop Knowledge Graph Reasoning with Reward Shaping[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 3243-3253.
[9]
Domeniconi G, Moro G, Pagliarani A, et al. Job Recommendation from Semantic Similarity of LinkedIn Users’ Skills[C]// Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods. 2016: 270-277.
[10]
Qin C, Zhu H S, Xu T, et al. Enhancing Person-Job Fit for Talent Recruitment: An Ability-aware Neural Network Approach[C]// Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. 2018: 25-34.
[11]
Meng Q X, Zhu H S, Xiao K L, et al. A Hierarchical Career-Path-Aware Neural Network for Job Mobility Prediction[C]// Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2019: 14-24.
[12]
Bordes A, Usunier N, García-Durán A, et al. Translating Embeddings for Modeling Multi-Relational Data[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013: 2787-2795.
[13]
Nickel M, Tresp V, Kriegel H. A Three-Way Model for Collective Learning on Multi-Relational Data[C]// Proceedings of the 28th International Conference on Machine Learning. 2011: 809-816.
[14]
Zhang F Z, Yuan J N, Lian D F, et al. Collaborative Knowledge Base Embedding for Recommender Systems[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016: 353-362.
[15]
Huang J, Zhao W X, Dou H J, et al. Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks[C]// Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 2018: 505-514.
[16]
Wang H W, Zhang F Z, Wang J L, et al. RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems[C]// Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2018: 417-426.
[17]
Ai Q Y, Azizi V, Chen X, et al. Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation[J]. Algorithms, 2018,11(9):137.
doi: 10.3390/a11090137
[18]
Wang X, Wang D X, Xu C R, et al. Explainable Reasoning over Knowledge Graphs for Recommendation[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019: 5329-5336.
[19]
Silver D, Huang A, Maddison C J, et al. Mastering the Game of Go with Deep Neural Networks and Tree Search[J]. Nature, 2016,529(7587):484-489.
doi: 10.1038/nature16961
pmid: 26819042
[20]
Theocharous G, Thomas P S, Ghavamzadeh M, et al. Personalized Ad Recommendation Systems for Life-Time Value Optimization with Guarantees[C]// Proceedings of the 24th International Conference on Artificial Intelligence. 2015: 1806-1812.
[21]
Zheng G J, Zhang F Z, Zheng Z H, et al. DRN: A Deep Reinforcement Learning Framework for News Recommendation[C]// Proceedings of the 2018 World Wide Web Conference. 2018: 167-176.
[22]
Wang X T, Chen Y R, Yang J, et al. A Reinforcement Learning Framework for Explainable Recommendation[C]// Proceedings of the 2018 IEEE International Conference on Data Mining. 2018: 587-596.
[23]
Xiong W H, Hoang T, Wang W Y. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 564-573.
[24]
Das R, Dhuliawala S, Zaheer M, et al. Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases Using Reinforcement Learning[C]// Proceedings of the 31st Conference on Neural Information Processing Systems. 2017.
(National Bureau of Statistics. 2017 National Economic Industry Classification (GB/T 4754-2017) [EB/OL]. (2017-09-29). [2020-09-30]. http://www.stats.gov.cn/tjsj/tjbz/hyflbz/201710/t20171012_1541679.html.)
(Ministry of Education of the People’s Republic of China. Mergers of Colleges and Universities Since 1990 (as of May 15, 2006) [EB/OL]. (2006-05-15). [2020-09-30]. http://www.moe.gov.cn/srcsite/A03/moe_634/200605/t20060515_88440.html.)
(Ministry of Education of the People’s Republic of China. Colleges and Universities Setting, The Ministry of Education Development Department [EB/OL]. (2020-08-28). [2020-09-30]. http://www.moe.gov.cn/s78/A03/ghs_left/s181/.)
[29]
Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv:1810.04805.
[30]
Lerer A, Wu L, Shen J J, et al. PyTorch-BigGraph: A Large-scale Graph Embedding System[C]// Proceedings of the 2nd Conference on Systems and Machine Learning. 2019.
[31]
Yu X, Gu Q Q, Zhou M W, et al. Citation Prediction in Heterogeneous Bibliographic Networks[C]// Proceedings of the 2012 SIAM International Conference on Data Mining. 2012: 1119-1130.
[32]
Frans K, Ho J, Chen X, et al. Meta Learning Shared Hierarchies[C]// Proceedings of the 6th International Conference on Learning Representations. 2018.
[33]
Mnih V, Badia A P, Mirza M, et al. Asynchronous Methods for Deep Reinforcement Learning[C]// Proceedings of the 33rd International Conference on Machine Learning. 2016: 1928-1937.
(Ifchange. “Resume Desensitization Dataset of Ifchange”[EB/OL]. [2020-06-10]. http://hdl.handle.net/20.500.12291/10226 V1 [Version].)
[35]
Wright R E. Logistic Regression[J]. Reading and Understanding Multivariate Statistics, 1995: 217-244.
[36]
Rendle S, Freudenthaler C, Gantner Z, et al. BPR: Bayesian Personalized Ranking from Implicit Feedback[C]// Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2009: 452-461.
[37]
Zhang Y F, Ai Q Y, Chen X, et al. Joint Representation Learning for Top-N Recommendation with Heterogeneous Information Sources[C]// Proceedings of the 2017 ACM Conference on Information and Knowledge Management. 2017: 1449-1458.
[38]
Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780.
pmid: 9377276
[39]
Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[OL]. arXiv Preprint, arXiv: 1412.6980.