Recommending Reviewer Groups for Research Projects Based on Topic Coverage
Liu Xiaoyu1(),Wang Xuefeng2,Zhu Donghua2
1Department of Management, Beijing Electronic Science & Technology Institute, Beijing 100070, China 2School of Management and Economics, Beijing Institute of Technology, Beijing 100081, China
[Objective] Aimed at the peer review process of scientific research projects, this paper measures the coverage of reviewers’ knowledge on research project topics and constructs expert groups of maximum topic coverage. [Methods] We proposed three principles for recommending reviewer groups for research projects: the maximum topic coverage principle, the maximum knowledge matching principle, and the appropriate workload principle. Then, we developed a method for identifying the research topics of reviewers and projects using the Overlapping K-means. To achieve maximum topic coverage, we constructed a reviewer group recommendation model based on topic coverage, transforming the recommendation problem into an optimization problem. [Results] In two controlled experiments, the reviewer groups constructed by the proposed method increased the topic coverage by 32.38% and 29.01%, respectively. [Limitations] We need to quantitatively explore how to achieve multi-objective optimization for recommending reviewers for research projects according to the three principles. [Conclusions] This research took the reviewer group recommendation for the National Natural Science Foundation of China project application as a case study. It verified the feasibility and effectiveness of the proposed method through qualitative and quantitative analysis.
刘晓豫, 汪雪锋, 朱东华. 基于主题覆盖度的科研项目评审专家组推荐方法研究*[J]. 数据分析与知识发现, 2024, 8(6): 132-143.
Liu Xiaoyu, Wang Xuefeng, Zhu Donghua. Recommending Reviewer Groups for Research Projects Based on Topic Coverage. Data Analysis and Knowledge Discovery, 2024, 8(6): 132-143.
(Hao Hongquan, Zhao Yinghong, Yang Haohao, et al. Proposal Application, Peer Review and Funding of National Natural Science Foundation of China in 2022: An Overview[J]. Bulletin of National Natural Science Foundation of China, 2023, 37(1): 3-6.)
(Zhang Hongliang, Fan Yonggang, Chen Qingyun, et al. Rational Thoughts on Improving the Peer-Review Mechanism During the Deepening Reform of the National Natural Science Foundation[J]. Bulletin of National Natural Science Foundation of China, 2022, 36(5): 685-692.)
[3]
Rahman A I M J, Guns R, Leydesdorff L, et al. Measuring the Match Between Evaluators and Evaluees: Cognitive Distances Between Panel Members and Research Groups at the Journal Level[J]. Scientometrics, 2016, 109(3): 1639-1663.
(Li Dong, Hao Yanni, Peng Shenghui, et al. Information Construction of the National Natural Science Foundation of China: Today and Intelligent Development Prospects[J]. Bulletin of National Natural Science Foundation of China, 2023, 37(2): 307-312.)
(Yang Haohao, Hao Hongquan, An Hao, et al. The Importance of the Reform on Classification Application and Evaluation of National Natural Science Foundation of China in the New Era[J]. Bulletin of National Natural Science Foundation of China, 2022, 36(5): 675-684.)
(Li Jiangtao, Liu Lei, Shi Ling, et al. Proposal Application, Peer Review and Funding of the Department of Management Sciences in 2022: An Overview[J]. Bulletin of National Natural Science Foundation of China, 2023, 37(1): 48-53.)
[7]
Wang Q, Ma J, Liao X W, et al. A Context-Aware Researcher Recommendation System for University-Industry Collaboration on R&D Projects[J]. Decision Support Systems, 2017, 103: 46-57.
[8]
Liu O, Wang J, Ma J, et al. An Intelligent Decision Support Approach for Reviewer Assignment in R&D Project Selection[J]. Computers in Industry, 2016, 76: 1-10.
[9]
van Arensbergen P, van der Weijden I, van den Besselaar P. The Selection of Talent as a Group Process. A Literature Review on the Social Dynamics of Decision Making in Grant Panels[J]. Research Evaluation, 2014, 23(4): 298-311.
[10]
Hoang D T, Nguyen N T, Hwang D. Recommendation of Expert Group to Question and Answer Sites Based on User Behaviors and Diversity[J]. Journal of Intelligent & Fuzzy Systems, 2019, 37(6): 7117-7129.
(Yu Xuan, Gao Ruiping. Give Play to the Role of the “Four Platforms” of the Joint Funds of NSFC, Improve the Diversified Input Mechanism for Basic Research[J]. Bulletin of National Natural Science Foundation of China, 2023, 37(2): 296-300.)
(Zhou Xiaomei, Nie Jianqing, Tang Fujie, et al. On the Joint Fund Management Mechanism of National Natural Science Foundation of China: Problems and Suggestions[J]. Bulletin of National Natural Science Foundation of China, 2023, 37(1): 131-135.)
(Li Zhengfeng, Wu Chenxiao, Huang Lu, et al. How Could National Natural Science Foundation of China Better Guide Diversified Investment in Basic Research?[J]. Bulletin of Chinese Academy of Sciences, 2021, 36(12): 1448-1455.)
[14]
Herrera F, Martı́nez L, Sánchez P J. Managing Non-Homogeneous Information in Group Decision Making[J]. European Journal of Operational Research, 2005, 166(1): 115-132.
[15]
Ceylan D, Saatçioǧlu Ö, Sepil C. An Algorithm for the Committee Construction Problem[J]. European Journal of Operational Research, 1994, 77(1): 60-69.
[16]
Silva T, Guo Z L, Ma J, et al. A Social Network-Empowered Research Analytics Framework for Project Selection[J]. Decision Support Systems, 2013, 55(4): 957-968.
[17]
Darling E S. Use of Double-Blind Peer Review to Increase Author Diversity[J]. Conservation Biology, 2015, 29(1): 297-299.
doi: 10.1111/cobi.12333
pmid: 25039807
(Zhao Songtao, Shen Qian, Dai Yafei, et al. Proposal Applications, Peer Review and Funding of the Department of Interdisciplinary Sciences in 2022: An Overview[J]. Bulletin of National Natural Science Foundation of China, 2023, 37(1): 54-56.)
(Li Wencong, Xu Jin, Shen Jie, et al. Interdisciplinary Research Funding Mechanisms of the UK Research and Innovation(UKRI) and Their Implications[J]. Acta Physico-Chimica Sinica, 2020, 36(11): 173-178.)
[20]
贺颖. 基于科学计量视角的同行评议专家遴选问题研究[D]. 天津: 天津大学, 2008.
[20]
(He Ying. Study on the Problems About Expert Selection of Peer Review Based on the Viewpoint of Scientometrics[D]. Tianjin: Tianjin University, 2008.)
(Li Jiang, Li Dong, Feng Peihua, et al. An Expert Recommendation Model Based on the Speciality, Scientific Impact of Experts, and Social Relationship Between Experts and Applicants[J]. Journal of the China Society for Scientific and Technical Information, 2017, 36(4): 338-345.)
(Wu Renke. Study on Intelligent Retrieval and Recommendation System for Science and Technology Project Experts[D]. Hangzhou: Hangzhou Dianzi University, 2014.)
[23]
Li X L, Watanabe T. Paper-to-Reviewer Assignment, Based on Expertise Degree of Reviewers and Relevance Degree Between Reviewers and Papers[J]. International Journal of Knowledge and Web Intelligence, 2014, 5(1): 1-20.
[24]
Protasiewicz J, Pedrycz W, Kozłowski M, et al. A Recommender System of Reviewers and Experts in Reviewing Problems[J]. Knowledge-Based Systems, 2016, 106: 164-178.
[25]
Liu X Y, Wang X F, Zhu D H. Reviewer Recommendation Method for Scientific Research Proposals: A Case for NSFC[J]. Scientometrics, 2022, 127(6): 3343-3366.
[26]
Kunaver M, Porl T. Diversity in Recommender Systems—A Survey[J]. Knowledge-Based Systems, 2017, 123: 154-162.
[27]
Nguyen N T. Advanced Methods for Inconsistent Knowledge Management[M]. London: Springer London, 2008.
[28]
Chen Y, Fan Z P, Ma J, et al. A Hybrid Grouping Genetic Algorithm for Reviewer Group Construction Problem[J]. Expert Systems with Applications, 2011, 38(3): 2401-2411.
[29]
Cleuziou G. An Extended Version of the K-Means Method for Overlapping Clustering[C]// Proceedings of the 19th International Conference on Pattern Recognition. IEEE, 2008.
[30]
De Mulder W. Optimal Clustering in the Context of Overlapping Cluster Analysis[J]. Information Sciences, 2013, 223: 56-74.
[31]
Ben n’cir C E, Cleuziou G, Essoussi N. Overview of Overlapping Partitional Clustering Methods[M]// CelebiM. PartitionalClustering Algorithms. Cham: Springer, 2015: 245-275.
[32]
Banerjee A, Krumpelman C, Ghosh J, et al. Model-Based Overlapping Clustering[C]// Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining. ACM, 2005: 532-537.
(Liu Xiaoyu, Zhu Donghua, Wang Xuefeng, et al. Multi-Expertise Researcher Identification: A Case Study of the Big Data[J]. Library and Information Service, 2018, 62(3): 55-63.)
doi: 10.13266/j.issn.0252-3116.2018.03.007
