Selecting Representative Papers Based on Innovation and Influence
Liu Jiacheng1,2,Ma Tingcan1,2,3(),Yue Mingliang1,2,3
1National Science Library(Wuhan), Chinese Academy of Sciences, Wuhan 430071, China 2Department of Information Resources Management, University of Chinese Academy of Sciences, Beijing 100190, China 3Hubei Key Laboratory of Big Data in Science and Technology, Wuhan 430071, China
[Objective] This study proposes a new method for selecting representative papers, considering their innovation and impact. [Methods] First, we designed a representative index based on each paper’s innovation and influence measurement. Then, we chose papers with higher representative indices as a scholar’s representative works. Taking Nobel Prize laureates in physics as an example, we selected their representative papers. We used the average ranking and precision as evaluation metrics to assess the effectiveness of the proposed model. We also analyzed Academician Zhang Tao and Professor Hirsch JE’s works using the new model and identified their representative papers. [Results] Compared with the other six methods, the proposed model ranks first in both average ranking (2.838) and precision (63.158%). [Limitations] Our metrics require citation accumulation and may not effectively identify a scholar’s recent contributions. [Conclusions] The proposed method is feasible for selecting representative papers.
刘佳程, 马廷灿, 岳名亮. 融合创新性与影响力的论文代表作遴选方法研究*[J]. 数据分析与知识发现, 2024, 8(4): 88-98.
Liu Jiacheng, Ma Tingcan, Yue Mingliang. Selecting Representative Papers Based on Innovation and Influence. Data Analysis and Knowledge Discovery, 2024, 8(4): 88-98.
(Ministry of Education of the People’s Republic of China. Guiding Opinions on Deepening the Reform of the Evaluation System for University Teachers[EB/OL]. [2023-02-15]. http://www.moe.gov.cn/srcsite/A10/s7151/201609/t20160920_281586.html.)
(General Office of the Central Committee of the Communist Party of China, General Office of the State Council of the People’s Republic of China. Opinions on Deepening the Reform of Project Evaluation, Talent Evaluation and Institutional Evaluation[EB/OL]. [2023-02-15]. http://www.gov.cn/zhengce/2018-07/03/content_5303251.htm.)
(Ministry of Science and Technology of the People’s Republic of China. Some Measures to Eliminate the Bad Orientation of “Papers only” in Scientific and Technological Evaluation(Trial)[EB/OL]. [2023-02-15]. http://www.most.gov.cn/xxgk/xinxifenlei/fdzdgknr/fgzc/gfxwj/gfxwj2020/202002/w020200716318617342543.pdf.)
(Ministry of Education of the People’s Republic of China. Some Opinions on Breaking the Bad Orientation of “Papers only” in the Evaluation of Philosophy and Social Sciences Research in Colleges and Universities[EB/OL]. [2023-02-15]. http://www.moe.gov.cn/srcsite/A13/moe_2557/s3103/202012/t20201215_505588.html.)
(General Office of the State Council of the People’s Republic of China. Guiding. Opinions on Improving the Evaluation Mechanism of Scientific and Technological Achievements[EB/OL]. [2023-02-15]. http://www.gov.cn/zhengce/content/2021-08/02/content_5628987.htm.)
(Yang Ruixian, Gao Xinning, Dong Ke. Research Progress on Evaluation of Academic Representative Works in China[J]. Library and Information Service, 2022, 66(17): 129-140.)
doi: 10.13266/j.issn.0252-3116.2022.17.012
(Xie Zhen, Ma Jianxia, Hu Wenjing. Evaluation of Academic Representative Works: Methods Review and Research Prospects[J]. Information Studies: Theory & Application, 2021, 44(12): 190-196.)
(Wang Shipeng, Bao Xiang, Song Yuqing, et al. Review of the Theory Research of the “Representative Work” System[J]. Technology Intelligence Engineering, 2022, 8(2): 97-108.)
[9]
Niu Q K, Zhou J L, Zeng A, et al. Which Publication is Your Representative Work?[J]. Journal of Informetrics, 2016, 10(3): 842-853.
(Xie Zhen, Ma Jianxia, Hu Wenjing. The Innovation Measurement Method for Academic Papers Oriented to the Evaluation of Representative Works[J]. Information Studies: Theory & Application, 2022, 45(7): 81-88.)
doi: 10.16353/j.cnki.1000-7490.2022.07.012
[12]
Zhou J L, Zeng A, Fan Y, et al. The Representative Works of Scientists[J]. Scientometrics, 2018, 17(3): 1721-1732.
(Ministry of Science and Technology of the People’s Republic of China, Ministry of Education of the People’s Republic of China, Ministry of Industry and Information Technology of the People’s Republic of China, et al. Work Plan for Conducting Pilot Reform of Science and Technology Talent Evaluation[EB/OL]. [2023-04-05]. https://www.most.gov.cn/xxgk/xinxifenlei/fdzdgknr/qtwj/qtwj2022/202211/t20221109_183356.html.)
[14]
俞吾金. “代表作”制度改变了什么[N]. 解放日报, 2012-06-09(005).
[14]
(Yu Wujin. What has the “Representative Work” System Changed[N]. Jiefang Daily, 2012-06-09(005).)
[15]
叶继元. 代表作制有益遏制学术评价数量化[N]. 中国教育报, 2012-03-28(003).
[15]
(Ye Jiyuan. The Representative Work System is Beneficial to Curb the Formalization and Quantification of Academic Evaluation[N]. China Education Daily, 2012-03-28(003).)
(Jiang Chunlin, Wei Qingxiao. Construction of Evaluation Index System for Representative Papers in Humanities and Social Sciences and Its Realization Mechanism[J]. Gansu Social Sciences, 2017(2): 97-106.)
(Ren Quane. Cognition and Interpretation of Representative Academic Achievement Evaluation in Library and Information Science[J]. Journal of Yangzhou University(Humanities & Social Sciences), 2021, 25(4): 119-128.)
(Ministry of Human Resources and Social Security of the People’s Republic of China, Ministry of Education of the People’s Republic of China. Guiding Opinions on Deepening the Reform of Teachers’ Professional Title System in Colleges and Universities[EB/OL]. [2023-02-15]. http://www.gov.cn/zhengce/zhengceku/2021-01/27/content_5583094.htm.)
[19]
REF2021. Panel Criteria and Working Methods[EB/OL]. [2023-01-05]. https://www.ref.ac.uk/media/1450/ref-2019_02-panel-criteria-and-working-methods.pdf.
[20]
杨兴林. 高校教师职务晋升的学术代表作评价研究[J]. 江苏高教, 2015(2): 34-37.
[20]
(Yang Xinglin. Research on the Evaluation of Academic Representative Works for the Promotion of University Teachers[J]. Jiangsu Higher Education, 2015(2): 34-37.)
(Tian Xianpeng. Evaluation System of Academic Representative Work of University Teachers: Agent, Challenges and Path[J]. China Higher Education Research, 2020(2): 85-91.)
Hirsch J E. An Index to Quantify an Individual’s Scientific Research Output[J]. Proceedings of the National Academy of Sciences of the United States of America, 2005, 102(46): 16569-16572.
doi: 10.1073/pnas.0507655102
pmid: 16275915
[24]
Schubert A. Using the h-Index for Assessing Single Publications[J]. Scientometrics, 2009, 78(3): 559-565.
(Liu Yunmei, Li Changling, Feng Zhigang, et al. Discussion of Improved p-Index Evaluating the Academic Quality of Single Paper[J]. Library and Information Service, 2017, 61(21): 106-113.)
doi: 10.13266/j.issn.0252-3116.2017.21.013
[26]
Ye F Y, Leydesdorff L. The “Academic Trace” of the Performance Matrix: A Mathematical Synthesis of the h-Index and the Integrated Impact Indicator(I3)[J]. Journal of the Association for Information Science and Technology, 2014, 65(4): 742-750.
(Sun Jinnan, Ding Zuoqi. Research on the Evaluation of University Teachers’ Representative Works Based on Z-index[J]. Pharmaceutical Education, 2021, 37(1): 1-6.)
[28]
Egghe L. Theory and Practise of the G-Index[J]. Scientometrics, 2006, 69(1): 131-152.
(Jiang Chunlin, Zhang Liwei, Zhang Chunbo. A Probe into Scientific Metrological Method-Aided Magnum Opus Evaluation[J]. Information and Documentation Services, 2014(3): 31-36.)
(Liu Xueli, Shen Lan, Guo Jia, et al. Method for Selecting Scholars’ Representative Works by Baselines of Citations: Based on the Articles of Henan Provincial Scholars[J]. Chinese Journal of Scientific and Technical Periodicals, 2020, 31(8): 941-947.)
doi: 10.11946/cjstp.202002280145
(Song Peiyan, Feng Chaohui, Long Chenxiang, et al. Study on Discovery of Outstanding Scientific and Technological Talents in Specific Domains Based on Optimized Disruptive Index[J]. Journal of Intelligence, 2022, 41(5): 61-65.)
(Yang Jing, Wang Fang, Bai Rujiang. Evaluation of a Single Academic Paper’s Innovation Based on the Research Level[J]. Information Studies:Theory & Application, 2017, 40(9): 105-111, 76.)
(Yang Jing, Wang Fang, Bai Rujiang. A Method to Evaluate Academic Papers’ Innovation Based on the Research Theme Comparing[J]. Library and Information Service, 2018, 62(17): 75-83.)
doi: 10.13266/j.issn.0252-3116.2018.17.010
(Ren Haiying, Wang Deying, Wang Feifei. Relationship Between Novelty of Key-Term Combinations and Papers’ Scientific Impact[J]. Library and Information Service, 2017, 61(9): 87-93.)
doi: 10.13266/j.issn.0252-3116.2017.09.011
[42]
Ding Y, Liu X Z, Guo C, et al. The Distribution of References Across Texts: Some Implications for Citation Analysis[J]. Journal of Informetrics, 2013, 7(3): 583-592.
(Geng Shuqing, Yang Jianlin. A Method to Evaluate the Academic Influence of Papers Based on Citation Sentiment[J]. Information Studies: Theory & Application, 2018, 41(12): 93-98.)
doi: 10.16353/j.cnki.1000-7490.2018.12.017
[44]
Altmetric. What are Altmetrics?[EB/OL]. [2023-02-15]. https://www.altmetric.com/about-altmetrics/what-are-altmetrics/.
[45]
Priem J, Taraborelli D, Groth P, et al. Altmetrics: A Manifesto[EB/OL]. [2023-02-15]. http://altmetrics.org/manifesto.
(Wang Xianwen, Fang Zhichao, Wang Hongyin. Study on the Continuous, Dynamic and Comprehensive Article-Level Evaluation System[J]. Science of Science and Management of S.&T., 2015, 36(8): 37-48.)
[47]
陈建青. 对我国学术论文创新性评审的几点思考[J]. 青年记者, 2013(18): 33-35.
[47]
(Chen Jianqing. Some Thoughts on the Innovation Evaluation of Chinese Academic Papers[J]. Youth Journalist, 2013(18): 33-35.)
(Luo Zhuoran, Wang Yuqi, Qian Jiajia, et al. Research Review on Innovation Evaluation of Academic Papers[J]. Journal of the China Society for Scientific and Technical Information, 2021, 40(7): 780-790.)
[49]
Uzzi B, Mukherjee S, Stringer M, et al. Atypical Combinations and Scientific Impact[J]. Science, 2013, 342(6157): 468-472.
doi: 10.1126/science.1240474
pmid: 24159044
[50]
Bornmann L, Tekles A. Disruptive Papers Published in Scientometrics[J]. Scientometrics, 2019, 120(1): 331-336.
doi: 10.1007/s11192-019-03113-z
[51]
Bu Y, Waltman L, Huang Y. A Multidimensional Framework for Characterizing the Citation Impact of Scientific Publications[J]. Quantitative Science Studies, 2021, 2(1): 155-183.
[52]
Bornmann L, Tekles A. Convergent Validity of Several Indicators Measuring Disruptiveness with Milestone Assignments to Physics Papers by Experts[J]. Journal of Informetrics, 2021, 15(3): 101159.
[53]
Leydesdorff L, Tekles A, Bornmann L. A Proposal to Revise the Disruption Indicator[J]. El Profesional de la Información, 2021, 30(1): e300121.
(Liu Xiaohui, Shen Zhesi, Liao Yu, et al. The Research about the Improved Disruption Index and Its Influencing Factors[J]. Library and Information Service, 2020, 64(24): 84-91.)
doi: 10.13266/j.issn.0252-3116.2020.24.010
(Liu Xiaohui, Liao Yu, Zhu Manman. A Preliminary Study on the Application of Disruption Index to Scientific Research Evaluation[J]. Information Studies: Theory & Application, 2021, 44(12): 34-40.)
APS. APS Data Sets for Research[EB/OL]. [2023-02-15]. https://journals.aps.org/datasets.
[58]
The Nobel Prize. All Nobel Prizes in Physics[EB/OL]. [2023-02-15]. https://www.nobelprize.org/prizes/lists/all-nobel-prizes-in-physics/.
[59]
Li J C, Yin Y A, Fortunato S, et al. A Dataset of Publication Records for Nobel Laureates[J]. Scientific Data, 2019, 6: 33.
doi: 10.1038/s41597-019-0033-6
pmid: 31000709
[60]
Sinatra R, Wang D S, Deville P, et al. Quantifying the Evolution of Individual Scientific Impact[J]. Science, 2016, 354(6312): aaf5239.
[61]
Shen H W, Barabási A L. Collective Credit Allocation in Science[J]. Proceedings of the National Academy of Science, 2014, 111(34): 12325-12330.
(Chinese Academy of Sciences. Academician Information[EB/OL]. [2023-02-15]. https://casad.cas.cn/sourcedb_ad_cas/zw2/ysxx/hxb/201312/t20131219_4001995.html.)
[63]
Wikipedia. Jorge E. Hirsch[EB/OL]. [2023-02-15]. https://en.wikipedia.org/wiki/Jorge_E._Hirsch.
[64]
Qiao B T, Wang A Q, Yang X F, et al. Single-Atom Catalysis of CO Oxidation Using Pt1/FeOx[J]. Nature Chemistry, 2011, 3(8): 634-641.
doi: 10.1038/nchem.1095
pmid: 21778984
[65]
Yang H B, Hung S F, Liu S, et al. Atomically Dispersed Ni(I) as the Active Site for Electrochemical CO2 Reduction[J]. Nature Energy, 2018, 3(2): 140-147.
[66]
Ji N, Zhang T, Zheng M Y, et al. Direct Catalytic Conversion of Cellulose into Ethylene Glycol Using Nickel-Promoted Tungsten Carbide Catalysts[J]. Angewandte Chemie(International Ed in English), 2008, 47(44): 8510-8513.
[67]
The Canadian Academy of Engineering. Fifty-Two New Fellows Elected into the Canadian Academy of Engineering[EB/OL]. [2023-02-15]. https://www.cae-acg.ca/fifty-two-new-fellows-elected-into-the-canadian-academy-of-engineering/.
[68]
Zhao X C, Zhang Q, Chen C M, et al. Aromatic Sulfide, Sulfoxide, and Sulfone Mediated Mesoporous Carbon Monolith for Use in Supercapacitor[J]. Nano Energy, 2012, 1(4): 624-630.
[69]
Liu X Y, Wang A Q, Wang X D, et al. Au-Cu Alloy Nanoparticles Confined in SBA-15 as a Highly Efficient Catalyst for CO Oxidation[J]. Chemical Communications, 2008(27): 3187-3189.
doi: 10.1039/b804362k
pmid: 18594737
[70]
Hirsch J E. Spin Hall Effect[J]. Physical Review Letters, 1999, 83(9): 1834-1837.
[71]
Wikipedia. Spin Hall Effect[EB/OL]. [2023-02-15]. https://en.wikipedia.org/wiki/Spin_Hall_effect.
[72]
Hirsch J E. Attractive Interaction and Pairing in Fermion Systems with Strong On-Site Repulsion[J]. Physical Review Letters, 1985, 54(12): 1317-1320.
pmid: 10030994
[73]
Hirsch J E, Fye R M. Monte Carlo Method for Magnetic Impurities in Metals[J]. Physical Review Letters, 1986, 56(23): 2521-2524.
pmid: 10033014