1(National Science Library, Chinese Academy of Sciences, Beijing 100190, China) 2(University of Chinese Academy of Sciences, Beijing 100049, China) 3(Chinese Academy of Social Science Evaluation Studies, Chinese Academy of Social Sciences, Beijing 100732, China)
[Objective] This paper aims to construct a novelty index to evaluate the academic achievements. [Methods] First, we proposed a model to calculate content eigenfactor based on deep learning (Doc2Vec) and Hidden Markov Model. Then, we built the topic novelty measure index. Finally, we examined the proposed method with academic papers published by three Chinese LIS journals in 2014. [Results] Compared with the existing methods, the proposed model measured the topic novelty more effectively. [Limitations] Our empirical research only examined abstracts of the academic papers. [Conclusions] The proposed method could help us evaluate and monitor scholarly research.
(Yang Feng, Liang Liang, Gou Qinglong, et al.Origination and Improvement for the Pitfalls of Peer Review Systems[J]. Studies in Science of Science, 2008, 26(3): 569-572.)
(Shen Lv.A General Equilibrium Theory of the Science & Technology Innovation: On Scientometrics Analysis of the Science & Technology Innovation[J]. Studies in Science of Science, 2003, 21(2): 205-209.)
(Zhu Daming.Main Roles of References and Appraisal of Innovation of Academic Papers[J]. Acta Editologica, 2004, 16(2): 91-92.)
doi: 10.3969/j.issn.1001-4314.2004.02.006
(Qi Yanli, Dou Xiqian, Xia Huichuan.Limitations of SCI Quantitative Evaluation Function & Its Modifications[J]. Information Studies: Theory & Application, 2009, 32(9): 48-52.
[9]
Seglen P O.Why the Impact Factor of Journals Should Not Be Used for Evaluating Research[J]. BMJ, 1997, 314(7079): 498-502.
doi: 10.1136/bmj.314.7079.498
pmid: 9056805
[10]
Leydesdorff L, Bornmann L, Comins J, et al. Citations: Indicators of Quality? The Impact Fallacy[J]. Frontiers in Research Metrics and Analysis, 2016, 1: Article 1.
(Shang Hairu, Feng Changgen, Sun Liang.Evaluation of Academic Papers with Academic Influence — Proposing Two New Indicators of Academic Inheritance Effect and Long-term Citation[J]. Chinese Science Bulletin, 2016, 61(26): 2853-2860.)
(Wu Qin.Research on Quality Evaluation in the Academic Articles Based on the Intensity of Citation[J]. Journal of the China Society for Scientific and Technical Information, 2007, 26(4): 522-526.)
doi: 10.3969/j.issn.1000-0135.2007.04.007
[13]
Hirsch J E.An Index to Quantify an Individual’s Scientific Research Output[J]. Proceedings of the National Academy of Sciences of the United States of America, 2005, 102(46): 16569-16572.
doi: 10.1073/pnas.0507655102
pmid: 16275915
(Qian Lingfei, Yang Jianlin, Zhang Li.Comparison of the Discipline Innovation Based on Keyword Analysis — Take the Information Science and Library Sciences as an Example[J]. Information Studies: Theory & Application, 2011, 34(1): 117-120.)
(Yang Jianlin, Qian Lingfei.A Novel Measurement Method of the Theme Based on Inverse Document Frequency of Keywords[J]. Information Studies: Theory & Application, 2013, 36(3): 99-102.)
[17]
Mase H, Matsubayashi T, Ogawa Y, et al.Proposal of Two-stage Patent Retrieval Method Considering the Claim Structure[J]. ACM Transactions on Asian Language Information Processing, 2005, 4(2): 190-206.
doi: 10.1145/1105696.1105702
(Liu Yuqin, Zhu Donghua, Lv Lin.Technology Maturity of Product Forecasting Based on Text Mining[J]. Computer Integrated Manufacturing Systems, 2008, 14(3): 506-510.)
doi: 10.1016/j.commatsci.2008.03.016
[19]
Meng L, Huang R, Gu J.A Review of Semantic Similarity Measures in WordNet[J]. International Journal of Hybrid Information Technology, 2013, 6(1): 1-12.
[20]
孙润志. 基于语义理解的文本相似度计算研究与实现[D]. 北京: 中国科学院大学, 2015.
[20]
(Sun Runzhi.Research and Implementation of Text Similarity Computing Based on Semantic Understanding[D]. Beijing: University of Chinese Academy of Sciences, 2015.)
[21]
Word2Vec [EB/OL]. [2017-12-05]..
[22]
Le Q V, Mikolov T.Distributed Representations of Sentences and Documents [OL]. arXiv Preprint, arXiv: 1405.405302.
[23]
Lee S, Jin X, Kim W.Sentiment Classification for Unlabeled Dataset Using Doc2Vec with JST[C]// Proceedings of the 18th Annual International Conference on Electronic Commerce: E-Commerce in Smart Connected World. ACM, 2016: 28.
[24]
Maslova N, Potapov V.Neural Network Doc2vec in Automated Sentiment Analysis for Short Informal Texts[A]// Lecture Notes in Computer Science[M]. Springer, 2017: 546-554.
(Lu Wanhui.Research on Measuring Academic Journals Topics Homogenization Based on Deep Learning[J]. Information and Documentation Services, 2017, 38(5): 105-112.)
doi: 10.3969/j.issn.1002-0314.2017.05.016
[26]
Sheldon D.Manipulation of Pagerank and Collective Hidden Markov Models[D]. Cornell University, 2010.
[27]
荆林波. 中国人文社会科学期刊评价报告[M]. 北京: 中国社会科学出版社, 2015.
[27]
(Jing Linbo.Evaluation Report of Chinese Humanities and Social Sciences Journals [M]. Beijing: China Social Sciences Press, 2015.)