Please wait a minute...
Data Analysis and Knowledge Discovery  2018, Vol. 2 Issue (6): 58-69    DOI: 10.11925/infotech.2096-3467.2018.0354
Current Issue | Archive | Adv Search |
The Correlation Between Altmetrics and Citations
Wu Pengmin1,2, Chen Ting1,2,3, Wang Xiaomei3()
1National Science Library, Chinese Academy of Sciences, Beijing 100190, China
2University of Chinese Academy of Sciences, Beijing 100049, China
3Institutes of Science and Development, Chinese Academy of Sciences, Beijing 100190, China
Download: PDF (4743 KB)   HTML ( 3 ) 
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper studies the characteristics of the Altmetrics for high quality journal articles, including their correlations with citation numbers, differences in disciplines, and the contribution of sub-indicators. These Altmetrics are also compared with previous results. [Methods] We selected 68 journals from Nature Index as data sources, and used machine learning method to classify papers published by them. Then, we used Spearman correlation test to find relationship between Altmetrics and traditional citation indexes, as well as the contributions of sub-indicators in various disciplines. Finally, we evaluated the effectiveness of using Altmetrics to identify highly-cited papers, with the help of ROC curve analysis. [Results] There were significant differences in the performance of Altmetrics among disciplines. In high-quality journals, the correlation between Altmetrics and citations were enhanced, and the contributions of News, Blog, and Twitter to the Altmetrics were also increased. Altmetrics could help us identify highly cited papers. [Limitations] The data collection period is short, and the data set needs to be expanded based on the characteristics of the disciplines. [Conclusions] Compared with previous research results of full data sets, Altmetrics for high-quality journal articles are unique, and the correlation between Altmetrics and citations is enhanced.
Key words： Altmetrics Indicators      Citation Counts      Correlation Analysis      ROC Curve Analysis     
Received: 29 March 2018      Published: 11 July 2018
ZTFLH:  P315 G312  
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Pengmin Wu
Ting Chen
Xiaomei Wang

Cite this article:

Wu Pengmin,Chen Ting,Wang Xiaomei. The Correlation Between Altmetrics and Citations. Data Analysis and Knowledge Discovery, 2018, 2(6): 58-69.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2018.0354     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2018/V2/I6/58

指标 TC AS MS NS BS PS WS TS
最大值 4 901 6 422 13 421 256 124 2 452 10 439
最小值 0.00 0.25 0.00 0.00 0.00 0.00 0.00 0.00
中位数 17 3 32 0 0 0 0 2
均 值 29.79 28.32 61.04 2.00 0.68 0.01 0.12 11.67
标准差 53.98 96.40 152.79 6.64 2.24 0.10 2.09 68.56
覆盖率 99.2% 100.0% 99.7% 29.6% 25.0% 1.0% 6.6% 92.7%
学科 N TC AS MS NS BS PS WS TS
化学 19 587 99.4% 100% 99.9% 15.2% 17.9% 0.3% 2.8% 91.7%
物理学 10 494 98.7% 100% 99.5% 27.9% 17.5% 0.2% 4.6% 89.6%
天文学与天体物理学 6 054 97.8% 100% 98.5% 23.5% 20.4% 0.1% 13.7% 87.8%
地球与环境科学 6 088 98.8% 100% 99.6% 41.6% 37.0% 0.3% 6.6% 93.8%
古生物学 639 99.4% 100% 100.0% 74.2% 66.2% 1.6% 35.4% 98.7%
生命科学 24 951 99.6% 100% 99.9% 39.1% 30.8% 2.3% 7.9% 95.5%
对比项目 TC AS NS BS PS WS TS MS
Kolmogorov-Smirnov Z 75.65 100.37 99.33 99.11 138.02 124.33 112.60 89.78
渐进显著性(双侧) 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Spearman相关系数 68种期刊 3种综合性期刊 增长率
AS .362** .458** 26.5%
MS .640** .691** 8.0%
NS .302** .386** 27.8%
BS .284** .404** 42.3%
PS .078** .128** 64.1%
WS .147** .247** 68.0%
TS .280** .451** 61.1%
学科 化学 物理学 天文学与天体物理学 地球与环境科学 古生物学 生命科学
AS 68种期刊 .227** .396** .271** .509** .395** .463**
3种综合性期刊 .494** .490** .458** .417** .482** .493**
增长率 117.6% 23.7% 69.0% -18.1% 22.0% 6.5%
MS 68种期刊 .642** .658** .633** .695** .674** .714**
3种综合性期刊 .738** .784** .533** .675** .703** .707**
增长率 15.0% 19.2% -15.8% -2.9% 4.3% -1.0%
NS 68种期刊 .209** .342** .237** .435** .365** .361**
3种综合性期刊 .437** .448** .442* .365** .460* .399**
增长率 109.1% 31.0% 86.5% -16.1% 26.0% 10.5%
BS 68种期刊 .216** .298** .230** .411** .408** .331**
3种综合性期刊 .410** .440** .434** .372** .483** .426**
增长率 89.8% 47.7% 88.7% -9.5% 18.4% 28.7%
PS 68种期刊 .026** 0.019 0.004 0.014 0.059 .113**
3种综合性期刊 0.006 -0.016 0.03 0.011 0.097 .170**
增长率 -76.9% -184.2% 650.0% -21.4% 64.4% 50.4%
WS 68种期刊 .081** .171** .151** .181** .355** .196**
3种综合性期刊 .206** .275** .343** .177** .412** .268**
增长率 154.3% 60.8% 127.2% -2.2% 16.1% 36.7%
TS 68种期刊 .069** .225** .187** .479** .387** .431**
3种综合性期刊 .465** .446** .282** .421** .495** .502**
增长率 573.9% 98.2% 50.8% -2.2% 27.9% 16.5%
指标 AS MS NS BS PS WS TS
AUC 0.796 0.934 0.754 0.747 0.532 0.654 0.742
学科 化学 物理学 天文学与天体物理学 地球与环境科学 古生物学 生命科学
AS 0.728 0.862 0.703 0.928 0.955 0.893
MS 0.955 0.961 0.930 0.961 0.999 0.943
NS 0.711 0.791 0.626 0.896 0.937 0.820
BS 0.646 0.783 0.659 0.906 0.956 0.840
PS 0.509 0.499 0.499 0.498 0.492 0.566
WS 0.568 0.729 0.627 0.684 0.818 0.702
TS 0.608 0.770 0.721 0.910 0.959 0.905
Spearman相关系数 68种期刊 3种综合性期刊 增长率
NS .780** .927** 18.8%
BS .672** .837** 24.6%
PS .093** .113** 21.5%
WS .253** .339** 34.0%
TS .674** .807** 19.7%
学科 化学 物理学 天文学与天体物理学 地球与环境科学 古生物学 生命科学
NS 68种期刊 .604** .782** .731** .866** .963** .837**
3种综合性期刊 .941** .959** .981** .954** .972** .895**
增长率 55.8% 22.6% 34.2% 10.2% 0.9% 6.9%
BS 68种期刊 .600** .606** .657** .747** .906** .712**
3种综合性期刊 .794** .813** .865** .840** .917** .812**
增长率 32.3% 34.2% 31.7% 12.4% 1.2% 14.0%
PS 68种期刊 .042** .035** .029* 0.022 .120** .111**
3种综合性期刊 0.018 0.050 0.034 0.024 0.110 .163**
增长率 -57.1% 42.9% 17.2% 9.1% -8.3% 46.8%
WS 68种期刊 .193** .234** .345** .269** .501** .238**
3种综合性期刊 .252** .322** .481** .327** .547** .299**
增长率 30.6% 37.6% 39.4% 21.6% 9.2% 25.6%
TS 68种期刊 .503** .486** .495** .761** .866** .769**
3种综合性期刊 .728** .701** .739** .775** .871** .819**
增长率 44.7% 44.2% 49.3% 1.8% 0.6% 6.5%
[1] Bornmann L.Do Altmetrics Point to the Broader Impact of Research? An Overview of Benefits and Disadvantages of Altmetrics[J]. Journal of Informetrics, 2014, 8(4): 895-903.
doi: 10.1016/j.joi.2014.09.005
[2] Altmetrics Top 100 Articles [DS/OL]. [2017-12-16]..
[3] 中国科学院科技战略咨询研究院. 2016全球最受公众关注的科学成果[R/OL]. (2017-06-17). [2017-12-16]. .
[3] (Institute of Science and Development, Chinese Academy of Sciences. 2016 The World’s Most Popular Scientific Achievements [R/OL]. (2017-06-17). [2017-12-16].
[4] 王鹏飞, 刘烜贞. 基于内容分析的Altmetrics 本质研究[J].图书情报工作, 2017, 61(2): 114-120.
[4] (Wang Pengfei, Liu Xuanzhen.Research on Altmetrics’Essence Based on Content Analysis[J]. Library and Information Service, 2017, 61(2): 114-120.)
[5] 王睿, 胡文静, 郭玮. 高Altmetrics指标科技论文学术影响力研究[J]. 图书情报工作, 2014, 58(21): 92-98.
doi: 10.13266/j.issn.0252-3116.2014.21.013
[5] (Wang Rui, Hu Wenjing, Guo Wei.Research on Academic Influence of High Altmetrics Sci-Tech Papers[J]. Library and Information Service, 2014, 58(21): 92-98.)
doi: 10.13266/j.issn.0252-3116.2014.21.013
[6] 卫垌圻, 谭宗颖. Altmetrics国内外研究中的问题与挑战[J].图书情报工作, 2015, 59(2): 93-99.
doi: 10.13266/j.issn.0252-3116.2015.02.015
[6] (Wei Tongqi, Tan Zongying.The Problems and Challenges of Research on Altmetrics[J]. Library and Information Service, 2015, 59(2): 93-99.)
doi: 10.13266/j.issn.0252-3116.2015.02.015
[7] Barilan J, Haustein S, Peters I, et al.Beyond Citations: Scholars’ Visibility on the Social Web[OL]. arXiv Preprint, arXiv: 1205.5611.
[8] Haustein S, Peters I, Sugimoto C R, et al.Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature[J]. Journal of the American Society for Information Science and Technology, 2014, 65(4): 656-669.
doi: 10.1002/asi.23101
[9] Ortega J L.Relationship Between Altmetric and Bibliometric Indicators Across Academic Social Sites: The Case of CSIC’s Members[J]. Journal of Informetrics, 2015, 9(1): 39-49.
doi: 10.1016/j.joi.2014.11.004
[10] Sotudeh H, Mazarei Z, Mirzabeigi M.CiteULike Bookmarks are Correlated to Citation at Journal and Author Levels in Library and Information Science[J]. Scientometrics, 2015, 105(3): 2237-2248.
doi: 10.1007/s11192-015-1745-9
[11] Costas R, Zahedi Z, Wouters P.Do “Altmetrics” Correlate with Citations? Extensive Comparison of Altmetric Indicators with Citations from a Multidisciplinary Perspective[J]. Journal of the Association for Information Science and Technology, 2015, 66(10): 2003-2019.
doi: 10.1002/asi.23309
[12] Costas R, Zahedi Z, Wouters P.The Thematic Orientation of Publications Mentioned on Social Media: Large-Scale Disciplinary Comparison of Social Media Metrics with Citations[J]. Aslib Journal of Information Management, 2015, 67(3): 260-288.
doi: 10.1108/AJIM-12-2014-0173
[13] Hassan S U, Imran M, Gillani U, et al.Measuring Social Media Activity of Scientific Literature: An Exhaustive Comparison of Scopus and Novel Altmetrics Big Data[J]. Scientometrics, 2017, 113(2): 1037-1057.
doi: 10.1007/s11192-017-2512-x
[14] 由庆斌, 汤珊红. 不同类型论文层面计量指标间的相关性研究[J]. 图书情报工作, 2014, 58(8): 79-84.
doi: 10.13266/j.issn.0252-3116.2014.08.013
[14] (You Qingbin, Tang Shanhong.Study on Correlation of Different Article-level Metrics[J]. Library and Information Service, 2014, 58(8): 79-84.)
doi: 10.13266/j.issn.0252-3116.2014.08.013
[15] 刘晓娟, 周建华, 尤斌. 基于Mendeley与WoS的选择性计量指标与传统科学计量指标相关性研究[J]. 图书情报工作, 2015, 59(3): 112-118.
doi: 10.13266/j.issn.0252-3116.2015.03.016
[15] (Liu Xiaojuan, Zhou Jianhua, You Bin.Study on Correlation Between Altmetrics Indicators and Traditional Scientometric Indicators Based on Mendeley and WoS[J]. Library and Information Service, 2015, 59(3): 112-118.)
doi: 10.13266/j.issn.0252-3116.2015.03.016
[16] 余厚强. 替代计量指标与引文量相关性的大规模跨学科研究——数值类型、指标种类与用户类别的影响[J]. 情报学报, 2017, 36(6): 606-617.
[16] (Yu Houqiang.Large Scale Cross-Disciplinary Study of Correlation Between Altmetrics Indicators and Citations-Influence of Data Types, Indicator Types, and User Categories[J]. Journal of the China Society for Scientific and Technical Information, 2017, 36(6): 606-617.)
[17] Nature Index[DS/OL].[2017-12-16]..
[18] 刘晓娟, 宋婉姿. 基于PLOS ALM的Altmetrics指标可用性分析[J]. 图书情报工作, 2016, 60(4): 93-101.
doi: 10.13266/j.issn.0252-3116.2016.04.013
[18] (Liu Xiaojuan, Song Wanzi.Analysis on the Usability of Altmetrics Indicators Based on PLOS ALM[J]. Library and Information Service, 2016, 60(4): 93-101.)
doi: 10.13266/j.issn.0252-3116.2016.04.013
[19] 刘艳民. Altmetrics指标与传统文献计量指标相关性研究[J]. 情报杂志, 2017, 36(9): 71-77.
doi: 10.3969/j.issn.1002-1965.2017.09.012
[19] (Liu Yanmin.The Correlation Research of Altmetrics and Bibliometric Indicators[J]. Journal of Intelligence, 2017, 36(9): 71-77.)
doi: 10.3969/j.issn.1002-1965.2017.09.012
[20] 刘晓娟, 宰冰欣. 图书情报领域文献的Altmetrics指标分析[J]. 图书情报工作, 2015, 59(18): 108-116.
doi: 10.13266/j.issn.0252-3116.2015.18.017
[20] (Liu Xiaojuan, Zai Bingxin.Analysis of Altmetrics Indicators in the Field of Library and Information Science[J]. Library and Information Service, 2015, 59(18): 108-116.)
doi: 10.13266/j.issn.0252-3116.2015.18.017
[21] 由庆斌, 韦博, 汤珊红. 基于补充计量学的论文影响力评价模型构建[J]. 图书情报工作, 2014, 58(22): 5-11.
doi: 10.13266/j.issn.0252-3116.2014.22.001
[21] (You Qingbin, Wei Bo, Tang Shanhong.Evaluation Model Construction to Evaluate Article’s Influence Based on Altmetrics[J]. Library and Information Service, 2014, 58(22): 5-11.)
doi: 10.13266/j.issn.0252-3116.2014.22.001
[22] 时立文. SPSS19.0统计分析从入门到精通[M]. 北京: 清华大学出版社, 2012.
[22] (Shi Liwen.SPSS19.0 Statistical Analysis from Getting Started to Proficient[M]. Beijing: Tsinghua University Press, 2012.)
[23] 王妍, 郭舒, 张建勇. 学者影响力评价指标的相关性研究[J]. 图书情报工作, 2015, 59(3): 106-112, 127.
[23] (Wang Yan, Guo Shu, Zhang Jianyong.Study on Correlation of Researcher Influence Indicators[J]. Library and Information Service, 2015, 59(3): 106-112,127.)
[24] 张建国. ROC曲线分析的基本原理以及在体质与健康促进研究中的应用[J]. 体育科学, 2008, 28(6): 62-66.
doi: 10.3969/j.issn.1000-677X.2008.06.010
[24] (Zhang Jianguo.The Basic Principle of ROC Curve Analysis and Application in Physical Fitness and Health Promotion Studies[J]. China Sport Science, 2008, 28(6): 62-66.)
doi: 10.3969/j.issn.1000-677X.2008.06.010
[25] Baynes R D, Cook J D, Bothwell T H, et al.Serum Transferrin Receptors in Hereditary Hemochromatosis and African Siderosis[J]. American Journal of Hematology, 1994, 45(4): 288-292.
doi: 10.1002/ajh.2830450404 pmid: 8178799
[26] 杨柳, 陈贡. Altmetrics视角下科研机构影响力评价指标的相关性研究[J]. 图书情报工作, 2015, 59(15): 106-114, 132.
doi: 10.13266/j.issn.0252-3116.2015.15.015
[26] (Yang Liu, Chen Gong.Correlation Study on Evaluation Indicators of Scientific Institution Influence from the Perspective of Altmetrics[J]. Library and Information Service, 2015, 59(15): 106-114, 132.)
doi: 10.13266/j.issn.0252-3116.2015.15.015
[1] Ming Yi,Tingting Zhang. Ranking Answer Quality of Popular Q&A Community[J]. 数据分析与知识发现, 2019, 3(6): 12-20.
[2] Jiang Wu,Yinghui Zhao,Jiahui Gao. Research on Weibo Opinion Leaders Identification and Analysis in Medical Public Opinion Incidents[J]. 数据分析与知识发现, 2019, 3(4): 53-62.
[3] Xinrui Wang,Yue He. Predicting Stock Market Fluctuations with Social Media Behaviors: Case Study of Sina Finance Blog[J]. 数据分析与知识发现, 2019, 3(11): 108-119.
[4] Wang Daoping,Jiang Zhongyang,Zhang Boqing. Collaborative Filtering Algorithm Based on Gray Correlation Analysis and Time Factor[J]. 数据分析与知识发现, 2018, 2(6): 102-109.
[5] Li Dong,Tong Shouchuan,Li Jiang. Analyzing Interdisciplinarity and Scientists’ Academic Impacts[J]. 数据分析与知识发现, 2018, 2(12): 1-11.
[6] He Yue,Zhu Can. Sentiment Analysis of Weibo Opinion Leaders——Case Study of “Illegal Vaccine” Event[J]. 数据分析与知识发现, 2017, 1(9): 65-73.
[7] Yu Liping. New Method to Evaluate Academic Journals: Case Study of Mathematics Journals[J]. 现代图书情报技术, 2016, 32(7-8): 94-100.
[8] Bi Dayu, Xia Xiaoxu, Wang Jing. The Analysis of Credit Standing of E-businessman Based on the Data Mining of Users' Online Evaluation[J]. 现代图书情报技术, 2014, 30(7): 77-83.
[9] Zhou Qun, Zuo Wenge, Chen Shiji. Review on Percentile-based Bibliometric Indicator[J]. 现代图书情报技术, 2013, 29(7/8): 82-88.
[10] Chen Shiji, Shi Liwen, Li Dongmei, Zuo Wenge. Review on Normalization Methods of Citation Counts[J]. 现代图书情报技术, 2012, 28(4): 54-60.
[11] Kan Detao. Design and Implementation of Service Quality Analysis System[J]. 现代图书情报技术, 2009, 25(5): 76-80.
[12] Huang Yuanyuan,Zhu Donghua,Ren Zhijun,Zhang Cheng . Applicatian Study of Correlation Analysis in Patent Information Analysis[J]. 现代图书情报技术, 2006, 1(10): 60-65.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn