Please wait a minute...
Data Analysis and Knowledge Discovery  2021, Vol. 5 Issue (12): 48-59    DOI: 10.11925/infotech.2096-3467.2021.0679
Current Issue | Archive | Adv Search |
Automatic Classification of Citation Sentiment and Purposes with AttentionSBGMC Model
Zhou Wenyuan,Wang Mingyang(),Jing Yu
College of Information and Computer Engineering, Northeast Forestry University, Harbin 150040, China
Download: PDF (1078 KB)   HTML ( 19 ) 
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper proposes a deep learning model——AttentionSBGMC to improve the automatic classification of citation sentiment and purposes. [Methods] First, we used the SciBERT pre-training model to obtain the semantic representation vector for the sentences. Then, according to the characteristics of the texts, we used the BiGRU neural network and the multi-scale convolutional neural network (Multi-CNN) to extract their temporal global features and local key features. Third, we utilized the attention model to highlight the key features by redistributing the extracted features’ weights. Finally, we finished the classification tasks with the help of linear layers. [Results] We examined the new method with two citation data sets. With Abu-Jbara data set the F1 values in three classification tasks (for subjective and objective citation emotion, positive and negative citation emotion, and citation purpose) were 86.74%, 91.14% and 84.92%, respectively. With Athar data set the F1 values in two classification tasks (for subjective and objective citation emotion, positive and negative citation emotion) were 88.50%, 86.59%, respectively. [Limitations] The proposed model was only examined on English data sets, which needs to be expanded in the future. [Conclusions] The proposed model could effectively extract the important corpus features, and automatically classify citation sentiment and purposes.
Key words： Citation Sentiment Classification      SciBERT      Attention Mechanism      BiGRU      Multi-CNN     
Received: 07 July 2021      Published: 20 January 2022
ZTFLH:  TP391  
Fund:National Natural Science Foundation of China(71473034)
Corresponding Authors: Wang Mingyang,ORCID：0000-0002-5022-6628     E-mail: wangmingyang@nefu.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Wenyuan Zhou
Mingyang Wang
Yu Jing

Cite this article:

Zhou Wenyuan, Wang Mingyang, Jing Yu. Automatic Classification of Citation Sentiment and Purposes with AttentionSBGMC Model. Data Analysis and Knowledge Discovery, 2021, 5(12): 48-59.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2021.0679     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2021/V5/I12/48

AttentionSBGMC Model
SciBERT Model
BiGRU Structure Diagram
Multi-CNN Structure Diagram
Schematic Diagram of Attention Mechanism
Schematic Diagram of the Original Data
Example of Citation Fragment Generation
分类任务 种类 比率/%
引用情感 正面 34.50
中性 51.10
负面 14.40
引用目的 批评 16.30
比较 8.10
使用
证实
基础
中立		 18.20
8.20
5.40
43.80
Data Set 1 Distribution
分类任务 种类 比率/%
正面 10.20
引用情感 中性 86.50
负面 3.30
Data Set 2 Distribution
Text Enhancement Example
实验参数 参数值
词嵌入维度 200
隐藏层大小 100
卷积核大小 1,2,3,4
注意力机制中单位数（维度） 64
注意力头数 5
优化器 Adam
Batch Size 16
Epoch 15
Dropout 0.25
Main Parameters
真实类别 预测类别
正例 反例
正例 TP（真正例） FN（假反例）
反例 FP（假正例） TN（真反例）
Confusion Matrix
实验方法 主客观分类
P/% R/% F1/%
GloVe-BiGRU 72.73 56.15 63.37
BERT-BiGRU 83.84 83.65 83.79
SciBERT-BiGRU 84.85 84.8 84.82
SciBERT-BiGRU-Multi-CNN 85.17 85.04 85.1
SciBERT-Multi-CNN-BiGRU-Attention 85.35 85.86 85.87
SciBERT-BiGRU-Multi-CNN-Attention 86.76 86.72 86.74
Subjective and Objective Classification of the Index Results of Each Model
实验方法 正负面分类
P/% R/% F1/%
GloVe-BiGRU 80.71 59.25 68.33
BERT-BiGRU 88.75 87.32 88.03
SciBERT-BiGRU 90.69 87.71 88.98
SciBERT-BiGRU-Multi-CNN 90.78 89.24 90.01
SciBERT-Multi-CNN-BiGRU-Attention 92.06 89.14 90.58
SciBERT-BiGRU-Multi-CNN-Attention 92.26 90.06 91.14
Positive and Negative Classification of the Index Results of Each Model
实验方法 引用目的分类
P/% R/% F1/%
Glove-BiGRU 68.26 52.18 59.15
BERT-BiGRU 82.79 79.98 81.26
SciBERT-BiGRU 83.26 80.39 81.80
SciBERT-BiGRU-Multi-CNN 84.68 81.59 83.11
SciBERT-Multi-CNN-BiGRU-Attention 85.58 82.75 84.14
SciBERT-BiGRU-Multi-CNN-Attention 86.67 83.24 84.92
The Citation Purpose Classification of the Index Results of Each Model
实验方法 引用情感分类
P/% R/% F1/%
NB with Syntactic Features[30] 69.00 62.50 64.40
SVM with Features [38] 67.10 70.60 68.80
SVM with TF-IDF [20] 77.90 76.30 77.10
SVM with Embedding [20] 81.30 75.40 77.30
CNN with Embedding [20] 82.00 75.90 78.80
LSTM[19] 80.08 74.30 77.40
BiLSTM [19] 80.40 77.56 79.10
SciBERT-BiGRU-Multi-CNN-Attention 83.76 82.63 83.19
The Citation Sentiment Classification of the Index Results of Each Model
实验方法 引用目的分类
P/% R/% F1/%
NB with Syntactic Features[30] 65.02 58.50 60.40
SVM with Features [38] 54.90 62.50 58.40
SVM with TF-IDF [20] 74.30 70.90 72.60
SVM with Embedding [20] 86.80 64.70 74.10
CNN with Embedding [20] 80.80 68.80 74.30
LSTM[19] 79.87 67.80 73.21
BiLSTM [19] 77.22 73.11 75.11
SciBERT-BiGRU-Multi-CNN-Attention 86.67 83.24 84.92
The Citation Purpose Classification of the Index Results of Each Model
实验任务 评价指标
P/% R/% F1/%
正负面分类 87.42 89.60 88.50
主客观分类 85.53 87.64 86.59
引用情感三分类 84.58 86.67 85.61
Classification Experiment Results of Data Set 2
[1] 吴勤. 基于引证强度的学术论文质量评价方法研究[J]. 情报学报, 2007, 26(4): 522-526.
[1] (Wu Qin. Research on Quality Evaluation in the Academic Articles Based on the Intensity of Citation[J]. Journal of the China Society for Scientific and Technical Information, 2007, 26(4): 522-526.)
[2] 王岩, 刘雅娟. 运用引文分析进行论文评价的方法初探: 引文分析中期刊影响因子的运用[J]. 科研管理, 2001, 22(1): 133-138.
[2] (Wang Yan, Liu Yajuan. Exploration on Assessing Paper by Citation Analysis: Using Impact Factors in Citation Analysis[J]. Science Research Management, 2001, 22(1): 133-138.)
[3] 钟文一, 陈云鹏. 基于引证系数的论文影响力评价方法研究[J]. 情报科学, 2011, 29(5): 706-712.
[3] (Zhong Wenyi, Chen Yunpeng. Research on Influence Evaluation in the Academic Articles Based on the Citation Index[J]. Information Science, 2011, 29(5): 706-712.)
[4] 叶鹰. 高品质论文被引数据及其对学术评价的启示[J]. 中国图书馆学报, 2010, 36(1): 100-103.
[4] (Ye Ying. An Outline of Academic Assessment with the Citation Data of High-Quality Papers[J]. Journal of Library Science in China, 2010, 36(1): 100-103.)
[5] Sendhilkumar S, Elakkiya E, Mahalakshmi G S. Citation Semantic Based Approaches to Identify Article Quality [C]// Proceedings of the 3rd International Conference on Computer Science, Engineering & Applications. 2013. DOI: 10.5121/CSIT.2013.3543.
doi: 10.5121/CSIT.2013.3543
[6] Parthasarathy G, Tomar D C. Sentiment Analyzer: Analysis of Journal Citations from Citation Databases [C]//Proceedings of the 5th International Conference-Confluence the Next Generation Information Technology Summit (Confluence). IEEE, 2014: 923-928.
[7] Goodarzi M, Mahmoudi M T, Zamani R. A Framework for Sentiment Analysis on Schema-Based Research Content via Lexica Analysis [C]//Proceedings of the 7th International Symposium on Telecommunications. IEEE, 2014: 405-411.
[8] Baccianella S, Esuli A, Sebastiani F. SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining [C]//Proceedings of the International Conference on Language Resources and Evaluation. 2010.
[9] 廖君华, 刘自强, 白如江, 等. 基于引文内容分析的引用情感识别研究[J]. 图书情报工作, 2018, 62(15): 112-121.
[9] (Liao Junhua, Liu Ziqiang, Bai Rujiang, et al. Citation Sentiment Recognition Method Based on Citation Content Analysis[J]. Library and Information Service, 2018, 62(15): 112-121.)
[10] Athar A. Sentiment Analysis of Citations Using Sentence Structure-Based Features [C]//Proceedings of the ACL 2011 Student Session. 2011: 81-87.
[11] Athar A, Teufel S. Detection of Implicit Citations for Sentiment Detection [C]//Proceedings of the Workshop on Detecting Structure in Scholarly Discourse. 2012: 18-26.
[12] Radev D R, Muthukrishnan P, Qazvinian V, et al. The ACL Anthology Network Corpus[J]. Language Resources and Evaluation, 2013, 47(4): 919-944.
doi: 10.1007/s10579-012-9211-2
[13] Sula C A, Miller M. Citations, Contexts, and Humanistic Discourse: Toward Automatic Extraction and Classification[J]. Literary and Linguistic Computing, 2014, 29(3): 452-464.
doi: 10.1093/llc/fqu019
[14] Kim I C, Thoma G R. Automated Classification of Author’s Sentiments in Citation Using Machine Learning Techniques: A Preliminary Study [C]//Proceedings of 2015 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology. IEEE, 2015: 1-7.
[15] Ikram M T, Afzal M T. Aspect Based Citation Sentiment Analysis Using Linguistic Patterns for Better Comprehension of Scientific Knowledge[J]. Scientometrics, 2019, 119(1): 73-95.
doi: 10.1007/s11192-019-03028-9
[16] 刘盛博, 丁堃. 基于引用内容的引文评价分析[C]// 第九届中国科技政策与管理学术年会论文集. 2013: 7.
[16] (Liu Shengbo, Ding Kun. Citation Evaluation Analysis Based on Citation Context[C]// Proceedings of the the 9th China Science and Technology Policy and Management Annual Conference. 2013: 7.)
[17] Xu J, Zhang Y, Wu Y, et al. Citation Sentiment Analysis in Clinical Trial Papers[J]. AMIA Annual Symposium Proceedings, 2015:1334-1341.
[18] 冷东天. 基于语义的引用内容情感分析及其应用研究[D]. 哈尔滨: 东北林业大学, 2020.
[18] (Leng Dongtian. Research on Citation Sentiment Analysis Based on Semantics in Citation Context and Its Application[D]. Harbin: Northeast Forestry University, 2020.)
[19] Munkhdalai T, Lalor J, Yu H. Citation Analysis with Neural Attention Models [C]//Proceedings of the the 7th International Workshop on Health Text Mining and Information Analysis. 2016: 69-77.
[20] Lauscher A, Glavaš G, Ponzetto S P, et al. Investigating Convolutional Networks and Domain-Specific Embeddings for Semantic Classification of Citations [C]//Proceedings of the 6th International Workshop on Mining Scientific Publications. 2017: 24-28.
[21] Vyas V, Ravi K, Ravi V, et al. Article Citation Study: Context Enhanced Citation Sentiment Detection[OL]. arXiv Preprint, arXiv:2005.04534.
[22] Brooks T A. Private Acts and Public Objects: An Investigation of Citer Motivations[J]. Journal of the American Society for Information Science, 1985, 36(4): 223-229.
doi: 10.1002/(ISSN)1097-4571
[23] Tang R, Safer M A. Author-Rated Importance of Cited References in Biology and Psychology Publications[J]. Journal of Documentation, 2008, 64(2): 246-272.
doi: 10.1108/00220410810858047
[24] 马凤, 武夷山. 关于论文引用动机的问卷调查研究: 以中国期刊研究界和情报学界为例[J]. 情报杂志, 2009, 28(6): 9-14.
[24] (Ma Feng, Wu Yishan. A Survey Study on Motivations for Citation[J]. Journal of Intelligence, 2009, 28(6): 9-14.)
[25] 邱均平, 陈晓宇, 何文静. 科研人员论文引用动机及相互影响关系研究[J]. 图书情报工作, 2015, 59(9): 36-44.
[25] (Qiu Junping, Chen Xiaoyu, He Wenjing. Study on Paper Citation Motivations and Mutual Influence of Researchers[J]. Library and Information Service, 2015, 59(9): 36-44.)
[26] Garfield E. Can Citation Indexing be Automated?[J]. Essays of an Information Scientist, 1962, 1: 84-90.
[27] Lipetz B A. Improvement of the Selectivity of Citation Indexes to Science Literature Through Inclusion of Citation Relationship Indicators[J]. American Documentation, 1965, 16(2): 81-90.
doi: 10.1002/(ISSN)1936-6108
[28] Moravcsik M J, Murugesan P. Some Results on the Function and Quality of Citations[J]. Social Studies of Science, 1975, 5(1): 86-92.
doi: 10.1177/030631277500500106
[29] Teufel S, Siddharthan A, Tidhar D. Automatic Classification of Citation Function [C]//Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. 2006: 103-110.
[30] Dong C L, Schäfer U. Ensemble-style Self-training on Citation Classification [C]//Proceedings of the 5th International Joint Conference on Natural Language Processing. 2011: 623-631.
[31] Hernández-Alvarez M, Gómez J M. Citation Impact Categorization: For Scientific Literature [C]//Proceedings of 2015 IEEE 18th International Conference on Computational Science and Engineering. IEEE, 2015: 307-313.
[32] 李卓, 赵梦圆, 柳嘉昊, 等. 基于引文内容的图书被引动机研究[J]. 图书与情报, 2019(3): 96-104.
[32] (Li Zhuo, Zhao Mengyuan, Liu Jiahao, et al. Citing Motivation of Book Based on Citation Content[J]. Library & Information, 2019(3): 96-104.)
[33] 蔡文靖, 王兴旺. 我国学术论文评价研究状况分析[J]. 情报探索, 2018(8): 103-108.
[33] (Cai Wenjing, Wang Xingwang. Analysis on Research Status of Academic Paper Evaluation in China[J]. Information Research, 2018(8): 103-108.)
[34] 尹莉, 郭璐, 李旭芬. 基于引用功能和引用极性的一个引用分类模型研究[J]. 情报杂志, 2018, 37(7): 139-145.
[34] (Yin Li, Guo Lu, Li Xufen. An Empirical Study on Citation Classification Based on Citation Function and Citation Polarity[J]. Journal of Intelligence, 2018, 37(7): 139-145.)
[35] Kolen J F, Kremer S C. Gradient Flow in Recurrent Nets: The Difficulty of Learning Long-Term Dependencies[A]// A Field Guide to Dynamical Recurrent Networks[M]. Willey, 2001.
[36] Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
pmid: 9377276
[37] Dey R, Salem F M. Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks [C]//Proceedings of 2017 IEEE 60th International Midwest Symposium on Circuits and Systems. IEEE, 2017: 1597-1600.
[38] Abu-Jbara A, Ezra J, Radev D. Purpose and Polarity of Citation: Towards NLP-Based Bibliometrics [C]//Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2013: 596-606.
[1] Yang Hanxun, Zhou Dequn, Ma Jing, Luo Yongcong. Detecting Rumors with Uncertain Loss and Task-level Attention Mechanism[J]. 数据分析与知识发现, 2021, 5(7): 101-110.
[2] Xie Hao,Mao Jin,Li Gang. Sentiment Classification of Image-Text Information with Multi-Layer Semantic Fusion[J]. 数据分析与知识发现, 2021, 5(6): 103-114.
[3] Yin Pengbo,Pan Weimin,Zhang Haijun,Chen Degang. Identifying Clickbait with BERT-BiGA Model[J]. 数据分析与知识发现, 2021, 5(6): 126-134.
[4] Yu Bengong,Zhu Xiaojie,Zhang Ziwei. A Capsule Network Model for Text Classification with Multi-level Feature Extraction[J]. 数据分析与知识发现, 2021, 5(6): 93-102.
[5] Han Pu,Zhang Zhanpeng,Zhang Mingtao,Gu Liang. Normalizing Chinese Disease Names with Multi-feature Fusion[J]. 数据分析与知识发现, 2021, 5(5): 83-94.
[6] Duan Jianyong,Wei Xiaopeng,Wang Hao. A Multi-Perspective Co-Matching Model for Machine Reading Comprehension[J]. 数据分析与知识发现, 2021, 5(4): 134-141.
[7] Wang Yuzhu,Xie Jun,Chen Bo,Xu Xinying. Multi-modal Sentiment Analysis Based on Cross-modal Context-aware Attention[J]. 数据分析与知识发现, 2021, 5(4): 49-59.
[8] Yu Bengong, Zhang Shuwen. Aspect-Level Sentiment Analysis Based on BAGCNN[J]. 数据分析与知识发现, 2021, 5(12): 37-47.
[9] Jiang Cuiqing,Wang Xiangxiang,Wang Zhao. Forecasting Car Sales Based on Consumer Attention[J]. 数据分析与知识发现, 2021, 5(1): 128-139.
[10] Huang Lu,Zhou Enguo,Li Daifeng. Text Representation Learning Model Based on Attention Mechanism with Task-specific Information[J]. 数据分析与知识发现, 2020, 4(9): 111-122.
[11] Yin Haoran,Cao Jinxuan,Cao Luzhe,Wang Guodong. Identifying Emergency Elements Based on BiGRU-AM Model with Extended Semantic Dimension[J]. 数据分析与知识发现, 2020, 4(9): 91-99.
[12] Shi Lei,Wang Yi,Cheng Ying,Wei Ruibin. Review of Attention Mechanism in Natural Language Processing[J]. 数据分析与知识发现, 2020, 4(5): 1-14.
[13] Xue Fuliang,Liu Lifang. Fine-Grained Sentiment Analysis with CRF and ATAE-LSTM[J]. 数据分析与知识发现, 2020, 4(2/3): 207-213.
[14] Qi Ruihua,Jian Yue,Guo Xu,Guan Jinghua,Yang Mingxin. Sentiment Analysis of Cross-Domain Product Reviews Based on Feature Fusion and Attention Mechanism[J]. 数据分析与知识发现, 2020, 4(12): 85-94.
[15] Xu Tongtong,Sun Huazhi,Ma Chunmei,Jiang Lifen,Liu Yichen. Classification Model for Few-shot Texts Based on Bi-directional Long-term Attention Features[J]. 数据分析与知识发现, 2020, 4(10): 113-123.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn