|
|
Recognizing Structure Functions of Academic Articles with Hierarchical Attention Network |
Qin Chenglei,Zhang Chengzhi() |
School of Economics and Management, Nanjing University of Science and Technology, Nanjing 210094, China |
|
|
Abstract [Objective] This paper proposes a new method using hierarchical attention network, aiming to effectively recognize structure functions of scholarly articles. [Methods] First, we constructed a network model with different-grained hierarchical attention to automatically identify the functions of text structures. Then, we examined the performance of our method with four datasets from PLoS. Same tests were also applied to traditional machine learning models with text feature vectors, as well as and Bert model. We also modified the proposed model in accordance with test results. Third, we evaluated the performance of the new model with articles from Atmospheric Chemistry and Physics and decided the compatibility of this model for other domains. [Results] At the sentence level, our model (using Bi-LSTM+Attention as the encoder) outperformed the others (Macro-F1: 0.866 1). However, this model did not perform well in un-related fields (minimum Macro-F1: 0.455 4). [Limitations] The model cannot recognize functions of mixed structure texts, as well as the logical relationship in these structures. [Conclusions] The proposed model could effectively recognize the structure functions at sentence level, which expands research of the full text scholarly literature.
|
Received: 27 April 2020
Published: 04 December 2020
|
|
Corresponding Authors:
Zhang Chengzhi
E-mail: zhangcz@njust.edu.cn
|
[1] |
Norris M, Oppenheim C, Rowland F. The Citation Advantage of Open-Access Articles[J]. Journal of the Association for Information Science & Technology, 2014,59(12):1963-1972.
|
[2] |
Wang X M, Liu C, Mao W L, et al. The Open Access Advantage Considering Citation, Article Usage and Social Media Attention[J]. Scientometrics, 2015,103(2):555-564.
|
[3] |
Haustein S, Piwowar H A, Priem J, et al. Data From: The State of OA: A Large-Scale Analysis of the Prevalence and Impact of Open Access Articles[J]. PeerJ, 2018,6(4):e4375.
|
[4] |
Zhang R, Guo J F, Fan Y X, et al. Outline Generation: Understanding the Inherent Content Structure of Documents[OL]. arXiv Preprint, arXiv: 1905. 10039.
|
[5] |
Sollaci L B, Pereira M G. The Introduction, Methods, Results, and Discussion (IMRAD) Structure: A Fifty-Year Survey[J]. Journal of the Medical Library Association, 2004,92(3):364.
pmid: 15243643
|
[6] |
Bertin M, Atanassova I, Gingras Y, et al. The Invariant Distribution of References in Scientific Articles[J]. Journal of the Association for Information Science & Technology, 2016,67(1):164-177.
|
[7] |
Bertin M, Atanassova I, Sugimoto C R, et al. The Linguistic Patterns and Rhetorical Structure of Citation Context:An Approach Using N-Grams[J]. Scientometrics, 2016,109(3):1417-1434.
|
[8] |
Hu Z G, Chen C M, Liu Z Y. Where are Citations Located in the Body of Scientific Articles? A Study of the Distributions of Citation Locations[J]. Journal of Informetrics, 2013,7(4):887-896.
doi: 10.1016/j.joi.2013.08.005
|
[9] |
Nair P K R, Nair V D. Scientific Writing and Communication in Agriculture and Natural Resources[M]. Springer, 2014.
|
[10] |
International Committee of Medical Journal Editors. Uniform Requirements for Manuscripts Submitted to Biomedical Journals[J]. The New England Journal of Medicine, 1991,324(6):424-428.
pmid: 1987468
|
[11] |
Devlin J, Chang M W, Lee K, et al. Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv: 1810. 04805.
|
[12] |
Noriko K. Text-Level Structure of Research Papers: Implications for Text-Based Information Processing Systems[C]// Proceedings of the 19th Annual BCS-IRSG Conference on Information Retrieval Research, Swindon, United Kingdom. 1997: 1-14.
|
[13] |
Santiago P. The Schematic Structure of Computer Science Research Articles[J]. English for Specific Purposes, 1999,18(2):139-160.
|
[14] |
Budsaba K. Rhetorical Structure of Biochemistry Research Articles[J]. English for Specific Purposes, 2005,24(3):269-292.
doi: 10.1016/j.esp.2004.08.003
|
[15] |
McKnight L, Srinivasan P. Categorization of Sentence Types in Medical Abstracts[C]// Proceedings of AMIA Annual Symposium,Washington, DC, USA. 2003: 440-444.
|
[16] |
Mizuta Y, Korhonen A, Mullen T, et al. Zone Analysis in Biology Articles as a Basis for Information Extraction[J]. International Journal of Medical Informatics, 2006,75(6):468-487.
doi: 10.1016/j.ijmedinf.2005.06.013
pmid: 16112609
|
[17] |
Agarwal S, Yu H. Automatically Classifying Sentences in Full-Text Biomedical Articles into Introduction, Methods, Results and Discussion[J]. Bioinformatics, 2009,25(23):3174-3180.
doi: 10.1093/bioinformatics/btp548
pmid: 19783830
|
[18] |
Ribeiro S S, Yao J T, Rezende D A. Discovering IMRAD Structure with Different Classifiers[C]// Proceedings of 2018 IEEE International Conference on Big Knowledge (ICBK,Singapore. 2018: 200-204.
|
[19] |
陆伟, 黄永, 程齐凯. 学术文本的结构功能识别——功能框架及基于章节标题的识别[J]. 情报学报, 2014,33(9):979-985.
|
[19] |
( Lu Wei, Huang Yong, Cheng Qikai. The Structure Function of Academic Text and Its Classification[J]. Journal of the China Society for Scientific and Technical Information, 2014,33(9):979-985.)
|
[20] |
黄永, 陆伟, 程齐凯. 学术文本的结构功能识别——基于章节内容的识别[J]. 情报学报, 2016,35(3):293-300.
|
[20] |
( Huang Yong, Lu Wei, Cheng Qikai. The Structure Function Recognition of Academic Text Chapter Content Based Recognition[J]. Journal of the China Society for Scientific and Technical Information, 2016,35(3):293-300.)
|
[21] |
黄永, 陆伟, 程齐凯, 等. 学术文本的结构功能识别——基于段落的识别[J]. 情报学报, 2016,35(5):530-538.
|
[21] |
( Huang Yong, Lu Wei, Cheng Qikai, et al. The Structure Function Recognition of Academic Text Paragraph-Based Recognition[J]. Journal of the China Society for Scientific and Technical Information, 2016,35(5):530-538.)
|
[22] |
王东波, 高瑞卿, 叶文豪, 等. 不同特征下的学术文本结构功能自动识别研究[J]. 情报学报, 2018,37(10):997-1008.
|
[22] |
( Wang Dongbo, Gao Ruiqing, Ye Wenhao, et al. Research on the Structure Recognition of Academic Texts Under Different Characteristics[J]. Journal of the China Society for Scientific and Technical Information, 2018,37(10):997-1008.)
|
[23] |
王佳敏, 陆伟, 刘家伟, 等. 多层次融合的学术文本结构功能识别研究[J]. 图书情报工作, 2019,63(13):95-104.
|
[23] |
( Wang Jiamin, Lu Wei, Liu Jiawei, et al. Research on Structure Function Recognition of Academic Text Based on Multi-Level Fusion[J]. Library and Information Service, 2019,63(13):95-104.)
|
[24] |
Joulin A, Grave E, Bojanowski P, et al. Bag of Tricks for Efficient Text Classification[OL]. arXiv Preprint, arXiv: 1607. 01759.
|
[25] |
Pappas N, Popescu-Belis A. Multilingual Hierarchical Attention Networks for Document Classification[OL]. arXiv Preprint, arXiv: 1707. 00896.
|
[26] |
Zhang X, Zhao J B, Lecun Y. Character-Level Convolutional Networks for Text Classification[OL]. arXiv Preprint, arXiv: 1509. 01626.
|
[27] |
Lee J Y, Dernoncourt F. Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks[OL]. arXiv Preprint, arXiv: 1603. 03827.
|
[28] |
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[OL]. arXiv Preprint, arXiv: 1706. 03762.
|
[29] |
Yang Z, Yang D, Dyer C, et al. Hierarchical Attention Networks for Document Classification[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vancouver, Canada. 2017: 1480-1489.
|
[30] |
Giorgino T. Computing and Visualizing Dynamic Time Warping Alignments in R: The DTW Package[J]. Journal of Statistical Software, 2009,31(7):1-25.
|
[31] |
Massey F J. The Kolmogorov-Smirnov Test for Goodness of Fit[J]. Publications of the American Statistical Association, 1951,46(253):68-78.
|
[32] |
Yang Y M. An Evaluation of Statistical Approaches to Text Categorization[J]. Information Retrieval, 1999,1(1/2):69-90.
doi: 10.1023/A:1009982220290
|
[33] |
Cherkassky V, Ma Y Q. Practical Selection of SVM Parameters and Noise Estimation for SVM Regression[J]. Neural Networks, 2004,17(1):113-126.
doi: 10.1016/S0893-6080(03)00169-2
pmid: 14690712
|
[34] |
Hernández-Lobato J M, Hernández-Lobato D, Suárez A. Expectation Propagation in Linear Regression Models with Spike- and-Slab Priors[J]. Machine Learning, 2015,99(3):437-487.
doi: 10.1007/s10994-014-5475-7
|
[35] |
Sebe N, Lew M S, Cohen I, et al. Emotion Recognition Using a Cauchy Naive Bayes Classifier[C]// Proceedings of the 16th International Conference on Pattern Recognition, Quebec, Canada. 2002: 17-20.
|
[36] |
Zhang H, Berg A C, Michael B, et al. SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition[C]// Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition,New York, USA. 2006: 17-22.
|
[37] |
Chen T Q, Guestrin C. XGBoost: A Scalable Tree Boosting System[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, USA. 2016: 785-794.
|
[38] |
Wu H C, Luk K, Wong K F, et al. Interpreting TF-IDF Term Weights as Making Relevance Decisions[J]. ACM Transactions on Information System, 2008,26(3):1-37.
|
[39] |
Satorra A, Bentler P M. A Scaled Difference Chi-Square Test Statistic for Moment Structure Analysis[J]. Psychometrika, 2001,66(4):507-514.
doi: 10.1007/BF02296192
|
[40] |
Kent J T. Information Gain and a General Measure of Correlation[J]. Biometrika, 1983,70(1):163-173.
doi: 10.1093/biomet/70.1.163
|
[41] |
Rigby A S. Statistical Methods in Epidemiology. v. Towards an Understanding of the Kappa Coefficient[J]. Disability & Rehabilitation, 2000,22(8):339-344.
doi: 10.1080/096382800296575
pmid: 10896093
|
[42] |
Gannon T, Madnick S E, Moulton A, et al. Framework for the Analysis of the Adaptability, Extensibility, and Scalability of Semantic Information Integration and the Context Mediation Approach[C]// Proceedings of the 42nd Hawaii International Conference on System Sciences. 2009: 1-11.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|