|
|
|
| Identifying Moves from Scientific Abstracts Based on Paragraph-BERT-CRF |
Guo Hangcheng,He Yanqing( ),Lan Tian,Wu Zhenfeng,Dong Cheng |
| Institute of Scientific and Technical Information of China, Beijing 100038, China |
|
|
|
|
Abstract [Objective] This paper tries to automatically identify the moves from scientific paper abstracts, aiming to find the purpose, method, results, and conclusion of the paper. It also helps readers quickly receive main contents of the literature and conduct semantic retrieval. [Methods] We proposed a neural network model for abstract move recognition based on the Paragraph-BERT-CRF framework, which fully uses the context information. We also added the attention mechanism and the transfer relationship between sequence move labels. [Results] We examined our model with 94,456 abstracts of scientific papers. The weighted average precision, recall and F1 values were 97.45%, 97.44% and 97.44%, respectively. Compared with the ablation experimental results of CRF, BiLSTM, BiLSTM-CRF, BERT, BERT-CRF and Paragraph-BERT, our new model is effective. [Limitations] We only used the basic BERT-based pre-trained language model. More research is needed to optimize the model parameters and add more pre-trained language model in the recognition of move information. [Conclusions] Attention mechanism and paragraph level contextual information can effectively improve the proposed model’s inference scores and identify move information from abstracts.
|
|
Received: 01 September 2021
Published: 28 February 2022
|
|
|
| Fund:Key Project of Institute of Scientific and Technical Information of China(ZD2021-17);Innovation Research Fund General Project of Institute of Scientific and Technical Information of China(MS2021-03);Innovation Research Fund Youth Project of Institute of Scientific and Technical Information of China(QN2021-12) |
|
Corresponding Authors:
He Yanqing,ORCID：0000-0002-8791-1581
E-mail: heyq@istic.ac.cn
|
| [1] |
科学技术报告、学位论文和学术论文的编写格式: GB 7713-1987[S]. 北京: 中国标准出版社, 1988.
|
| [1] |
(Presentation of Scientific and Technical Reports, Dissertations and Scientific Papers: GB 7713-1987[S]. Beijing: Standards Press of China, 1988.)
|
| [2] |
钱寿初. 从传统摘要到结构式摘要[J]. 编辑学报, 1990, 2(1):56-60.
|
| [2] |
( Qian Shouchu. From Traditional Summary to Structured Summary[J]. Acta Editologica, 1990, 2(1):56-60.)
|
| [3] |
Milward D, Bjäreland M, Hayes W, et al. Ontology-Based Interactive Information Extraction from Scientific Abstracts[J]. Comparative and Functional Genomics, 2005, 6:251456.
|
| [4] |
Cross C, Oppenheim C. A Genre Analysis of Scientific Abstracts[J]. Journal of Documentation, 2006, 62(4):428-446.
doi: 10.1108/00220410610700953
|
| [5] |
杜圣梅, 朱礼军, 徐硕. 面向循证医学的科技文献摘要结构化表示研究[J]. 中国科技资源导刊, 2018, 50(6):94-100.
|
| [5] |
( Du Shengmei, Zhu Lijun, Xu Shuo. Research on Structured Presentation of Scientific Literature Abstracts for Evidence-Based Medicine[J]. China Science & Technology Resources Review, 2018, 50(6):94-100.)
|
| [6] |
郑梦悦, 秦春秀, 马续补. 面向中文科技文献非结构化摘要的知识元表示与抽取研究——基于知识元本体理论[J]. 情报理论与实践, 2020, 43(2):157-163.
|
| [6] |
( Zheng Mengyue, Qin Chunxiu, Ma Xubu. Research on Knowledge Unit Representation and Extraction for Unstructured Abstracts of Chinese Scientific and Technical Literature: Ontology Theory Based on Knowledge Unit[J]. Information Studies: Theory & Application, 2020, 43(2):157-163.)
|
| [7] |
李小乐, 王玉琢, 章成志. 针对特定任务的方法实体评估研究[J]. 情报工程, 2021, 7(4):13-26.
|
| [7] |
( Li Xiaole, Wang Yuzhuo, Zhang Chengzhi. Evaluation of Method Entities for a Special Task[J]. Technology Intelligence Engineering, 2021, 7(4):13-26.)
|
| [8] |
Anthony L. A Machine Learning System for the Automatic Identification of Text Structure, and Application to Research Article Abstracts in Computer Science[D]. University of Birmingham, 2002.
|
| [9] |
Wu J C, Chang Y C, Liou H C, et al. Computational Analysis of Move Structures in Academic Abstracts[C]// Proceedings of the COLING/ACL on Interactive Presentation Sessions. Association for Computational Linguistics, 2006: 41-44.
|
| [10] |
McKnight L, Srinivasan P. Categorization of Sentence Types in Medical Abstracts[J]. AMIA Annual Symposium Proceedings, 2003: 440-444.
|
| [11] |
丁良萍, 张智雄, 刘欢. 影响支持向量机模型语步自动识别效果的因素研究[J]. 数据分析与知识发现, 2019, 3(11):16-23.
|
| [11] |
( Ding Liangping, Zhang Zhixiong, Liu Huan. Factors Affecting Rhetorical Move Recognition with SVM Model[J]. Data Analysis and Knowledge Discovery, 2019, 3(11):16-23.)
|
| [12] |
Hirohata K, Okazaki N, Ananiadou S, et al. Identifying Sections in Scientific Abstracts Using Conditional Random Fields[C]// Proceedings of the 3rd International Joint Conference on Natural Language Processing. 2008: 381-388.
|
| [13] |
沈思, 胡昊天, 叶文豪, 等. 基于全字语义的摘要结构功能自动识别研究[J]. 情报学报, 2019, 38(1):79-88.
|
| [13] |
( Shen Si, Hu Haotian, Ye Wenhao, et al. Research on Abstract Structure Function Automatic Recognition Based on Full Character Semantics[J]. Journal of the China Society for Scientific and Technical Information, 2019, 38(1):79-88.)
|
| [14] |
张智雄, 刘欢, 丁良萍, 等. 不同深度学习模型的科技论文摘要语步识别效果对比研究[J]. 数据分析与知识发现, 2019, 3(12):1-9.
|
| [14] |
( Zhang Zhixiong, Liu Huan, Ding Liangping, et al. Identifying Moves of Research Abstracts with Deep Learning Methods[J]. Data Analysis and Knowledge Discovery, 2019, 3(12):1-9.)
|
| [15] |
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 5998-6008.
|
| [16] |
Yan H, Deng B C, Li X N, et al. TENER: Adapting Transformer Encoder for Named Entity Recognition[OL]. arXiv Preprint, arXiv:1911.04474.
|
| [17] |
Yu G H, Zhang Z X, Liu H, et al. Masked Sentence Model Based on BERT for Move Recognition in Medical Scientific Abstracts[J]. Journal of Data and Information Science, 2019, 4(4):42-55.
doi: 10.2478/jdis-2019-0020
|
| [18] |
丁龙, 文雯, 林强. 基于预训练BERT字嵌入模型的领域实体识别[J]. 情报工程, 2019, 5(6):65-74.
|
| [18] |
( Ding Long, Wen Wen, Lin Qiang. Domain Entity Recognition Based on Pre-trained BERT Character Embedding[J]. Technology Intelligence Engineering, 2019, 5(6):65-74.)
|
| [19] |
王末, 崔运鹏, 陈丽, 等. 基于深度学习的学术论文语步结构分类方法研究[J]. 数据分析与知识发现, 2020, 4(6):60-68.
|
| [19] |
( Wang Mo, Cui Yunpeng, Chen Li, et al. A Deep Learning-Based Method of Argumentative Zoning for Research Articles[J]. Data Analysis and Knowledge Discovery, 2020, 4(6):60-68.)
|
| [20] |
郭晨睿, 王佳敏, 崔浩冉, 等. 基于SciBERT模型的引文上下文识别系统优化[J]. 情报工程, 2021, 7(5):3-14.
|
| [20] |
( Guo Chenrui, Wang Jiamin, Cui Haoran, et al. Optimization of Citation Context Recognition System Based on SciBERT Model[J]. Technology Intelligence Engineering, 2021, 7(5):3-14.)
|
| [21] |
Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv: 1810.04805.
|
| [22] |
Lafferty J, McCallum A, Pereira F. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data[C]// Proceedings of the 18th International Conference on Machine Learning. 2001: 282-289.
|
| [23] |
Li B H, Zhou H, He J X, et al. On the Sentence Embeddings from Pre-Trained Language Models[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA, USA: Association for Computational Linguistics, 2020: 9119-9130.
|
| [24] |
宗成庆, 夏睿, 张家俊. 文本数据挖掘[M]. 北京: 清华大学出版社, 2019.
|
| [24] |
( Zong Chengqing, Xia Rui, Zhang Jiajun. Text Data Mining[M]. Beijing: Tsinghua University Press, 2019.)
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
