Please wait a minute...
Data Analysis and Knowledge Discovery  2019, Vol. 3 Issue (12): 1-9    DOI: 10.11925/infotech.2096-3467.2019.0266
Current Issue | Archive | Adv Search |
Identifying Moves of Research Abstracts with Deep Learning Methods
Zhixiong Zhang1,2,3,4(),Huan Liu1,2,4,Liangping Ding1,2,4,Pengmin Wu1,2,Gaihong Yu1,2
1 National Science Library, Chinese Academy of Sciences, Beijing 100190, China
2 Department of Library Information and Archives Management, University of Chinese Academy of Sciences, Beijing 100190, China
3 Wuhan Library, Chinese Academy of Sciences, Wuhan 430071, China
4 Hubei Key Laboratory of Big Data in Science and Technology, Wuhan 430071, China
Download: PDF(566 KB)   HTML ( 46
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper compares the performance of move recognition methods with different deep learning algorithms. [Methods] Firstly, we built a large training corpus. Then, we used the traditional machine learning method SVM as a benchmark, and developed four moves recognition models based on DNN, LSTM, Attention-BiLSTM and LSTM. Finally, we conducted two rounds of experiments with sample size of 10,000 and 50,000. [Results] Attention-BiLSTM method achieved the best results in both experiments over the four methods (F1=0.9375 with the larger sample). SVM method outperformed DNN and LSTM in both experiments. While changing sample size from 10,000 to 50,000, SVM received the least increase of F1 score (0.0125), and LSTM had the largest increase of F1 score (0.1125). [Limitations] There is no universal test corpus for similar research. Therefore, our results could not be compared with the results of other studies. [Conclusions] The bi-directional LSTM network structure and attention mechanism can significantly improve the performance of move recognition. The deep learning methods work better with larger sample size.

Key wordsDeep Learning      Neural Network      Moves Recognition      Support Vector Machine     
Received: 07 March 2019      Published: 25 January 2020
ZTFLH:  G202 TP393  
Corresponding Authors: Zhixiong Zhang     E-mail: zhangzhx@mail.las.ac.cn

Cite this article:

Zhixiong Zhang,Huan Liu,Liangping Ding,Pengmin Wu,Gaihong Yu. Identifying Moves of Research Abstracts with Deep Learning Methods. Data Analysis and Knowledge Discovery, 2019, 3(12): 1-9.

URL:

http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2019.0266     OR     http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2019/V3/I12/1

对比项 SciAnnoDoc[2,3] CoreSC[4] AZ[5]/AZ-II[6] Multi-Layer[7]
类别数 5 11 7/15 5
语料领域 人文性别研究 生物化学 计算语言学/生物化学 计算机图形学
自动分类算法 基于规则的算法 SVM、CRF NB LR和SVM
特定类别上的最好效果(F1值) Methodology59% Experiment75% OWN85% Approach 87.6%
类别 P R F1值
Purpose 0.8900 0.8800 0.8900
Methods 0.9000 0.9400 0.9200
Results 0.8700 0.9200 0.8900
Conclusions 0.8900 0.8200 0.8600
平均值 0.8875 0.8900 0.8900
类别 P R F1值
Purpose 0.9000 0.9200 0.9100
Methods 0.9300 0.9200 0.9300
Results 0.8700 0.9200 0.9000
Conclusions 0.9100 0.8400 0.8700
平均值 0.9025 0.9000 0.9025
类别 P R F1值
Purpose 0.8300 0.7900 0.8100
Methods 0.8600 0.8700 0.8700
Results 0.8000 0.8300 0.8100
Conclusions 0.7900 0.7900 0.7900
平均值 0.8200 0.8200 0.8200
类别 P R F1值
Purpose 0.8500 0.8400 0.8500
Methods 0.8800 0.9000 0.8900
Results 0.8800 0.8100 0.8400
Conclusions 0.7800 0.8400 0.8100
平均值 0.8475 0.8475 0.8475
类别 P R F1值
Purpose 0.7900 0.7300 0.7600
Methods 0.8000 0.9000 0.8500
Results 0.8200 0.7100 0.7700
Conclusions 0.7200 0.7800 0.7500
平均值 0.7825 0.7800 0.7825
类别 P R F1值
Purpose 0.9000 0.9400 0.9200
Methods 0.9100 0.9200 0.9200
Results 0.8800 0.8700 0.8700
Conclusions 0.9000 0.8500 0.8700
平均值 0.8975 0.8950 0.8950
类别 P R F1值
Purpose 0.9200 0.9300 0.9300
Methods 0.9300 0.9400 0.9300
Results 0.9200 0.9200 0.9200
Conclusions 0.9100 0.9000 0.9000
平均值 0.9200 0.9225 0.9200
类别 P R F1值
Purpose 0.9600 0.9500 0.9500
Methods 0.9400 0.9500 0.9400
Results 0.9400 0.9100 0.9300
Conclusions 0.9200 0.9300 0.9300
平均值 0.9400 0.9350 0.9375
样本量 类别 SVM
模型
DNN
模型
LSTM
模型
Att-BiLSTM
模型
10 000 Purpose 0.8900 0.8100 0.7600 0.9300
Methods 0.9200 0.8700 0.8500 0.9300
Results 0.8900 0.8100 0.7700 0.9200
Conclusions 0.8600 0.7900 0.7500 0.9000
50 000 Purpose 0.9100 0.8500 0.9200 0.9500
Methods 0.9300 0.8900 0.9200 0.9400
Results 0.9000 0.8400 0.8700 0.9300
Conclusions 0.8700 0.8100 0.8700 0.9300
因子 自由度 离差平方和 均方 F统计量 p
样本量
(整体)
1.0 0.014450 0.014450 5.14006 0.03075
样本量
(SVM方法)
1.0 0.000313 0.000313 0.51020 0.50188
样本量
(DNN方法)
1.0 0.001513 0.001513 1.32 0.29432
样本量
(LSTM方法)
1.0 0.025313 0.025313 17.30769 0.00594
样本量
(Att-BiLSTM方法)
1.0 0.000612 0.000612 4.2 0.08632
因子 自由度 离差平方和 均方 F统计量 p
SVM,
DNN, LSTM,
Att-BiLSTM
3.0 0.050837 0.016946 9.895377 0.000129
SVM, DNN 1.0 0.015625 0.015625 17.676768 0.000883
SVM, LSTM 1.0 0.013225 0.013225 4.862771 0.044665
SVM,
Att-BiLSTM
1.0 0.004225 0.004225 10.803653 0.005401
DNN, LSTM 1.0 0.000100 0.000100 0.032961 0.858538
DNN,
Att-BiLSTM
1.0 0.036100 0.036100 51.179747 0.000005
LSTM,
Att-BiLSTM
1.0 0.032400 0.032400 12.750527 0.003071
[1] Swales J M . Research Genres: Explorations and Applications [M]. Cambridge: Cambridge University Press, 2004: 228-229.
[2] Ribaupierre H D, Falquet G . User-Centric Design and Evaluation of a Semantic Annotation Model for Scientific Documents [C]//Proceedings of the 14th International Conference on Knowledge Technologies and Data-driven Business. New York: ACM, 2014: 40.
[3] Ribaupierre H D, Falquet G . An Automated Annotation Process for the SciDocAnnot Scientific Document Model [C] //Proceedings of the 5th International Workshop on Semantic Digital Archives. Osaka: International Workshop on Semantic Digital Archives, 2015: 30-41.
[4] Liakata M, Teufel S, Siddharthan A , et al. Corpora for the Conceptualisation and Zoning of Scientific Papers [C] //Proceedings of the 2010 International Conference on Language Resources and Evaluation. Valletta: European Languages Resources Association (ELRA), 2010: 105-108.
[5] Teufel S, Carletta J, Moens M . An Annotation Scheme for Discourse-level Argumentation in Research Articles [C] //Proceedings of the 9th Conference on European Chapter of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 1999: 110-117.
[6] Teufel S, Batchelor C . Towards Discipline-independent Argumentative Zoning: Evidence from Chemistry and Computational Linguistics [C]// Proceeding of the Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, 2009: 1493-1502.
[7] Fisas B, Ronzano F, Saggion H . A Multi-layered Annotated Corpus of Scientific Papers [C]//Proceedings of the 10th International Conference on Language Resources and Evaluation. European Language Resources Association, 2016: 3081-3088.
[8] 刘霞 . 英语学术论文摘要语步结构自动识别模型的构建[D]. 北京: 北京外国语大学, 2016.
[8] ( Liu Xia . Constructing a Model for the Automatic Identification of Move Structure in English Research Article Abstracts[D]. Beijing: Beijing Foreign Studies University, 2016.)
[9] 王立非, 刘霞 . 英语学术论文摘要语步结构自动识别模型的构建[J]. 外语电化教学, 2017(2):47-52, 66.
[9] ( Wang Lifei, Liu Xia . Constructing a Model for the Automatic Identification of Move Structure in English Research Article Abstracts[J]. Technology Enhance Foreign Language Education, 2017(2):47-52, 66.)
[10] Mikolov T, Chen K, Corrado G , et al. Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint, arXiv:1301.3781v3.
[11] Bengio Y, Delalleau O . On the Expressive Power of Deep Architectures [C]//Proceeding of the International Conference on Algorithmic Learning Theory. Springer-Verlag, 2011: 18-36.
[12] Hochreiter S, Schmidhuber J . Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780.
[13] Irsoy O, Cardie C . Opinion Mining with Deep Recurrent Neural Networks [C]//Proceeding of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 720-728.
[14] Mnih V, Heess N, Graves A , et al. Recurrent Models of Visual Attention[OL]. arXiv Preprint , arXiv: 1406.6247.
[1] Weimin Nie,Yongzhou Chen,Jing Ma. A Text Vector Representation Model Merging Multi-Granularity Information[J]. 数据分析与知识发现, 2019, 3(9): 45-52.
[2] Heran Qin,Liu Liu,Bin Li,Dongbo Wang. Automatic Classification of Ancient Classics with Entity Features[J]. 数据分析与知识发现, 2019, 3(9): 68-76.
[3] Ruojia Wang,Lu Zhang,Jimin Wang. Automatic Triage of Online Doctor Services Based on Machine Learning[J]. 数据分析与知识发现, 2019, 3(9): 88-97.
[4] Zhenyu He,Xiangxiang Dong,Qinghua Zhu. Classifying Baidu Encyclopedia Entries with User Behaviors[J]. 数据分析与知识发现, 2019, 3(6): 117-122.
[5] Qingtian Zeng,Mingdi Dai,Chao Li,Hua Duan,Zhongying Zhao. Discovering Important Locations with User Representation and Trace Data[J]. 数据分析与知识发现, 2019, 3(6): 75-82.
[6] Kan Liu,Lu Chen. Deep Neural Network Learning for Medical Triage[J]. 数据分析与知识发现, 2019, 3(6): 99-108.
[7] Mengji Zhang,Wanyu Du,Nan Zheng. Predicting Stock Trends Based on News Events[J]. 数据分析与知识发现, 2019, 3(5): 11-18.
[8] Cheng Zhou,Hongqin Wei. Evaluating and Classifying Patent Values Based on Self-Organizing Maps and Support Vector Machine[J]. 数据分析与知识发现, 2019, 3(5): 117-124.
[9] Wancheng Chen,Haoran Dai,Yinghan Jin. Appraising Home Prices with HEDONIC Model: Case Study of Seattle, U.S.[J]. 数据分析与知识发现, 2019, 3(5): 19-26.
[10] Jingjing Pei,Xiaoqiu Le. Identifying Coordinate Text Blocks in Discourses[J]. 数据分析与知识发现, 2019, 3(5): 51-56.
[11] Juhua Wu,Shuo Zhang,Lei Tao,Shunjun Jiang. Predicting Stroke Risks with Neural Network[J]. 数据分析与知识发现, 2019, 3(12): 70-75.
[12] Liangping Ding,Zhixiong Zhang,Huan Liu. Factors Affecting Rhetorical Move Recognition with SVM Model[J]. 数据分析与知识发现, 2019, 3(11): 16-23.
[13] Li Yu,Li Qian,Changlei Fu,Huaming Zhao. Extracting Fine-grained Knowledge Units from Texts with Deep Learning[J]. 数据分析与知识发现, 2019, 3(1): 38-45.
[14] Changlei Fu,Li Qian,Huaping Zhang,Huaming Zhao,Jing Xie. Mining Innovative Topics Based on Deep Learning[J]. 数据分析与知识发现, 2019, 3(1): 46-54.
[15] Bengong Yu,Peihang Zhang,Qingtang Xu. Selecting Products Based on F-BiGRU Sentiment Analysis[J]. 数据分析与知识发现, 2018, 2(9): 22-30.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn