Please wait a minute...
Data Analysis and Knowledge Discovery  2019, Vol. 3 Issue (12): 1-9    DOI: 10.11925/infotech.2096-3467.2019.0266
Current Issue | Archive | Adv Search |
Identifying Moves of Research Abstracts with Deep Learning Methods
Zhixiong Zhang1,2,3,4(),Huan Liu1,2,4,Liangping Ding1,2,4,Pengmin Wu1,2,Gaihong Yu1,2
1 National Science Library, Chinese Academy of Sciences, Beijing 100190, China
2 Department of Library Information and Archives Management, University of Chinese Academy of Sciences, Beijing 100190, China
3 Wuhan Library, Chinese Academy of Sciences, Wuhan 430071, China
4 Hubei Key Laboratory of Big Data in Science and Technology, Wuhan 430071, China
Download: PDF (566 KB)   HTML ( 68
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper compares the performance of move recognition methods with different deep learning algorithms. [Methods] Firstly, we built a large training corpus. Then, we used the traditional machine learning method SVM as a benchmark, and developed four moves recognition models based on DNN, LSTM, Attention-BiLSTM and LSTM. Finally, we conducted two rounds of experiments with sample size of 10,000 and 50,000. [Results] Attention-BiLSTM method achieved the best results in both experiments over the four methods (F1=0.9375 with the larger sample). SVM method outperformed DNN and LSTM in both experiments. While changing sample size from 10,000 to 50,000, SVM received the least increase of F1 score (0.0125), and LSTM had the largest increase of F1 score (0.1125). [Limitations] There is no universal test corpus for similar research. Therefore, our results could not be compared with the results of other studies. [Conclusions] The bi-directional LSTM network structure and attention mechanism can significantly improve the performance of move recognition. The deep learning methods work better with larger sample size.

Key wordsDeep Learning      Neural Network      Moves Recognition      Support Vector Machine     
Received: 07 March 2019      Published: 25 December 2019
ZTFLH:  G202 TP393  
Corresponding Authors: Zhixiong Zhang     E-mail: zhangzhx@mail.las.ac.cn

Cite this article:

Zhixiong Zhang,Huan Liu,Liangping Ding,Pengmin Wu,Gaihong Yu. Identifying Moves of Research Abstracts with Deep Learning Methods. Data Analysis and Knowledge Discovery, 2019, 3(12): 1-9.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2019.0266     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2019/V3/I12/1

对比项 SciAnnoDoc[2,3] CoreSC[4] AZ[5]/AZ-II[6] Multi-Layer[7]
类别数 5 11 7/15 5
语料领域 人文性别研究 生物化学 计算语言学/生物化学 计算机图形学
自动分类算法 基于规则的算法 SVM、CRF NB LR和SVM
特定类别上的最好效果(F1值) Methodology59% Experiment75% OWN85% Approach 87.6%
类别 P R F1值
Purpose 0.8900 0.8800 0.8900
Methods 0.9000 0.9400 0.9200
Results 0.8700 0.9200 0.8900
Conclusions 0.8900 0.8200 0.8600
平均值 0.8875 0.8900 0.8900
类别 P R F1值
Purpose 0.9000 0.9200 0.9100
Methods 0.9300 0.9200 0.9300
Results 0.8700 0.9200 0.9000
Conclusions 0.9100 0.8400 0.8700
平均值 0.9025 0.9000 0.9025
类别 P R F1值
Purpose 0.8300 0.7900 0.8100
Methods 0.8600 0.8700 0.8700
Results 0.8000 0.8300 0.8100
Conclusions 0.7900 0.7900 0.7900
平均值 0.8200 0.8200 0.8200
类别 P R F1值
Purpose 0.8500 0.8400 0.8500
Methods 0.8800 0.9000 0.8900
Results 0.8800 0.8100 0.8400
Conclusions 0.7800 0.8400 0.8100
平均值 0.8475 0.8475 0.8475
类别 P R F1值
Purpose 0.7900 0.7300 0.7600
Methods 0.8000 0.9000 0.8500
Results 0.8200 0.7100 0.7700
Conclusions 0.7200 0.7800 0.7500
平均值 0.7825 0.7800 0.7825
类别 P R F1值
Purpose 0.9000 0.9400 0.9200
Methods 0.9100 0.9200 0.9200
Results 0.8800 0.8700 0.8700
Conclusions 0.9000 0.8500 0.8700
平均值 0.8975 0.8950 0.8950
类别 P R F1值
Purpose 0.9200 0.9300 0.9300
Methods 0.9300 0.9400 0.9300
Results 0.9200 0.9200 0.9200
Conclusions 0.9100 0.9000 0.9000
平均值 0.9200 0.9225 0.9200
类别 P R F1值
Purpose 0.9600 0.9500 0.9500
Methods 0.9400 0.9500 0.9400
Results 0.9400 0.9100 0.9300
Conclusions 0.9200 0.9300 0.9300
平均值 0.9400 0.9350 0.9375
样本量 类别 SVM
模型
DNN
模型
LSTM
模型
Att-BiLSTM
模型
10 000 Purpose 0.8900 0.8100 0.7600 0.9300
Methods 0.9200 0.8700 0.8500 0.9300
Results 0.8900 0.8100 0.7700 0.9200
Conclusions 0.8600 0.7900 0.7500 0.9000
50 000 Purpose 0.9100 0.8500 0.9200 0.9500
Methods 0.9300 0.8900 0.9200 0.9400
Results 0.9000 0.8400 0.8700 0.9300
Conclusions 0.8700 0.8100 0.8700 0.9300
因子 自由度 离差平方和 均方 F统计量 p
样本量
(整体)
1.0 0.014450 0.014450 5.14006 0.03075
样本量
(SVM方法)
1.0 0.000313 0.000313 0.51020 0.50188
样本量
(DNN方法)
1.0 0.001513 0.001513 1.32 0.29432
样本量
(LSTM方法)
1.0 0.025313 0.025313 17.30769 0.00594
样本量
(Att-BiLSTM方法)
1.0 0.000612 0.000612 4.2 0.08632
因子 自由度 离差平方和 均方 F统计量 p
SVM,
DNN, LSTM,
Att-BiLSTM
3.0 0.050837 0.016946 9.895377 0.000129
SVM, DNN 1.0 0.015625 0.015625 17.676768 0.000883
SVM, LSTM 1.0 0.013225 0.013225 4.862771 0.044665
SVM,
Att-BiLSTM
1.0 0.004225 0.004225 10.803653 0.005401
DNN, LSTM 1.0 0.000100 0.000100 0.032961 0.858538
DNN,
Att-BiLSTM
1.0 0.036100 0.036100 51.179747 0.000005
LSTM,
Att-BiLSTM
1.0 0.032400 0.032400 12.750527 0.003071
[1] Swales J M . Research Genres: Explorations and Applications [M]. Cambridge: Cambridge University Press, 2004: 228-229.
[2] Ribaupierre H D, Falquet G . User-Centric Design and Evaluation of a Semantic Annotation Model for Scientific Documents [C]//Proceedings of the 14th International Conference on Knowledge Technologies and Data-driven Business. New York: ACM, 2014: 40.
[3] Ribaupierre H D, Falquet G . An Automated Annotation Process for the SciDocAnnot Scientific Document Model [C] //Proceedings of the 5th International Workshop on Semantic Digital Archives. Osaka: International Workshop on Semantic Digital Archives, 2015: 30-41.
[4] Liakata M, Teufel S, Siddharthan A , et al. Corpora for the Conceptualisation and Zoning of Scientific Papers [C] //Proceedings of the 2010 International Conference on Language Resources and Evaluation. Valletta: European Languages Resources Association (ELRA), 2010: 105-108.
[5] Teufel S, Carletta J, Moens M . An Annotation Scheme for Discourse-level Argumentation in Research Articles [C] //Proceedings of the 9th Conference on European Chapter of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 1999: 110-117.
[6] Teufel S, Batchelor C . Towards Discipline-independent Argumentative Zoning: Evidence from Chemistry and Computational Linguistics [C]// Proceeding of the Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, 2009: 1493-1502.
[7] Fisas B, Ronzano F, Saggion H . A Multi-layered Annotated Corpus of Scientific Papers [C]//Proceedings of the 10th International Conference on Language Resources and Evaluation. European Language Resources Association, 2016: 3081-3088.
[8] 刘霞 . 英语学术论文摘要语步结构自动识别模型的构建[D]. 北京: 北京外国语大学, 2016.
[8] ( Liu Xia . Constructing a Model for the Automatic Identification of Move Structure in English Research Article Abstracts[D]. Beijing: Beijing Foreign Studies University, 2016.)
[9] 王立非, 刘霞 . 英语学术论文摘要语步结构自动识别模型的构建[J]. 外语电化教学, 2017(2):47-52, 66.
[9] ( Wang Lifei, Liu Xia . Constructing a Model for the Automatic Identification of Move Structure in English Research Article Abstracts[J]. Technology Enhance Foreign Language Education, 2017(2):47-52, 66.)
[10] Mikolov T, Chen K, Corrado G , et al. Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint, arXiv:1301.3781v3.
[11] Bengio Y, Delalleau O . On the Expressive Power of Deep Architectures [C]//Proceeding of the International Conference on Algorithmic Learning Theory. Springer-Verlag, 2011: 18-36.
[12] Hochreiter S, Schmidhuber J . Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780.
[13] Irsoy O, Cardie C . Opinion Mining with Deep Recurrent Neural Networks [C]//Proceeding of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 720-728.
[14] Mnih V, Heess N, Graves A , et al. Recurrent Models of Visual Attention[OL]. arXiv Preprint , arXiv: 1406.6247.
[1] Zhou Zeyu,Wang Hao,Zhao Zibo,Li Yueyan,Zhang Xiaoqin. Construction and Application of GCN Model for Text Classification with Associated Information[J]. 数据分析与知识发现, 2021, 5(9): 31-41.
[2] Gu Yaowen, Zhang Bowen, Zheng Si, Yang Fengchun, Li Jiao. Predicting Drug ADMET Properties Based on Graph Attention Network[J]. 数据分析与知识发现, 2021, 5(8): 76-85.
[3] Xu Yuemei, Wang Zihou, Wu Zixin. Predicting Stock Trends with CNN-BiLSTM Based Multi-Feature Integration Model[J]. 数据分析与知识发现, 2021, 5(7): 126-138.
[4] Zhang Le, Leng Jidong, Lv Xueqiang, Cui Zhuo, Wang Lei, You Xindong. RLCPAR: A Rewriting Model for Chinese Patent Abstracts Based on Reinforcement Learning[J]. 数据分析与知识发现, 2021, 5(7): 59-69.
[5] Zhao Danning,Mu Dongmei,Bai Sen. Automatically Extracting Structural Elements of Sci-Tech Literature Abstracts Based on Deep Learning[J]. 数据分析与知识发现, 2021, 5(7): 70-80.
[6] Huang Mingxuan,Jiang Caoqing,Lu Shoudong. Expanding Queries Based on Word Embedding and Expansion Terms[J]. 数据分析与知识发现, 2021, 5(6): 115-125.
[7] Zhong Jiawa,Liu Wei,Wang Sili,Yang Heng. Review of Methods and Applications of Text Sentiment Analysis[J]. 数据分析与知识发现, 2021, 5(6): 1-13.
[8] Zhang Guobiao,Li Jie. Detecting Social Media Fake News with Semantic Consistency Between Multi-model Contents[J]. 数据分析与知识发现, 2021, 5(5): 21-29.
[9] Han Pu,Zhang Zhanpeng,Zhang Mingtao,Gu Liang. Normalizing Chinese Disease Names with Multi-feature Fusion[J]. 数据分析与知识发现, 2021, 5(5): 83-94.
[10] Wang Nan,Li Hairong,Tan Shuru. Predicting of Public Opinion Reversal with Improved SMOTE Algorithm and Ensemble Learning[J]. 数据分析与知识发现, 2021, 5(4): 37-48.
[11] Chang Chengyang,Wang Xiaodong,Zhang Shenglei. Polarity Analysis of Dynamic Political Sentiments from Tweets with Deep Learning Method[J]. 数据分析与知识发现, 2021, 5(3): 121-131.
[12] Feng Yong,Liu Yang,Xu Hongyan,Wang Rongbing,Zhang Yonggang. Recommendation Model Incorporating Neighbor Reviews for GRU Products[J]. 数据分析与知识发现, 2021, 5(3): 78-87.
[13] Cheng Bin,Shi Shuicai,Du Yuncheng,Xiao Shibin. Keyword Extraction for Journals Based on Part-of-Speech and BiLSTM-CRF Combined Model[J]. 数据分析与知识发现, 2021, 5(3): 101-108.
[14] Hu Haotian,Ji Jinfeng,Wang Dongbo,Deng Sanhong. An Integrated Platform for Food Safety Incident Entities Based on Deep Learning[J]. 数据分析与知识发现, 2021, 5(3): 12-24.
[15] Zhang Qi,Jiang Chuan,Ji Youshu,Feng Minxuan,Li Bin,Xu Chao,Liu Liu. Unified Model for Word Segmentation and POS Tagging of Multi-Domain Pre-Qin Literature[J]. 数据分析与知识发现, 2021, 5(3): 2-11.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn