Please wait a minute...
Data Analysis and Knowledge Discovery  2020, Vol. 4 Issue (5): 46-53    DOI: 10.11925/infotech.2096-3467.2019.1321
Current Issue | Archive | Adv Search |
Coreference Resolution Based on Dynamic Semantic Attention
Deng Siyi,Le Xiaoqiu()
National Science Library, Chinese Academy of Sciences, Beijing 100190, China
Department of Library, Information and Archives Management, School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190, China
Download: PDF(839 KB)   HTML ( 15
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper tries to more effectively identify the coreference, aiming to address the issues of ambiguous anaphor meaning and complex antecedent structure.[Methods] We established an end-to-end framework and used score ranking to identify the coreference relationships. Firstly, we calculated scores of all spans to retrieve the “mentions”. Then, we used scores of the candidate mention pairs to determine coreference relationship. We also built span representation with external multiple semantic representations. Finally, we combined scores of the two parts to generate the final list.[Results] We examined our model with the OntoNotes benchmark datasets. The precision, recall and F1 values of our model were 2.02%, 0.42% and 1.14% higher than those of the SOTA model.[Limitations] The training data sets only collected news, talk shows, or weblogs. More sci-tech literature is needed to further improve the model’s performance.[Conclusions] The proposed model could more effectively identify coreferences.

Key wordsCoreference Resolution      Dynamic Semantic Attention      Ranking Model      Deep Learning     
Received: 20 November 2019      Published: 15 June 2020
ZTFLH:  G35  
Corresponding Authors: Le Xiaoqiu     E-mail: lexq@mail.las.ac.cn

Cite this article:

Deng Siyi,Le Xiaoqiu. Coreference Resolution Based on Dynamic Semantic Attention. Data Analysis and Knowledge Discovery, 2020, 4(5): 46-53.

URL:

http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2019.1321     OR     http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2020/V4/I5/46

Coreference Resolution Model Based on Dynamic Semantic Attention
模型 平均准确率(%) 平均召回率(%) 平均F1值(%)
E2E模型[5] 72.58 65.12 68.64
本文模型 74.60 65.54 69.78
Δ +2.02 +0.42 +1.14
Models Performance
[1] Steinberger J, Poesio M, Kabadjov M A, et al. Two Uses of Anaphora Resolution in Summarization[J]. Information Processing and Management, 2007,43(6):1663-1680.
[2] Gabbard R, Freedman M, Weischedel R . Coreference for Learning to Extract Relations: Yes, Virginia, Coreference Matters [C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011.
[3] Mitkov R, Evans R, Orăsan C , et al. Coreference Resolution: To What Extent Does It Help NLP Applications? [C]// Proceedings of the 2012 International Conference on Text, Speech and Dialogue. 2012.
[4] Kilicoglu H, Fiszman M, Demnerfushman D . Interpreting Consumer Health Questions: The Role of Anaphora and Ellipsis [C]// Proceedings of the 2013 Workshop on Biomedical Natural Language Processing. 2013.
[5] Lee K, He L, Lewis M, et al. End-to-End Neural Coreference Resolution[OL]. arXiv Preprint, arXiv: 1707.07045, 2017.
[6] Lappin S, Leass H J. An Algorithm for Pronominal Anaphora Resolution[J]. Computational Linguistics, 1994,20(4):535-561.
[7] Ng V. Machine Learning for Coreference Resolution: From Local Classification to Global Ranking[C]//Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05). 2005.
[8] Ng V . Supervised Ranking for Pronoun Resolution: Some Recent Improvements [C]// Proceedings of the 20th National Conference on Artificial Intelligence. 2005.
[9] Li D, Miller T, Schuler W . A Pronoun Anaphora Resolution System Based on Factorial Hidden Markov Models [C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011.
[10] Zhang H, Song Y, Song Y Q. Incorporating Context and External Knowledge for Pronoun Coreference Resolution[OL]. arXiv Preprint, arXiv:1905.10238, 2019.
[11] Subramanian S, Roth D. Improving Generalization in Coreference Resolution via Adversarial Training[OL]. arXiv Preprint, arXiv:1908.04728, 2019.
[12] Zhang R, Santos C N D, Yasunaga M, et al. Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering[OL]. arXiv Preprint, arXiv:1805. 04893, 2018.
[13] Peng H, Khashabi D, Roth D. Solving Hard Coreference Problems[OL]. arXiv Preprint, arXiv: 1907. 05524, 2019.
[14] Jindal P, Roth D . End-to-End Coreference Resolution for Clinical Narratives [C]// Proceedings of the 23rd International Joint Conference on Artificial Intelligence. 2013.
[15] Trieu L, Nguyen N, Miwa M , et al. Investigating Domain-Specific Information for Neural Coreference Resolution on Biomedical Texts [C]// Proceedings of the 2018 Workshop on Biomedical Natural Language Processing. 2018.
[16] Rahman A, Ng V . Coreference Resolution with World Knowledge [C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011.
[17] Zhang H, Song Y, Song Y, et al. Knowledge-Aware Pronoun Coreference Resolution[OL]. arXiv Preprint, arXiv:1907.03663, 2019.
[18] Joshi M, Chen D, Liu Y, et al. SpanBERT: Improving Pre-training by Representing and Predicting Spans[OL]. arXiv Preprint, arXiv: 1907.10529, 2019.
[19] Song Y, Shi S . Complementary Learning of Word Embeddings [C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018.
[20] Song Y, Shi S, Li J , et al. Directional Skip-Gram: Explicitly Distinguishing Left and Right Context for Word Embeddings [C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. 2018.
[21] Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780.
[22] Bahdanau D, Cho K, Bengio Y . Neural Machine Translation by Jointly Learning to Align and Translate[OL]. arXiv Preprint, arXiv:1409.0473, 2014.
[23] Pennington J, Socher R, Manning C . GloVe: Global Vectors for Word Representation [C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014.
[24] Peters M E, Neumann M, Iyyer M, et al. Deep Contextualized Word Representations[OL]. arXiv Preprint, arXiv:1802.05365, 2018.
[25] Zhang X, Zhao J, Lecun Y. Character-level Convolutional Networks for Text Classification[OL]. arXiv Preprint, arXiv:1509.01626, 2015.
[26] Pradhan S, Moschitti A, Xue N , et al. CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes [C]// Proceedings of Joint Conference on EMNLP and CoNLL-Shared Task. 2012.
[27] Vilain M B, Burger J D, Aberdeen J S , et al. A Model-Theoretic Coreference Scoring Scheme [C]// Proceedings of the 6th Conference on Message Understanding. 1995.
[28] Bagga A, Baldwin B . Algorithms for Scoring Coreference Chains [C]// Proceedings of the 1st International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference. 1998.
[29] Luo X . On Coreference Resolution Performance Metrics [C]// Proceedings of the 2005 Conference on Human Language Technology and Empirical Methods in Natural Language Processing. 2005.
[30] Nair V, Hinton G E. Rectified Linear Units Improve Restricted Boltzmann Machines[C]//Proceedings of the 27th International Conference on Machine Learning (ICML-10). 2010.
[31] Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[OL]. arXiv Preprint, arXiv:1412.6980, 2014.
[1] Yu Chuanming,Yuan Sai,Zhu Xingyu,Lin Hongjun,Zhang Puliang,An Lu. Research on Deep Learning Based Topic Representation of Hot Events[J]. 数据分析与知识发现, 2020, 4(4): 1-14.
[2] Su Chuandong,Huang Xiaoxi,Wang Rongbo,Chen Zhiqun,Mao Junyu,Zhu Jiaying,Pan Yuhao. Identifying Chinese / English Metaphors with Word Embedding and Recurrent Neural Network[J]. 数据分析与知识发现, 2020, 4(4): 91-99.
[3] Liu Tong,Ni Weijian,Sun Yujian,Zeng Qingtian. Predicting Remaining Business Time with Deep Transfer Learning[J]. 数据分析与知识发现, 2020, 4(2/3): 134-142.
[4] Chuanming Yu,Haonan Li,Manyi Wang,Tingting Huang,Lu An. Knowledge Representation Based on Deep Learning:Network Perspective[J]. 数据分析与知识发现, 2020, 4(1): 63-75.
[5] Mengji Zhang,Wanyu Du,Nan Zheng. Predicting Stock Trends Based on News Events[J]. 数据分析与知识发现, 2019, 3(5): 11-18.
[6] Jingjing Pei,Xiaoqiu Le. Identifying Coordinate Text Blocks in Discourses[J]. 数据分析与知识发现, 2019, 3(5): 51-56.
[7] Zhixiong Zhang,Huan Liu,Liangping Ding,Pengmin Wu,Gaihong Yu. Identifying Moves of Research Abstracts with Deep Learning Methods[J]. 数据分析与知识发现, 2019, 3(12): 1-9.
[8] Li Yu,Li Qian,Changlei Fu,Huaming Zhao. Extracting Fine-grained Knowledge Units from Texts with Deep Learning[J]. 数据分析与知识发现, 2019, 3(1): 38-45.
[9] Changlei Fu,Li Qian,Huaping Zhang,Huaming Zhao,Jing Xie. Mining Innovative Topics Based on Deep Learning[J]. 数据分析与知识发现, 2019, 3(1): 46-54.
[10] Bengong Yu,Peihang Zhang,Qingtang Xu. Selecting Products Based on F-BiGRU Sentiment Analysis[J]. 数据分析与知识发现, 2018, 2(9): 22-30.
[11] Wei Lu,Mengqi Luo,Heng Ding,Xin Li. Image Annotation Tags by Deep Learning and Real Users: A Comparative Study[J]. 数据分析与知识发现, 2018, 2(5): 1-10.
[12] Guoming Feng,Xiaodong Zhang,Suhui Liu. Classifying Chinese Texts with CapsNet[J]. 数据分析与知识发现, 2018, 2(12): 68-76.
[13] Yanhui Xiao,Xin Wang,Wen’gang Feng,Huawei Tian,Shaozhong Wu,Lihua Li. Predicting Crime Locations Based on Long Short Term Memory and Convolutional Neural Networks[J]. 数据分析与知识发现, 2018, 2(10): 15-20.
[14] Wengang Feng,Jing Huang. Early Warning for Civil Aviation Security Checks Based on Deep Learning[J]. 数据分析与知识发现, 2018, 2(10): 46-53.
[15] Jiaheng Hu,Yonghua Cen,Chengyao Wu. Constructing Sentiment Dictionary with Deep Learning: Case Study of Financial Data[J]. 数据分析与知识发现, 2018, 2(10): 95-102.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn