Coreference Resolution Based on Dynamic Semantic Attention
Deng Siyi,Le Xiaoqiu()
National Science Library, Chinese Academy of Sciences, Beijing 100190, China Department of Library, Information and Archives Management, School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190, China
[Objective] This paper tries to more effectively identify the coreference, aiming to address the issues of ambiguous anaphor meaning and complex antecedent structure.[Methods] We established an end-to-end framework and used score ranking to identify the coreference relationships. Firstly, we calculated scores of all spans to retrieve the “mentions”. Then, we used scores of the candidate mention pairs to determine coreference relationship. We also built span representation with external multiple semantic representations. Finally, we combined scores of the two parts to generate the final list.[Results] We examined our model with the OntoNotes benchmark datasets. The precision, recall and F1 values of our model were 2.02%, 0.42% and 1.14% higher than those of the SOTA model.[Limitations] The training data sets only collected news, talk shows, or weblogs. More sci-tech literature is needed to further improve the model’s performance.[Conclusions] The proposed model could more effectively identify coreferences.
邓思艺,乐小虬. 基于动态语义注意力的指代消解方法[J]. 数据分析与知识发现, 2020, 4(5): 46-53.
Deng Siyi,Le Xiaoqiu. Coreference Resolution Based on Dynamic Semantic Attention. Data Analysis and Knowledge Discovery, 2020, 4(5): 46-53.
Steinberger J, Poesio M, Kabadjov M A, et al. Two Uses of Anaphora Resolution in Summarization[J]. Information Processing and Management, 2007,43(6):1663-1680.
[2]
Gabbard R, Freedman M, Weischedel R . Coreference for Learning to Extract Relations: Yes, Virginia, Coreference Matters [C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011.
[3]
Mitkov R, Evans R, Orăsan C , et al. Coreference Resolution: To What Extent Does It Help NLP Applications? [C]// Proceedings of the 2012 International Conference on Text, Speech and Dialogue. 2012.
[4]
Kilicoglu H, Fiszman M, Demnerfushman D . Interpreting Consumer Health Questions: The Role of Anaphora and Ellipsis [C]// Proceedings of the 2013 Workshop on Biomedical Natural Language Processing. 2013.
[5]
Lee K, He L, Lewis M, et al. End-to-End Neural Coreference Resolution[OL]. arXiv Preprint, arXiv: 1707.07045, 2017.
[6]
Lappin S, Leass H J. An Algorithm for Pronominal Anaphora Resolution[J]. Computational Linguistics, 1994,20(4):535-561.
[7]
Ng V. Machine Learning for Coreference Resolution: From Local Classification to Global Ranking[C]//Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05). 2005.
[8]
Ng V . Supervised Ranking for Pronoun Resolution: Some Recent Improvements [C]// Proceedings of the 20th National Conference on Artificial Intelligence. 2005.
[9]
Li D, Miller T, Schuler W . A Pronoun Anaphora Resolution System Based on Factorial Hidden Markov Models [C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011.
[10]
Zhang H, Song Y, Song Y Q. Incorporating Context and External Knowledge for Pronoun Coreference Resolution[OL]. arXiv Preprint, arXiv:1905.10238, 2019.
[11]
Subramanian S, Roth D. Improving Generalization in Coreference Resolution via Adversarial Training[OL]. arXiv Preprint, arXiv:1908.04728, 2019.
[12]
Zhang R, Santos C N D, Yasunaga M, et al. Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering[OL]. arXiv Preprint, arXiv:1805. 04893, 2018.
[13]
Peng H, Khashabi D, Roth D. Solving Hard Coreference Problems[OL]. arXiv Preprint, arXiv: 1907. 05524, 2019.
[14]
Jindal P, Roth D . End-to-End Coreference Resolution for Clinical Narratives [C]// Proceedings of the 23rd International Joint Conference on Artificial Intelligence. 2013.
[15]
Trieu L, Nguyen N, Miwa M , et al. Investigating Domain-Specific Information for Neural Coreference Resolution on Biomedical Texts [C]// Proceedings of the 2018 Workshop on Biomedical Natural Language Processing. 2018.
[16]
Rahman A, Ng V . Coreference Resolution with World Knowledge [C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011.
[17]
Zhang H, Song Y, Song Y, et al. Knowledge-Aware Pronoun Coreference Resolution[OL]. arXiv Preprint, arXiv:1907.03663, 2019.
[18]
Joshi M, Chen D, Liu Y, et al. SpanBERT: Improving Pre-training by Representing and Predicting Spans[OL]. arXiv Preprint, arXiv: 1907.10529, 2019.
[19]
Song Y, Shi S . Complementary Learning of Word Embeddings [C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018.
[20]
Song Y, Shi S, Li J , et al. Directional Skip-Gram: Explicitly Distinguishing Left and Right Context for Word Embeddings [C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. 2018.
[21]
Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780.
[22]
Bahdanau D, Cho K, Bengio Y . Neural Machine Translation by Jointly Learning to Align and Translate[OL]. arXiv Preprint, arXiv:1409.0473, 2014.
[23]
Pennington J, Socher R, Manning C . GloVe: Global Vectors for Word Representation [C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014.
[24]
Peters M E, Neumann M, Iyyer M, et al. Deep Contextualized Word Representations[OL]. arXiv Preprint, arXiv:1802.05365, 2018.
[25]
Zhang X, Zhao J, Lecun Y. Character-level Convolutional Networks for Text Classification[OL]. arXiv Preprint, arXiv:1509.01626, 2015.
[26]
Pradhan S, Moschitti A, Xue N , et al. CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes [C]// Proceedings of Joint Conference on EMNLP and CoNLL-Shared Task. 2012.
[27]
Vilain M B, Burger J D, Aberdeen J S , et al. A Model-Theoretic Coreference Scoring Scheme [C]// Proceedings of the 6th Conference on Message Understanding. 1995.
[28]
Bagga A, Baldwin B . Algorithms for Scoring Coreference Chains [C]// Proceedings of the 1st International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference. 1998.
[29]
Luo X . On Coreference Resolution Performance Metrics [C]// Proceedings of the 2005 Conference on Human Language Technology and Empirical Methods in Natural Language Processing. 2005.
[30]
Nair V, Hinton G E. Rectified Linear Units Improve Restricted Boltzmann Machines[C]//Proceedings of the 27th International Conference on Machine Learning (ICML-10). 2010.
[31]
Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[OL]. arXiv Preprint, arXiv:1412.6980, 2014.
