Please wait a minute...
Data Analysis and Knowledge Discovery  2020, Vol. 4 Issue (5): 46-53    DOI: 10.11925/infotech.2096-3467.2019.1321
Current Issue | Archive | Adv Search |
Coreference Resolution Based on Dynamic Semantic Attention
Deng Siyi,Le Xiaoqiu()
National Science Library, Chinese Academy of Sciences, Beijing 100190, China
Department of Library, Information and Archives Management, School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190, China
Download: PDF (839 KB)   HTML ( 18
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper tries to more effectively identify the coreference, aiming to address the issues of ambiguous anaphor meaning and complex antecedent structure.[Methods] We established an end-to-end framework and used score ranking to identify the coreference relationships. Firstly, we calculated scores of all spans to retrieve the “mentions”. Then, we used scores of the candidate mention pairs to determine coreference relationship. We also built span representation with external multiple semantic representations. Finally, we combined scores of the two parts to generate the final list.[Results] We examined our model with the OntoNotes benchmark datasets. The precision, recall and F1 values of our model were 2.02%, 0.42% and 1.14% higher than those of the SOTA model.[Limitations] The training data sets only collected news, talk shows, or weblogs. More sci-tech literature is needed to further improve the model’s performance.[Conclusions] The proposed model could more effectively identify coreferences.

Key wordsCoreference Resolution      Dynamic Semantic Attention      Ranking Model      Deep Learning     
Received: 20 November 2019      Published: 15 June 2020
ZTFLH:  G35  
Corresponding Authors: Le Xiaoqiu     E-mail: lexq@mail.las.ac.cn

Cite this article:

Deng Siyi,Le Xiaoqiu. Coreference Resolution Based on Dynamic Semantic Attention. Data Analysis and Knowledge Discovery, 2020, 4(5): 46-53.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2019.1321     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2020/V4/I5/46

Coreference Resolution Model Based on Dynamic Semantic Attention
模型 平均准确率(%) 平均召回率(%) 平均F1值(%)
E2E模型[5] 72.58 65.12 68.64
本文模型 74.60 65.54 69.78
Δ +2.02 +0.42 +1.14
Models Performance
[1] Steinberger J, Poesio M, Kabadjov M A, et al. Two Uses of Anaphora Resolution in Summarization[J]. Information Processing and Management, 2007,43(6):1663-1680.
[2] Gabbard R, Freedman M, Weischedel R . Coreference for Learning to Extract Relations: Yes, Virginia, Coreference Matters [C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011.
[3] Mitkov R, Evans R, Orăsan C , et al. Coreference Resolution: To What Extent Does It Help NLP Applications? [C]// Proceedings of the 2012 International Conference on Text, Speech and Dialogue. 2012.
[4] Kilicoglu H, Fiszman M, Demnerfushman D . Interpreting Consumer Health Questions: The Role of Anaphora and Ellipsis [C]// Proceedings of the 2013 Workshop on Biomedical Natural Language Processing. 2013.
[5] Lee K, He L, Lewis M, et al. End-to-End Neural Coreference Resolution[OL]. arXiv Preprint, arXiv: 1707.07045, 2017.
[6] Lappin S, Leass H J. An Algorithm for Pronominal Anaphora Resolution[J]. Computational Linguistics, 1994,20(4):535-561.
[7] Ng V. Machine Learning for Coreference Resolution: From Local Classification to Global Ranking[C]//Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05). 2005.
[8] Ng V . Supervised Ranking for Pronoun Resolution: Some Recent Improvements [C]// Proceedings of the 20th National Conference on Artificial Intelligence. 2005.
[9] Li D, Miller T, Schuler W . A Pronoun Anaphora Resolution System Based on Factorial Hidden Markov Models [C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011.
[10] Zhang H, Song Y, Song Y Q. Incorporating Context and External Knowledge for Pronoun Coreference Resolution[OL]. arXiv Preprint, arXiv:1905.10238, 2019.
[11] Subramanian S, Roth D. Improving Generalization in Coreference Resolution via Adversarial Training[OL]. arXiv Preprint, arXiv:1908.04728, 2019.
[12] Zhang R, Santos C N D, Yasunaga M, et al. Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering[OL]. arXiv Preprint, arXiv:1805. 04893, 2018.
[13] Peng H, Khashabi D, Roth D. Solving Hard Coreference Problems[OL]. arXiv Preprint, arXiv: 1907. 05524, 2019.
[14] Jindal P, Roth D . End-to-End Coreference Resolution for Clinical Narratives [C]// Proceedings of the 23rd International Joint Conference on Artificial Intelligence. 2013.
[15] Trieu L, Nguyen N, Miwa M , et al. Investigating Domain-Specific Information for Neural Coreference Resolution on Biomedical Texts [C]// Proceedings of the 2018 Workshop on Biomedical Natural Language Processing. 2018.
[16] Rahman A, Ng V . Coreference Resolution with World Knowledge [C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011.
[17] Zhang H, Song Y, Song Y, et al. Knowledge-Aware Pronoun Coreference Resolution[OL]. arXiv Preprint, arXiv:1907.03663, 2019.
[18] Joshi M, Chen D, Liu Y, et al. SpanBERT: Improving Pre-training by Representing and Predicting Spans[OL]. arXiv Preprint, arXiv: 1907.10529, 2019.
[19] Song Y, Shi S . Complementary Learning of Word Embeddings [C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018.
[20] Song Y, Shi S, Li J , et al. Directional Skip-Gram: Explicitly Distinguishing Left and Right Context for Word Embeddings [C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. 2018.
[21] Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780.
[22] Bahdanau D, Cho K, Bengio Y . Neural Machine Translation by Jointly Learning to Align and Translate[OL]. arXiv Preprint, arXiv:1409.0473, 2014.
[23] Pennington J, Socher R, Manning C . GloVe: Global Vectors for Word Representation [C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014.
[24] Peters M E, Neumann M, Iyyer M, et al. Deep Contextualized Word Representations[OL]. arXiv Preprint, arXiv:1802.05365, 2018.
[25] Zhang X, Zhao J, Lecun Y. Character-level Convolutional Networks for Text Classification[OL]. arXiv Preprint, arXiv:1509.01626, 2015.
[26] Pradhan S, Moschitti A, Xue N , et al. CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes [C]// Proceedings of Joint Conference on EMNLP and CoNLL-Shared Task. 2012.
[27] Vilain M B, Burger J D, Aberdeen J S , et al. A Model-Theoretic Coreference Scoring Scheme [C]// Proceedings of the 6th Conference on Message Understanding. 1995.
[28] Bagga A, Baldwin B . Algorithms for Scoring Coreference Chains [C]// Proceedings of the 1st International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference. 1998.
[29] Luo X . On Coreference Resolution Performance Metrics [C]// Proceedings of the 2005 Conference on Human Language Technology and Empirical Methods in Natural Language Processing. 2005.
[30] Nair V, Hinton G E. Rectified Linear Units Improve Restricted Boltzmann Machines[C]//Proceedings of the 27th International Conference on Machine Learning (ICML-10). 2010.
[31] Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[OL]. arXiv Preprint, arXiv:1412.6980, 2014.
[1] Zhou Zeyu,Wang Hao,Zhao Zibo,Li Yueyan,Zhang Xiaoqin. Construction and Application of GCN Model for Text Classification with Associated Information[J]. 数据分析与知识发现, 2021, 5(9): 31-41.
[2] Tan Ying, Tang Yifei. Extracting Citation Contents with Coreference Resolution[J]. 数据分析与知识发现, 2021, 5(8): 25-33.
[3] Zhao Danning,Mu Dongmei,Bai Sen. Automatically Extracting Structural Elements of Sci-Tech Literature Abstracts Based on Deep Learning[J]. 数据分析与知识发现, 2021, 5(7): 70-80.
[4] Xu Yuemei, Wang Zihou, Wu Zixin. Predicting Stock Trends with CNN-BiLSTM Based Multi-Feature Integration Model[J]. 数据分析与知识发现, 2021, 5(7): 126-138.
[5] Huang Mingxuan,Jiang Caoqing,Lu Shoudong. Expanding Queries Based on Word Embedding and Expansion Terms[J]. 数据分析与知识发现, 2021, 5(6): 115-125.
[6] Zhong Jiawa,Liu Wei,Wang Sili,Yang Heng. Review of Methods and Applications of Text Sentiment Analysis[J]. 数据分析与知识发现, 2021, 5(6): 1-13.
[7] Zhang Guobiao,Li Jie. Detecting Social Media Fake News with Semantic Consistency Between Multi-model Contents[J]. 数据分析与知识发现, 2021, 5(5): 21-29.
[8] Chang Chengyang,Wang Xiaodong,Zhang Shenglei. Polarity Analysis of Dynamic Political Sentiments from Tweets with Deep Learning Method[J]. 数据分析与知识发现, 2021, 5(3): 121-131.
[9] Feng Yong,Liu Yang,Xu Hongyan,Wang Rongbing,Zhang Yonggang. Recommendation Model Incorporating Neighbor Reviews for GRU Products[J]. 数据分析与知识发现, 2021, 5(3): 78-87.
[10] Hu Haotian,Ji Jinfeng,Wang Dongbo,Deng Sanhong. An Integrated Platform for Food Safety Incident Entities Based on Deep Learning[J]. 数据分析与知识发现, 2021, 5(3): 12-24.
[11] Zhang Qi,Jiang Chuan,Ji Youshu,Feng Minxuan,Li Bin,Xu Chao,Liu Liu. Unified Model for Word Segmentation and POS Tagging of Multi-Domain Pre-Qin Literature[J]. 数据分析与知识发现, 2021, 5(3): 2-11.
[12] Lv Xueqiang,Luo Yixiong,Li Jiaquan,You Xindong. Review of Studies on Detecting Chinese Patent Infringements[J]. 数据分析与知识发现, 2021, 5(3): 60-68.
[13] Cheng Bin,Shi Shuicai,Du Yuncheng,Xiao Shibin. Keyword Extraction for Journals Based on Part-of-Speech and BiLSTM-CRF Combined Model[J]. 数据分析与知识发现, 2021, 5(3): 101-108.
[14] Li Danyang, Gan Mingxin. Music Recommendation Method Based on Multi-Source Information Fusion[J]. 数据分析与知识发现, 2021, 5(2): 94-105.
[15] Yu Chuanming, Zhang Zhengang, Kong Lingge. Comparing Knowledge Graph Representation Models for Link Prediction[J]. 数据分析与知识发现, 2021, 5(11): 29-44.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn