|
|
Joint Extraction Model for Entities and Events with Multi-task Deep Learning |
Yu Chuanming(),Lin Hongjun,Zhang Zhengang |
School of Information and Safety Engineering, Zhongnan University of Economics and Law,Wuhan 430073, China |
|
|
Abstract [Objective] The study tries to improve the performance of entity and event extraction with the help of their correlation. [Methods] Based on the multi-task deep learning, we proposed a joint entity and event extraction model (MDL-J3E), which had the shared layer, the private layer, and the decoding layer. The shared layer generated common features. The private layer had the named entity recognition and event detection modules, which extracted features of the two subtasks based on their general features. The decoding layer analyzed features of each task and generated tag sequence following the constraint rules. [Results] We examined our model with the ACE2005 dataset. The F1 values were 84.15% in the named entity recognition task and 70.96% in the event detection task. [Limitations] We did not evaluate the proposed model with other information extraction scenarios. [Conclusions] Compared with the single task model, our multi-task model has better performance in both named entity recognition and event detection tasks.
|
Received: 31 August 2021
Published: 14 April 2022
|
|
Fund:National Natural Science Foundation of China(71974202);National Natural Science Foundation of China(71790612) |
Corresponding Authors:
Yu Chuanming,ORCID:0000-0001-7099-0853
E-mail: yucm@zuel.edu.cn
|
[1] |
宗成庆. 统计自然语言处理[M]. 北京: 清华大学出版社, 2008: 379-388.
|
[1] |
( Zong Chengqing. Statistical Natural Language Processing[M]. Beijing: Tsinghua University Press, 2008: 379-388.)
|
[2] |
郭剑毅, 薛征山, 余正涛, 等. 基于层叠条件随机场的旅游领域命名实体识别[J]. 中文信息学报, 2009, 23(5):47-52.
|
[2] |
( Guo Jianyi, Xue Zhengshan, Yu Zhengtao, et al. Named Entity Recognition for the Tourism Domain Based on Cascaded Conditional Random Fields[J]. Journal of Chinese Information Processing, 2009, 23(5):47-52.)
|
[3] |
冯元勇, 孙乐, 李文波, 等. 基于单字提示特征的中文命名实体识别快速算法[J]. 中文信息学报, 2008, 22(1):104-110.
|
[3] |
( Feng Yuanyong, Sun Le, Li Wenbo, et al. A Rapid Algorithm to Chinese Named Entity Recognition Based on Single Character Hints[J]. Journal of Chinese Information Processing, 2008, 22(1):104-110.)
|
[4] |
陈美杉, 夏晨曦. 肝癌患者在线提问的命名实体识别研究: 一种基于迁移学习的方法[J]. 数据分析与知识发现, 2019, 3(12):61-69.
|
[4] |
( Chen Meishan, Xia Chenxi. Identifying Entities of Online Questions from Cancer Patients Based on Transfer Learning[J]. Data Analysis and Knowledge Discovery, 2019, 3(12):61-69.)
|
[5] |
喻雪寒, 何琳, 徐健. 基于RoBERTa-CRF的古文历史事件抽取方法研究[J]. 数据分析与知识发现, 2021, 5(7):26-35.
|
[5] |
( Yu Xuehan, He Lin, Xu Jian. Extracting Events from Ancient Books Based on RoBERTa-CRF[J]. Data Analysis and Knowledge Discovery, 2021, 5(7):26-35.)
|
[6] |
Zhang Y, Yang Q. An Overview of Multi-task Learning[J]. National Science Review, 2018, 5(1):30-43.
doi: 10.1093/nsr/nwx105
|
[7] |
Dai J F, He K M, Sun J. Instance-aware Semantic Segmentation via Multi-task Network Cascades[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016: 3150-3158.
|
[8] |
Misra I, Shrivastava A, Gupta A, et al. Cross-stitch Networks for Multi-task Learning[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2016: 3994-4003.
|
[9] |
Cipolla R, Gal Y, Kendall A. Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 7482-7491.
|
[10] |
Li Q, Ji H, Huang L. Joint Event Extraction via Structured Prediction with Global Features[C]// Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 2013: 73-82.
|
[11] |
Liu J, Chen Y B, Liu K, et al. Event Detection via Gated Multilingual Attention Mechanism[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 4865-4872.
|
[12] |
王吉地, 郭军军, 黄于欣, 等. 融合依存信息和卷积神经网络的越南语新闻事件检测[J]. 南京大学学报(自然科学), 2020, 56(1):125-131.
|
[12] |
( Wang Jidi, Guo Junjun, Huang Yuxin, et al. Vietnamese News Event Detection Based on Converge Dependent Information and Convolutional Neural Networks[J]. Journal of Nanjing University (Natural Science), 2020, 56(1):125-131.)
|
[13] |
Nguyen T H, Grishman R. Graph Convolutional Networks with Argument-Aware Pooling for Event Detection[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 5900-5907.
|
[14] |
Liu S L, Chen Y B, Liu K, et al. Exploiting Argument Information to Improve Event Detection via Supervised Attention Mechanisms[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 1789-1798.
|
[15] |
Ji Y Z, Lin Y F, Gao J W, et al. Exploiting the Entity Type Sequence to Benefit Event Detection[C]// Proceedings of the 23rd Conference on Computational Natural Language Learning. 2019: 613-623.
|
[16] |
贺瑞芳, 段绍杨. 基于多任务学习的中文事件抽取联合模型[J]. 软件学报, 2019, 30(4):1015-1030.
|
[16] |
( He Ruifang, Duan Shaoyang. Joint Chinese Event Extraction Based Multi-Task Learning[J]. Journal of Software, 2019, 30(4):1015-1030.)
|
[17] |
仲伟峰, 杨航, 陈玉博, 等. 基于联合标注和全局推理的篇章级事件抽取[J]. 中文信息学报, 2019, 33(9):88-95, 106.
|
[17] |
( Zhong Weifeng, Yang Hang, Chen Yubo, et al. Document-Level Event Extraction Based on Joint Labeling and Global Reasoning[J]. Journal of Chinese Information Processing, 2019, 33(9):88-95, 106.)
|
[18] |
曹晓民, 史瑞刚. 多任务神经网络药物不良反应检测算法[J]. 控制工程, 2020, 27(7):1151-1156.
|
[18] |
( Cao Xiaomin, Shi Ruigang. Multi-Task Based Neural Network Algorithm for Detection of Drug Adverse Event[J]. Control Engineering of China, 2020, 27(7):1151-1156.)
|
[19] |
张贺, 刘茂福, 胡慧君, 等. 基于信息单元融合的新闻原子事件抽取[J]. 武汉大学学报(理学版), 2015, 61(2):139-144.
|
[19] |
( Zhang He, Liu Maofu, Hu Huijun, et al. Atomic Event Extraction Based on Information Unit Fusion[J]. Journal of Wuhan University (Natural Science Edition), 2015, 61(2):139-144.)
|
[20] |
Lin Y, Yang S Q, Stoyanov V, et al. A Multi-lingual Multi-task Architecture for Low-resource Sequence Labeling[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 799-809.
|
[21] |
Wang J, Kulkarni M, Preotiuc-Pietro D. Multi-domain Named Entity Recognition with Genre-aware and Agnostic Inference[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 8476-8488.
|
[22] |
杨晓辉, 毕雪华, 张琳琳, 等. 基于多任务的中文电子病历中命名实体识别研究[J]. 东北师大学报(自然科学版), 2020, 52(1):81-87.
|
[22] |
( Yang Xiaohui, Bi Xuehua, Zhang Linlin, et al. Multi-Task Based Chinese Electronic Medical Record Entity Recognition[J]. Journal of Northeast Normal University (Natural Science Edition), 2020, 52(1):81-87.)
|
[23] |
罗凌, 杨志豪, 宋雅文, 等. 基于笔画ELMo和多任务学习的中文电子病历命名实体识别研究[J]. 计算机学报, 2020, 43(10):1943-1957.
|
[23] |
( Luo Ling, Yang Zhihao, Song Yawen, et al. Chinese Clinical Named Entity Recognition Based on Stroke ELMo and Multi-Task Learning[J]. Chinese Journal of Computers, 2020, 43(10):1943-1957.)
|
[24] |
李青青, 杨志豪, 罗凌, 等. 基于多任务学习的生物医学实体关系抽取[J]. 中文信息学报, 2019, 33(8):84-92.
|
[24] |
( Li Qingqing, Yang Zhihao, Luo Ling, et al. A Multi-Task Learning Approach to Biomedical Entity Relation Extraction[J]. Journal of Chinese Information Processing, 2019, 33(8):84-92.)
|
[25] |
刘宗林, 张梅山, 甄冉冉, 等. 融入罪名关键词的法律判决预测多任务学习模型[J]. 清华大学学报(自然科学版), 2019, 59(7):497-504.
|
[25] |
( Liu Zonglin, Zhang Meishan, Zhen Ranran, et al. Multi-Task Learning Model for Legal Judgment Predictions with Charge Keywords[J]. Journal of Tsinghua University (Science and Technology), 2019, 59(7):497-504.)
|
[26] |
余传明, 李浩男, 安璐. 基于多任务深度学习的文本情感原因分析[J]. 广西师范大学学报(自然科学版), 2019, 37(1):50-61.
|
[26] |
( Yu Chuanming, Li Haonan, An Lu. Analysis of Text Emotion Cause Based on Multi-Task Deep Learning[J]. Journal of Guangxi Normal University (Natural Science Edition), 2019, 37(1):50-61.)
|
[27] |
Yang B S, Mitchell T M. Joint Extraction of Events and Entities within a Document Context[OL]. arXiv Preprint, arXiv: 1609.03632.
|
[28] |
Kruengkrai C, Nguyen T H, Aljunied S M, et al. Improving Low-Resource Named Entity Recognition Using Joint Sentence and Token Labeling[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 5898-5905.
|
[29] |
Martins P H, Marinho Z, Martins A F T. Joint Learning of Named Entity Recognition and Entity Linking[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop. 2019: 190-196.
|
[30] |
吴文涛, 李培峰, 朱巧明. 基于混合神经网络的实体和事件联合抽取方法[J]. 中文信息学报, 2019, 33(8):77-83.
|
[30] |
( Wu Wentao, Li Peifeng, Zhu Qiaoming. Joint Extraction of Entities and Events by a Hybrid Neural Network[J]. Journal of Chinese Information Processing, 2019, 33(8):77-83.)
|
[31] |
Martínez A H, Plank B. When is Multitask Learning Effective? Semantic Sequence Prediction under Varying Data Conditions[C]// Proceedings of the Conference of the 15th European Chapter of the Association for Computational Linguistics. 2017: 44-53.
|
[32] |
Strubell E, Verga P, Belanger D, et al. Fast and Accurate Entity Recognition with Iterated Dilated Convolutions[OL]. arXiv Preprint, arXiv: 1702.02098.
|
[33] |
Ju M Z, Miwa M, Ananiadou S. A Neural Layered Model for Nested Named Entity Recognition[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018: 1446-1459.
|
[34] |
Lin H Y, Lu Y J, Han X P, et al. Gazetteer-Enhanced Attentive Neural Networks for Named Entity Recognition[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019: 6232-6237.
|
[35] |
Luo Y, Zhao H. Bipartite Flat-Graph Network for Nested Named Entity Recognition[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 6408-6418.
|
[36] |
Fisher J, Vlachos A. Merge and Label: A Novel Neural Network Architecture for Nested NER[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 5840-5850.
|
[37] |
Chen Y B, Xu L H, Liu K, et al. Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015: 167-176.
|
[38] |
Liu S L, Chen Y B, He S Z, et al. Leveraging FrameNet to Improve Automatic Event Detection[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 2134-2143.
|
[39] |
Chen Y B, Liu S L, He S Z, et al. Event Extraction via Bidirectional Long Short-Term Memory Tensor Neural Networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 190-203.
|
[40] |
Nguyen T H, Cho K, Grishman R. Joint Event Extraction via Recurrent Neural Networks[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016: 300-309.
|
[41] |
Liu S B, Cheng R, Yu X M, et al. Exploiting Contextual Information via Dynamic Memory Network for Event Detection[OL]. arXiv Preprint, arXiv: 1810.03449.
|
[42] |
邱盈盈, 洪宇, 周文瑄, 等. 面向事件抽取的深度与主动联合学习方法[J]. 中文信息学报, 2018, 32(6):98-106.
|
[42] |
( Qiu Yingying, Hong Yu, Zhou Wenxuan, et al. Combining Deep Learning and Active Learning for Event Extraction[J]. Journal of Chinese Information Processing, 2018, 32(6):98-106.)
|
[43] |
Zhang T T, Ji H, Sil A. Joint Entity and Event Extraction with Generative Adversarial Imitation Learning[J]. Data Intelligence, 2019, 1(2):99-120.
doi: 10.1162/dint_a_00014
|
[44] |
陈斌, 周勇, 刘兵. 基于卷积双向长短期记忆网络的事件触发词抽取[J]. 计算机工程, 2019, 45(1):153-158.
|
[44] |
( Chen Bin, Zhou Yong, Liu Bing. Event Trigger Word Extraction Based on Convolutional Bidirectional Long Short Term Memory Network[J]. Computer Engineering, 2019, 45(1):153-158.)
|
[45] |
余传明, 王峰, 张贞港, 等. 基于表示学习的知识库问答模型研究[J]. 科技情报研究, 2021, 3(1):56-70.
|
[45] |
( Yu Chuanming, Wang Feng, Zhang Zhengang, et al. Research on Knowledge Graph Question Answering Model Based on Representation Learning[J]. Scientific Information Research, 2021, 3(1):56-70.)
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|