[Objective] The study tries to improve the performance of entity and event extraction with the help of their correlation. [Methods] Based on the multi-task deep learning, we proposed a joint entity and event extraction model (MDL-J3E), which had the shared layer, the private layer, and the decoding layer. The shared layer generated common features. The private layer had the named entity recognition and event detection modules, which extracted features of the two subtasks based on their general features. The decoding layer analyzed features of each task and generated tag sequence following the constraint rules. [Results] We examined our model with the ACE2005 dataset. The F1 values were 84.15% in the named entity recognition task and 70.96% in the event detection task. [Limitations] We did not evaluate the proposed model with other information extraction scenarios. [Conclusions] Compared with the single task model, our multi-task model has better performance in both named entity recognition and event detection tasks.
余传明, 林虹君, 张贞港. 基于多任务深度学习的实体和事件联合抽取模型*[J]. 数据分析与知识发现, 2022, 6(2/3): 117-128.
Yu Chuanming, Lin Hongjun, Zhang Zhengang. Joint Extraction Model for Entities and Events with Multi-task Deep Learning. Data Analysis and Knowledge Discovery, 2022, 6(2/3): 117-128.
( Guo Jianyi, Xue Zhengshan, Yu Zhengtao, et al. Named Entity Recognition for the Tourism Domain Based on Cascaded Conditional Random Fields[J]. Journal of Chinese Information Processing, 2009, 23(5):47-52.)
( Feng Yuanyong, Sun Le, Li Wenbo, et al. A Rapid Algorithm to Chinese Named Entity Recognition Based on Single Character Hints[J]. Journal of Chinese Information Processing, 2008, 22(1):104-110.)
( Chen Meishan, Xia Chenxi. Identifying Entities of Online Questions from Cancer Patients Based on Transfer Learning[J]. Data Analysis and Knowledge Discovery, 2019, 3(12):61-69.)
( Yu Xuehan, He Lin, Xu Jian. Extracting Events from Ancient Books Based on RoBERTa-CRF[J]. Data Analysis and Knowledge Discovery, 2021, 5(7):26-35.)
[6]
Zhang Y, Yang Q. An Overview of Multi-task Learning[J]. National Science Review, 2018, 5(1):30-43.
doi: 10.1093/nsr/nwx105
[7]
Dai J F, He K M, Sun J. Instance-aware Semantic Segmentation via Multi-task Network Cascades[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016: 3150-3158.
[8]
Misra I, Shrivastava A, Gupta A, et al. Cross-stitch Networks for Multi-task Learning[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2016: 3994-4003.
[9]
Cipolla R, Gal Y, Kendall A. Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 7482-7491.
[10]
Li Q, Ji H, Huang L. Joint Event Extraction via Structured Prediction with Global Features[C]// Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 2013: 73-82.
[11]
Liu J, Chen Y B, Liu K, et al. Event Detection via Gated Multilingual Attention Mechanism[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 4865-4872.
( Wang Jidi, Guo Junjun, Huang Yuxin, et al. Vietnamese News Event Detection Based on Converge Dependent Information and Convolutional Neural Networks[J]. Journal of Nanjing University (Natural Science), 2020, 56(1):125-131.)
[13]
Nguyen T H, Grishman R. Graph Convolutional Networks with Argument-Aware Pooling for Event Detection[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 5900-5907.
[14]
Liu S L, Chen Y B, Liu K, et al. Exploiting Argument Information to Improve Event Detection via Supervised Attention Mechanisms[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 1789-1798.
[15]
Ji Y Z, Lin Y F, Gao J W, et al. Exploiting the Entity Type Sequence to Benefit Event Detection[C]// Proceedings of the 23rd Conference on Computational Natural Language Learning. 2019: 613-623.
( Zhong Weifeng, Yang Hang, Chen Yubo, et al. Document-Level Event Extraction Based on Joint Labeling and Global Reasoning[J]. Journal of Chinese Information Processing, 2019, 33(9):88-95, 106.)
( Cao Xiaomin, Shi Ruigang. Multi-Task Based Neural Network Algorithm for Detection of Drug Adverse Event[J]. Control Engineering of China, 2020, 27(7):1151-1156.)
( Zhang He, Liu Maofu, Hu Huijun, et al. Atomic Event Extraction Based on Information Unit Fusion[J]. Journal of Wuhan University (Natural Science Edition), 2015, 61(2):139-144.)
[20]
Lin Y, Yang S Q, Stoyanov V, et al. A Multi-lingual Multi-task Architecture for Low-resource Sequence Labeling[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 799-809.
[21]
Wang J, Kulkarni M, Preotiuc-Pietro D. Multi-domain Named Entity Recognition with Genre-aware and Agnostic Inference[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 8476-8488.
( Yang Xiaohui, Bi Xuehua, Zhang Linlin, et al. Multi-Task Based Chinese Electronic Medical Record Entity Recognition[J]. Journal of Northeast Normal University (Natural Science Edition), 2020, 52(1):81-87.)
( Luo Ling, Yang Zhihao, Song Yawen, et al. Chinese Clinical Named Entity Recognition Based on Stroke ELMo and Multi-Task Learning[J]. Chinese Journal of Computers, 2020, 43(10):1943-1957.)
( Li Qingqing, Yang Zhihao, Luo Ling, et al. A Multi-Task Learning Approach to Biomedical Entity Relation Extraction[J]. Journal of Chinese Information Processing, 2019, 33(8):84-92.)
( Liu Zonglin, Zhang Meishan, Zhen Ranran, et al. Multi-Task Learning Model for Legal Judgment Predictions with Charge Keywords[J]. Journal of Tsinghua University (Science and Technology), 2019, 59(7):497-504.)
( Yu Chuanming, Li Haonan, An Lu. Analysis of Text Emotion Cause Based on Multi-Task Deep Learning[J]. Journal of Guangxi Normal University (Natural Science Edition), 2019, 37(1):50-61.)
[27]
Yang B S, Mitchell T M. Joint Extraction of Events and Entities within a Document Context[OL]. arXiv Preprint, arXiv: 1609.03632.
[28]
Kruengkrai C, Nguyen T H, Aljunied S M, et al. Improving Low-Resource Named Entity Recognition Using Joint Sentence and Token Labeling[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 5898-5905.
[29]
Martins P H, Marinho Z, Martins A F T. Joint Learning of Named Entity Recognition and Entity Linking[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop. 2019: 190-196.
( Wu Wentao, Li Peifeng, Zhu Qiaoming. Joint Extraction of Entities and Events by a Hybrid Neural Network[J]. Journal of Chinese Information Processing, 2019, 33(8):77-83.)
[31]
Martínez A H, Plank B. When is Multitask Learning Effective? Semantic Sequence Prediction under Varying Data Conditions[C]// Proceedings of the Conference of the 15th European Chapter of the Association for Computational Linguistics. 2017: 44-53.
[32]
Strubell E, Verga P, Belanger D, et al. Fast and Accurate Entity Recognition with Iterated Dilated Convolutions[OL]. arXiv Preprint, arXiv: 1702.02098.
[33]
Ju M Z, Miwa M, Ananiadou S. A Neural Layered Model for Nested Named Entity Recognition[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018: 1446-1459.
[34]
Lin H Y, Lu Y J, Han X P, et al. Gazetteer-Enhanced Attentive Neural Networks for Named Entity Recognition[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019: 6232-6237.
[35]
Luo Y, Zhao H. Bipartite Flat-Graph Network for Nested Named Entity Recognition[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 6408-6418.
[36]
Fisher J, Vlachos A. Merge and Label: A Novel Neural Network Architecture for Nested NER[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 5840-5850.
[37]
Chen Y B, Xu L H, Liu K, et al. Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015: 167-176.
[38]
Liu S L, Chen Y B, He S Z, et al. Leveraging FrameNet to Improve Automatic Event Detection[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 2134-2143.
[39]
Chen Y B, Liu S L, He S Z, et al. Event Extraction via Bidirectional Long Short-Term Memory Tensor Neural Networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 190-203.
[40]
Nguyen T H, Cho K, Grishman R. Joint Event Extraction via Recurrent Neural Networks[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016: 300-309.
[41]
Liu S B, Cheng R, Yu X M, et al. Exploiting Contextual Information via Dynamic Memory Network for Event Detection[OL]. arXiv Preprint, arXiv: 1810.03449.
( Qiu Yingying, Hong Yu, Zhou Wenxuan, et al. Combining Deep Learning and Active Learning for Event Extraction[J]. Journal of Chinese Information Processing, 2018, 32(6):98-106.)
[43]
Zhang T T, Ji H, Sil A. Joint Entity and Event Extraction with Generative Adversarial Imitation Learning[J]. Data Intelligence, 2019, 1(2):99-120.
doi: 10.1162/dint_a_00014
( Chen Bin, Zhou Yong, Liu Bing. Event Trigger Word Extraction Based on Convolutional Bidirectional Long Short Term Memory Network[J]. Computer Engineering, 2019, 45(1):153-158.)
( Yu Chuanming, Wang Feng, Zhang Zhengang, et al. Research on Knowledge Graph Question Answering Model Based on Representation Learning[J]. Scientific Information Research, 2021, 3(1):56-70.)