[Objective] This study proposes a joint event extraction model employing an automatically constructed template to leverage the knowledge of pre-trained language models, aiming to improve the existing event extraction models relying on sequence labeling and text generation. [Methods] Firstly, we designed an automatic template construction strategy based on the Event Prompt to generate unified prompt templates. Then, we introduced the Event Prompt Embedding layer for the Event Prompt at the encoding level. Next, we used the BART model to capture the semantic information of the sentence and generated the corresponding prediction sequence. Finally, we jointly extracted trigger words and event arguments from the prediction sequences. [Results] In a dataset containing complex event information, the F1 values for event trigger and argument extraction reached 77.67% and 65.06%, which were 2.43% and 1.62% higher than the optimal baseline method. [Limitations] The proposed model could only work with sentence-level texts and optimize the Event Prompt at the encoding layer. [Conclusions] The proposed model can reduce the template construction cost while maintaining the same or even better performance. The model could recognize text with complex event information and improve the multi-label classification for event elements.
陈诺, 李旭晖. 一种基于模板提示学习的事件抽取方法*[J]. 数据分析与知识发现, 2023, 7(6): 86-98.
Chen Nuo, Li Xuhui. An Event Extraction Method Based on Template Prompt Learning. Data Analysis and Knowledge Discovery, 2023, 7(6): 86-98.
Hogenboom F, Frasincar F, Kaymak U, et al. A Survey of Event Extraction Methods from Text for Decision Support Systems[J]. Decision Support Systems, 2016, 85(C): 12-22.
doi: 10.1016/j.dss.2016.02.006
[2]
Ahn D. The Stages of Event Extraction[C]// Proceedings of the Workshop on Annotating and Reasoning about Time and Events. 2006: 1-8.
[3]
Jiang Z B, Xu F F, Araki J, et al. How Can We Know What Language Models Know?[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 423-438.
doi: 10.1162/tacl_a_00324
[4]
Riloff E, Shoen J. Automatically Acquiring Conceptual Patterns Without an Annotated Corpus[C]// Proceedings of the 3rd Workshop on Very Large Corpora. 1995: 148-161.
[5]
Riloff E. Automatically Constructing a Dictionary for Information Extraction Tasks[C]// Proceedings of the 11th National Conference on Artificial Intelligence. 1993: 811-816.
[6]
Feldman R, Rosenfeld B, Bar-Haim R, et al. The Stock Sonar—Sentiment Analysis of Stocks Based on a Hybrid Approach[C]// Proceedings of the 23rd Innovative Applications of Artificial Intelligence Conference. 2011: 1642-1647.
[7]
Chieu H L, Ng H T. A Maximum Entropy Approach to Information Extraction from Semi-structured and Free Text[C]// Proceedings of the 18th National Conference on Artificial Intelligence. 2002: 786-791.
(Zhao Yanyan, Qin Bing, Che Wanxiang, et al. Research on Chinese Event Extraction[J]. Journal of Chinese Information Processing, 2008, 22(1): 3-8.)
[9]
Chen Y B, Xu L H, Liu K, et al. Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1:Long Papers). 2015: 167-176.
[10]
Nguyen T H, Cho K, Grishman R. Joint Event Extraction via Recurrent Neural Networks[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2016: 300-309.
[11]
Zeng Y, Feng Y S, Ma R, et al. Scale up Event Extraction Learning via Automatic Training Data Generation[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018: 6045-6052.
[12]
Yang H, Chen Y B, Liu K, et al. Dcfee: A Document-Level Chinese Financial Event Extraction System Based on Automatically Labeled Training Data[C]// Proceedings of ACL 2018, System Demonstrations. 2018: 50-55.
(Jia Zhen, Ding Zehua, Chen Yanping, et al. Research on Event Extraction Method for Judicial Data[J]. Computer Engineering and Applications, 2023, 59(6): 277-282.)
doi: 10.3778/j.issn.1002-8331.2109-0497
[14]
Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1 (Long and Short Papers). 2019: 4171-4186.
[15]
Yang S, Feng D W, Qiao L B, et al. Exploring Pre-trained Language Models for Event Extraction and Generation[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 5284-5294.
[16]
Lin Y, Ji H, Huang F, et al. A Joint Neural Model for Information Extraction with Global Features[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 7999-8009.
(Chen Xingyue, Ni Liping, Ni Zhiwei. Extracting Financial Events with ELECTRA and Part-of-Speech[J]. Data Analysis and Knowledge Discovery, 2021, 5(7): 36-47.)
(Gao Su, Tao Hu, Jiang Yanzhao, et al. Sentence-Level Joint Event Extraction of Traditional Chinese Medical Literature[J]. Technology Intelligence Engineering, 2021, 7(5): 15-29.)
(Yu Chuanming, Lin Hongjun, Zhang Zhengang. Joint Extraction Model for Entities and Events with Multi-Task Deep Learning[J]. Data Analysis and Knowledge Discovery, 2022, 6(2/3): 117-128.)
[20]
Brown T B, Mann B, Ryder N, et al. Language Models are Few-Shot Learners[C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020: 1877-1901.
[21]
Liu P, Yuan W, Fu J, et al. Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing[OL]. arXiv Preprint, arXiv: 2107.13586.
[22]
Zhong R Q, Lee K, Zhang Z, et al. Adapting Language Models for Zero-Shot Learning by Meta-Tuning on Dataset and Prompt Collections[C]// Proceedings of Findings of the Association for Computational Linguistics:EMNLP 2021. 2021: 2856-2878.
[23]
Shin T, Razeghi Y, Logan R L, et al. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020: 4222-4235.
[24]
Liu X, Zheng Y, Du Z, et al. GPT Understands, Too[OL]. arXiv Preprint, arXiv: 2103.10385.
[25]
Du X Y, Cardie C. Event Extraction by Answering (Almost) Natural Questions[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020: 671-683.
(Liu Zeyi, Yu Wenhua, Hong Zhiyong, et al. Chinese Event Extraction Using Question Answering[J]. Computer Engineering and Applications, 2023, 59(2): 153-160.)
doi: 10.3778/j.issn.1002-8331.2107-0157
[27]
Li S, Ji H, Han J W. Document-Level Event Argument Extraction by Conditional Generation[C]// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2021: 894-908.
[28]
Lewis M, Liu Y H, Goyal N, et al. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 7871-7880.
[29]
Lin J J, Jian J, Chen Q. Eliciting Knowledge from Language Models for Event Extraction[OL]. arXiv Preprint, arXiv: 2109.05190.
[30]
Raffel C, Shazeer N, Roberts A, et al. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer[J]. Journal of Machine Learning Research, 2020, 21(1): 5485-5551.
[31]
Allen-Zhu Z, Li Y, Song Z. A Convergence Theory for Deep Learning via Over-parameterization[C]// Proceedings of the 36th International Conference on Machine Learning. 2019: 242-252.
[32]
Shao Y, Geng Z, Liu Y, et al. CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation[OL]. arXiv Preprint, arXiv: 2109.05729.
(Li Xuhui, Cheng Wei, Tang Xiaoya, et al. A Joint Extraction Method of Financial Events Based on Multi-Layer Convolutional Neural Networks[J]. Library and Information Service, 2021, 65(24): 89-99.)
doi: 10.13266/j.issn.0252-3116.2021.24.010
[34]
Lample G, Ballesteros M, Subramanian S, et al. Neural Architectures for Named Entity Recognition[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2016: 260-270.
