Please wait a minute...
Data Analysis and Knowledge Discovery  2023, Vol. 7 Issue (6): 86-98    DOI: 10.11925/infotech.2096-3467.2022.0495
Current Issue | Archive | Adv Search |
An Event Extraction Method Based on Template Prompt Learning
Chen Nuo1,Li Xuhui1,2()
1School of Information Management, Wuhan University, Wuhan 430072, China
2Big Data Institute, Wuhan University, Wuhan 430072, China
Download: PDF (1794 KB)   HTML ( 15 ) 
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This study proposes a joint event extraction model employing an automatically constructed template to leverage the knowledge of pre-trained language models, aiming to improve the existing event extraction models relying on sequence labeling and text generation. [Methods] Firstly, we designed an automatic template construction strategy based on the Event Prompt to generate unified prompt templates. Then, we introduced the Event Prompt Embedding layer for the Event Prompt at the encoding level. Next, we used the BART model to capture the semantic information of the sentence and generated the corresponding prediction sequence. Finally, we jointly extracted trigger words and event arguments from the prediction sequences. [Results] In a dataset containing complex event information, the F1 values for event trigger and argument extraction reached 77.67% and 65.06%, which were 2.43% and 1.62% higher than the optimal baseline method. [Limitations] The proposed model could only work with sentence-level texts and optimize the Event Prompt at the encoding layer. [Conclusions] The proposed model can reduce the template construction cost while maintaining the same or even better performance. The model could recognize text with complex event information and improve the multi-label classification for event elements.
Key words： Chinese Event Extraction      Pre-trained Language Model      Prompt Learning      Joint Learning     
Received: 16 May 2022      Published: 09 August 2023
ZTFLH:  TP183  
  G202  
Fund:National Natural Science Foundation of China(91646206);National Social Science Fund of China(21&ZD334)
Corresponding Authors: Li Xuhui，ORCID：0000-0002-1155-3597，E-mail：lixuhui@whu.edu.cn。   
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Nuo Chen
Xuhui Li

Cite this article:

Chen Nuo, Li Xuhui. An Event Extraction Method Based on Template Prompt Learning. Data Analysis and Knowledge Discovery, 2023, 7(6): 86-98.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022.0495     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2023/V7/I6/86

Process of Event Extraction Based on Automatic Template Construction
Definition of the Strategy for Automatic Template Construction
Example of Template Construction
Structure of KEACT Model
Structure of Encoding Layer
数据集 样本总数 质押 股份股权转让 起诉 投资 减持 标注事件总数
训练集 1 906 549 1 088 382 766 451 3 236
验证集 271 93 140 50 94 64 441
测试集 546 171 300 98 210 147 926
总计 2 723 813 1 528 530 1 070 662 4 603
Dataset Sample Distribution
实验环境 环境配置
操作系统 Ubuntu16.04.7 LTS x86_64
CPU AMD Ryzen ThreadRipper 3970x
GPU GeForce RTX 2080Ti×2
内存 256GB
Python 3.8.3
深度学习框架 PyTorch（1.7.1） + Transformers（4.2.2）
Experimental Environment
参数 参数值
Epoch 100
Batch Size 8
Max Sequence Length 512
Learning Rate 2e-5
Optimizer AdamW
Model Parameters
模型 事件触发词抽取 事件论元抽取
P（%） R（%） F1（%） P（%） R（%） F1（%）
BERT-CRF 88.00 64.15 74.20 68.41 49.87 57.69
BERT-CNN-CRF 87.75 62.63 73.09 70.23 50.13 58.50
BERT-BiLSTM-CRF 89.83 63.93 74.70 71.11 50.61 59.13
EEQA无模板 85.73 65.55 74.30 69.76 50.02 58.26
EEQA有模板 89.69 64.79 75.24 73.44 49.89 59.42
CondiGen - - - 75.62 54.63 63.44
KEACT单事件模板 85.95 67.39 75.54 72.32 56.70 63.57
KEACT多事件模板 78.35 77.00 77.67 65.63 64.50 65.06
Experimental Results
模型 事件触发词抽取 事件论元抽取
P（%） R（%） F1（%） P（%） R（%） F1（%）
KEACT 78.35 77.00 77.67 65.63 64.50 65.06
-BiLSTM 80.38 73.87 76.98 65.07 59.80 62.33
-MLP 76.37 78.19 77.27 62.28 63.76 63.01
Experimental Results of the Action of Event Prompt Embedding Layer
Effect of Different Number of the Event Prompt on F1
Examples of Event Extraction Results of Different Algorithms
[1] Hogenboom F, Frasincar F, Kaymak U, et al. A Survey of Event Extraction Methods from Text for Decision Support Systems[J]. Decision Support Systems, 2016, 85(C): 12-22.
doi: 10.1016/j.dss.2016.02.006
[2] Ahn D. The Stages of Event Extraction[C]// Proceedings of the Workshop on Annotating and Reasoning about Time and Events. 2006: 1-8.
[3] Jiang Z B, Xu F F, Araki J, et al. How Can We Know What Language Models Know?[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 423-438.
doi: 10.1162/tacl_a_00324
[4] Riloff E, Shoen J. Automatically Acquiring Conceptual Patterns Without an Annotated Corpus[C]// Proceedings of the 3rd Workshop on Very Large Corpora. 1995: 148-161.
[5] Riloff E. Automatically Constructing a Dictionary for Information Extraction Tasks[C]// Proceedings of the 11th National Conference on Artificial Intelligence. 1993: 811-816.
[6] Feldman R, Rosenfeld B, Bar-Haim R, et al. The Stock Sonar—Sentiment Analysis of Stocks Based on a Hybrid Approach[C]// Proceedings of the 23rd Innovative Applications of Artificial Intelligence Conference. 2011: 1642-1647.
[7] Chieu H L, Ng H T. A Maximum Entropy Approach to Information Extraction from Semi-structured and Free Text[C]// Proceedings of the 18th National Conference on Artificial Intelligence. 2002: 786-791.
[8] 赵妍妍, 秦兵, 车万翔, 等. 中文事件抽取技术研究[J]. 中文信息学报, 2008, 22(1): 3-8.
[8] (Zhao Yanyan, Qin Bing, Che Wanxiang, et al. Research on Chinese Event Extraction[J]. Journal of Chinese Information Processing, 2008, 22(1): 3-8.)
[9] Chen Y B, Xu L H, Liu K, et al. Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1:Long Papers). 2015: 167-176.
[10] Nguyen T H, Cho K, Grishman R. Joint Event Extraction via Recurrent Neural Networks[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2016: 300-309.
[11] Zeng Y, Feng Y S, Ma R, et al. Scale up Event Extraction Learning via Automatic Training Data Generation[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018: 6045-6052.
[12] Yang H, Chen Y B, Liu K, et al. Dcfee: A Document-Level Chinese Financial Event Extraction System Based on Automatically Labeled Training Data[C]// Proceedings of ACL 2018, System Demonstrations. 2018: 50-55.
[13] 贾阵, 丁泽华, 陈艳平, 等. 面向司法数据的事件抽取方法研究[J]. 计算机工程与应用, 2023, 59(6): 277-282.
doi: 10.3778/j.issn.1002-8331.2109-0497
[13] (Jia Zhen, Ding Zehua, Chen Yanping, et al. Research on Event Extraction Method for Judicial Data[J]. Computer Engineering and Applications, 2023, 59(6): 277-282.)
doi: 10.3778/j.issn.1002-8331.2109-0497
[14] Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1 (Long and Short Papers). 2019: 4171-4186.
[15] Yang S, Feng D W, Qiao L B, et al. Exploring Pre-trained Language Models for Event Extraction and Generation[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 5284-5294.
[16] Lin Y, Ji H, Huang F, et al. A Joint Neural Model for Information Extraction with Global Features[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 7999-8009.
[17] 陈星月, 倪丽萍, 倪志伟. 基于ELECTRA模型与词性特征的金融事件抽取方法研究[J]. 数据分析与知识发现, 2021, 5(7): 36-47.
[17] (Chen Xingyue, Ni Liping, Ni Zhiwei. Extracting Financial Events with ELECTRA and Part-of-Speech[J]. Data Analysis and Knowledge Discovery, 2021, 5(7): 36-47.)
[18] 高甦, 陶浒, 蒋彦钊, 等. 中医文献的句子级联合事件抽取[J]. 情报工程, 2021, 7(5): 15-29.
[18] (Gao Su, Tao Hu, Jiang Yanzhao, et al. Sentence-Level Joint Event Extraction of Traditional Chinese Medical Literature[J]. Technology Intelligence Engineering, 2021, 7(5): 15-29.)
[19] 余传明, 林虹君, 张贞港. 基于多任务深度学习的实体和事件联合抽取模型[J]. 数据分析与知识发现, 2022, 6(2/3): 117-128.
[19] (Yu Chuanming, Lin Hongjun, Zhang Zhengang. Joint Extraction Model for Entities and Events with Multi-Task Deep Learning[J]. Data Analysis and Knowledge Discovery, 2022, 6(2/3): 117-128.)
[20] Brown T B, Mann B, Ryder N, et al. Language Models are Few-Shot Learners[C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020: 1877-1901.
[21] Liu P, Yuan W, Fu J, et al. Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing[OL]. arXiv Preprint, arXiv: 2107.13586.
[22] Zhong R Q, Lee K, Zhang Z, et al. Adapting Language Models for Zero-Shot Learning by Meta-Tuning on Dataset and Prompt Collections[C]// Proceedings of Findings of the Association for Computational Linguistics:EMNLP 2021. 2021: 2856-2878.
[23] Shin T, Razeghi Y, Logan R L, et al. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020: 4222-4235.
[24] Liu X, Zheng Y, Du Z, et al. GPT Understands, Too[OL]. arXiv Preprint, arXiv: 2103.10385.
[25] Du X Y, Cardie C. Event Extraction by Answering (Almost) Natural Questions[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020: 671-683.
[26] 刘泽旖, 余文华, 洪智勇, 等. 基于问题回答模式的中文事件抽取[J]. 计算机工程与应用, 2023, 59(2): 153-160.
doi: 10.3778/j.issn.1002-8331.2107-0157
[26] (Liu Zeyi, Yu Wenhua, Hong Zhiyong, et al. Chinese Event Extraction Using Question Answering[J]. Computer Engineering and Applications, 2023, 59(2): 153-160.)
doi: 10.3778/j.issn.1002-8331.2107-0157
[27] Li S, Ji H, Han J W. Document-Level Event Argument Extraction by Conditional Generation[C]// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2021: 894-908.
[28] Lewis M, Liu Y H, Goyal N, et al. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 7871-7880.
[29] Lin J J, Jian J, Chen Q. Eliciting Knowledge from Language Models for Event Extraction[OL]. arXiv Preprint, arXiv: 2109.05190.
[30] Raffel C, Shazeer N, Roberts A, et al. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer[J]. Journal of Machine Learning Research, 2020, 21(1): 5485-5551.
[31] Allen-Zhu Z, Li Y, Song Z. A Convergence Theory for Deep Learning via Over-parameterization[C]// Proceedings of the 36th International Conference on Machine Learning. 2019: 242-252.
[32] Shao Y, Geng Z, Liu Y, et al. CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation[OL]. arXiv Preprint, arXiv: 2109.05729.
[33] 李旭晖, 程威, 唐小雅, 等. 基于多层卷积神经网络的金融事件联合抽取方法[J]. 图书情报工作, 2021, 65(24): 89-99.
doi: 10.13266/j.issn.0252-3116.2021.24.010
[33] (Li Xuhui, Cheng Wei, Tang Xiaoya, et al. A Joint Extraction Method of Financial Events Based on Multi-Layer Convolutional Neural Networks[J]. Library and Information Service, 2021, 65(24): 89-99.)
doi: 10.13266/j.issn.0252-3116.2021.24.010
[34] Lample G, Ballesteros M, Subramanian S, et al. Neural Architectures for Named Entity Recognition[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2016: 260-270.
[1] Li Daifeng, Lin Kaixin, Li Xuting. Identifying Named Entities of Adverse Drug Reaction with Adversarial Transfer Learning[J]. 数据分析与知识发现, 2023, 7(3): 121-130.
[2] Ye Han,Sun Haichun,Li Xin,Jiao Kainan. Classification Model for Long Texts with Attention Mechanism and Sentence Vector Compression[J]. 数据分析与知识发现, 2022, 6(6): 84-94.
[3] Jing Shenqi, Zhao Youlin. Extracting Medical Entity Relationships with Domain-Specific Knowledge and Distant Supervision[J]. 数据分析与知识发现, 2022, 6(6): 105-114.
[4] Wang Yizhen,Ou Shiyan,Chen Jinju. Automatic Abstracting Civil Judgment Documents with Two-Stage Procedure[J]. 数据分析与知识发现, 2021, 5(5): 104-114.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn