Extracting Financial Events with ELECTRA and Part-of-Speech
Chen Xingyue,Ni Liping(),Ni Zhiwei
School of Management, Hefei University of Technology, Hefei 230009, China Key Laboratory of Process Optimization & Intelligent Decision-making, Ministry of Education, Hefei University of Technology, Hefei 230009, China
[Objective] This paper proposes a method to extract financial events based on the ELECTRA model and part-of-speech, aiming to address the issues of blurred entity boundaries and inaccurate extractions. [Methods] First, we input corpus to two models pre-trained by ELECTRA, which identified key entities, the original semantic information, and part-of-speech. Then, we used the BiGRU model to extract contextual semantic dependency and generated the original sequence tags. Finally, we addressed the issues of label deviation with the CRF model and extracted the financial events. [Results] We examined the new model with financial event dataset and found its F-value reached 70.96%, which was 20.74 percentage point higher than the BiLSTM-CRF model. [Limitations] The number of events in the dataset needs to be increased. The size of pre-trained model is large, which might be limited by the memory of GPU/TPU. [Conclusions] The model based on ELECTRA and part-of-speech could effectively identify the relationships among financial events to extract them.
陈星月, 倪丽萍, 倪志伟. 基于ELECTRA模型与词性特征的金融事件抽取方法研究*[J]. 数据分析与知识发现, 2021, 5(7): 36-47.
Chen Xingyue, Ni Liping, Ni Zhiwei. Extracting Financial Events with ELECTRA and Part-of-Speech. Data Analysis and Knowledge Discovery, 2021, 5(7): 36-47.
Ding X, Liao K, Liu T, et al. Event Representation Learning Enhanced with External Commonsense Knowledge[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019: 4894-4903.
[2]
Liang X, Cheng D, Yang F, et al. F-HMTC: Detecting Financial Events for Investment Decisions Based on Neural Hierarchical Multi-Label Text Classification[C]// Proceedings of the 29th International Joint Conference on Artificial Intelligence. 2020:4490-4496.
[3]
Wollmer M, Weninger F, Knaup T, et al. YouTube Movie Reviews: Sentiment Analysis in an Audio-Visual Context[J]. IEEE Intelligent Systems, 2013, 28(3):46-53.
doi: 10.1109/MIS.2013.34
[4]
Feldman R, Rosenfeld B, Bar-Haim R, et al. The Stock Sonar - Sentiment Analysis of Stocks Based on a Hybrid Approach[C]// Proceedings of the 23rd Conference on Innovative Applications of Artificial Intelligence. 2011: 1642-1647.
[5]
Arendarenko E, Kakkonen T. Ontology-Based Information and Event Extraction for Business Intelligence[C]// Proceedings of the 15th International Conference on Artificial Intelligence: Methodology, Systems, and Applications. Springer Berlin Heidelberg, 2012:89-102.
[6]
Hogenboom A, Hogenboom F, Frasincar F, et al. Semantics-based Information Extraction for Detecting Economic Events[J]. Multimedia Tools and Applications, 2012, 64(1):27-52.
doi: 10.1007/s11042-012-1122-0
[7]
Ronnqvist S, Sarlin P. Bank Distress in the News: Describing Events Through Deep Learning[J]. Neurocomputing, 2016, 264(11):57-70.
doi: 10.1016/j.neucom.2016.12.110
[8]
Jacobs G, Lefever E, Hoste V. Economic Event Detection in Company-Specific News Text[C]// Proceedings of the 1st Workshop on Economics and Natural Language Processing. 2018:1-10.
[9]
Yang H, Chen Y, Liu K, et al. DCFEE: A Document-Level Chinese Financial Event Extraction System Based on Automatically Labeled Training Data[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 50-55.
[10]
Zheng S, Cao W, Xu W, et al. Doc2EDAG: An End-to-End Document-level Framework for Chinese Financial Event Extraction[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019: 337-346.
[11]
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017). 2017: 5998-6008.
[12]
Mikolov T, Chen K, Corrado G, et al. Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint, arXiv: 1301.3781.
[13]
Pennington J, Socher R, Manning C. Glove: Global Vectors for Word Representation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014:1532-1543.
[14]
Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv: 1810.04805.
[15]
Liu Y, Ott M, Goyal N, et al. RoBERTa: A Robustly Optimized BERT Pretraining Approach[OL]. arXiv Preprint, arXiv: 1907. 11692.
[16]
Yang S, Feng D, Qiao L, et al. Exploring Pre-trained Language Models for Event Extraction and Generation[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 5284-5294.
[17]
Du X, Cardie C. Event Extraction by Answering (Almost) Natural Questions[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP). 2020: 671-683.
[18]
Liu J, Chen Y, Liu K, et al. Event Extraction as Machine Reading Comprehension[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020: 1641-1651.
[19]
Ein-Dor L, Gera A, Toledo-Ronen O, et al. Financial Event Extraction Using Wikipedia-Based Weak Supervision[C]// Proceedings of the 2nd Workshop on Economics and Natural Language Processing. 2019: 10-15.
[20]
Li Q, Zhang Q. A Unified Model for Financial Event Classification, Detection and Summarization[C]// Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI-20). 2020:4668-4678.)
[21]
Zhao L, Li L, Zheng X. A BERT Based Sentiment Analysis and Key Entity Detection Approach for Online Financial Texts[OL]. arXiv Preprint, arXiv: 2001. 05326.
Clark K, Luong M T, Le Q V, et al. Electra: Pre-training Text Encoders as Discriminators Rather Than Generators[OL]. arXiv Preprint, arXiv: 2003. 10555.
[24]
Cho K, van Merrienboer B, Gulcehre C, et al. Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation[C]// Proceedings of the19th Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014: 1724-1734.
[25]
Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997, 9(8):1735-1780.
pmid: 9377276
[26]
Du X, Cardie C. Document-Level Event Role Filler Extraction Using Multi-Granularity Contextualized Encoding[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics(ACL). 2020: 8010-8020.
(Li Ni, Guan Huanmei, Yang Piao, et al. BERT-IDCNN-CRF for Named Entity Recognition in Chinese[J]. Journal of Shandong University (Natural Science), 2020, 55(1):102-109.)
[28]
Lample G, Ballesteros M, Subramanian S, et al. Neural Architectures for Named Entity Recognition[C]// Proceedings of the 2016 Conference on North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016: 260-270.)
[29]
Lan Z, Chen M, Goodman S, et al. ALBERT: A Lite BERT for Self-Supervised Learning of Language Representations[OL]. arXiv Preprint, arXiv: 1909. 11942.
[30]
Wei J, Ren X, Li X, et al. NEZHA: Neural Contextualized Representation for Chinese Language Understanding[OL]. arXiv Preprint, arXiv: 1909. 00204.
[31]
You Y, Li J, Hseu J, et al. Reducing BERT Pre-Training Time from 3 Days to 76 Minutes[OL]. arXiv Preprint, arXiv: 1904. 00962.
[32]
Cui Y, Che W, Liu T, et al. Revisiting Pre-Trained Models for Chinese Natural Language Processing[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings (EMNLP). 2020: 657-668.
