Please wait a minute...
Data Analysis and Knowledge Discovery  2023, Vol. 7 Issue (2): 86-96    DOI: 10.11925/infotech.2096-3467.2022.0985
Current Issue | Archive | Adv Search |
Extracting Emotion-Cause Pairs Based on Multi-Label Seq2Seq Model
Zhang Siyang1,Wei Subo2,3,Sun Zhengyan4,Zhang Shunxiang2,3(),Zhu Guangli2,3,Wu Houyue2,3
1School of Mathematics and Big Data, Anhui University of Science and Technology, Huainan 232001, China
2School of Computer Science and Engineering, Anhui University of Science and Technology, Huainan 232001, China
3Artificial Intelligence Research Institute, Hefei Comprehensive National Science Center, Hefei 230026, China
4School of Computer Science, Huainan Normal University, Huainan 232038, China
Download: PDF (917 KB)   HTML ( 15
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper explores new algorithms to extract emotion-cause pairs based on multi-label Seq2Seq model. [Methods] First, we used the BERT pre-training to obtain semantically rich word vectors. Then, we utilized the Bi-GRU and LSTM to obtain the global and local features of the texts. Finally, we introduced the hybrid attention mechanism to merge the features and improve the integrity of these semantic features. [Results] Compared with the latest FSS-GCN model, the F1 value of our new model for emotional cause pairs increased by 0.98 percentage point and 11.60 percentage point on two data sets. The F1 value of emotion extraction increased by 0.87 percentage point and 1.10 percentage point, while the F1 value for cause extraction increased by 0.79 percentage point and 2.31 percentage point respectively. [Limitations] Our new model mainly examined the explicit emotion-cause pairs and did not explore implicit emotion-cause pairs. [Conclusions] The proposed model improves the F1 values of extracting emotion-cause pairs.

Key wordsEmotion-Cause Pair Extraction      Multi-Label      Seq2Seq Model      BERT     
Received: 20 September 2022      Published: 28 March 2023
ZTFLH:  TP393 G250  
Fund:National Natural Science Foundation of China(62076006);University Synergy Innovation Program of Anhui Province(GXXT-2021-008);Anhui Provincial Natural Science Foundation(1908085MF189)
Corresponding Authors: Zhang Shunxiang,ORCID: 0000-0002-0540-7593,E-mail: sxzhang@aust.edu.cn。   

Cite this article:

Zhang Siyang, Wei Subo, Sun Zhengyan, Zhang Shunxiang, Zhu Guangli, Wu Houyue. Extracting Emotion-Cause Pairs Based on Multi-Label Seq2Seq Model. Data Analysis and Knowledge Discovery, 2023, 7(2): 86-96.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022.0985     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2023/V7/I2/86

Framework of Emotion-Cause Pair Extraction for Multi-Label Seq2Seq Model
数据集 情感原因对 数量 比例
中文 一个情感原因对 1 746 89.77%
多个情感原因对 199 10.23%
英文 一个情感原因对 2 537 89.24%
多个情感原因对 306 10.76%
Dataset Description
模型 中文数据集 英文数据集
P(%) R(%) F1(%) P(%) R(%) F1(%)
ECPA 70.96 70.32 70.64 69.12 71.45 70.27
E-CNN 57.14 54.37 55.72 56.98 55.37 56.16
Indep 68.32 50.82 58.18 69.12 53.71 60.45
Inter-CE 69.02 51.35 59.01 70.26 55.68 62.13
Inter-EC 67.21 57.05 61.28 71.48 59.72 65.08
UTOS 73.89 70.62 72.03 71.54 63.97 67.54
TDG-ECPA 73.74 63.07 67.99 68.51 60.46 64.23
OG-ECPE 77.90 70.82 74.17 73.69 53.84 62.22
MAM-SD 69.63 57.99 63.20 68.46 57.13 62.28
ECPE-2D 68.97 62.54 65.59 70.08 48.72 57.37
FSS-GCN 78.61 75.72 77.14 67.43 53.03 59.48
MLS-ECPE 78.94 77.31 78.12 71.38 70.79 71.08
Experimental Results of Models Performance
模型 情绪抽取 原因抽取
P(%) R(%) F1(%) P(%) R(%) F1(%)
FSS-GCN 86.14 80.46 83.20 70.52 63.93 67.06
MAM-SD 85.54 81.41 83.39 72.02 62.75 67.07
UTOS 82.07 77.07 79.47 67.81 60.64 63.98
MLS-ECPE 86.95 81.37 84.07 73.67 62.89 67.85
Emotion Extraction and Cause Extraction (Chinese Dataset)
模型 情绪抽取 原因抽取
P(%) R(%) F1(%) P(%) R(%) F1(%)
FSS-GCN 83.75 77.86 80.70 68.37 56.78 62.04
MAM-SD 79.64 76.43 78.00 65.29 55.18 59.81
UTOS 78.48 76.09 77.27 60.31 54.76 57.40
MLS-ECPE 84.16 79.57 81.80 70.15 59.43 64.35
Emotion Extraction and Cause Extraction (English Dataset)
情绪-原因对 模型 P(%) R(%) F1(%)
一对 FSS-GCN 83.12 75.81 79.30
MLS-ECPE 85.49 77.58 81.34
多对 FSS-GCN 73.84 60.75 66.66
MLS-ECPE 75.92 62.83 68.76
Analysis of the Number of Emotion-Cause Pairs
P(%) R(%) F1(%)
组别 Full model 78.94 77.31 78.12
目标信息组 -BERT 77.15 75.81 76.47
-Bi-GRU+Att 66.78 58.93 62.61
-LSTM+Att 70.46 72.84 71.63
上下文信息组 -HA 73.84 70.75 72.26
标签信息组 -LM 73.92 69.83 71.82
Experimental Study of Ablation
[1] 刁宇峰, 杨亮, 林鸿飞, 等. 基于ECPA神经网络的情绪原因识别方法[J]. 中文信息学报, 2021, 35(6): 85-92.
[1] (Diao Yufeng, Yang Liang, Lin Hongfei, et al. Emotion Cause Recognition Based on Emotion Context Position Attention(ECPA) Neural Network[J]. Journal of Chinese Information Processing, 2021, 35(6): 85-92.)
[2] 慕永利, 李旸, 王素格. 基于E-CNN的情绪原因识别方法[J]. 中文信息学报, 2018, 32(2): 120-128.
[2] (Mu Yongli, Li Yang, Wang Suge. Emotion Cause Detection Based on Ensembled Convolution Neural Networks[J]. Journal of Chinese Information Processing, 2018, 32(2): 120-128.)
[3] 杨帆, 叶潇, 曹存根, 等. 基于场合和角色的情绪知识获取与分析[J]. 计算机工程, 2006, 32(15): 197-199.
doi: 10.1063/1.30805
[3] (Yang Fan, Ye Xiao, Cao Cungen, et al. Acquisition and Analysis of Emotion Knowledge Based on Context and Role[J]. Computer Engineering, 2006, 32(15): 197-199.)
doi: 10.1063/1.30805
[4] Xia R, Ding Z X. Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Texts[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 1003-1012.
[5] Cheng Z F, Jiang Z W, Yin Y F, et al. A Unified Target-Oriented Sequence-to-Sequence Model for Emotion-Cause Pair Extraction[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 2779-2791.
doi: 10.1109/TASLP.2021.3102194
[6] Fan C, Yuan C F, Du J C, et al. Transition-Based Directed Graph Construction for Emotion-Cause Pair Extraction[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 3707-3717.
[7] Poria S, Majumder N, Hazarika D, et al. Recognizing Emotion Cause in Conversations[J]. Cognitive Computation, 2021, 13(5): 1317-1332.
doi: 10.1007/s12559-021-09925-7
[8] Gao K, Xu H, Wang J. A Rule-Based Approach to Emotion Cause Detection for Chinese Micro-Blogs[J]. Expert Systems with Applications, 2015, 42(9): 4517-4528.
doi: 10.1016/j.eswa.2015.01.064
[9] 张晨, 钱涛, 姬东鸿. 基于神经网络的微博情绪识别与诱因抽取联合模型[J]. 计算机应用, 2018, 38(9): 2464-2468.
doi: 10.11772/j.issn.1001-9081.2018020481
[9] (Zhang Chen, Qian Tao, Ji Donghong. Joint Model of Microblog Emotion Recognition of Emoticons and Emotion Cause Detection Based on Neural Network[J]. Journal of Computer Applications, 2018, 38(9): 2464-2468.)
doi: 10.11772/j.issn.1001-9081.2018020481
[10] Xu R F, Hu J N, Lu Q, et al. An Ensemble Approach for Emotion Cause Detection with Event Extraction and Multi-Kernel SVMS[J]. Tsinghua Science and Technology, 2017, 22(6): 646-659.
doi: 10.23919/TST.2017.8195347
[11] Xiao X L, Wei P H, Mao W J, et al. Context-Aware Multi-View Attention Networks for Emotion Cause Extraction[C]// Proceedings of the 2019 IEEE International Conference on Intelligence and Security Informatics. 2019: 128-133.
[12] Fan C, Yan H Y, Du J C, et al. A Knowledge Regularized Hierarchical Approach for Emotion Cause Analysis[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019: 5614-5624.
[13] Gui L, Hu J N, He Y L, et al. A Question Answering Approach for Emotion Cause Extraction[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 1594-1603.
[14] Cheng X Y, Chen Y, Cheng B X, et al. An Emotion Cause Corpus for Chinese Microblogs with Multiple-User Structures[J]. ACM Transactions on Asian and Low-Resource Language Information Processing, 2017, 17(1): 1-19.
[15] Li X J, Song K S, Feng S, et al. A Co-Attention Neural Network Model for Emotion Cause Analysis with Emotional Context Awareness[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 4752-4757.
[16] Li X J, Feng S, Wang D L, et al. Context-Aware Emotion Cause Analysis with Multi-Attention-Based Neural Network[J]. Knowledge-Based Systems, 2019, 174: 205-218.
doi: 10.1016/j.knosys.2019.03.008
[17] Fan W, Zhu Y X, Wei Z Y, et al. Order-Guided Deep Neural Network for Emotion-Cause Pair Prediction[J]. Applied Soft Computing, 2021, 112: 107818.
doi: 10.1016/j.asoc.2021.107818
[18] Yu J X, Liu W Y, He Y J, et al. A Mutually Auxiliary Multitask Model with Self-Distillation for Emotion-Cause Pair Extraction[J]. IEEE Access, 2021, 9: 26811-26821.
doi: 10.1109/ACCESS.2021.3057880
[19] Liu W F, Pang J M, Li N, et al. Research on Multi-Label Text Classification Method Based on tALBERT-CNN[J]. International Journal of Computational Intelligence Systems, 2021, 14(1): 1-12.
doi: 10.2991/ijcis.d.201027.001
[20] 王嫄, 徐涛, 王世龙, 等. 层级标签语义引导的极限多标签文本分类策略[J]. 中文信息学报, 2021, 35(10): 110-118.
[20] (Wang Yuan, Xu Tao, Wang Shilong, et al. An Extreme Multi-Label Text Classification Strategy via Hierarchical Label Semantic Guidance[J]. Journal of Chinese Information Processing, 2021, 35(10): 110-118.)
[21] 王进, 徐巍, 丁一, 等. 基于图嵌入和区域注意力的多标签文本分类[J]. 江苏大学学报(自然科学版), 2022, 43(3): 310-318.
[21] (Wang Jin, Xu Wei, Ding Yi, et al. Multi-Label Text Classification Based on Graph Embedding and Region Attention[J]. Journal of Jiangsu University(Natural Science Edition), 2022, 43(3): 310-318.)
[22] 陆泉, 何超, 陈静, 等. 基于两阶段迁移学习的多标签分类模型研究[J]. 数据分析与知识发现, 2021, 5(7): 91-100.
[22] (Lu Quan, He Chao, Chen Jing, et al. A Multi-Label Classification Model with Two-Stage Transfer Learning[J]. Data Analysis and Knowledge Discovery, 2021, 5(7): 91-100.)
[23] 吕学强, 彭郴, 张乐, 等. 融合BERT与标签语义注意力的文本多标签分类方法[J]. 计算机应用, 2022, 42(1): 57-63.
doi: 10.11772/j.issn.1001-9081.2021020366
[23] (Lyu Xueqiang, Peng Chen, Zhang Le, et al. Text Multi-Label Classification Method Incorporating BERT and Label Semantic Attention[J]. Journal of Computer Applications, 2022, 42(1): 57-63.)
doi: 10.11772/j.issn.1001-9081.2021020366
[24] Huang Y, Li Z X, Deng W, et al. D-BERT: Incorporating Dependency-Based Attention into BERT for Relation Extraction[J]. CAAI Transactions on Intelligence Technology, 2021, 6(4): 417-425.
doi: 10.1049/cit2.v6.4
[25] Zhou P, Shi W, Tian J, et al. Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 207-212.
[26] Ding Z X, Xia R, Yu J F. ECPE-2D: Emotion-Cause Pair Extraction Based on Joint Two-Dimensional Representation, Interaction and Prediction[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 3161-3170.
[27] Hu G M, Lu G M, Zhao Y. FSS-GCN: A Graph Convolutional Networks with Fusion of Semantic and Structure for Emotion Cause Analysis[J]. Knowledge-Based Systems, 2021, 212: 106584.
doi: 10.1016/j.knosys.2020.106584
[1] Su Mingxing, Wu Houyue, Li Jian, Huang Ju, Zhang Shunxiang. AEMIA:Extracting Commodity Attributes Based on Multi-level Interactive Attention Mechanism[J]. 数据分析与知识发现, 2023, 7(2): 108-118.
[2] Zhao Yiming, Pan Pei, Mao Jin. Recognizing Intensity of Medical Query Intentions Based on Task Knowledge Fusion and Text Data Enhancement[J]. 数据分析与知识发现, 2023, 7(2): 38-47.
[3] Wang Yufei, Zhang Zhixiong, Zhao Yang, Zhang Mengting, Li Xuesi. Designing and Implementing Automatic Title Generation System for Sci-Tech Papers[J]. 数据分析与知识发现, 2023, 7(2): 61-71.
[4] Wang Weijun, Ning Zhiyuan, Du Yi, Zhou Yuanchun. Identifying Interdisciplinary Sci-Tech Literature Based on Multi-Label Classification[J]. 数据分析与知识发现, 2023, 7(1): 102-112.
[5] Shi Yunmei, Yuan Bo, Zhang Le, Lv Xueqiang. IMTS: Detecting Fake Reviews with Image and Text Semantics[J]. 数据分析与知识发现, 2022, 6(8): 84-96.
[6] Zheng Jie, Huang Hui, Qin Yongbin. Matching Similar Cases with Legal Knowledge Fusion[J]. 数据分析与知识发现, 2022, 6(7): 99-106.
[7] Wu Jiang, Liu Tao, Liu Yang. Mining Online User Profiles and Self-Presentations: Case Study of NetEase Music Community[J]. 数据分析与知识发现, 2022, 6(7): 56-69.
[8] Pan Huiping, Li Baoan, Zhang Le, Lv Xueqiang. Extracting Keywords from Government Work Reports with Multi-feature Fusion[J]. 数据分析与知识发现, 2022, 6(5): 54-63.
[9] Xiao Yuejun, Li Honglian, Zhang Le, Lv Xueqiang, You Xindong. Classifying Chinese Patent Texts with Feature Fusion[J]. 数据分析与知识发现, 2022, 6(4): 49-59.
[10] Yang Lin, Huang Xiaoshuo, Wang Jiayang, Ding Lingling, Li Zixiao, Li Jiao. Identifying Subtypes of Clinical Trial Diseases with BERT-TextCNN[J]. 数据分析与知识发现, 2022, 6(4): 69-81.
[11] Guo Hangcheng, He Yanqing, Lan Tian, Wu Zhenfeng, Dong Cheng. Identifying Moves from Scientific Abstracts Based on Paragraph-BERT-CRF[J]. 数据分析与知识发现, 2022, 6(2/3): 298-307.
[12] Wang Yongsheng, Wang Hao, Yu Wei, Zhou Zeyu. Extracting Relationship Among Characters from Local Chronicles with Text Structures and Contents[J]. 数据分析与知识发现, 2022, 6(2/3): 318-328.
[13] Zhang Yunqiu, Wang Yang, Li Bocheng. Identifying Named Entities of Chinese Electronic Medical Records Based on RoBERTa-wwm Dynamic Fusion Model[J]. 数据分析与知识发现, 2022, 6(2/3): 242-250.
[14] Shang Lili, Tang Huayun, Wang Yanzhao, Zuo Meiyun. Identifying Actionable Information from Online Reviews[J]. 数据分析与知识发现, 2022, 6(12): 1-12.
[15] Yan Dongmei, He Wenxin, Chen Zhi. Predicting Stock Prices Based on RoBERTa-TCN and Sentimental Characteristics[J]. 数据分析与知识发现, 2022, 6(12): 123-134.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn