Please wait a minute...
Advanced Search
数据分析与知识发现  2021, Vol. 5 Issue (12): 37-47     https://doi.org/10.11925/infotech.2096-3467.2021.0554
  研究论文 本期目录 | 过刊浏览 | 高级检索 |
基于BAGCNN的方面级别情感分析研究*
余本功1,2(),张书文1
1合肥工业大学管理学院 合肥 230009
2合肥工业大学过程优化与智能决策教育部重点实验室 合肥 230009
Aspect-Level Sentiment Analysis Based on BAGCNN
Yu Bengong1,2(),Zhang Shuwen1
1School of Management, Hefei University of Technology, Hefei 230009, China
2Key Laboratory of Process Optimization & Intelligent Decision-making (Hefei University of Technology),Ministry of Education, Hefei 230009, China
全文: PDF (884 KB)   HTML ( 20
输出: BibTeX | EndNote (RIS)      
摘要 

【目的】 解决传统方面级别情感分析模型在词嵌入过程中未将上下文与方面词信息融合、需以复杂的下游结构提取特征等问题。【方法】 提出一种基于BERT的注意力门控卷积模型(BAGCNN),该模型由预训练BERT模型生成融合上下文语义的文本和方面词特征表示,并引入多头自注意力机制解决方面词长距离依赖问题,最后利用门控卷积网络并行地选择性提取与方面词信息相关的多层次上下文特征。【结果】 实验结果表明,与使用循环神经网络中效果最好的基准模型相比,本文模型精度在Restaurant、Laptop和Twitter三个数据集上分别提升4.24、4.01和3.89个百分点,且模型下游并行结构尺寸减小了1.27MB。【局限】 本文模型在文本长度差异大的数据集中分类效果较差。【结论】 在BERT和多头自注意力机制辅助下,BAGCNN模型中门控卷积网络可有效过滤与方面词无关的上下文信息。

服务
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章
余本功
张书文
关键词 方面级别情感分析BERT多头自注意力机制门控卷积网络    
Abstract

[Objective] This paper proposes a BERT-based Attention Gated Convolution Neural Network model (BAGCNN), aiming to improve the traditional aspect-level sentiment analysis algorithm. [Methods] First, the pre-trained BERT model generated feature representation for the texts and aspect words. Then, we introduced the Multi Head Self-attention Mechanism to solve the problem of long-distance dependence of aspect words. Finally, we selectively extracted the multi-level context features paralleling the aspect words with the Gated Convolution Neural Network. [Results] Compared to the benchmark model, the accuracy of our new model was improved by 4.24, 4.01 and 3.89 percentage points on restaurant, laptop and twitter datasets. The size of the downstream parallel structure of the model was also reduced by 1.27 MB. [Limitations] The proposed model did not work well with data sets having significantly different text length. [Conclusions] The new BAGCNN model could effectively remove the context information irrelevant to the aspect words.

Key wordsAspect-Level Sentiment Analysis    BERT    Multi Head Self-attention Mechanism    Gated Convolution Neural Network
收稿日期: 2021-06-13      出版日期: 2022-01-20
ZTFLH:  TP391  
基金资助:* 国家自然科学基金项目(72071061);国家自然科学基金项目(71671057)
通讯作者: 余本功,ORCID:0000-0003-4170-2335     E-mail: bgyu@hfut.edu.cn
引用本文:   
余本功, 张书文. 基于BAGCNN的方面级别情感分析研究*[J]. 数据分析与知识发现, 2021, 5(12): 37-47.
Yu Bengong, Zhang Shuwen. Aspect-Level Sentiment Analysis Based on BAGCNN. Data Analysis and Knowledge Discovery, 2021, 5(12): 37-47.
链接本文:  
https://manu44.magtech.com.cn/Jwk_infotech_wk3/CN/10.11925/infotech.2096-3467.2021.0554      或      https://manu44.magtech.com.cn/Jwk_infotech_wk3/CN/Y2021/V5/I12/37
Fig.1  基于BAGCNN的方面级别情感分析模型
数据集 类型 positive neural negative
Restaurant train 2 164 637 807
test 728 196 196
Laptop train 994 464 879
test 341 169 128
Twitter train 1 561 3 127 1 560
test 173 346 173
Table 1  数据统计
参数
dropout rate 0.1
batch size 32
epoch 6
optimizer Adam
learning rate 2e-5
Table 2  超参数设置
Model Restaurant Laptop Twitter
Accuracy F1 Accuracy F1 Accuracy F1
GCAE 77.32 63.78 66.04 57.35 72.16 69.87
AGCN 78.84 68.78 72.10 67.52 70.28 68.56
Mul-AT-CNN 79.46* - 75.39* - 71.25* -
AOA 79.46 68.80 73.51 68.18 71.96 69.91
IAN 79.46 69.75 72.47 67.18 69.85 68.27
MAN 80.71* 70.95* 74.13* 71.93* 72.12* 70.13*
ASGCN 81.28 71.74 74.81 70.74 72.40 70.68
BERT-SPC 83.02 74.55 78.06 73.61 73.12 71.83
BERT-AEN 81.98 72.16 77.18 73.83 72.98 71.74
TD-BERT 84.11 76.96 77.43 73.54 75.43 74.38
Our(BAGCNN) 84.95 77.90 78.14 74.14 76.01 74.61
Table 3  模型对比结果(%)
Fig.2  卷积核尺寸对模型影响
Fig.3  不同词嵌入编码方式对模型影响
Fig.4  不同特征提取方式对模型影响
Models Params× 10 6 Memory(MB)
ATAE-LSTM 2.53 13.76
AOA 2.89 15.15
IAN 2.17 12.40
GloVe-LSTM-ATT 2.07 16.90
AEN-GloVe 1.16 11.04
ASGCN 0.45 5.84
AGCN 1.36 13.42
GCAE 0.82 11.38
Our(GloVe-AGCNN) 1.02 11.13
Table 4  模型尺寸
Fig.5  门控卷积效果
文本 方法 结果
1. Even when the chef is not in the house, the food and service are right on target. Aspect chef(O) food(P) service(P)
BERT-SPC ×
BERT-AEN × ×
TD-BERT ×
Our(BAGCNN)
2. Food was average and creme brulee was awful - the sugar was charred, not caramelized and smelled of kerosene. Aspect Food(O) brulee(N) sugar(N)
BERT-SPC ×
BERT-AEN ×
TD-BERT ×
Our(BAGCNN) ×
3. The food is uniformly exceptional, with a very capable kitchen which will proudly whip up whatever you feel like eating, whether it 's on the menu or not. Aspect food(P) kitchen(P) menu(O)
BERT-SPC ×
BERT-AEN ×
TD-BERT
Our(BAGCNN)
Table 5  模型预测结果
[1] 赵明清, 武圣强. 基于微博情感分析的股市加权预测方法研究[J]. 数据分析与知识发现, 2019, 3(2): 43-51.
[1] (Zhao Mingqing, Wu Shengqiang. Research on Stock Market Weighted Prediction Method Based on Micro-blog Sentiment Analysis[J]. Data Analysis and Knowledge Discovery, 2019, 3(2): 43-51.)
[2] 李铁军, 颜端武, 杨雄飞. 基于情感加权关联规则的微博推荐研究[J]. 数据分析与知识发现, 2020, 4(4): 27-33.
[2] (Li Tiejun, Yan Duanwu, Yang Xiongfei. Recommending Microblogs Based on Emotion-Weighted Association Rules[J]. Data Analysis and Knowledge Discovery, 2020, 4(4): 27-33.)
[3] 曾子明, 万品玉. 基于双层注意力和Bi-LSTM的公共安全事件微博情感分析[J]. 情报科学, 2019, 37(6): 23-29.
[3] (Zeng Ziming, Wan Pinyu. Sentiment Analysis of Public Safety Events in Micro-blog Based on Double-layered Attention and Bi-LSTM[J]. Information Science, 2019, 37(6): 23-29.)
[4] Kiritchenko S, Zhu X, Cherry C, et al. Nrc-canada-2014: Detecting Aspects and Sentiment in Customer Reviews [C]//Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). 2014: 437-442.
[5] Mikolov T, Sutskever I, Chen K, et al. Distributed Representations of Words and Phrases and Their Compositionality [C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013: 3111-3119.
[6] Pennington J, Socher R, Manning C D. GloVe: Global Vectors for Word Representation [C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014: 1532-1543.
[7] Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding [C]//Proceeding of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019: 4171-4186.
[8] Peters M E, Neumann M, Iyyer M, et al. Deep Contextualized Word Representations [C]//Proceeding of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018: 2227-2237.
[9] Hoang M, Bihorac O A, Rouces J. Aspect-Based Sentiment Analysis Using BERT [C]//Proceedings of the 22nd Nordic Conference on Computational Linguistics. 2019: 187-196.
[10] Gao Z, Feng A, Song X, et al. Target-dependent Sentiment Classification with BERT[J]. IEEE Access, 2019, 7: 154290-154299.
doi: 10.1109/Access.6287639
[11] Xu Q, Zhu L, Dai T, et al. Aspect-Based Sentiment Classification with Multi-Attention Network[J]. Neurocomputing, 2020, 388: 135-143.
doi: 10.1016/j.neucom.2020.01.024
[12] Hu M, Zhao S, Zhang L, et al. CAN: Constrained Attention Networks for Multi-Aspect Sentiment Analysis [C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019: 4601-4610.
[13] Song Y, Wang J, Jiang T, et al. Targeted Sentiment Classification with Attentional Encoder Network [C]//Proceedings of the 28th International Conference on Artificial Neural Networks. Springer, Cham, 2019: 93-103.
[14] Zhao F, Wu Z, Dai X. Attention Transfer Network for Aspect-level Sentiment Classification [C]//Proceedings of the 28th International Conference on Computational Linguistics. 2020: 811-821.
[15] Zhang C, Li Q, Song D. Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks [C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019: 4568-4578.
[16] Chen C, Teng Z, Zhang Y. Inducing Target-Specific Latent Structures for Aspect Sentiment Classification [C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020: 5596-5607.
[17] Wang K, Shen W, Yang Y, et al. Relational Graph Attention Network for Aspect-based Sentiment Analysis [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 3229-3238.
[18] Kipf T N, Welling M. Semi-Supervised Classification with Graph Convolutional Networks[OL]. arXiv Preprint, arXiv: 1609.02907.
[19] Veličković P, Cucurull G, Casanova A, et al. Graph Attention Networks[OL]. arXiv Preprint, arXiv: 1710.10903.
[20] Xue W, Li T. Aspect Based Sentiment Analysis with Gated Convolutional Networks [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 2514-2523.
[21] Phan M H, Ogunbona P O. Modelling Context and Syntactical Features for Aspect-Based Sentiment Analysis [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 3211-3220.
[22] Zhang S, Xu X, Pang Y, et al. Multi-layer Attention Based CNN for Target-Dependent Sentiment Classification[J]. Neural Processing Letters, 2020, 51(3): 2089-2103.
doi: 10.1007/s11063-019-10017-9
[23] 曹卫东, 李嘉琪, 王怀超. 采用注意力门控卷积网络模型的目标情感分析[J]. 西安电子科技大学学报, 2019, 46(6): 30-36.
[23] (Cao Weidong, Li Jiaqi, Wang Huaichao. Analysis of Targeted Sentiment by the Attention Gated Convolutional Network Model[J]. Journal of Xidian University, 2019, 46(6): 30-36.)
[24] Liu N, Shen B. Aspect-based Sentiment Analysis with Gated Alternate Neural Network[J]. Knowledge-Based Systems, 2020, 188: 105010.
doi: 10.1016/j.knosys.2019.105010
[25] Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 6000-6010.
[26] Dong L, Wei F, Tan C, et al. Adaptive Recursive Neural Network for Target-Dependent Twitter Sentiment Classification [C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014: 49-54.
[27] Huang B, Ou Y, Carley K M. Aspect Level Sentiment Classification with Attention-over-Attention Neural Networks [C]//Proceedings of the 11th International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation. Springer, Cham, 2018: 197-206.
[28] Ma D, Li S, Zhang X, et al. Interactive Attention Networks for Aspect-Level Sentiment Classification [C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017: 4068-4074.
[29] Jiang N, Tian F, Li J, et al. MAN: Mutual Attention Neural Networks Model for Aspect-Level Sentiment Classification in SIoT[J]. IEEE Internet of Things Journal, 2020, 7(4): 2901-2913.
doi: 10.1109/JIoT.6488907
[30] Wang Y, Huang M, Zhu X, et al. Attention-based LSTM for Aspect-Level Sentiment Classification [C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016: 606-615.
[1] 陈杰,马静,李晓峰. 融合预训练模型文本特征的短文本分类方法*[J]. 数据分析与知识发现, 2021, 5(9): 21-30.
[2] 周泽聿,王昊,赵梓博,李跃艳,张小琴. 融合关联信息的GCN文本分类模型构建及其应用研究*[J]. 数据分析与知识发现, 2021, 5(9): 31-41.
[3] 马江微, 吕学强, 游新冬, 肖刚, 韩君妹. 融合BERT与关系位置特征的军事领域关系抽取方法*[J]. 数据分析与知识发现, 2021, 5(8): 1-12.
[4] 李文娜, 张智雄. 基于联合语义表示的不同知识库中的实体对齐方法研究*[J]. 数据分析与知识发现, 2021, 5(7): 1-9.
[5] 王昊, 林克柔, 孟镇, 李心蕾. 文本表示及其特征生成对法律判决书中多类型实体识别的影响分析[J]. 数据分析与知识发现, 2021, 5(7): 10-25.
[6] 喻雪寒, 何琳, 徐健. 基于RoBERTa-CRF的古文历史事件抽取方法研究*[J]. 数据分析与知识发现, 2021, 5(7): 26-35.
[7] 陆泉, 何超, 陈静, 田敏, 刘婷. 基于两阶段迁移学习的多标签分类模型研究*[J]. 数据分析与知识发现, 2021, 5(7): 91-100.
[8] 刘文斌, 何彦青, 吴振峰, 董诚. 基于BERT和多相似度融合的句子对齐方法研究*[J]. 数据分析与知识发现, 2021, 5(7): 48-58.
[9] 尹鹏博,潘伟民,张海军,陈德刚. 基于BERT-BiGA模型的标题党新闻识别研究*[J]. 数据分析与知识发现, 2021, 5(6): 126-134.
[10] 宋若璇,钱力,杜宇. 基于科技论文中未来工作句集的学术创新构想话题自动生成方法研究*[J]. 数据分析与知识发现, 2021, 5(5): 10-20.
[11] 胡昊天,吉晋锋,王东波,邓三鸿. 基于深度学习的食品安全事件实体一体化呈现平台构建*[J]. 数据分析与知识发现, 2021, 5(3): 12-24.
[12] 王倩,王东波,李斌,许超. 面向海量典籍文本的深度学习自动断句与标点平台构建研究*[J]. 数据分析与知识发现, 2021, 5(3): 25-34.
[13] 常城扬,王晓东,张胜磊. 基于深度学习方法对特定群体推特的动态政治情感极性分析*[J]. 数据分析与知识发现, 2021, 5(3): 121-131.
[14] 周文远, 王名扬, 井钰. 基于AttentionSBGMC模型的引文情感和引文目的自动分类研究*[J]. 数据分析与知识发现, 2021, 5(12): 48-59.
[15] 董淼, 苏中琪, 周晓北, 兰雪, 崔志刚, 崔雷. 利用Text-CNN改进PubMedBERT在化学诱导性疾病实体关系分类效果的尝试[J]. 数据分析与知识发现, 2021, 5(11): 145-152.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
版权所有 © 2015 《数据分析与知识发现》编辑部
地址:北京市海淀区中关村北四环西路33号 邮编:100190
电话/传真:(010)82626611-6626,82624938
E-mail:jishu@mail.las.ac.cn