Please wait a minute...
Data Analysis and Knowledge Discovery  2023, Vol. 7 Issue (3): 58-68    DOI: 10.11925/infotech.2096-3467.2022.0332
Current Issue | Archive | Adv Search |
Chinese Text Sentiment Analysis Based on Dual Channel Attention Network with Hybrid Word Embedding
Zhou Ning(),Zhong Na,Jin Gaoya,Liu Bin
School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
Download: PDF (970 KB)   HTML ( 39 ) 
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper addresses the challenges facing the traditional static word vector embedding method, aiming to handle polysemy in Chinese texts effectively. It also excavates the contextual emotional features and internal semantic association structure. [Methods] In one channel, we integrated the sentiment elements related to the text into Word2Vec and FastText word vectors through rough data reasoning. We also used CNN to extract the local features of the text. In the other channel, we employed BERT for word embedding supplement and used BiLSTM to obtain the global features of the texts. Finally, we added the attention calculation module for the deep interaction of dual channel features. [Results] The experiment on three Chinese datasets achieved the highest accuracy of 92.43%, representing an improvement of 0.81% over the best value of the benchmark model. [Limitations] The selected datasets are only for modelling coarse-grained sentiment classification. We did not conduct experiments in the fine-grained domain. [Conclusions] The proposed model could effectively improve the performance of Chinese text sentiment classification.
Key words： Rough Data Reasoning      Dynamic Word Vector      Attention Mechanism      Text Sentiment Analysis     
Received: 11 April 2022      Published: 13 April 2023
ZTFLH:  TP391  
Fund:Research Results of National Natural Science Foundation of China(61650207);Research Results of National Natural Science Foundation of China(61963023);Tianyou Innovation Team of Lanzhou Jiaotong University(TY202003)
Corresponding Authors: Zhou Ning，ORCID:0000-0001-7466-8925，E-mail:zhouning@mail.lzjtu.cn。   
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Ning Zhou
Na Zhong
Gaoya Jin
Bin Liu

Cite this article:

Zhou Ning, Zhong Na, Jin Gaoya, Liu Bin. Chinese Text Sentiment Analysis Based on Dual Channel Attention Network with Hybrid Word Embedding. Data Analysis and Knowledge Discovery, 2023, 7(3): 58-68.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022.0332     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2023/V7/I3/58

Chinese Text Sentiment Analysis Model Based on Dual Channel Attention Network
Schematic Diagram of Rough Reasoning
Structure Diagram of BERT Model
Model of Classification of BERT-BiLSTM Channel
数据集 训练集/条 测试集/条 类别数
ChnSentiCorp_htl 4 800 1 200 2
Book_review 42 182 16 872 3
Weibo_senti_100k 95 990 23 998 2
Experimental Data Set Information
参数 ChnSentiCorp_htl Book_review Weibo_senti_100k
词向量维度 100维 100维 100维
学习率 1e-5 1e-3 1e-5
卷积核宽度 2、3、4 3、4、5 3、4、5
卷积核个数 64 64 64
Dropout 0.5 0.5 0.5
Batch_size 32 32 32
n_epoch 10 10 10
Experimental Parameter Setting
词向量种类 分类准确率
ChnSentiCorp_htl Book_review Weibo_senti_100k
Word2Vec 0.831 3 0.817 4 0.818 2
FastText 0.827 6 0.818 6 0.812 7
BERT 0.882 6 0.901 3 0.907 6
Classification Accuracy of Various Word Vectors on Different Data Sets
模型 数据集
ChnSenti
Corp_htl		 Book_
review		 Weibo_senti_
100k
RS-WvGv-LR 0.842 0 — —
BiLSTM 0.831 3 0.817 4 0.818 2
CNN 0.805 0 0.709 7 0.745 4
BiLSTM-ATT 0.870 9 0.812 7 0.824 7
CNN-BiLSTM 0.850 0 0.811 3 0.821 3
CNN-BiLSTM-ATT 0.853 3 0.818 2 0.826 2
MCNN-MA 0.863 2 — —
BERT-BiLSTM 0.875 0 — —
DC-GCNN-SL 0.908 2 0.916 2 —
RCBN-BM 0.901 2 0.924 3 0.915 5
Comparison of Accuracy of RCBN-BM Model and Baseline Model on Each Data Set
模型 数据集
ChnSentiCorp_
htl		 Book_
review		 Weibo_senti_100k
RCBN-BM（R） 0.877 5 0.899 2 0.907 6
RCBN-BM（BM） 0.861 8 0.829 0 0.839 8
RCBN-BM（A） 0.894 9 0.918 6 0.904 0
RCBN-BM 0.901 2 0.924 3 0.915 5
Classification Accuracy of Ablation Experimental Model on Each Data Set
数据集 ChnSentiCorp_htl Book_review Weibo_senti_100k
P R F P R F P R F
RCBN-BM（R） 0.870 0 0.875 0 0.872 4 0.895 0 0.900 0 0.897 5 0.905 0 0.905 0 0.905 0
RCBN-BM（BM） 0.860 0 0.865 0 0.862 4 0.820 0 0.825 0 0.822 5 0.835 0 0.840 0 0.837 5
RCBN-BM（A） 0.895 0 0.900 0 0.897 5 0.916 7 0.916 7 0.916 7 0.910 0
0.915 0		 0.905 0 0.907 5
RCBN-BM 0.900 0 0.905 0 0.902 5 0.923 3 0.926 7 0.925 0 0.915 0 0.915 0
Ablation Experimental Model on Each Data Set
[1] Hearst M A. Direction-Based Text Interpretation as an Information Access Refinement[A]// Text-Based Intelligent Systems: Current Research and Practice in Information Extraction and Retrieval[M]. 1992: 257-274.
[2] Xie R B, Yuan X C, Liu Z Y, et al. Lexical Sememe Prediction via Word Embeddings and Matrix Factorization[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence. ACM, 2017: 4200-4206.
[3] Pang B, Lee L, Vaithyanathan S. Thumbs up? Sentiment Classification Using Machine Learning Techniques[C]// Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing. ACM, 2002: 79-86.
[4] 陈钊, 徐睿峰, 桂林, 等. 结合卷积神经网络和词语情感序列特征的中文情感分析[J]. 中文信息学报, 2015, 29(6): 172-178.
[4] ( Chen Zhao, Xu Ruifeng, Gui Lin, et al. Combining Convolutional Neural Networks and Word Sentiment Sequence Features for Chinese Text Sentiment Analysis[J]. Journal of Chinese Information Processing, 2015, 29(6): 172-178.)
[5] Tang D Y, Qin B, Feng X C, et al. Effective LSTMS for Target-Dependent Sentiment Classification[OL]. arXiv Preprint, arXiv: 1512.01100.
[6] Lu W J, Duan Y, Song Y T. Self-Attention-Based Convolutional Neural Networks for Sentence Classification[C]// Proceedings of the 6th International Conference on Computer and Communications. IEEE, 2021: 2065-2069.
[7] Mikolov T, Chen K, Corrado G, et al. Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint, arXiv: 1301.3781.
[8] Pennington J, Socher R, Manning C. GloVe: Global Vectors for Word Representation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1532-1543.
[9] Bojanowski P, Grave E, Joulin A, et al. Enriching Word Vectors with Subword Information[J]. Transactions of the Association for Computational Linguistics, 2017, 5: 135-146.
doi: 10.1162/tacl_a_00051
[10] Choi J, Lee S W. Improving FastText with Inverse Document Frequency of Subwords[J]. Pattern Recognition Letters, 2020, 133: 165-172.
doi: 10.1016/j.patrec.2020.03.003
[11] Kim Y. Convolutional Neural Networks for Sentence Classification[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1746-1751.
[12] Dong M, Li Y F, Tang X, et al. Variable Convolution and Pooling Convolutional Neural Network for Text Sentiment Classification[J]. IEEE Access, 2020, 8: 16174-16186.
doi: 10.1109/Access.6287639
[13] Li L Q, Yang L, Zeng Y Y. Improving Sentiment Classification of Restaurant Reviews with Attention-Based Bi-GRU Neural Network[J]. Symmetry, 2021, 13(8): 1517.
doi: 10.3390/sym13081517
[14] 程艳, 孙欢, 陈豪迈, 等. 融合卷积神经网络与双向GRU的文本情感分析胶囊模型[J]. 中文信息学报, 2021, 35(5): 118-129.
[14] ( Cheng Yan, Sun Huan, Chen Haomai, et al. Text Sentiment Analysis Capsule Model Combining Convolutional Neural Network and Bidirectional GRU[J]. Journal of Chinese Information Processing, 2021, 35(5): 118-129.)
[15] 韩普, 张伟, 张展鹏, 等. 基于特征融合和多通道的突发公共卫生事件微博情感分析[J]. 数据分析与知识发现, 2021, 5(11): 68-79.
[15] ( Han Pu, Zhang Wei, Zhang Zhanpeng, et al. Sentiment Analysis of Weibo Posts on Public Health Emergency with Feature Fusion and Multi-Channel[J]. Data Analysis and Knowledge Discovery, 2021, 5(11): 68-79.)
[16] Cheng Y, Yao L B, Xiang G X, et al. Text Sentiment Orientation Analysis Based on Multi-Channel CNN and Bidirectional GRU with Attention Mechanism[J]. IEEE Access, 2020, 8: 134964-134975.
doi: 10.1109/Access.6287639
[17] Peters M, Neumann M, Iyyer M, et al. Deep Contextualized Word Representations[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2018: 2227-2237.
[18] Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv: 1810.04805.
[19] Man R, Lin K. Sentiment Analysis Algorithm Based on BERT and Convolutional Neural Network[C]// Proceedings of the 2021 IEEE Asia-Pacific Conference on Image Processing, Electronics and Computers. IEEE, 2021: 769-772.
[20] Mihalcea R, Tarau P. TextRank: Bringing Order into Text[C]// Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing. 2004: 404-411.
[21] Yan S, Yan L, Wu J Z. Rough Data-Deduction Based on the Upper Approximation[J]. Information Sciences, 2016, 373: 308-320.
doi: 10.1016/j.ins.2016.09.011
[22] 闫硕. 基于上近似的粗糙数据推理研究及应用[D]. 北京: 北京交通大学, 2017.
[22] Yan Shuo. Rough Data-Deduction Based on the Upper Approximation and Its Applications[D]. Beijing: Beijing Jiaotong University, 2017.)
[23] 周宁, 石雯茜, 朱昭昭. 基于粗糙数据推理的TextRank关键词提取算法[J]. 中文信息学报, 2020, 34(9): 44-52.
[23] ( Zhou Ning, Shi Wenqian, Zhu Zhaozhao. TextRank Keyword Extraction Algorithm Based on Rough Data-Deduction[J]. Journal of Chinese Information Processing, 2020, 34(9): 44-52.)
[24] Turney P D, Littman M L. Unsupervised Learning of Semantic Orientation from a Hundred-Billion-Word Corpus[OL]. arXiv Preprint, arXiv: 0212012.
[25] 朱新华, 马润聪, 孙柳, 等. 基于知网与词林的词语语义相似度计算[J]. 中文信息学报, 2016, 30(4): 29-36.
[25] ( Zhu Xinhua, Ma Runcong, Sun Liu, et al. Word Semantic Similarity Computation Based on HowNet and CiLin[J]. Journal of Chinese Information Processing, 2016, 30(4): 29-36.)
[26] 谢思雅, 施一萍, 胡佳玲, 等. 基于BiLSTM-ATT的微博用户情感分类研究[J]. 传感器与微系统, 2021, 40(2): 26-29.
[26] ( Xie Siya, Shi Yiping, Hu Jialing, et al. Research on Emotional Classification of Weibo Users Based on BiLSTM-ATT[J]. Transducer and Microsystem Technologies, 2021, 40(2): 26-29.)
[27] Tan S B, Zhang J. An Empirical Study of Sentiment Analysis for Chinese Documents[J]. Expert Systems with Applications, 2008, 34(4): 2622-2629.
doi: 10.1016/j.eswa.2007.05.028
[28] 陈波, 谢珺, 苗夺谦, 等. 基于粗糙集和多通道词向量的中文文本情感特征分析[J]. 中文信息学报, 2020, 34(8): 94-104.
[28] ( Chen Bo, Xie Jun, Miao Duoqian, et al. Chinese Text Sentiment Feature Analysis Based on Rough Set and Multi Channel Word Vector[J]. Journal of Chinese Information Processing, 2020, 34(8): 94-104.)
[29] Feng Y, Cheng Y. Short Text Sentiment Analysis Based on Multi-Channel CNN with Multi-Head Attention Mechanism[J]. IEEE Access, 2021, 9: 19854-19863.
doi: 10.1109/Access.6287639
[30] 杨杰, 杨文军. 基于BERT模型的文本评论情感分析[J]. 天津理工大学学报, 2021, 37(2): 12-16.
[30] ( Yang Jie, Yang Wenjun. Text Comment Sentiment Analysis Based on BERT Model[J]. Journal of Tianjin University of Technology, 2021, 37(2): 12-16.)
[31] 杨璐, 何明祥. 基于门控机制和卷积神经网络的中文文本情感分析模型[J]. 计算机应用, 2021, 41(10): 2842-2848.
doi: 10.11772/j.issn.1001-9081.2020122043
[31] ( Yang Lu, He Mingxiang. Chinese Text Sentiment Analysis Model Based on Gated Mechanism and Convolutional Neural Network[J]. Journal of Computer Applications, 2021, 41(10): 2842-2848.)
doi: 10.11772/j.issn.1001-9081.2020122043
[1] Han Pu, Zhong Yule, Lu Haojie, Ma Shiwen. Identifying Named Entities of Adverse Drug Reaction with Adversarial Transfer Learning[J]. 数据分析与知识发现, 2023, 7(3): 131-141.
[2] Su Mingxing, Wu Houyue, Li Jian, Huang Ju, Zhang Shunxiang. AEMIA:Extracting Commodity Attributes Based on Multi-level Interactive Attention Mechanism[J]. 数据分析与知识发现, 2023, 7(2): 108-118.
[3] Wang jinzheng, Yang Ying, Yu Bengong. Classifying Customer Complaints Based on Multi-head Co-attention Mechanism[J]. 数据分析与知识发现, 2023, 7(1): 128-137.
[4] Peng Cheng, Zhang Chunxia, Zhang Xin, Guo Jingtao, Niu Zhendong. Reasoning Model for Temporal Knowledge Graph Based on Entity Multiple Unit Coding[J]. 数据分析与知识发现, 2023, 7(1): 138-149.
[5] Zhao Ruijie, Tong Xinyu, Liu Xiaohua, Lu Yonghe. Entity Recognition and Labeling for Medical Literature Based on Neural Network[J]. 数据分析与知识发现, 2022, 6(9): 100-112.
[6] Chen Yuanyuan, Ma Jing. Detecting Multimodal Sarcasm Based on SC-Attention Mechanism[J]. 数据分析与知识发现, 2022, 6(9): 40-51.
[7] Tang Jiao, Zhang Lisheng, Sang Chunyan. News Recommendation with Latent Topic Distribution and Long and Short-Term User Representations[J]. 数据分析与知识发现, 2022, 6(9): 52-64.
[8] Zhao Pengwu, Li Zhiyi, Lin Xiaoqi. Identifying Relationship of Chinese Characters with Attention Mechanism and Convolutional Neural Network[J]. 数据分析与知识发现, 2022, 6(8): 41-51.
[9] Zhang Ruoqi, Shen Jianfang, Chen Pinghua. Session Sequence Recommendation with GNN, Bi-GRU and Attention Mechanism[J]. 数据分析与知识发现, 2022, 6(6): 46-54.
[10] Ye Han,Sun Haichun,Li Xin,Jiao Kainan. Classification Model for Long Texts with Attention Mechanism and Sentence Vector Compression[J]. 数据分析与知识发现, 2022, 6(6): 84-94.
[11] Guo Hangcheng, He Yanqing, Lan Tian, Wu Zhenfeng, Dong Cheng. Identifying Moves from Scientific Abstracts Based on Paragraph-BERT-CRF[J]. 数据分析与知识发现, 2022, 6(2/3): 298-307.
[12] Xu Yuemei, Fan Zuwei, Cao Han. A Multi-Task Text Classification Model Based on Label Embedding of Attention Mechanism[J]. 数据分析与知识发现, 2022, 6(2/3): 105-116.
[13] Yan Dongmei, He Wenxin, Chen Zhi. Predicting Stock Prices Based on RoBERTa-TCN and Sentimental Characteristics[J]. 数据分析与知识发现, 2022, 6(12): 123-134.
[14] Fan Tao, Wang Hao, Zhang Wei, Li Xiaomin. Extracting Entities from Intangible Cultural Heritage Texts Based on Machine Reading Comprehension[J]. 数据分析与知识发现, 2022, 6(12): 70-79.
[15] Bai Simeng,Niu Zhendong,He Hui,Shi Kaize,Yi Kun,Ma Yuanchi. Biomedical Text Classification Method Based on Hypergraph Attention Network[J]. 数据分析与知识发现, 2022, 6(11): 13-24.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn