Please wait a minute...
Data Analysis and Knowledge Discovery  2020, Vol. 4 Issue (12): 85-94    DOI: 10.11925/infotech.2096-3467.2020.0535
Current Issue | Archive | Adv Search |
Sentiment Analysis of Cross-Domain Product Reviews Based on Feature Fusion and Attention Mechanism
Qi Ruihua1,2(),Jian Yue1,2,Guo Xu2,Guan Jinghua2,Yang Mingxin1,2
1Research Center for Language Intelligence, Dalian University of Foreign Languages, Dalian 116044, China
2School of Software Engineering, Dalian University of Foreign Languages, Dalian 116044, China
Download: PDF (1050 KB)   HTML ( 21 ) 
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper tries to address the issues of labelled data shortage, aiming to distinguish the weights of sentiment characteristics in cross-domain sentiment classification. [Methods] We proposed a sentiment classification model for cross-domain product reviews based on feature fusion representation and the attention mechanism. First, this model integrated Bert and cross-domain word vectors to generate cross-domain unified feature space. Then, it extracted the weights of global and local features through attention mechanism. [Results] We examined our model with public review data from Amazon and found the average accuracy of the proposed model was up-to 95.93%, which was 9.33% higher than the existing model. [Limitations] More research is needed to evaluate our model with large-scale multi-domain data sets. [Conclusions] The proposed model could effectively analyze sentiment information.
Key words： Feature Fusion      Attention Mechanism      Cross-Domain      Sentiment Classification     
Received: 08 June 2020      Published: 25 December 2020
ZTFLH:  TP393  
Corresponding Authors: Qi Ruihua     E-mail: rhqi@dlufl.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Ruihua Qi
Yue Jian
Xu Guo
Jinghua Guan
Mingxin Yang

Cite this article:

Qi Ruihua,Jian Yue,Guo Xu,Guan Jinghua,Yang Mingxin. Sentiment Analysis of Cross-Domain Product Reviews Based on Feature Fusion and Attention Mechanism. Data Analysis and Knowledge Discovery, 2020, 4(12): 85-94.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2020.0535     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2020/V4/I12/85

Sentiment Classification Model for Cross-Domain Product Reviews Based on Feature Fusion and Attention Mechanism
领域 积极评论 消极评论 无标注文本
Books 3 000 3 000 9 750
DVD disk 3 000 3 000 11 843
Electronics 3 000 3 000 17 009
Kitchen appliances 3 000 3 000 13 856
Videos 3 000 3 000 30 180
Experimental Data Set
参数名称 值 参数名称 值
最大长度 120 优化器 Adam
词向量维度 768,1 068,
1 068		 损失函数 binary_
crossentropy
LSTM隐藏单元 128 batch_size 32
全连接层1 128, activation=“tanh” 全连接层2 2, activation=“softmax”
Dropout 0.5 输出层激活函数 Softmax
Experimental Parameters of Cross-Domain Sentiment Classification
源领域 目标领域 S-only SFA DANN mSDA HATN CDSA-B CDSA-F CDSA-F-Att
Book DVD 80.57% 82.85% 83.42% 86.12% 87.07% 93.20% 94.96% 95.46%
Book Electronic 73.65% 76.38% 76.27% 79.02% 85.75% 93.68% 95.46% 95.43%
Book Kitchen 71.63% 78.10% 77.90% 81.05% 87.03% 94.65% 95.20% 96.11%
Book Video 81.45% 82.95% 83.23% 84.98% 87.80% 95.56% 96.10% 96.53%
DVD Book 76.45% 80.20% 80.77% 85.17% 87.78% 93.41% 94.55% 94.86%
DVD Electronic 73.12% 76.00% 76.35% 76.17% 86.32% 92.86% 95.08% 95.35%
DVD Kitchen 73.43% 77.50% 78.15% 82.60% 87.47% 94.76% 96.28% 96.51%
DVD Video 82.75% 85.95% 85.95% 83.80% 89.12% 96.15% 96.65% 97.10%
Electronic Book 68.87% 72.35% 73.53% 79.92% 84.03% 93.15% 94.70% 94.86%
Electronic DVD 72.60% 75.93% 76.27% 82.63% 84.32% 93.38% 95.68% 96.13%
Electronic Kitchen 84.63% 86.50% 84.53% 85.80% 90.08% 95.70% 96.41% 96.60%
Electronic Video 72.48% 75.65% 77.20% 81.70% 84.08% 95.48% 96.23% 96.71%
Kitchen Book 71.53% 73.97% 74.17% 80.55% 84.88% 93.68% 94.03% 95.00%
Kitchen DVD 73.32% 75.67% 75.32% 82.18% 84.72% 93.40% 95.81% 96.03%
Kitchen Electronic 83.15% 85.38% 85.53% 88.00% 89.33% 94.63% 96.00% 96.01%
Kitchen Video 76.08% 77.97% 76.37% 81.47% 84.85% 95.90% 96.91% 96.80%
Video Book 77.03% 79.48% 80.03% 83.00% 87.10% 93.88% 94.18% 94.19%
Video DVD 82.43% 83.65% 84.15% 85.90% 87.90% 95.30% 95.96% 96.50%
Video Electronic 71.87% 75.93% 75.72% 77.67% 85.98% 94.00% 94.98% 95.86%
Video Kitchen 71.33% 74.78% 75.22% 79.52% 86.45% 95.80% 96.28% 96.71%
平均准确率 75.92% 78.69% 79.00% 82.36% 86.60% 94.42% 95.57% 95.93%
Experimental Accuracy on Amazon Review Data Set
Average Accuracy of Eight Models in Source-Target Domain Experiments
Average Accuracy of CDSA-F Model in Source-Target Domain Experiments
Average Accuracy of CDSA-F-Att Model in Source-Target Domain Experiments
[1] Al-Moslmi T, Omar N, Abdullah S , et al. Approaches to Cross-Domain Sentiment Analysis: A Systematic Literature Review[J]. IEEE Access, 2017,5:16173-16192.
[2] Lai S W, Liu K, He S Z , et al. How to Generate a Good Word Embedding[J]. IEEE Intelligent Systems, 2016,31(6):5-14.
[3] Weiss K, Khoshgoftaar T M, Wang D D . A Survey of Transfer Learning[J]. Journal of Big Data, 2016, 3(1): Article No. 9.
[4] Tahmoresnezhad J, Hashemi S . Visual Domain Adaptation via Transfer Feature Learning[J]. Knowledge and Information Systems, 2017,50(2):585-605.
[5] Sun M, Tan Q, Ding R W, et al. Cross-Domain Sentiment Classification Using Deep Learning Approach [C]//Proceedings of the 3rd International Conference on Cloud Computing and Intelligence Systems. IEEE, 2014: 60-64.
[6] 余传明 . 基于深度循环神经网络的跨领域文本情感分析[J]. 图书情报工作, 2018,62(11):23-34.
[6] ( Yu Chuanming . A Cross-Domain Text Sentiment Analysis Based on Deep Recurrent Neural Network[J]. Library and Information Service, 2018,62(11):23-34.)
[7] Ganin Y, Ustinova E, Ajakan H , et al. Domain-Adversarial Training of Neural Networks[J]. The Journal of Machine Learning Research, 2016,17(1):2096-2130.
[8] Gamon M, Aue A. Automatic Identification of Sentiment Vocabulary: Exploiting Low Association with Known Sentiment Terms [C]//Proceedings of the ACL Workshop on Feature Engineering for Machine Learning in Natural Language Processing. 2005: 57-64.
[9] Dai W Y, Yang Q, Xue G R, et al. Boosting for Transfer Learning [C]//Proceedings of the 24th International Conference on Machine Learning. 2007: 193-200.
[10] Zhang S W, Liu H L, Yang L, et al. A Cross-domain Sentiment Classification Method Based on Extraction of Key Sentiment Sentence [C]//Proceedings of the 4th CCF Conference on Natural Language Processing and Chinese Computing-Volume 9362. 2015: 90-101.
[11] Blitzer J, McDonald R, Pereira F. Domain Adaptation with Structural Correspondence Learning [C]//Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. 2006: 120-128.
[12] Blitzer J, Dredze M, Pereira F. Biographies, Bollywood, Boom-Boxes and Blenders: Domain Adaptation for Sentiment Classification [C]//Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. 2007: 440-447.
[13] Pan S J, Ni X, Sun J T, et al. Cross-Domain Sentiment Classification via Spectral Feature Alignment [C]// Proceedings of the 19th International Conference on World Wide Web. 2010: 751-760.
[14] Wei X C, Lin H F, Yang L . Cross-Domain Sentiment Classification via Constructing Semantic Correlation[J]. IAENG International Journal of Computer Science, 2017,44(2):172-179.
[15] Glorot X, Bordes A, Bengio Y. Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach [C]//Proceedings of the 28th International Conference on Machine Learning. 2011: 513-520.
[16] Chen M M, Xu Z X, Weinberger K Q, et al. Marginalized Denoising Autoencoders for Domain Adaptation [C]//Proceedings of the 29th International Conference on Machine Learning. 2012: 1627-1634.
[17] Mikolov T, Yih W T, Zweig G. Linguistic Regularities in Continuous Space Word Representations [C]//Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2013: 746-751.
[18] Akhtar M S, Kumar A, Ekbal A, et al. A Hybrid Deep Learning Architecture for Sentiment Analysis [C]// Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers. 2016: 482-493.
[19] Yu J F, Jiang J. Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification [C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016: 236-246.
[20] Li Y T, Baldwin T, Cohn T. What’s in a Domain? Learning Domain-Robust Text Representations Using Adversarial Training [C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2. 2018: 474-479.
[21] Zhang L W, Tu K W, Zhang Y. Latent Variable Sentiment Grammar [C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 4642-4651.
[22] Tai K S, Socher R, Manning C D. Improved Semantic Representations from Tree-Structured Long Short-Term Memory Networks [C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1). 2015: 1556-1566.
[23] Qian Q, Huang M L, Lei J H, et al. Linguistically Regularized LSTM for Sentiment Classification [C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1). 2017: 1679-1689.
[24] Wei X C, Lin H F, Yang L , et al. A Convolution-LSTM-Based Deep Neural Network for Cross-Domain MOOC Forum Post Classification[J]. Information, 2017,8(3). DOI: 10.3390/info8030092.
[25] Raffel C, Ellis D P W . Feed-Forward Networks with Attention can Solve Some Long-Term Memory Problems[OL]. arXiv Preprint, arXiv: 1512.08756.
[26] Yang Z C, Yang D Y, Dyer C, et al. Hierarchical Attention Networks for Document Classification [C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016: 1480-1489.
[27] Gu Y, Yang K N, Fu S Y, et al. Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1). 2018: 2225-2235.
[28] Li Z, Zhang Y, Wei Y, et al. End-to-End Adversarial Memory Network for Cross-Domain Sentiment Classification [C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017.
[29] Li Z, Wei Y, Zhang Y, et al. Hierarchical Attention Transfer Network for Cross-Domain Sentiment Classification [C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 5852-5859.
[30] Ben-David S, Blitzer J, Crammer K , et al. A Theory of Learning from Different Domains[J]. Machine Learning, 2010,79(1-2):151-175.
doi: 10.1007/s10994-009-5152-4
[31] Jatnika D, Bijaksana M A, Suryani A A . Word2Vec Model Analysis for Semantic Similarities in English Words[J]. Procedia Computer Science, 2019,157:160-167.
doi: 10.1016/j.procs.2019.08.153
[32] Hochreiter S, Schmidhuber J . Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780.
doi: 10.1162/neco.1997.9.8.1735 pmid: 9377276
[33] Zhou P, Shi W, Tian J, et al. Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification [C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2). 2016: 207-212.
[34] Buja A, Stuetzle W, Shen Y. Loss Functions for Binary Class Probability Estimation and Classification: Structure and Applications[EB/OL]. [ 2020- 07- 12]. http://www-stat.wharton.upenn.edu/~buja/.
[1] Chen Jie,Ma Jing,Li Xiaofeng. Short-Text Classification Method with Text Features from Pre-trained Models[J]. 数据分析与知识发现, 2021, 5(9): 21-30.
[2] Yang Hanxun, Zhou Dequn, Ma Jing, Luo Yongcong. Detecting Rumors with Uncertain Loss and Task-level Attention Mechanism[J]. 数据分析与知识发现, 2021, 5(7): 101-110.
[3] Yin Pengbo,Pan Weimin,Zhang Haijun,Chen Degang. Identifying Clickbait with BERT-BiGA Model[J]. 数据分析与知识发现, 2021, 5(6): 126-134.
[4] Xie Hao,Mao Jin,Li Gang. Sentiment Classification of Image-Text Information with Multi-Layer Semantic Fusion[J]. 数据分析与知识发现, 2021, 5(6): 103-114.
[5] Meng Zhen,Wang Hao,Yu Wei,Deng Sanhong,Zhang Baolong. Vocal Music Classification Based on Multi-category Feature Fusion[J]. 数据分析与知识发现, 2021, 5(5): 59-70.
[6] Zhang Guobiao,Li Jie. Detecting Social Media Fake News with Semantic Consistency Between Multi-model Contents[J]. 数据分析与知识发现, 2021, 5(5): 21-29.
[7] Han Pu,Zhang Zhanpeng,Zhang Mingtao,Gu Liang. Normalizing Chinese Disease Names with Multi-feature Fusion[J]. 数据分析与知识发现, 2021, 5(5): 83-94.
[8] Duan Jianyong,Wei Xiaopeng,Wang Hao. A Multi-Perspective Co-Matching Model for Machine Reading Comprehension[J]. 数据分析与知识发现, 2021, 5(4): 134-141.
[9] Wang Yuzhu,Xie Jun,Chen Bo,Xu Xinying. Multi-modal Sentiment Analysis Based on Cross-modal Context-aware Attention[J]. 数据分析与知识发现, 2021, 5(4): 49-59.
[10] Lin Kerou,Wang Hao,Gong Lijuan,Zhang Baolong. Disambiguation of Chinese Author Names with Multiple Features[J]. 数据分析与知识发现, 2021, 5(4): 90-102.
[11] Han Pu, Zhang Wei, Zhang Zhanpeng, Wang Yuxin, Fang Haoyu. Sentiment Analysis of Weibo Posts on Public Health Emergency with Feature Fusion and Multi-Channel[J]. 数据分析与知识发现, 2021, 5(11): 68-79.
[12] Jiang Cuiqing,Wang Xiangxiang,Wang Zhao. Forecasting Car Sales Based on Consumer Attention[J]. 数据分析与知识发现, 2021, 5(1): 128-139.
[13] Yin Haoran,Cao Jinxuan,Cao Luzhe,Wang Guodong. Identifying Emergency Elements Based on BiGRU-AM Model with Extended Semantic Dimension[J]. 数据分析与知识发现, 2020, 4(9): 91-99.
[14] Huang Lu,Zhou Enguo,Li Daifeng. Text Representation Learning Model Based on Attention Mechanism with Task-specific Information[J]. 数据分析与知识发现, 2020, 4(9): 111-122.
[15] Shi Lei,Wang Yi,Cheng Ying,Wei Ruibin. Review of Attention Mechanism in Natural Language Processing[J]. 数据分析与知识发现, 2020, 4(5): 1-14.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn