1Research Center for Language Intelligence, Dalian University of Foreign Languages, Dalian 116044, China 2School of Software Engineering, Dalian University of Foreign Languages, Dalian 116044, China
[Objective] This paper tries to address the issues of labelled data shortage, aiming to distinguish the weights of sentiment characteristics in cross-domain sentiment classification. [Methods] We proposed a sentiment classification model for cross-domain product reviews based on feature fusion representation and the attention mechanism. First, this model integrated Bert and cross-domain word vectors to generate cross-domain unified feature space. Then, it extracted the weights of global and local features through attention mechanism. [Results] We examined our model with public review data from Amazon and found the average accuracy of the proposed model was up-to 95.93%, which was 9.33% higher than the existing model. [Limitations] More research is needed to evaluate our model with large-scale multi-domain data sets. [Conclusions] The proposed model could effectively analyze sentiment information.
Al-Moslmi T, Omar N, Abdullah S , et al. Approaches to Cross-Domain Sentiment Analysis: A Systematic Literature Review[J]. IEEE Access, 2017,5:16173-16192.
[2]
Lai S W, Liu K, He S Z , et al. How to Generate a Good Word Embedding[J]. IEEE Intelligent Systems, 2016,31(6):5-14.
[3]
Weiss K, Khoshgoftaar T M, Wang D D . A Survey of Transfer Learning[J]. Journal of Big Data, 2016, 3(1): Article No. 9.
[4]
Tahmoresnezhad J, Hashemi S . Visual Domain Adaptation via Transfer Feature Learning[J]. Knowledge and Information Systems, 2017,50(2):585-605.
[5]
Sun M, Tan Q, Ding R W, et al. Cross-Domain Sentiment Classification Using Deep Learning Approach [C]//Proceedings of the 3rd International Conference on Cloud Computing and Intelligence Systems. IEEE, 2014: 60-64.
( Yu Chuanming . A Cross-Domain Text Sentiment Analysis Based on Deep Recurrent Neural Network[J]. Library and Information Service, 2018,62(11):23-34.)
[7]
Ganin Y, Ustinova E, Ajakan H , et al. Domain-Adversarial Training of Neural Networks[J]. The Journal of Machine Learning Research, 2016,17(1):2096-2130.
[8]
Gamon M, Aue A. Automatic Identification of Sentiment Vocabulary: Exploiting Low Association with Known Sentiment Terms [C]//Proceedings of the ACL Workshop on Feature Engineering for Machine Learning in Natural Language Processing. 2005: 57-64.
[9]
Dai W Y, Yang Q, Xue G R, et al. Boosting for Transfer Learning [C]//Proceedings of the 24th International Conference on Machine Learning. 2007: 193-200.
[10]
Zhang S W, Liu H L, Yang L, et al. A Cross-domain Sentiment Classification Method Based on Extraction of Key Sentiment Sentence [C]//Proceedings of the 4th CCF Conference on Natural Language Processing and Chinese Computing-Volume 9362. 2015: 90-101.
[11]
Blitzer J, McDonald R, Pereira F. Domain Adaptation with Structural Correspondence Learning [C]//Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. 2006: 120-128.
[12]
Blitzer J, Dredze M, Pereira F. Biographies, Bollywood, Boom-Boxes and Blenders: Domain Adaptation for Sentiment Classification [C]//Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. 2007: 440-447.
[13]
Pan S J, Ni X, Sun J T, et al. Cross-Domain Sentiment Classification via Spectral Feature Alignment [C]// Proceedings of the 19th International Conference on World Wide Web. 2010: 751-760.
[14]
Wei X C, Lin H F, Yang L . Cross-Domain Sentiment Classification via Constructing Semantic Correlation[J]. IAENG International Journal of Computer Science, 2017,44(2):172-179.
[15]
Glorot X, Bordes A, Bengio Y. Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach [C]//Proceedings of the 28th International Conference on Machine Learning. 2011: 513-520.
[16]
Chen M M, Xu Z X, Weinberger K Q, et al. Marginalized Denoising Autoencoders for Domain Adaptation [C]//Proceedings of the 29th International Conference on Machine Learning. 2012: 1627-1634.
[17]
Mikolov T, Yih W T, Zweig G. Linguistic Regularities in Continuous Space Word Representations [C]//Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2013: 746-751.
[18]
Akhtar M S, Kumar A, Ekbal A, et al. A Hybrid Deep Learning Architecture for Sentiment Analysis [C]// Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers. 2016: 482-493.
[19]
Yu J F, Jiang J. Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification [C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016: 236-246.
[20]
Li Y T, Baldwin T, Cohn T. What’s in a Domain? Learning Domain-Robust Text Representations Using Adversarial Training [C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2. 2018: 474-479.
[21]
Zhang L W, Tu K W, Zhang Y. Latent Variable Sentiment Grammar [C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 4642-4651.
[22]
Tai K S, Socher R, Manning C D. Improved Semantic Representations from Tree-Structured Long Short-Term Memory Networks [C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1). 2015: 1556-1566.
[23]
Qian Q, Huang M L, Lei J H, et al. Linguistically Regularized LSTM for Sentiment Classification [C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1). 2017: 1679-1689.
[24]
Wei X C, Lin H F, Yang L , et al. A Convolution-LSTM-Based Deep Neural Network for Cross-Domain MOOC Forum Post Classification[J]. Information, 2017,8(3). DOI: 10.3390/info8030092.
[25]
Raffel C, Ellis D P W . Feed-Forward Networks with Attention can Solve Some Long-Term Memory Problems[OL]. arXiv Preprint, arXiv: 1512.08756.
[26]
Yang Z C, Yang D Y, Dyer C, et al. Hierarchical Attention Networks for Document Classification [C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016: 1480-1489.
[27]
Gu Y, Yang K N, Fu S Y, et al. Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1). 2018: 2225-2235.
[28]
Li Z, Zhang Y, Wei Y, et al. End-to-End Adversarial Memory Network for Cross-Domain Sentiment Classification [C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017.
[29]
Li Z, Wei Y, Zhang Y, et al. Hierarchical Attention Transfer Network for Cross-Domain Sentiment Classification [C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 5852-5859.
[30]
Ben-David S, Blitzer J, Crammer K , et al. A Theory of Learning from Different Domains[J]. Machine Learning, 2010,79(1-2):151-175.
doi: 10.1007/s10994-009-5152-4
[31]
Jatnika D, Bijaksana M A, Suryani A A . Word2Vec Model Analysis for Semantic Similarities in English Words[J]. Procedia Computer Science, 2019,157:160-167.
doi: 10.1016/j.procs.2019.08.153
Zhou P, Shi W, Tian J, et al. Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification [C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2). 2016: 207-212.
[34]
Buja A, Stuetzle W, Shen Y. Loss Functions for Binary Class Probability Estimation and Classification: Structure and Applications[EB/OL]. [ 2020- 07- 12]. http://www-stat.wharton.upenn.edu/~buja/.
