|
|
Analyzing Text Sentiments Based on Patch Attention and Involution |
Lin Zhe(),Chen Pinghua |
School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China |
|
|
Abstract [Objective] Once the width of the convolution kernel is the same as the dimension of the word vector, the convolution layer will have too many parameters. The sparse connection of convolution operation, the spatial invariance, and the channel specificity of convolution are not suitable for text tasks. This paper will address these issues. [Methods] We proposed a sentiment analysis model for texts based on patch attention mechanism and Involution. The model first transformed the single-word vector after word segmentation and transformed the one-dimensional word vector into n×n word matrix blocks. Then, we spliced the word matrix blocks of multiple words in the sentence into a sentence matrix. Third, the patch attention mechanism layer enhanced the sentence matrix’s context relevance and position order information of text features. Fourth, we used the involution with spatial specificity and channel invariance to extract the sentence matrix features. Finally, we used the full connection layer for text sentiment classification. [Results] We examined the proposed model with three public data sets waimai_10k, IMDB, and Tweet. Its classification precision reached 88.47%, 86.22%, and 94.42%, respectively, which were 6.47%, 7.72%, 9.35% and 1.07%, 1.01%, 0.59% higher than Bi-LSTM model in word vector convolution network and recurrent neural network. [Limitations] The classification accuracy of this model on large datasets is not as high as on small and medium-sized datasets. [Conclusions] The proposed model solves the problems of excessive parameters, sparse connection of convolution operation, spatial invariance, and channel specificity of convolution, which yield better performance than the traditional convolution models.
|
Received: 09 September 2022
Published: 28 March 2023
|
|
Fund:Guangdong Provincial Key R&D Programme(2020B0101100001);Guangdong Provincial Key R&D Programme(2021B0101200002) |
Corresponding Authors:
Lin Zhe,ORCID:0000-0003-0894-5159,E-mail:1417505493@qq.com。
|
[1] |
王婷, 杨文忠. 文本情感分析方法研究综述[J]. 计算机工程与应用, 2021, 57(12): 11-24.
doi: 10.3778/j.issn.1002-8331.2101-0022
|
[1] |
(Wang Ting, Yang Wenzhong. Review of Text Sentiment Analysis Methods[J]. Computer Engineering and Applications, 2021, 57(12): 11-24.)
doi: 10.3778/j.issn.1002-8331.2101-0022
|
[2] |
杨书新, 张楠. 融合情感词典与上下文语言模型的文本情感分析[J]. 计算机应用, 2021, 41(10): 2829-2834.
doi: 10.11772/j.issn.1001-9081.2020121900
|
[2] |
(Yang Shuxin, Zhang Nan. Text Sentiment Analysis Based on Sentiment Lexicon and Context Language Model[J]. Journal of Computer Applications, 2021, 41(10): 2829-2834.)
doi: 10.11772/j.issn.1001-9081.2020121900
|
[3] |
欧阳继红, 刘燕辉, 李熙铭, 等. 基于LDA的多粒度主题情感混合模型[J]. 电子学报, 2015, 43(9): 1875-1880.
doi: 10.3969/j.issn.0372-2112.2015.09.029
|
[3] |
(Ouyang Jihong, Liu Yanhui, Li Ximing, et al. Multi-Grain Sentiment/Topic Model Based on LDA[J]. Acta Electronica Sinica, 2015, 43(9): 1875-1880.)
doi: 10.3969/j.issn.0372-2112.2015.09.029
|
[4] |
Kim Y. Convolutional Neural Networks for Sentence Classification[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014:1746-1751.
|
[5] |
杨丽, 吴雨茜, 王俊丽, 等. 循环神经网络研究综述[J]. 计算机应用, 2018, 38(S2): 1-6.
|
[5] |
(Yang Li, Wu Yuxi, Wang Junli, et al. Research on Recurrent Neural Network[J]. Journal of Computer Applications, 2018, 38(S2): 1-6.)
|
[6] |
邢长征, 李珊. 文本情感分析的深度学习方法[J]. 计算机应用与软件, 2018, 35(8): 102-106.
|
[6] |
(Xing Changzheng, Li Shan. Deep Learning Method for Text Sentiment Analysis[J]. Computer Applications and Software, 2018, 35(8): 102-106.)
|
[7] |
Basnet A, Timalsina A K. Improving Nepali News Recommendation Using Classification Based on LSTM Recurrent Neural Networks[C]// Proceedings of the IEEE 3rd International Conference on Computing, Communication and Security. 2018: 138-142.
|
[8] |
滕金保, 孔韦韦, 田乔鑫, 等. 基于CNN和LSTM的多通道注意力机制文本分类模型[J]. 计算机工程与应用, 2021, 57(23): 154-162.
doi: 10.3778/j.issn.1002-8331.2104-0212
|
[8] |
(Teng Jinbao, Kong Weiwei, Tian Qiaoxin, et al. Multi-Channel Attention Mechanism Text Classification Model Based on CNN and LSTM[J]. Computer Engineering and Applications, 2021, 57(23): 154-162.)
doi: 10.3778/j.issn.1002-8331.2104-0212
|
[9] |
王友卫, 朱晨, 朱建明, 等. 基于用户兴趣词典和LSTM的个性化情感分类方法[J]. 计算机科学, 2021, 48(S2): 251-257.
|
[9] |
(Wang Youwei, Zhu Chen, Zhu Jianming, et al. User Interest Dictionary and LSTM Based Method for Personalized Emotion Classification[J]. Computer Science, 2021, 48(S2): 251-257.)
|
[10] |
Lai S W, Xu L H, Liu K, et al. Recurrent Convolutional Neural Networks for Text Classification[C]// Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015: 2267-2273.
|
[11] |
朱烨, 陈世平. 融合卷积神经网络和注意力的评论文本情感分析[J]. 小型微型计算机系统, 2020, 41(3): 551-557.
|
[11] |
(Zhu Ye, Chen Shiping. Commentary Text Sentiment Analysis Combining Convolution Neural Network and Attention[J]. Journal of Chinese Computer Systems, 2020, 41(3): 551-557.)
|
[12] |
吴迪, 姜丽婷, 王路路, 等. 结合多头注意力机制的旅游问句分类研究[J]. 计算机工程与应用, 2022, 58(3): 165-171.
doi: 10.3778/j.issn.1002-8331.2008-0151
|
[12] |
(Wu Di, Jiang Liting, Wang Lulu, et al. Research on Classification of Tourist Questions Combined with Multi-Head Attention Mechanism[J]. Computer Engineering and Applications, 2022, 58(3): 165-171.)
doi: 10.3778/j.issn.1002-8331.2008-0151
|
[13] |
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 6000-6010.
|
[14] |
Mnih V, Heess N, Graves A, et al. Recurrent Models of Visual Attention[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2. 2014: 2204-2212.
|
[15] |
Bahdanau D, Cho K, Bengio Y. Neural Machine Translation by Jointly Learning to Align and Translate[OL]. arXiv Preprint, arXiv: 1409.0473.
|
[16] |
程淑玉, 郭泽颖, 刘威, 等. 融合Attention多粒度句子交互自然语言推理研究[J]. 小型微型计算机系统, 2019, 40(6): 1215-1220.
|
[16] |
(Cheng Shuyu, Guo Zeying, Liu Wei, et al. Research on Multi-Granularity Sentence Interaction Natural Language Inference Based on Attention Mechanism[J]. Journal of Chinese Computer Systems, 2019, 40(6): 1215-1220.)
|
[17] |
Li D, Hu J, Wang C H, et al. Involution: Inverting the Inherence of Convolution for Visual Recognition[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 12316-12325.
|
[18] |
杨洪刚, 陈洁洁, 徐梦飞. 双线性内卷神经网络用于眼底疾病图像分类[J]. 计算机应用, 2023, 43(1): 259-264.
doi: 10.11772/j.issn.1001-9081.2021111932
|
[18] |
(Yang Honggang, Chen Jiejie, Xu Mengfei. Bilinear Involution Neural Network for Image Classification of Fundus Diseases[J]. Journal of Computer Applications, 2023, 43(1): 259-264.)
doi: 10.11772/j.issn.1001-9081.2021111932
|
[19] |
林志洁, 郑秋岚, 梁涌, 等. 基于内卷U-Net的医学图像分割模型[J]. 计算机工程, 2022, 48(8): 180-186.
doi: 10.19678/j.issn.1000-3428.0062023
|
[19] |
(Lin Zhijie, Zheng Qiulan, Liang Yong, et al. Medical Image Segmentation Model Based on Involution U-Net[J]. Computer Engineering, 2022, 48(8): 180-186.)
doi: 10.19678/j.issn.1000-3428.0062023
|
[20] |
Mikolov T, Chen K, Corrado G, et al. Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint, arXiv: 1301.3781.
|
[21] |
Sharfuddin A A, Tihami M N, Islam M S. A Deep Recurrent Neural Network with BiLSTM Model for Sentiment Classification[C]// Proceedings of the International Conference on Bangla Speech and Language Processing. 2018: 1-4.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|