1School of Computer Science, Hunan University of Technology and Business,Changsha 410205, China 2School of Frontier Crossover Studies, Hunan University of Technology and Business,Changsha 410205, China
[Objective] This study aims to solve problems such as text feature extraction bias and difficult separation of ambiguous semantics caused by the unique expressions and semantic drift phenomenon in Chinese. [Methods] This paper proposes a supervised contrastive learning semantic optimization method, which first uses a pre-trained model to generate semantic vectors, then designs a supervised joint self-supervised method to construct contrastive sample pairs, and finally constructs a supervised contrastive loss for semantic space measurement and optimization. [Results] On the ChnSentiCorp dataset, the five mainstream neural network models optimized by this method achieved F1 value improvements of 2.77%-3.82%. [Limitations] Due to limited hardware resources, a larger number of contrastive learning sample pairs were not constructed. [Conclusions] The semantic optimization method can effectively solve problems such as text feature extraction bias and difficult separation of ambiguous semantics, and provide new research ideas for text sentiment analysis tasks.
熊曙初, 李轩, 吴佳妮, 周赵宏, 孟晗. 基于有监督对比学习的文本情感语义优化方法研究*[J]. 数据分析与知识发现, 2024, 8(6): 69-81.
Xiong Shuchu, Li Xuan, Wu Jiani, Zhou Zhaohong, Meng Han. Research on Text Sentiment Semantic Optimization Method Based on Supervised Contrastive Learning. Data Analysis and Knowledge Discovery, 2024, 8(6): 69-81.
(Wang Xiwei, Wang Xiaotian, Li Yueqi. Research on the Construction of the Event Evolution Graph of UGC of Network Public Opinions for Major Emergencies——Taking the Natural Disaster 7·20 Torrential Rain in Henan as an Example[J]. Library and Information Service, 2022, 66(16): 13-23.)
doi: 10.13266/j.issn.0252-3116.2022.16.002
(Ji Chen, Ye Huiwen, Wang Zimin, et al. Social Text Sentiment Analysis Based on BiLSTM-CapsNet Hybrid Model[J]. Journal of Nanjing University of Science and Technology, 2022, 46(2): 205-210.)
(Yang Qing, Zhang Yawen, Zhu Li, et al. Text Sentiment Analysis Based on Fusion of Attention Mechanism and BiGRU[J]. Computer Science, 2021, 48(11): 307-311.)
doi: 10.11896/jsjkx.201000075
(Wang Yingjie, Zhu Jiuqi, Wang Zumin, et al. Review of Applications of Natural Language Processing in Text Sentiment Analysis[J]. Journal of Computer Applications, 2022, 42(4): 1011-1020.)
doi: 10.11772/j.issn.1001-9081.2021071262
[7]
Hearst M A. Direction-Based Text Interpretation as an Information Access Refinement[M]// Text-Based Intelligent Systems:Current Research and Practice in Information Extraction and Retrieval. Mahwah, NJ: Lawrence Erlbaum, 1992: 257-274.
[8]
Turney P D. Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews[OL]. arXiv Preprint, arXiv: cs/0212032.
[9]
Asghar M Z, Khan A, Ahmad S, et al. Lexicon-Enhanced Sentiment Analysis Framework Using Rule-Based Classification Scheme[J]. PLoS One, 2017, 12(2): e0171649.
(Li Yuqing, Li Xin, Han Xu, et al. A Bilingual Lexicon-Based Multi-class Semantic Orientation Analysis for Microblogs[J]. Acta Electronica Sinica, 2016, 44(9): 2068-2073.)
doi: 10.3969/j.issn.0372-2112.2016.09.007
[11]
马耀红. 基于机器学习的多粒度文本情感分类方法研究[D]. 武汉: 武汉大学, 2021.
[11]
(Ma Yaohong. Multi-granular Text Sentiment Classification for Method Research Based on Machine Learning[D]. Wuhan: Wuhan University, 2021.)
[12]
Pang B, Lee L, Vaithyanathan S. Thumbs Up? Sentiment Classification Using Machine Learning Techniques[OL]. arXiv Preprint, arXiv: cs/0205070.
[13]
Manek A S, Shenoy P D, Mohan M C, et al. Aspect Term Extraction for Sentiment Analysis in Large Movie Reviews Using Gini Index Feature Selection Method and SVM Classifier[J]. World Wide Web, 2017, 20(2): 135-154.
(Yang Shuang, Chen Fen. Analyzing Sentiments of Micro-blog Posts Based on Support Vector Machine[J]. Data Analysis and Knowledge Discovery, 2017, 1(2): 73-79.)
(Shi Ronghua, Jin Xin, Hu Chao. Aspect-Level Text Emotion Analysis Based on Graph Attention Network[J]. Computer Engineering, 2022, 48(2): 34-39.)
doi: 10.19678/j.issn.1000-3428.0060517
(Wang Ting, Yang Wenzhong. Review of Text Sentiment Analysis Methods[J]. Computer Engineering and Applications, 2021, 57(12): 11-24.)
doi: 10.3778/j.issn.1002-8331.2101-0022
(Teng Fei, Zheng Chaomei, Li Wen. Multidimensional Topic Model for Oriented Sentiment Analysis Based on Long Short-Term Memory[J]. Journal of Computer Applications, 2016, 36(8): 2252-2256.)
doi: 10.11772/j.issn.1001-9081.2016.08.2252
(Zhang Jun, Zhang Li, Shen Fanfan, et al. Implicit Sentiment Analysis Based on RoBERTa Fused with BiLSTM and Attention Mechanism[J]. Computer Engineering and Applications, 2022, 58(23): 142-150.)
doi: 10.3778/j.issn.1002-8331.2203-0505
(Zhang Mingquan, Zhou Hui, Cao Jingang. Dual BERT Directed Sentiment Text Classification Based on Attention Mechanism[J]. CAAI Transactions on Intelligent Systems, 2022, 17(6): 1220-1227.)
(Li Xi, Liu Xiping, Li Wangcai, et al. Survey on Contrastive Learning Research[J]. Journal of Chinese Computer Systems, 2023, 44(4): 787-797.)
[22]
Liu X, Zhang F J, Hou Z Y, et al. Self-Supervised Learning: Generative or Contrastive[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 35(1): 857-876.
[23]
Chen T, Kornblith S, Norouzi M, et al. A Simple Framework for Contrastive Learning of Visual Representations[C]// Proceedings of the 37th International Conference on Machine Learning. 2020: 1597-1607.
[24]
He K M, Fan H Q, Wu Y X, et al. Momentum Contrast for Unsupervised Visual Representation Learning[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 9726-9735.
[25]
Chen T, Kornblith S, Swersky K, et al. Big Self-Supervised Models are Strong Semi-Supervised Learners[C]// Proceedings of the 34th Conference on Neural Information Processing Systems. 2020.
[26]
Chen X L, He K M. Exploring Simple Siamese Representation Learning[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 15745-15753.
[27]
Wang D, Ding N, Li P J, et al. CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding[OL]. arXiv Preprint, arXiv: 2107.00440.
[28]
Gao T Y, Yao X C, Chen D Q. SimCSE: Simple Contrastive Learning of Sentence Embeddings[OL]. arXiv Preprint, arXiv: 2104.08821.
[29]
Pan X, Wang M X, Wu L W, et al. Contrastive Learning for Many-to-Many Multilingual Neural Machine Translation[OL]. arXiv Preprint, arXiv: 2105.09501.
[30]
Xiong L E, Xiong C Y, Li Y, et al. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval[OL]. arXiv Preprint, arXiv: 2007.00808.
[31]
Xu S S, Zhang X X, Wu Y, et al. Sequence Level Contrastive Learning for Text Summarization[C]// Proceedings of the 36th AAAI Conference on Artificial Intelligence. 2022: 11556-11565.
[32]
Mao X L, Shan Y R, Li F F, et al. CLSpell: Contrastive Learning with Phonological and Visual Knowledge for Chinese Spelling Check[J]. Neurocomputing, 2023, 554: 126468.
(Guo Dongen, Xia Ying, Luo Xiaobo, et al. Remote Sensing Image Scene Classification Based on Supervised Contrastive Learning[J]. Acta Photonica Sinica, 2021, 50(7): 0710002.)
[34]
Sedghamiz H, Raval S, Santus E, et al. SupCL-Seq: Supervised Contrastive Learning for Downstream Optimized Sequence Representations[OL]. arXiv Preprint, arXiv: 2109.07424.
[35]
Mikolov T, Chen K, Corrado G, et al. Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint, arXiv: 1301.3781.
[36]
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 6000-6010.
[37]
Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv: 1810.04805.
