Detecting Sarcasm from Travel Reviews Based on Cross-Modal Deep Learning
Liu Yang1,2(),Ma Lili3,Zhang Wen1,Hu Zhongyi1,2,Wu Jiang1,2
1School of Information Management, Wuhan University, Wuhan 430072,China 2Center for E-commerce Research and Development, Wuhan University, Wuhan 430072, China 3Economics and Management School, Wuhan University, Wuhan 430072, China
[Objective] Based on the cross-modal deep learning method, this paper analyzes consumers’ sentiments in travel reviews and identifies their sarcastic expression. [Methods] First, we encoded multi-modal information. Then, we extracted the interaction information between texts and pictures with the graph neural network. Finally, we used the attention mechanism to identify multi-modal features and sarcasm. [Results] We examined the proposed model with travel reviews from Yelp. The accuracy of sarcasm detection reached 88.77%, which is better than the baseline models. [Limitations] We only examined the proposed model with reviews on Hilton hotels, which needs to be expanded in the future. [Conclusions] The proposed model could extract interaction information between different modal of data, that effectively improve the accuracy of sarcasm detection.
刘洋, 马莉莉, 张雯, 胡忠义, 吴江. 基于跨模态深度学习的旅游评论反讽识别*[J]. 数据分析与知识发现, 2022, 6(12): 23-31.
Liu Yang, Ma Lili, Zhang Wen, Hu Zhongyi, Wu Jiang. Detecting Sarcasm from Travel Reviews Based on Cross-Modal Deep Learning. Data Analysis and Knowledge Discovery, 2022, 6(12): 23-31.
We stayed at Hilton Palace for one night while we stopped and enjoyed Disney Springs while we drove through Florida. For the price we paid， I'm disappointed. The room was tiny， outdated， sliding window was must have had the gases released through the window panes that it fogged it completely. Slept on a mattress cover and not sheets. which was weird. I wouldn't stay here again. I guess I expected more out of Hilton across from Disney World.
0
1
Christina Z
2021.8.6
I was here for work. There was a convention at the Orange County Convention Center. The hotel was very clean. Covid rules were very relaxed， but I expected that in Florida. Their room service is still suspended but their restaurant David's does take out. The staff was extremely friendly. Their WiFi was awful in the rooms and made it challenging to do work. Likewise the TV would keep going in and out. We weren't having any storms so I'm not sure what the problem was. I would stay here again. I just hope they fix their internet
(Chen Xiaohong, Li Yangyang, Song Lijie, et al. Theoretical Framework and Research Prospect of Digital Economy[J]. Journal of Management World, 2022, 38(2): 208-224.)
[2]
Alarcón-del-Amo M D C, Rialp-Criado A, Rialp-Criado J. Examining the Impact of Managerial Involvement with Social Media on Exporting Firm Performance[J]. International Business Review, 2018, 27(2): 355-366.
doi: 10.1016/j.ibusrev.2017.09.003
[3]
Goldin P R, McRae K, Ramel W, et al. The Neural Bases of Emotion Regulation: Reappraisal and Suppression of Negative Emotion[J]. Biological Psychiatry, 2008, 63(6): 577-586.
doi: 10.1016/j.biopsych.2007.05.031
pmid: 17888411
[4]
Braunstein L M, Gross J J, Ochsner K N. Explicit and Implicit Emotion Regulation: A Multi-Level Framework[J]. Social Cognitive and Affective Neuroscience, 2017, 12(10): 1545-1557.
doi: 10.1093/scan/nsx096
pmid: 28981910
[5]
Joshi A, Bhattacharyya P, Carman M J. Automatic Sarcasm Detection: A Survey[J]. ACM Computing Surveys, 2017, 50(5): 1-22.
[6]
Gibbs R W. On the Psycholinguistics of Sarcasm[J]. Journal of Experimental Psychology: General, 1986, 115(1): 3-15.
doi: 10.1037/0096-3445.115.1.3
[7]
Ordenes F V, Ludwig S, de Ruyter K, et al. Unveiling What is Written in the Stars: Analyzing Explicit, Implicit, and Discourse Patterns of Sentiment in Social Media[J]. Journal of Consumer Research, 2017, 43(6): 875-894.
[8]
Eke C I, Norman A A, Shuib L, et al. Sarcasm Identification in Textual Data: Systematic Review, Research Challenges and Open Directions[J]. Artificial Intelligence Review, 2020, 53(6): 4215-4258.
doi: 10.1007/s10462-019-09791-8
[9]
Riloff E, Qadir A, Surve P, et al. Sarcasm as Contrast Between a Positive Sentiment and Negative Situation[C]// Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 2013: 704-714.
[10]
Bharti S K, Babu K S, Jena S K. Parsing-Based Sarcasm Sentiment Recognition in Twitter Data[C]// Proceedings of 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining. 2015: 1373-1380.
[11]
Bing L. Sentiment Analysis and Opinion Mining[J]. Synthesis Lectures on Human Language Technologies, 2012, 5(1): 1-167.
[12]
Bharti S K, Pradhan R, Babu K S, et al. Sarcasm Analysis on Twitter Data Using Machine Learning Approaches[J]. Trends in Social Network Analysis, 2017: 51-76.
[13]
González-Ibánez R, Muresan S, Wacholder N. Identifying Sarcasm in Twitter: A Closer Look[C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies. 2011: 581-586.
[14]
Zhang Y Z, Liu Y C, Li Q C, et al. CFN: A Complex-Valued Fuzzy Network for Sarcasm Detection in Conversations[J]. IEEE Transactions on Fuzzy Systems, 2021, 29(12): 3696-3710.
doi: 10.1109/TFUZZ.2021.3072492
[15]
Ghosh A, Veale D T. Fracking Sarcasm Using Neural Network[C]// Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. 2016: 161-169.
[16]
McFee B, Lanckriet G, Jebara T. Learning Multi-modal Similarity[J]. Journal of Machine Learning Research, 2011, 12: 491-523.
[17]
Baltrušaitis T, Ahuja C, Morency L P. Multimodal Machine Learning: A Survey and Taxonomy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(2): 423-443.
doi: 10.1109/TPAMI.2018.2798607
pmid: 29994351
(Liu Jinqiao, Wu Jinqiang. Development of Machine Vision System and Its Application[J]. Mechanical Engineering ＆ Automation, 2010(1): 215-216.)
[19]
Law R, Li G, Fong D K C, et al. Tourism Demand Forecasting: A Deep Learning Approach[J]. Annals of Tourism Research, 2019, 75: 410-423.
doi: 10.1016/j.annals.2019.01.014
[20]
Sun S L, Wei Y J, Tsui K L, et al. Forecasting Tourist Arrivals with Machine Learning and Internet Search Index[J]. Tourism Management, 2019, 70: 1-10.
doi: 10.1016/j.tourman.2018.07.010
[21]
Yang Y, Tang J Y, Luo H, et al. Hotel Location Evaluation: A Combination of Machine Learning Tools and Web GIS[J]. International Journal of Hospitality Management, 2015, 47: 14-24.
doi: 10.1016/j.ijhm.2015.02.008
[22]
Deng N, Li X. Feeling a Destination Through the “Right” Photos: A Machine Learning Model for DMOs’ Photo Selection[J]. Tourism Management, 2018, 65: 267-278.
doi: 10.1016/j.tourman.2017.09.010
[23]
O’Connor P. Managing a Hotel’s Image on TripAdvisor[J]. Journal of Hospitality Marketing & Management, 2010, 19(7): 754-772.
[24]
Ma Y F, Xiang Z, Du Q Z, et al. Effects of User-Provided Photos on Hotel Review Helpfulness: An Analytical Approach with Deep Leaning[J]. International Journal of Hospitality Management, 2018, 71: 120-131.
doi: 10.1016/j.ijhm.2017.12.008
[25]
Tenney I, Das D, Pavlick E. BERT Rediscovers the Classical NLP Pipeline[OL]. arXiv Preprint, arXiv:1905.05950.
[26]
Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF Models for Sequence Tagging[OL]. arXiv Preprint, arXiv:1508.01991.
[27]
He K M, Zhang X Y, Ren S Q, et al. Identity Mappings in Deep Residual Networks[C]// Proceedings of European Conference on Computer Vision. 2016: 630-645.
[28]
Scarselli F, Gori M, Tsoi A C, et al. The Graph Neural Network Model[J]. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80.
doi: 10.1109/TNN.2008.2005605
pmid: 19068426
[29]
Shaw P, Uszkoreit J, Vaswani A. Self-attention with Relative Position Representations[OL]. arXiv Preprint, arXiv:1803.02155.
[30]
Zhang Y, Wallace B. A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification[OL]. arXiv Preprint, arXiv:1510.03820.
[31]
Tan M X, Le Q V. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks[C]// Proceedings of the 36th International Conference on Machine Learning. 2019: 6105-6114.
[32]
Vlad G A, Zaharia G E, Cercel D C, et al. UPB at SemEval-2020 Task 8: Joint Textual and Visual Modeling in a Multi-task Learning Architecture for Memotion Analysis[OL]. arXiv Preprint, arXiv:2009.02779.
[33]
Pan H L, Lin Z, Fu P, et al. Modeling Intra and Inter-modality Incongruity for Multi-modal Sarcasm Detection[C]// Findings of the Association for Computational Linguistics:EMNLP 2020. 2020: 1383-1392.
