A Deep Recommendation Model with Multi-Layer Interaction and Sentiment Analysis

doi:10.11925/infotech.2096-3467.2022.0228

Data Analysis and Knowledge Discovery

2023, Vol. 7

Issue (3): 43-57 DOI: 10.11925/infotech.2096-3467.2022.0228

Current Issue | Archive | Adv Search

A Deep Recommendation Model with Multi-Layer Interaction and Sentiment Analysis

Li Haojun(

),Lv Yun,Wang Xuhui,Huang Jieya

College of Education Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China

Download: PDF (1549 KB) HTML ( 30 )
Export: BibTeX | EndNote (RIS)

Abstract

[Objective] This paper proposes a deep recommendation model with multi-layer interaction and sentiment analysis. It tries to improve the traditional recommendation algorithms which rely on single user ratings to infer user preferences and ignore the impacts of sentiments. [Methods] First, we used the BRET word vector to represent the reviews, and utilized the bidirectional recurrent neural network to quantify their sentiments. Then, we updated the rating matrix using the sentiment values, and mapped the shallow features of users and resources. Fourth, we captured the deep features of users and resources from reviews with the convolutional neural network and the self-attention mechanism. Finally, we merged the shallow and deep features, and used the multi-layer perceptron to model the complex nonlinear interaction between users and resources to predict the rating of recommended resources. [Results] We examined the model with Amazon dataset and found the MAE and RMSE metrics were upto 7.93% and 9.37% lower than those of the baseline models. [Limitations] Our model did not include the temporal dynamics of user sentiment and ignore the domain adaptiveness of sentiment analysis methods. [Conclusions] The recommendation model incorporating sentiment analysis can more accurately reflect users’ real interests and preferences, and then effectively improve the recommendation accuracy.

Key words： Sentiment Analysis Feature Mining Neural Networks Multi-Layer Interaction Recommendation Models

Received: 18 March 2022 Published: 13 April 2023

ZTFLH:

TP391

Fund:National Natural Science Foundation of China(62077043);Zhejiang Provincial Philosophy and Social Sciences Planning Project(22JCXK05Z)

Corresponding Authors: Li Haojun，ORCID:0000-0003-0318-6528，E-mail:zgdlhj@zjut.edu.cn。

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Haojun Li
	Yun Lv
	Xuhui Wang
	Jieya Huang

Cite this article:

Li Haojun, Lv Yun, Wang Xuhui, Huang Jieya. A Deep Recommendation Model with Multi-Layer Interaction and Sentiment Analysis. Data Analysis and Knowledge Discovery, 2023, 7(3): 43-57.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022.0228 OR https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2023/V7/I3/43

符号	定义
$r u, i$	用户 $u$ 对资源 $i$ 的真实评分
$r^u, i$	用户 $u$ 对资源 $i$ 的预测评分
$R e u, i$	用户 $u$ 对资源 $i$ 的评论
$U L F M$	用户浅层特征
$I L F M$	资源浅层特征
$U T E X T$	用户深层特征
$I T E X T$	资源深层特征
$a u, m$	用户 $u$ 第 $m$ 条评论对应的贡献度权值
$a i, n$	资源 $i$ 第 $n$ 条评论对应的贡献度权值

Symbol Definition

Framework of DeepSAMI Model

Structure of Shallow Feature Mining of Comment Text Based on Sentiment Analysis

Structure of Deep Feature Mining of Comment Text Combining CNN and Attention Mechanism

Basic Information of the Datasets

Comparison of Baseline Models

Experimental Results of Sentiment Analysis

Comparison of Some Scoring Data

MAE of Each Model on 4 Datasets

Comparison of RMSE of Each Model on 4 Datasets

Effect of MLP Hidden Layer Depth on Model Performance

α on Model Performance
">

Effect of Equilibrium Factor $α$ on Model Performance

Experimental Results of Variant Algorithms

[1]	Du Y, Wang L, Peng Z, et al. Review-Based Hierarchical Attention Cooperative Neural Networks for Recommendation[J]. Neurocomputing, 2021, 447: 38-47. doi: 10.1016/j.neucom.2021.03.098
[2]	杨辰, 陈晓虹, 王楚涵, 等. 基于用户细粒度属性偏好聚类的推荐策略[J]. 数据分析与知识发现, 2021, 5(10): 94-102.
[2]	( Yang Chen, Chen Xiaohong, Wang Chuhan, et al. Recommendation Strategy Based on Users’ Preferences for Fine-Grained Attributes[J]. Data Analysis and Knowledge Discovery, 2021, 5(10): 94-102.)
[3]	Fan J, Jiang Y C, Liu Y Z, et al. Interpretable MOOC Recommendation: A Multi-Attention Network for Personalized Learning Behavior Analysis[J]. Internet Research, 2022, 32(2): 588-605. doi: 10.1108/INTR-08-2020-0477
[4]	Xiao W Y, Zhao H, Pan H J, et al. Social Explorative Attention Based Recommendation for Content Distribution Platforms[J]. Data Mining and Knowledge Discovery, 2021, 35(2): 533-567. doi: 10.1007/s10618-020-00729-1
[5]	Mitra S, Jenamani M. Helpfulness of Online Consumer Reviews: A Multi-Perspective Approach[J]. Information Processing & Management, 2021, 58(3): 102538. doi: 10.1016/j.ipm.2021.102538
[6]	张宜浩, 朱小飞, 徐传运, 等. 基于用户评论的深度情感分析和多视图协同融合的混合推荐方法[J]. 计算机学报, 2019, 42(6): 1316-1333.
[6]	( Zhang Yihao, Zhu Xiaofei, Xu Chuanyun, et al. Hybrid Recommendation Approach Based on Deep Sentiment Analysis of User Reviews and Multi-View Collaborative Fusion[J]. Chinese Journal of Computers, 2019, 42(6): 1316-1333.)
[7]	Sun P, Li J S, Li G H. Research on Collaborative Filtering Recommendation Algorithm Based on Sentiment Analysis and Topic Model[C]// Proceedings of the 4th International Conference on Big Data and Computing. 2019: 169-178.
[8]	Liu Y Y, Yang B, Pei H B, et al. Neural Explainable Recommender Model Based on Attributes and Reviews[J]. Journal of Computer Science and Technology, 2020, 35(6): 1446-1460. doi: 10.1007/s11390-020-0152-8
[9]	Koren Y, Bell R, Volinsky C. Matrix Factorization Techniques for Recommender Systems[J]. Computer, 2009, 42(8): 30-37.
[10]	Salakhutdinov R, Mnih A. Probabilistic Matrix Factorization[C]// Proceedings of the 20th International Conference on Neural Information Processing Systems. 2007: 1257-1264.
[11]	Liu D H, Li J, Du B, et al. A Hybrid Neural Network Approach to Combine Textual Information and Rating Information for Item Recommendation[J]. Knowledge and Information Systems, 2021, 63(3): 621-646. doi: 10.1007/s10115-020-01528-2
[12]	Jakob N, Weber S H, Müller M C, et al. Beyond the Stars: Exploiting Free-Text User Reviews to Improve the Accuracy of Movie Recommendations[C]// Proceeding of the 1st International CIKM Workshop on Topic-Sentiment Analysis for Mass Opinion. 2009: 57-64.
[13]	Wang C, Blei D M. Collaborative Topic Modeling for Recommending Scientific Articles[C]// Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2011: 448-456.
[14]	Bao Y, Fang H, Zhang J. TopicMF: Simultaneously Exploiting Ratings and Reviews for Recommendation[C]// Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014: 2-8.
[15]	Kim D, Park C, Oh J, et al. Convolutional Matrix Factorization for Document Context-Aware Recommendation[C]// Proceedings of the 10th ACM Conference on Recommender Systems. 2016: 233-240.
[16]	Zheng L, Noroozi V, Yu P S. Joint Deep Modeling of Users and Items Using Reviews for Recommendation[C]// Proceedings of the 10th ACM International Conference on Web Search and Data Mining. 2017: 425-434.
[17]	Seo S, Huang J, Yang H, et al. Interpretable Convolutional Neural Networks with Dual Local and Global Attention for Review Rating Prediction[C]// Proceedings of the 11th ACM Conference on Recommender Systems. 2017: 297-305.
[18]	Chen C, Zhang M, Liu Y Q, et al. Neural Attentional Rating Regression with Review-Level Explanations[C]// Proceedings of the 2018 World Wide Web Conference. 2018: 1583-1592.
[19]	Dong X, Ni J C, Cheng W, et al. Asymmetrical Hierarchical Networks with Attentive Interactions for Interpretable Review-Based Recommendation[C]// Proceedings of the 2020 AAAI Conference on Artificial Intelligence. 2020: 7667-7674.
[20]	Guo S, Wang Y, Yuan H, et al. TAERT: Triple-Attentional Explainable Recommendation with Temporal Convolutional Network[J]. Information Sciences, 2021, 567: 185-200. doi: 10.1016/j.ins.2021.03.034
[21]	Li W, Xu B. Aspect-Based Fashion Recommendation with Attention Mechanism[J]. IEEE Access, 2020, 8: 141814-141823. doi: 10.1109/ACCESS.2020.3013639
[22]	冯兴杰, 曾云泽. 基于评分矩阵与评论文本的深度推荐模型[J]. 计算机学报, 2020, 43(5): 884-900.
[22]	( Feng Xingjie, Zeng Yunze. Joint Deep Modeling of Rating Matrix and Reviews for Recommendation[J]. Chinese Journal of Computers, 2020, 43(5): 884-900.)
[23]	Ren Q F, Zheng Y, Guo G S, et al. Resource Recommendation Algorithm Based on Text Semantics and Sentiment Analysis[C]// Proceedings of the 3rd IEEE International Conference on Robotic Computing. 2019: 363-368.
[24]	Kim T Y, Pan S B, Kim S H. Sentiment Digitization Modeling for Recommendation System[J]. Sustainability, 2020, 12(12): 5191. doi: 10.3390/su12125191
[25]	王红霞, 陈健, 程艳芬. 采用评论挖掘修正用户评分的改进协同过滤算法[J]. 浙江大学学报(工学版), 2019, 53(3): 522-532.
[25]	( Wang Hongxia, Chen Jian, Cheng Yanfen. Improved Collaborative Filtering Algorithm to Revise Users’ Rating by Review Mining[J]. Journal of Zhejiang University(Engineering Science), 2019, 53(3): 522-532.)
[26]	Da’u A, Salim N. Sentiment-Aware Deep Recommender System with Neural Attention Networks[J]. IEEE Access, 2019, 7: 45472-45484. doi: 10.1109/ACCESS.2019.2907729
[27]	Jiang L, Liu L, Yao J J, et al. A Hybrid Recommendation Model in Social Media Based on Deep Emotion Analysis and Multi-Source View Fusion[J]. Journal of Cloud Computing, 2020, 9: Article No.57.
[28]	Zhang Y, Liu Z, Sang C. Unifying Paragraph Embeddings and Neural Collaborative Filtering for Hybrid Recommendation[J]. Applied Soft Computing, 2021, 106: 107345. doi: 10.1016/j.asoc.2021.107345
[29]	Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2019: 4171-4186.
[30]	Zhang X F, Yan K P. An Algorithm of Bidirectional RNN for Offline Handwritten Chinese Text Recognition[C]// Proceedings of the 15th International Conference on Intelligent Computing. 2019: 423-431.
[31]	Zhang Y, Liu Z, Sang C. Unifying Paragraph Embeddings and Neural Collaborative Filtering for Hybrid Recommendation[J]. Applied Soft Computing, 2021, 106: 107345. doi: 10.1016/j.asoc.2021.107345
[32]	Conneau A, Schwenk H, Barrault L, et al. Very Deep Convolutional Networks for Text Classification[C]// Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. 2017: 1107-1116.
[33]	Zhang X, Zhao J, LeCun Y. Character-Level Convolutional Networks for Text Classification[OL]. arXiv Preprint, arXiv: 1509.01626.
[34]	Hu B, Lu Z, Li H, et al. Convolutional Neural Network Architectures for Matching Natural Language Sentences[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014: 2042-2050.
[35]	Al-Sabahi K, Zhang Z P, Nadher M. A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS)[J]. IEEE Access, 2018, 6: 24205-24212. doi: 10.1109/ACCESS.2018.2829199
[36]	Wang X, He X N, Nie L Q, et al. Item Silk Road: Recommending Items from Information Domains to Social Users[C]// Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017: 185-194.
[37]	He X N, Liao L Z, Zhang H W, et al. Neural Collaborative Filtering[C]// Proceedings of the 26th International Conference on World Wide Web. 2017: 173-182.
[38]	He X N, Chua T S. Neural Factorization Machines for Sparse Predictive Analytics[C]// Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017: 355-364.
[39]	Li P J, Wang Z H, Ren Z C, et al. Neural Rating Regression with Abstractive Tips Generation for Recommendation[C]// Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017: 345-354.
[40]	Cai Y, Ke W, Cui E, et al. A Deep Recommendation Model of Cross-Grained Sentiments of User Reviews and Ratings[J]. Information Processing & Management, 2022, 59(2): 102842. doi: 10.1016/j.ipm.2021.102842
[41]	Du Y, Wang L, Peng Z, et al. Review-Based Hierarchical Attention Cooperative Neural Networks for Recommendation[J]. Neurocomputing, 2021, 447: 38-47. doi: 10.1016/j.neucom.2021.03.098
[42]	Koren Y. Factorization Meets the Neighborhood: A Multifaceted Collaborative Filtering Model[C]// Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2008: 426-434.
[43]	McAuley J, Leskovec J. Hidden Factors and Hidden Topics: Understanding Rating Dimensions with Review Text[C]// Proceedings of the 7th ACM Conference on Recommender Systems. 2013: 165-172.
[44]	Sun P, Li J S, Li G H. Research on Collaborative Filtering Recommendation Algorithm Based on Sentiment Analysis and Topic Model[C]// Proceedings of the 4th International Conference on Big Data and Computing. 2019: 169-178.

[1]	Zhou Ning, Zhong Na, Jin Gaoya, Liu Bin. Chinese Text Sentiment Analysis Based on Dual Channel Attention Network with Hybrid Word Embedding[J]. 数据分析与知识发现, 2023, 7(3): 58-68.
[2]	Shen Lining, Yang Jiayi, Pei Jiaxuan, Cao Guang, Chen Gongzheng. A Fine-Grained Sentiment Recognition Method Based on OCC Model and Triggering Events[J]. 数据分析与知识发现, 2023, 7(2): 72-85.
[3]	Wang Hao, Gong Lijuan, Zhou Zeyu, Fan Tao, Wang Yongsheng. Detecting Mis/Dis-information from Social Media with Semantic Enhancement[J]. 数据分析与知识发现, 2023, 7(2): 48-60.
[4]	Xu Yuemei, Cao Han, Wang Wenqing, Du Wanze, Xu Chengyang. Cross-Lingual Sentiment Analysis: A Survey[J]. 数据分析与知识发现, 2023, 7(1): 1-21.
[5]	Xiao Yuhan, Lin Huiping. Mining Differentiated Demands with Aspect Word Extraction: Case Study of Smartphone Reviews[J]. 数据分析与知识发现, 2023, 7(1): 63-75.
[6]	Xiao Hanqiong, Zhang Xinyu, Xiao Yuhan, Lin Huiping. Creating Consumer Psychology Portrait with Aspect Words[J]. 数据分析与知识发现, 2022, 6(6): 22-31.
[7]	Shang Rongxuan, Zhang Bin, Mi Jianing. End-to-End Aspect-Level Sentiment Analysis for E-Government Applications Based on BRNN[J]. 数据分析与知识发现, 2022, 6(2/3): 364-375.
[8]	Feng Xiaodong, Hui Kangxin. Topic Clustering for Social Media Texts with Heterogeneous Graph Neural Networks[J]. 数据分析与知识发现, 2022, 6(10): 9-19.
[9]	Sun Yu, Qiu Jiangnan. Studying Opinion Leaders with Network Analysis and Text Mining[J]. 数据分析与知识发现, 2022, 6(1): 69-79.
[10]	Xu Yuemei, Wang Zihou, Wu Zixin. Predicting Stock Trends with CNN-BiLSTM Based Multi-Feature Integration Model[J]. 数据分析与知识发现, 2021, 5(7): 126-138.
[11]	Zhong Jiawa,Liu Wei,Wang Sili,Yang Heng. Review of Methods and Applications of Text Sentiment Analysis[J]. 数据分析与知识发现, 2021, 5(6): 1-13.
[12]	Liu Tong,Liu Chen,Ni Weijian. A Semi-Supervised Sentiment Analysis Method for Chinese Based on Multi-Level Data Augmentation[J]. 数据分析与知识发现, 2021, 5(5): 51-58.
[13]	Wang Yuzhu,Xie Jun,Chen Bo,Xu Xinying. Multi-modal Sentiment Analysis Based on Cross-modal Context-aware Attention[J]. 数据分析与知识发现, 2021, 5(4): 49-59.
[14]	Li Feifei,Wu Fan,Wang Zhongqing. Sentiment Analysis with Reviewer Types and Generative Adversarial Network[J]. 数据分析与知识发现, 2021, 5(4): 72-79.
[15]	Chang Chengyang,Wang Xiaodong,Zhang Shenglei. Polarity Analysis of Dynamic Political Sentiments from Tweets with Deep Learning Method[J]. 数据分析与知识发现, 2021, 5(3): 121-131.

Viewed

Full text

Abstract

Cited

Shared

Discussed