Please wait a minute...
Data Analysis and Knowledge Discovery  2022, Vol. 6 Issue (6): 46-54    DOI: 10.11925/infotech.2096-3467.2021.1105
Current Issue | Archive | Adv Search |
Session Sequence Recommendation with GNN, Bi-GRU and Attention Mechanism
Zhang Ruoqi,Shen Jianfang(),Chen Pinghua
School of Computers, Guangdong University of Technology, Guangzhou 510006, China
Download: PDF (1602 KB)   HTML ( 19 ) 
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This study aims to improve the traditional session sequence recommendation algorithm, whose one-time modeling technique cannot represent the product’s comprehensive information or capture the user’s global/short-term interests. [Methods] First, we constructed a directed session graph based on the historical sessions, and used the GNN to learn their node information representation to enrich node embeddings. Then, we captured the user’s global and short-term interests in session sequences with Bi-GRU and attention mechanism to generate recommendation lists. [Results] We examined our new algorithm with the Yoochoose and the Diginetica datasets. Compared with the suboptimal model, the Mean Reciprocal Rank of our algorithm improved by 1.02%, and the precision improved by 2.11%, respectively. [Limitations] The proposed model did not work well with the long sequences. [Conclusions] Our new algorithm can more effectively model the user behavior sequence, predict the user’s possible actions, and improve the recommendation lists.
Key words： Sequential Recommendation      Session      GNN      Bi-GRU      Attention Mechanism     
Received: 27 September 2021      Published: 28 July 2022
ZTFLH:  TP391  
Fund:Guangdong Key Area R&D Program(2020B0101100001);Guangdong Key Area R&D Program(2021B0101200002);Science and Technology Planning Project of Guangdong Province(2020B1010010010)
Corresponding Authors: Shen Jianfang     E-mail: tysjf@gdut.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Ruoqi Zhang
Jianfang Shen
Pinghua Chen

Cite this article:

Zhang Ruoqi, Shen Jianfang, Chen Pinghua. Session Sequence Recommendation with GNN, Bi-GRU and Attention Mechanism. Data Analysis and Knowledge Discovery, 2022, 6(6): 46-54.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2021.1105     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2022/V6/I6/46

Model Architecture
Directed Session Graph
Adjacency Matrix
Bidirectional Gated Recurrent Unit
Gated Recurrent Unit
数据 Yoochoose Diginetica
点击数 557 248 982 961
项目数 16 766 43 097
平均长度 6.16 5.12
Statistics of Datasets
方法

评价指标		 Yoochoose Diginetica
P @ 20 M R R @ 20 P @ 20 M R R @ 20
Item-KNN 51.60 21.81 28.35 9.45
FPMC 45.62 15.01 31.55 8.92
GRU4Rec 60.64 22.89 43.82 15.46
STAMP 68.74 29.67 45.64 14.32
SRGNN 69.27 29.67 49.04 16.53
GNN-DBLGRU-TA 70.23 30.31 50.13 16.63
GNN-BiGRU-TA 70.54 30.69 51.15 17.34
Model Comparison on Yoochoose and Diginetica
属性

数据集		 Diginetica 1 Diginetica 3 Diginatica 5 Diginatica 7
训练集
会话数		 612 858 573 133 455 168 333 119
测试集
会话数		 51 751 48 432 48 432 27 978
项目数 39 056 37 096 31 987 26 336
序列平均
长度		 4.995 3 5.890 2 7.322 3 8.647 4
Different Length Sequence Attributes of Diginetica
The Effect of Sequence Length on Model Performance
Model Comparison
[1] Scarselli F, Gori M, Tsoi A C, et al. The Graph Neural Network Model[J]. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80.
doi: 10.1109/TNN.2008.2005605 pmid: 19068426
[2] Yap G E, Li X L, Yu P S. Effective Next-Items Recommendation via Personalized Sequential Pattern Mining[C]// Proceedings of the 2012 Database Systems for Advanced Applications, 2012: 48-64.
[3] Garcin F, Dimitrakakis C, Faltings B. Personalized News Recommendation with Context Trees[C]// Proceedings of the 7th ACM Conference on Recommender Systems. ACM, 2013: 105-112.
[4] Feng S S, Li X T, Zeng Y F, et al. Personalized Ranking Metric Embedding for Next New POI Recommendation[C]// Proceedings of the 24th International Joint Conference on Artificial Intelligence. AAAI Press, 2015: 2069-2075.
[5] Rendle S, Freudenthaler C, Schmidt-Thieme L. Factorizing Personalized Markov Chains for Next-Basket Recommendation[C]// Proceedings of the 19th International Conference on World Wide Web. ACM Press, 2010: 811-820.
[6] Wang P F, Guo J F, Lan Y Y, et al. Learning Hierarchical Representation Model for Next Basket Recommendation[C]// Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 2015: 403-412.
[7] Wang S J, Hu L, Cao L B. Attention Based Transactional Context Embedding for Next-Item Recommendation[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. AAAI Press, 2018: 2532-2539.
[8] He R N, Kang W C, McAuley J. Translation-Based Recommendation: A Scalable Method for Modeling Sequential Behavior[C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018: 5264-5268.
[9] Wu C Y, Ahmed A, Beutel A, et al. Recurrent Recommender Networks[C]// Proceedings of the 10th ACM International Conference on Web Search and Data Mining. ACM, 2017: 495-503.
[10] Hidasi B, Karatzoglou A, Baltrunas L, et al. Session-Based Recommendations with Recurrent Neural Networks[C]// Proceedings of the 4th International Conference on Learning Representations. ICLR, 2016:1-10.
[11] Tang J X, Wang K. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding[C]// Proceedings of the 11th ACM International Conference on Web Search and Data Mining. ACM, 2018: 565-573.
[12] Yuan F J, Karatzoglou A, Arapakis I, et al. A Simple Convolutional Generative Network for Next Item Recommendation[C]// Proceedings of the 12th ACM International Conference on Web Search and Data Mining. ACM, 2019: 582-590.
[13] 倪维健, 郭浩宇, 刘彤, 等. 基于多头自注意力神经网络的购物篮推荐方法[J]. 数据分析与知识发现, 2020, 4(2/3): 68-77.
[13] (Ni Weijian, Guo Haoyu, Liu Tong, et al. Online Product Recommendation Based on Multi-Head Self-Attention Neural Networks[J]. Data Analysis and Knowledge Discovery, 2020, 4(2/3): 68-77.)
[14] Zhou G R, Song C R, Zhu X Q, et al. Deep Interest Network for Click-through Rate Prediction[OL]. arXiv Preprint, arXiv: 1706.06978.
[15] Zhou C, Bai J Z, Song J S, et al. ATRANK: An Attention-Based User Behavior Modeling Framework for Recommendation[C]// Proceedings of the 32nd Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence. AAAI Press, 2018: 4564-4571.
[16] Zhou G R, Mou N, Fan Y, et al. Deep Interest Evolution Network for Click-through Rate Prediction[C]// Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33: 5941-5948.
[17] Lv F Y, Jin T W, Yu C L, et al. SDM: Sequential Deep Matching Model for Online Large-Scale Recommender System[C]// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. ACM, 2019: 2635-2643.
[18] Ma C, Ma L H, Zhang Y X. Memory Augmented Graph Neural Networks for Sequential Recommendation[C]// Proceedings of the 34th Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence. AAAI Press, 2020: 5045-5052.
[19] Wu S, Tang Y Y, Zhu Y Q, et al. Session-Based Recommendation with Graph Neural Networks[C]// Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33: 346-353.
[20] Zhou G R, Song C R, Zhu X Q, et al. Deep Interest Network for Click-Through Rate Prediction[OL]. arXiv Preprint, arXiv: 1706.06978.
[21] Wang Z Y, Wei W, Cong G, et al. Global Context Enhanced Graph Neural Networks for Session-Based Recommendation[C]// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020: 169-178.
[22] Xu C F, Zhao P P, Liu Y C, et al. Graph Contextualized Self-Attention Network for Session-Based Recommendation[C]// Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019: 3940-3946.
[23] Chung J, Gulcehre C, Cho K, et al. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling[OL]. arXiv Preprint, arXiv: 1412.3555.
[24] Liu Q, Zeng Y F, Mokhosi R, et al. STAMP: Short-Term Attention/Memory Priority Model for Session-Based Recommendation[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2018: 1831-1839.
[25] Sarwar B, Karypis G, Konstan J, et al. Item-Based Collaborative Filtering Recommendation Algorithms[C]// Proceedings of the 10th International Conference on World Wide Web. ACM Press, 2001: 285-295.
[1] Ye Han,Sun Haichun,Li Xin,Jiao Kainan. Classification Model for Long Texts with Attention Mechanism and Sentence Vector Compression[J]. 数据分析与知识发现, 2022, 6(6): 84-94.
[2] Wang Yongsheng, Wang Hao, Yu Wei, Zhou Zeyu. Extracting Relationship Among Characters from Local Chronicles with Text Structures and Contents[J]. 数据分析与知识发现, 2022, 6(2/3): 318-328.
[3] Guo Hangcheng, He Yanqing, Lan Tian, Wu Zhenfeng, Dong Cheng. Identifying Moves from Scientific Abstracts Based on Paragraph-BERT-CRF[J]. 数据分析与知识发现, 2022, 6(2/3): 298-307.
[4] Xu Yuemei, Fan Zuwei, Cao Han. A Multi-Task Text Classification Model Based on Label Embedding of Attention Mechanism[J]. 数据分析与知识发现, 2022, 6(2/3): 105-116.
[5] Yang Hanxun, Zhou Dequn, Ma Jing, Luo Yongcong. Detecting Rumors with Uncertain Loss and Task-level Attention Mechanism[J]. 数据分析与知识发现, 2021, 5(7): 101-110.
[6] Xie Hao,Mao Jin,Li Gang. Sentiment Classification of Image-Text Information with Multi-Layer Semantic Fusion[J]. 数据分析与知识发现, 2021, 5(6): 103-114.
[7] Yin Pengbo,Pan Weimin,Zhang Haijun,Chen Degang. Identifying Clickbait with BERT-BiGA Model[J]. 数据分析与知识发现, 2021, 5(6): 126-134.
[8] Han Pu,Zhang Zhanpeng,Zhang Mingtao,Gu Liang. Normalizing Chinese Disease Names with Multi-feature Fusion[J]. 数据分析与知识发现, 2021, 5(5): 83-94.
[9] Duan Jianyong,Wei Xiaopeng,Wang Hao. A Multi-Perspective Co-Matching Model for Machine Reading Comprehension[J]. 数据分析与知识发现, 2021, 5(4): 134-141.
[10] Wang Yuzhu,Xie Jun,Chen Bo,Xu Xinying. Multi-modal Sentiment Analysis Based on Cross-modal Context-aware Attention[J]. 数据分析与知识发现, 2021, 5(4): 49-59.
[11] Yu Bengong, Zhang Shuwen. Aspect-Level Sentiment Analysis Based on BAGCNN[J]. 数据分析与知识发现, 2021, 5(12): 37-47.
[12] Zhou Wenyuan, Wang Mingyang, Jing Yu. Automatic Classification of Citation Sentiment and Purposes with AttentionSBGMC Model[J]. 数据分析与知识发现, 2021, 5(12): 48-59.
[13] Jiang Cuiqing,Wang Xiangxiang,Wang Zhao. Forecasting Car Sales Based on Consumer Attention[J]. 数据分析与知识发现, 2021, 5(1): 128-139.
[14] Huang Lu,Zhou Enguo,Li Daifeng. Text Representation Learning Model Based on Attention Mechanism with Task-specific Information[J]. 数据分析与知识发现, 2020, 4(9): 111-122.
[15] Yin Haoran,Cao Jinxuan,Cao Luzhe,Wang Guodong. Identifying Emergency Elements Based on BiGRU-AM Model with Extended Semantic Dimension[J]. 数据分析与知识发现, 2020, 4(9): 91-99.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn