Please wait a minute...
Data Analysis and Knowledge Discovery  2021, Vol. 5 Issue (2): 94-105    DOI: 10.11925/infotech.2096-3467.2020.0521
Current Issue | Archive | Adv Search |
Music Recommendation Method Based on Multi-Source Information Fusion
Li Danyang,Gan Mingxin()
School of Economics and Management, University of Science and Technology Beijing, Beijing 100083, China
Download: PDF (1639 KB)   HTML ( 16
Export: BibTeX | EndNote (RIS)      

[Objective] This paper creates a musical feature system based on multi-source information, aiming to address the cold start issue facing music recommendation and provide personalized services. [Methods] We proposed a two-stage model with multi-source information fused by neural network algorithm. Then, we built the musical feature system and predicted the potential factor vectors of music. Finally, we generated the TopN recommendation list for the users. [Results] We examined our model with the Million Song Dataset. Compared with other models such as CNN, the F1 value was improved by 9.13%, and the RMSE, MAE values were reduced by 8.08% and 3.91%, respectively. [Limitations] Our new method encounters more limits than the end-to-end training ones. And training with the Mel-frequency spectrum demands much more memory. [Conclusions] The proposed model improves the performance of music recommendation services.

Key wordsMusic Recommendation      Deep Learning      Information Fusion      Neural Network     
Received: 04 June 2020      Published: 11 March 2021
ZTFLH:  G350  
Fund:National Natural Science Foundation of China(71871019);National Natural Science Foundation of China(71471016);National Natural Science Foundation of China(71531013)
Corresponding Authors: Gan Mingxin ORCID:0000-0001-8751-0780     E-mail:

Cite this article:

Li Danyang, Gan Mingxin. Music Recommendation Method Based on Multi-Source Information Fusion. Data Analysis and Knowledge Discovery, 2021, 5(2): 94-105.

URL:     OR

General Architecture of Music Recommendation Method
The Architecture of Neural Network Model
Flow Chart of Audio Data Processing
The Architecture of LSTM Unit
Track_id Title Song_id Release
TRACCVZ128F4291A8A Deep Sea Creature SOVLGJY12A8C13FBED Call of the Mastodon
TRACCMH128F428E4CD No Quieras Marcharte SOGDQZK12A8C13F37C Adelante
TRACCJA128F149A144 Segredo SODPNJR12A6D4FA52D Joao Voz E Violato
TRACCUS128F92E1FEB Bedroom Acoustics SOMMSMW12A8C13FCCC Plug In Baby
TRACJDX12903CD4917 Church Hangover SOSJVGP12AB0185B95 El Mas Chingon
Artist_id Artist_mbid Artist_name Year
ARMQHX71187B9890D3 bc5e2ad6-0a4a-4d90-b911-e9a7e6861727 Mastodon 2001
AR2PT4M1187FB55B1A d54ea4a6-0b9c-4e47-bed0-289ae9ff4037 Los Chichos 1984
AR3THYK1187B999F1F 286ec4c2-b5ca-4f85-b331-280a6d73dd14 Jo?o Gilberto 2000
ARR3ONV1187B9A2F59 fd857293-5ab8-40de-b29e-55a69d4e4d0f Muse 0
ARYF20K1187B9B76BD 4a41c200-153b-4158-87b3-db0fea627e2c George Lopez 2006
Metadata Information of Musics
User_id Song_id Play_num
d3f65887d5eb4eac8ca14d0f3085c5021e79a8c5 SOAWFKO129F06933A7 2
4fd90e488869fffe6e2186a489311bdb19ebce46 SOUGUKH12A8C13FBB0 22
8e62f55bc5830dfcbe92339afd86fda1d7fccb5e SOOQUVZ12A6D4F7536 3
747155a884e6c96299a1a052e89ead19f31ad235 SONDICG12AB0184D30 6
3446fc0b811c4970eafe30866f873e44567d371c SOODZFJ12A6D4F8D03 1
Listening Records of Users
Artist_id Tag
AR002UA1187B9A637D garage rock
AR002UA1187B9A637D free jazz
AR002UA1187B9A637D pop
AR003FB1187B994355 rock
AR003FB1187B994355 hip hop
Labels of Singers
CNN 1.0 156 0.8 734 0.4 580 0.1 099
Year 1.0 129 0.8 727 0.4 599 0.1 098
CNN+LSTM 0.9 668 0.9 450 0.4 833 0.1 073
Category 0.9 648 0.8 580 0.4 548 0.1 035
Artist 0.8 821 0.8 232 0.4 457 0.0 921
本文方法 0.7 942 0.8 028 0.4 401 0.0 894
Comparisons on Errors of Different Models
Results of Precision
Results of Recall
Results of F1 Value
模型 准确率 召回率 F1
N=3 N=4 N=3 N=4 N=3 N=4
CNN+LSTM 0.6 075 0.5 543 0.7 160 0.8 229 0.5 992 0.6 065
CNN 0.6 104 0.5 552 0.7 237 0.8 307 0.6 038 0.6 094
Year 0.6 269 0.5 612 0.7 517 0.8 480 0.6 230 0.6 176
Category 0.6 445 0.5 739 0.7 746 0.8 661 0.6 426 0.6 318
Artist 0.6 595 0.5 867 0.7 724 0.8 665 0.6 491 0.6 397
本文方法 0.6 682 0.5 888 0.7 843 0.8 711 0.6 589 0.6 431
Comparisons on Evaluation Indexes of Recommendation
Three-Dimensional Music Cluster
Visualized Results of Potential Factor Vectors
[1] Zheng E, Kondo G Y, Zilora S, et al. Tag-aware Dynamic Music Recommendation[J]. Expert Systems with Applications, 2018,106:244-251.
doi: 10.1016/j.eswa.2018.04.014
[2] Hu Y F, Koren Y, Volinsky C. Collaborative Filtering for Implicit Feedback Datasets[C]//Proceedings of the 8th IEEE International Conference on Data Mining. 2008: 263-272.
[3] van den Oord A, Dieleman S, Schrauwen B. Deep Content-based Music Recommendation[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013: 2643-2651.
[4] Bertin-Mahieux T, Ellis P W D, Whitman B, et al. The Million Song Dataset[C]//Proceedings of the 12th International Society for Music Information Retrieval Conference. 2011: 591-596.
[5] 牛滨, 孔令志, 罗森林, 等. 基于MFCC和GMM的个性音乐推荐模型[J]. 北京理工大学学报, 2009,29(4):351-355.
[5] ( Niu Bin, Kong Lingzhi, Luo Senlin, et al. Individuality Music Recommendation Model Based on MFCC and GMM[J]. Transactions of Beijing Institute of Technology, 2009,29(4):351-355.)
[6] Liu C L, Chen Y C. Background Music Recommendation Based on Latent Factors and Moods[J]. Knowledge-Based Systems, 2018,159:158-170.
doi: 10.1016/j.knosys.2018.07.001
[7] Li T, Choi M, Fu K M, et al. Music Sequence Prediction with Mixture Hidden Markov Models[C]//Proceedings of 2019 IEEE International Conference on Big Data. Los Angeles, CA, USA, 2018. DOI: 10.1109/BigData47090.2019.9005695.
[8] Flexer A, Stevens J. Mutual Proximity Graphs for Improved Reachability in Music Recommendation[J]. Journal of New Music Research, 2018,47(1):17-28.
doi: 10.1080/09298215.2017.1354891 pmid: 29348779
[9] McFee B, Barrington L, Lanckriet G. Learning Content Similarity for Music Recommendation[J]. IEEE Transactions on Audio, Speech, and Language Processing, 2012,20(8):2207-2218.
doi: 10.1109/TASL.2012.2199109
[10] Lecun Y, Bottou L, Bengio Y, et al. Gradient-based Learning Applied to Document Recognition[J]. Proceedings of the IEEE, 1998,86(11):2278-2324.
doi: 10.1109/5.726791
[11] Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012: 1097-1105.
[12] Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[C]//Proceedings of International Conference on Learning Representations, 2015.
[13] Lee J, Lee K, Park J, et al. Deep Content-User Embedding Model for Music Recommendation[OL]. arXiv Preprint, arXiv: 1807. 06786.
[14] Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780.
doi: 10.1162/neco.1997.9.8.1735 pmid: 9377276
[15] Chung J, Gulcehre C, Cho K, et al. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling [OL]. arXiv Preprint, arXiv: 1412. 3555.
[16] Balakrishnan A, Dixit K. DeepPlaylist: Using Recurrent Neural Networks to Predict Song Similarity [EB/OL]. [2019-05-16].
[17] Sachdeva N, Gupta K, Pudi V. Attentive Neural Architecture Incorporating Song Features for Music Recommendation[C]//Proceedings of the 12th ACM Conference on Recommender Systems. 2018: 417-421.
[18] Wang D J, Deng S G, Xu G D. Sequence-Based Context-Aware Music Recommendation[J]. Information Retrieval, 2018,21(2/3):230-252.
doi: 10.1007/s10791-017-9317-7
[19] Zou W. Design and Application of Incremental Music Recommendation System Based on Slope One Algorithm[J]. Wireless Personal Communications, 2018,102(4):2785-2795.
doi: 10.1007/s11277-018-5303-7
[20] Deng S G, Wang D J, Li X T, et al. Exploring User Emotion in Microblogs for Music Recommendation[J]. Expert Systems with Applications, 2015,42(23):9284-9293.
doi: 10.1016/j.eswa.2015.08.029
[21] Ren J, Kauffman R, King D. Two-Sided Value-Based Music Artist Recommendation in Streaming Music Services[C]//Proceedings of the 52nd Hawaii International Conference on System Sciences. 2019: 2679-2688.
[22] Chen J P, Ying P G, Zou M. Improving Music Recommendation by Incorporating Social Influence[J]. Multimedia Tools and Applications, 2019,78(3):2667-2687.
doi: 10.1007/s11042-018-5745-7
[23] Cheng Z Y, Shen J L. On Effective Location-Aware Music Recommendation[J]. ACM Transactions on Information Systems, 2016, 34(2): Article 13.
[24] 李洋, 赵鸣, 徐梦瑶, 等. 多源信息融合技术研究综述[J]. 智能计算机与应用, 2019,9(5):186-189.
[24] ( Li Yang, Zhao Ming, Xu Mengyao, et al. A Survey of Research on Multi-Source Information Fusion Technology[J]. Intelligent Computer and Applications, 2019,9(5):186-189.)
[25] Mikolov T, Chen K, Corrado G, et al. Efficient Estimation of Word Representations in Vector Space[C]//Proceedings of Workshop at International Conference on Learning Representations, 2013.
[1] Huang Lu,Zhou Enguo,Li Daifeng. Text Representation Learning Model Based on Attention Mechanism with Task-specific Information[J]. 数据分析与知识发现, 2020, 4(9): 111-122.
[2] Yin Haoran,Cao Jinxuan,Cao Luzhe,Wang Guodong. Identifying Emergency Elements Based on BiGRU-AM Model with Extended Semantic Dimension[J]. 数据分析与知识发现, 2020, 4(9): 91-99.
[3] Yu Chuanming, Wang Manyi, Lin Hongjun, Zhu Xingyu, Huang Tingting, An Lu. A Comparative Study of Word Representation Models Based on Deep Learning[J]. 数据分析与知识发现, 2020, 4(8): 28-40.
[4] Xu Chenfei, Ye Haiying, Bao Ping. Automatic Recognition of Produce Entities from Local Chronicles with Deep Learning[J]. 数据分析与知识发现, 2020, 4(8): 86-97.
[5] Zhao Yang, Zhang Zhixiong, Liu Huan, Ding Liangping. Classification of Chinese Medical Literature with BERT Model[J]. 数据分析与知识发现, 2020, 4(8): 41-49.
[6] Qiu Erli,He Hongwei,Yi Chengqi,Li Huiying. Research on Public Policy Support Based on Character-level CNN Technology[J]. 数据分析与知识发现, 2020, 4(7): 28-37.
[7] Wang Xinyun,Wang Hao,Deng Sanhong,Zhang Baolong. Classification of Academic Papers for Periodical Selection[J]. 数据分析与知识发现, 2020, 4(7): 96-109.
[8] Wang Mo,Cui Yunpeng,Chen Li,Li Huan. A Deep Learning-based Method of Argumentative Zoning for Research Articles[J]. 数据分析与知识发现, 2020, 4(6): 60-68.
[9] Jiao Qihang,Le Xiaoqiu. Generating Sentences of Contrast Relationship[J]. 数据分析与知识发现, 2020, 4(6): 43-50.
[10] Liu Weijiang,Wei Hai,Yun Tianhe. Evaluation Model for Customer Credits Based on Convolutional Neural Network[J]. 数据分析与知识发现, 2020, 4(6): 80-90.
[11] Deng Siyi,Le Xiaoqiu. Coreference Resolution Based on Dynamic Semantic Attention[J]. 数据分析与知识发现, 2020, 4(5): 46-53.
[12] Yu Chuanming,Yuan Sai,Zhu Xingyu,Lin Hongjun,Zhang Puliang,An Lu. Research on Deep Learning Based Topic Representation of Hot Events[J]. 数据分析与知识发现, 2020, 4(4): 1-14.
[13] Yan Chun,Liu Lu. Classifying Non-life Insurance Customers Based on Improved SOM and RFM Models[J]. 数据分析与知识发现, 2020, 4(4): 83-90.
[14] Su Chuandong,Huang Xiaoxi,Wang Rongbo,Chen Zhiqun,Mao Junyu,Zhu Jiaying,Pan Yuhao. Identifying Chinese / English Metaphors with Word Embedding and Recurrent Neural Network[J]. 数据分析与知识发现, 2020, 4(4): 91-99.
[15] Liu Tong,Ni Weijian,Sun Yujian,Zeng Qingtian. Predicting Remaining Business Time with Deep Transfer Learning[J]. 数据分析与知识发现, 2020, 4(2/3): 134-142.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938