|
|
Recognition of Emotions and Analysis of Emotional Changes in Chinese Folk Songs |
Zhao Meng1,2,Wang Hao1,2(),Li Xiaomin1,2 |
1School of Information Management, Nanjing University, Nanjing 210023,China 2Jiangsu Key Laboratory of Data Engineering and Knowledge Service, Nanjing 210023,China |
|
|
Abstract [Objective] This paper aims to achieve the automatic recognition of rich emotions in Chinese folk songs and to explore their emotional context and fluctuation patterns digitally. [Methods] We adopted Hevner's emotion model in the field of music and introduced external Chinese knowledge for the semantic enhancement of emotion words. The automatic mapping of artificially labelled tags is then realized by semantic distance calculation. We constructed a multimodal multitag emotion recognition model (MMERM) that fuses features of lyrics and audios for automatic emotion recognition. The model is also transferred to recognize changes of emotions in songs, based on which statistical analysis and visualization of emotional context and fluctuation patterns can be conducted. [Results] The semantic enhancement and mapping effectively improve the concentration and differentiation of tags in emotion recognition. MMERM performs well on both complete songs and fragments, with a precision of 82.29%. Regularity analysis indicates a changing trend of lightness to sadness and sacredness from the beginning to the end of the songs. Furthermore, the fluctuation pattern of Chinese folk songs is found to differ remarkably from that of Western music. [Limitations] The information of folk songs is insufficient, and emotional characteristics under different temporal and spatial conditions are not analyzed. [Conclusions] This paper provides a new paradigm for the research of traditional music from the perspective of digital humanities.
|
Received: 03 July 2022
Published: 07 September 2023
|
|
Fund:National Natural Science Foundation of China(72074108);Fundamental Research Funds for the Central Universities(010814370113) |
Corresponding Authors:
Wang Hao,ORCID:0000-0002-0131-0823,E-mail: ywhaowang@nju.edu.cn。
|
[1] |
张彤彤. “十四五”时期中华优秀传统民歌的发展路径探究[J]. 黄河之声, 2022(2): 27-29.
|
[1] |
(Zhang Tongtong. Exploring the Development Path of Excellent Chinese Traditional Folk Songs in the 14th Five-Year Plan Period[J]. Song of the Yellow River, 2022(2): 27-29.)
|
[2] |
章玺. 神圣与世俗:从情感表达看壮族民歌抒情空间的嬗变[J]. 歌海, 2020(6): 76-80.
|
[2] |
(Zhang Xi. Sacred and Secular: The Transmutation of the Lyrical Space of Zhuang Folk Songs from the Perspective of Emotional Expression[J]. Songs Bimonthly, 2020(6):76-80.)
|
[3] |
郝婷婷. 陕北民歌《兰花花》的风格探析[D]. 西安: 西安音乐学院, 2018.
|
[3] |
(Hao Tingting. Investigation and Analysis on the Musical Style of the Classical Folk Song: The Orchid Flower[D]. Xi'an: Xi'an Conservatory of Music, 2018.)
|
[4] |
王海龙, 柳林, 纪文璐, 等. 蒙古族音乐资源数据库的建设[J]. 陕西师范大学学报(自然科学版), 2021, 49(5): 109-116.
|
[4] |
(Wang Hailong, Liu Lin, Ji Wenlu, et al. The Construction of Mongolian Music Resource Database[J]. Journal of Shaanxi Normal University (Natural Science Edition), 2021, 49(5): 109-116.)
|
[5] |
康健, 王海龙, 苏贵斌, 等. 音乐情感识别研究综述[J]. 计算机工程与应用, 2022, 58(4): 64-72.
doi: 10.3778/j.issn.1002-8331.2108-0538
|
[5] |
(Kang Jian, Wang Hailong, Su Guibin, et al. Survey of Music Emotion Recognition[J]. Computer Engineering and Applications, 2022, 58(4): 64-72.)
doi: 10.3778/j.issn.1002-8331.2108-0538
|
[6] |
Han D, Kong Y, Han J, et al. A Survey of Music Emotion Recognition[J]. Frontiers of Computer Science, 2022, 16(6): Article No. 166335.
|
[7] |
颜程祎. 基于证据理论和语义细胞模型的多标签音乐情感识别研究[D]. 杭州: 浙江大学, 2012.
|
[7] |
(Yan Chengyi. Research on Multi-Label Music Emotion Based on Evidence Theory and Information Cell Model[D]. Hangzhou: Zhejiang University, 2012.)
|
[8] |
Hevner K. Experimental Studies of the Elements of Expression in Music[J]. The American Journal of Psychology, 1936, 48(2): 246-268.
doi: 10.2307/1415746
|
[9] |
刘涛. 音乐情感认知模型与交互技术研究[D]. 杭州: 浙江大学, 2006.
|
[9] |
(Liu Tao. Research on Music Emotion Recognition Model and Interactive Technology[D]. Hangzhou: Zhejiang University, 2006.)
|
[10] |
蒋盛益, 阳垚, 廖静欣. 中文音乐情感词典构建及情感分类方法研究[J]. 计算机工程与应用, 2014, 50(24): 118-121.
|
[10] |
(Jiang Shengyi, Yang Yao, Liao Jingxin. Research of Building Chinese Musical Emotional Lexicon and Emotional Classification[J]. Computer Engineering and Applications, 2014, 50(24): 118-121.)
|
[11] |
邢白夕, 谢磷海, 朱伯翰, 等. 基于情感特征的中华诗词-民乐跨媒体联觉匹配的研究[J]. 复旦学报(自然科学版), 2020, 59(5): 558-564.
|
[11] |
(Xing Baixi, Xie Linhai, Zhu Bohan, et al. Research on the Cross-Media Synesthesia Matching of Chinese Poetry and Folk Music Based on Emotional Characteristics[J]. Journal of Fudan University (Natural Science), 2020, 59(5): 558-564.)
|
[12] |
郭纪莹, 刘京宇, 蒋玉暕. 音乐声的音色感知特征与情感的关联研究[J]. 复旦学报(自然科学版), 2021, 60(3): 347-352.
|
[12] |
(Guo Jiying, Liu Jingyu, Jiang Yujian. A Study on the Relationship Between Timbre Perception Features and Emotions in Musical Sounds[J]. Journal of Fudan University (Natural Science), 2021, 60(3): 347-352.)
|
[13] |
Han E, Cha H. Audio Feature Extraction for Effective Emotion Classification[J]. IEIE Transactions on Smart Processing & Computing, 2019, 8(2): 100-107.
|
[14] |
Li J, Han L, Li X, et al. An Evaluation of Deep Neural Network Models for Music Classification Using Spectrograms[J]. Multimedia Tools and Applications, 2022, 81(4): 4621-4647.
doi: 10.1007/s11042-020-10465-9
|
[15] |
Gordon R L, Schön D, Magne C, et al. Words and Melody are Intertwined in Perception of Sung Words: EEG and Behavioral Evidence[J]. PLoS ONE, 2010, 5(3): e9889.
doi: 10.1371/journal.pone.0009889
|
[16] |
Xu L, Sun Z, Wen X, et al. Using Machine Learning Analysis to Interpret the Relationship Between Music Emotion and Lyric Features[J]. PeerJ Computer Science, 2021, 7: e785.
doi: 10.7717/peerj-cs.785
pmid: 34901433
|
[17] |
赵剑, 刘华平, 梁晓晶, 等. 基于知识蒸馏与迁移学习结合的多模态音乐情感识别[J]. 复旦学报(自然科学版), 2021, 60(3): 309-314.
|
[17] |
(Zhao Jian, Liu Huaping, Liang Xiaojing, et al. Multimodal Music Emotion Recognition Based on Knowledge Distillation and Transfer Learning[J]. Journal of Fudan University (Natural Science), 2021, 60(3): 309-314.)
|
[18] |
Deng J J, Leung C H C. Dynamic Time Warping for Music Retrieval Using Time Series Modeling of Musical Emotions[J]. IEEE Transactions on Affective Computing, 2015, 6(2): 137-151.
doi: 10.1109/TAFFC.2015.2404352
|
[19] |
Zhang Z, Xie L, Zhao Z. Temporal Integration of Emotion Perception for Cross-Cultural and Multi-Emotion Music[C]// Proceedings of 2021 International Conference on Culture-oriented Science & Technology (ICCST). IEEE, 2021: 198-203.
|
[20] |
Hasanzadeh F, Annabestani M, Moghimi S. Continuous Emotion Recognition During Music Listening Using EEG Signals: A Fuzzy Parallel Cascades Model[J]. Applied Soft Computing, 2021, 101: 107028.
doi: 10.1016/j.asoc.2020.107028
|
[21] |
杨媛. 数字音乐学研究综述[J]. 北方音乐, 2020(2): 4-5, 14.
|
[21] |
(Yang Yuan. A Review of Digital Musicology Research[J]. Journal of the Northern Music, 2020(2): 4-5, 14.)
|
[22] |
Yang Y H, Chen H H. Machine Recognition of Music Emotion: A Review[J]. ACM Transactions on Intelligent Systems and Technology (TIST), 2012, 3(3): Article No.40.
|
[23] |
Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv:1810.04805.
|
[24] |
Liu G, Guo J. Bidirectional LSTM with Attention Mechanism and Convolutional Layer for Text Classification[J]. Neurocomputing, 2019, 337: 325-338.
doi: 10.1016/j.neucom.2019.01.078
|
[25] |
Logan B. Mel Frequency Cepstral Coefficients for Music Modeling[C]// Proceedings of the International Symposium on Music Information Retrieval. 2000.
|
[26] |
Baziotis C, Pelekis N, Doulkeridis C. DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis[C]// Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). ACM Press, 2017: 747-754.
|
[27] |
Zia T, Zahid U. Long Short-Term Memory Recurrent Neural Network Architectures for Urdu Acoustic Modeling[J]. International Journal of Speech Technology, 2019, 22(1): 21-30.
doi: 10.1007/s10772-018-09573-7
|
[28] |
Pearl R, Reed L J. On the Rate of Growth of the Population of the United States Since 1790 and Its Mathematical Representation[J]. PNAS, 1920, 6(6): 275-288.
doi: 10.1073/pnas.6.6.275
pmid: 16576496
|
[29] |
张驰名, 王庆凤, 刘志勤, 等. 基于深度迁移学习的肺结节辅助诊断方法[J]. 计算机工程, 2020, 46(1): 271-278.
doi: 10.19678/j.issn.1000-3428.0053340
|
[29] |
(Zhang Chiming, Wang Qingfeng, Liu Zhiqin, et al. Pulmonary Nodule Auxiliary Diagnosis Method Based on Deep Transfer Learning[J]. Computer Engineering, 2020, 46(1): 271-278.)
doi: 10.19678/j.issn.1000-3428.0053340
|
[30] |
张艺. 中西方音乐异同初探[J]. 海南师范大学学报(社会科学版), 2012, 25(5): 142-144.
|
[30] |
(Zhang Yi. A Probe into Similarities and Dissimilarities Between Chinese Music and Western Music[J]. Journal of Hainan Normal University (Social Sciences), 2012, 25(5): 142-144.)
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|