[go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Music recommender using deep embedding-based features and behavior-based reinforcement learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the rapid increase of digital music on online music platforms, it has become difficult for users to find unknown but interesting songs. Although many collaborative filtering or content based recommendation methods have been proposed, they have various relatively serious some problems, including cold start, diversity of recommendations. etc. Therefore, we propose a reinforcement personal music recommendation system (RPMRS) to address these problems. RPMRS comprises two main components. First, deep representation of audio and lyrics extracted by WaveNet and Word2Vec models, respectively, and apply a proposed content based recommendation method from these. Second, we employ reinforcement learning is to learn user preferences from their song playing log. Experimental results confirm, that hybrid features are superior to audio or lyrics based features for content recommendation, largely because independent audio features significantly outperform lyrics features; and reinforcement learning improves personalized recommendations. Overall, the proposed RPMRS provides dynamic and personalized music recommendations for the user.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Notes

  1. Note: cite in Magenta | Make Music and Art Using Machine Learning. (Date Published: 2017, April 05). Retrieved from https://magenta.tensorflow.org/

  2. Pre-trained WaveNet model, downloadable from NSynth – Neural Audio Synthesis: http://download.magenta.tensorflow.org/models/nsynth/wavenet-ckpt.tar

  3. Note: cite in Magenta | The NSynth Dataset. (Date Published: 2017, April 05). Retrieved from https://magenta.tensorflow.org/datasets/nsynth#files

References

  1. Hurley N, Zhang M (2011) Novelty and diversity in top-n recommendation – analysis and evaluation. ACM Trans Internet Technol (TOIT) 10(4):14

    Article  Google Scholar 

  2. Cheng Z, Shen J, Nie L, Chua TS, Kankanhalli M (2017) Exploring user-specific information in music retrieval. In: Proceedings of the 40th international ACM SIGIR conference on Research and Development in Information Retrieval. ACM, pp 655–664

  3. Wang X, Wang Y (2014) Improving content-based and hybrid music recommendation using deep learning. In: Proceedings of the 22nd ACM international conference on Multimedia. ACM, pp 627–636

  4. Logan B (2004) Music recommendation from song sets. In: ISMIR, pp 425–428

  5. Ricci F, Rokach L, Shapira B (2011) Introduction to recommender systems handbook. In: Recommender systems handbook. Springer, Boston, pp 1–35

    Chapter  Google Scholar 

  6. Liang D, Zhan M, Ellis DP (2015) Content-aware collaborative music recommendation using pre-trained neural networks. In: ISMIR, pp 295–301

  7. Gao H, Tang J, Hu X, Liu H (2015) Content-aware point of interest recommendation on location-based social networks. In: Twenty-ninth AAAI conference on Artificial Intelligence

  8. Lu Z, Dou Z, Lian J, Xie X, Yang Q (2015) Content-based collaborative filtering for news topic recommendation. In: Twenty-ninth AAAI conference on Artificial Intelligence

  9. Soares M, Viana P (2015) Tuning metadata for better movie content-based recommendation systems. Multimed Tools Appl 74(17):7015–7036

    Article  Google Scholar 

  10. Yeh CH, Tseng WY, Chen CY, Lin YD, Tsai YR, Bi HI, Lin YC, Lin HY (2014) Popular music representation: chorus detection & emotion recognition. Multimed Tools Appl 73(3):2103–2128. https://doi.org/10.1007/s11042-013-1687-2

    Article  Google Scholar 

  11. Tzanetakis G (2007) Marsyas submissions to MIREX 2007. In: Proceedings of the international conference on Music Information Retrieval

  12. Peeters G (2008) A generic training and classification system for MIREX08 classification tasks: audio music mood, audio genre, audio artist and audio tag. In: Proceedings of the international conference on Music Information Retrieval

  13. Mokhsin MB, Rosli NB, Wan Adnan WA, Abdul Manaf N (2014). Automatic music emotion classification using artificial neural network based on vocal and instrumental sound timbres. In: New trends in software methodologies, tools and techniques – Proceedings of the 13th SoMeT 2014. Frontiers in artificial intelligence and applications, vol 265, IOS Press, pp 3–14. https://doi.org/10.3233/978-1-61499-434-3-3

  14. Kim Y, Schmidt E, Migneco R, Morton B, Richardson P, Scott J et al (2010) Music emotion recognition: a state of the art review. In: Proceedings of the 11th international Society for Music Information Retrieval Conference (ISMIR). ISMIR, Utrecht, pp 255–266

    Google Scholar 

  15. Li T, Ogihara M (2003) Detecting emotion in music

    Google Scholar 

  16. Van Den Oord A, Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, Kavukcuoglu K (2016) WaveNet: a generative model for raw audio. SSW 125

  17. Engel J, Resnick C, Roberts A, Dieleman S, Norouzi M, Eck D, Simonyan K (2017) Neural audio synthesis of musical notes with wavenet autoencoders. In: Proceedings of the 34th international conference on Machine Learning-Volume 70, pp 1068–1077. JMLR.org

  18. Huang JW, Chiang CW, Chang JW (2018) Email security level classification of imbalanced data using artificial neural network: the real case in a world-leading enterprise. Eng Appl Artif Intell 75:11–21

    Article  Google Scholar 

  19. Chang JW, Lee MC, Su CY, Wang TI (2017) Effects of using self-explanation on a web-based Chinese sentence-learning system. Comput Assist Lang Learn 30(1–2):44–63

    Article  Google Scholar 

  20. Chang JW, Lee MC, Wang TI (2016) Integrating a semantic-based retrieval agent into case-based reasoning systems: a case study of an online bookstore. Comput Ind 78:29–42

    Article  Google Scholar 

  21. Huang PS, Chiu PS, Chang JW, Huang YM, Lee MC (2019) A study of using syntactic cues in short-text similarity measure. J Internet Technol 20(3):839–850

    Google Scholar 

  22. Lee MC, Chang JW, Hsieh TC (2014) A grammar-based semantic similarity algorithm for natural language sentences. Sci World J:2014

  23. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119

  24. Choi K, Downie JS (2018) Exploratory investigation of word embedding in song lyric topic classification: promising preliminary results. In: JCDL, pp 327–328

  25. Delbouys R, Hennequin R, Piccoli F, Royo-Letelier J, Moussallam M (2018) Music mood detection based on audio and lyrics with deep neural net. arXiv preprint arXiv:1809.07276

  26. Watkins CJCH (1989) Learning from delayed rewards

    Google Scholar 

  27. King J, Imbrasaitė V (2015) Generating music playlists with hierarchical clustering and Q-learning. In: European conference on Information Retrieval. Springer, Cham, pp 315–326

    Google Scholar 

  28. Zhao X, Zhang L, Ding Z, Xia L, Tang J, Yin D (2018) Recommendations with negative feedback via pairwise deep reinforcement learning. In: Proceedings of the 24th ACM SIGKDD international conference on Knowledge Discovery & Data Mining. ACM, pp 1040–1048

  29. Taghipour N, Kardan A (2008) A hybrid web recommender system based on q-learning. In: Proceedings of the 2008 ACM symposium on Applied Computing. ACM, pp 1164–1168

  30. Srivihok A, Sukonmanee P (2005) E-commerce intelligent agent: personalization travel support agent using Q Learning. In: Proceedings of the 7th international conference on Electronic Commerce. ACM, pp 287–292

  31. Zheng G, Zhang F, Zheng Z, Xiang Y, Yuan NJ, Xie X, Li Z (2018). DRN: a deep reinforcement learning framework for news recommendation. In Proceedings of the 2018 World Wide Web conference on World Wide Web. International World Wide Web Conferences Steering Committee, pp 167–176

  32. Iyyer M, Manjunatha V, Boyd-Graber J, Daumé III H (2015) Deep unordered composition rivals syntactic methods for text classification. In: Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th international joint conference on Natural Language Processing (Volume 1: Long Papers), vol 1, pp 1681–1691

Download references

Acknowledgements

This work was supported by the Ministry of Science and Technology, Taiwan, R.O.C. [grant number MOST 108-2218-E-025-002-MY3], [grant number MOST 108-2218-E-001-001].

Author information

Authors and Affiliations

  1. National Taichung University of Science and Technology, Taichung City, Taiwan

    Jia-Wei Chang, Jia-Yi Liao, Ying-Kai Hung, Chien-Che Huang & Ying-Hung Pu

  2. National Chung Hsing University, Taichung City, Taiwan

    Ching-Yi Chiou & Kuan-Cheng Lin

Authors
  1. Jia-Wei Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ching-Yi Chiou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jia-Yi Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying-Kai Hung
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chien-Che Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kuan-Cheng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ying-Hung Pu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying-Hung Pu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chang, JW., Chiou, CY., Liao, JY. et al. Music recommender using deep embedding-based features and behavior-based reinforcement learning. Multimed Tools Appl 80, 34037–34064 (2021). https://doi.org/10.1007/s11042-019-08356-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08356-9

Keywords

Search

Navigation