Machine Learning for Creative AI Applications in Music

Abstract

In this talk, I will briefly introduce three latest projects in our lab at Academia Sinica on creative applications in music, including the singing voice separation project, GenMusic (music generation) project, and the DJnet project. The first project is about separating the singing voice from the musical accompaniments, which can be used as a pre-processing step for many music related applications. The second project is about learning from massive collection of MIDI files to generate multi-track music by a generative adversarial network (GAN). The generative model can be used for generating music either from scratch, or by accompanying a given (instrument) track. The third project is about creating an AI DJ that knows how to manipulate, sample, and sequence musical pieces to create a personalized playlist. The goal of these projects is to enrich the way people create and interact with music in their daily lives, using the latest machine learning (deep learning) techniques.