iSMuG - Senior Design

Interactive Smart Music Generator (iSMuG) is a novel music generator based on human states observed via computer vision (CV) and biomarker sensors. Senior Design Project by Sai Sravan Gogulapati, Kenny Moc, Arushi Arora, and Jade Whiting.

Our interactive smart music generator employs novel AI music generation and interactive feedback systems based on a user’s human state, which is detected via computer vision and physiological sensors like a heart rate monitor or galvanic skin response sensor. One of the main reasons we came up with this idea was because of our shared interests in both machine learning and its applications and music in general. We also thought this project would serve as a great proof of concept for companies like startups for autonomous cars, which are currently working on interactive music recommendation systems based on the passengers’ emotional states as monitored through computer vision. Our project can additionally be used as an open-ended tool for other research pertaining to the psychology of music and its effect on people’s emotions, which is an area that’s widely researched and of a lot of interest in the research world.

Both before and during our project work, we read a lot of the literature pertaining to the intersection of music or art and AI. Emotions are often vague, complex, and ambiguous so it’s hard to adjust generated art and music to match a subjective taste and incorporate preference learning with that. Since our project revolves around identifying features in the music that we can tweak to ensure it reflects the user’s mood, we read about features like dynamics, pitch, rhythm, and things like that to see how that influences people’s emotions. In a lot of these prior works, the ground-truth mood of a piece of music was typically extracted via low-level audio features like the ones mentioned, along with metadata about the piece.