Empowering music creation using AI
Music creation is a resource-intensive and costly process, demanding extensive time commitments. An opportunity lies in reimagining the creation workflow. Generative AI holds the potential to usher in a transformative revolution comparable to the impact of the introduction of the MIDI keyboard in the music industry in the 1980s. We are building a tool that enables our in-house musicians to create music with AI by giving prompts.
My role: Foundational research, identifying product market fit, creation of user stories, preliminary concepts, data collection, usability testing of the concepts
Timeline: October 2023 - January 2024
The team: 1 designer, 1 sound designer, 1 PM, 5 data scientists, 2 engineers
🌏 In the news
Case study 01
Designing a prompt experience to generate music
Background
In October of 2023, the technology office at Zee Entertainment organised a Gen AI hackathon, Prompt Play 1.0 (in association with Microsoft). Multiple teams were formed within the data science department to come up with ideas that could solve for in-house processes. These included processes like subtitling, dubbing, marketing asset generation, background music generation etc.
Since Gen AI is on the rise, I volunteered and became part of the team that was trying to generate background music, using the available large music models but for Indian music preference. As Zee is a media company, it uses background music frequently for its daily TV shows on multiple channels, short form content etc.
Post the hackathon, the company decided to pursue the project, and hence the project actually began!
Research Methodology
To understand the music creation process from a professional lens, I talked to music producers who create music for TV shows. I understood the process of receiving a music brief from the channel team and show producer as the show is being conceptualised, till creating music for each episode on regular basic. I also conducted brainstorming session with in-house teammates who are musicians to better understand music creation and identify opportunity areas. I also looked at applications that are already using AI to create music like Beatoven, Aiva etc to understand the current market offerings.
Task flow of music creation
The tool designed (Music Studio) assists the creation and validation aspect of the journey. For this case study, I will talk about the creation aspect, with focus on prompt input experience.
Identified user needs for music generation tool
Goals
Scenarios
HMW design an input that enables user to specify the kind of music they want and get expected results?
Concept Testing
I conducted a user testing with assistance from a senior researcher to understand the following:
1. Which options capture the music brief efficiently?
2. Which options allow for experimentation and creation of diverse music?
Findings
1. Uploading a reference was really appreciated. However we were able to encounter technical issues, if the reference had vocals, the output wasn’t great.
2. The option without a text input allowed for less expression as user could only input certain combinations.
3. The generated output was good only up-to 60 seconds, post that music quality was not good.
Final Design
Based on the user insights, understanding of technical limitations, and feedback from the design team, the final design for the MVP was conceptualised.
Experiences of the MVP will constantly change with better understanding of product market fit. Hence, I arrived at these guidelines for designing platforms that utilise AI generation.
