This Machine Learning Google AI Duet Project Jams With You

Imagine a computer that listens to the notes you play on your MIDI keyboard and plays musical responses in real time. Using Google's Magenta machine learning tech that's what AI Duet does.  

If you find yourself playing music alone and missing playing with others, this Google AI Duet project looks to solve that. AI Duet is a browser-based machine learning tool using the Google Magenta and TensorFlow software library technologies that jams with you.

Simply play some notes using the on-screen or a connected MIDI keyboard and the computer responds to your melody. Even if you don't know how to play piano it’s a fun way to experiment and listen to what the computer comes back with. It doesn't have to be a call and reponse style of duet either. You can get interesting results playing while the computer plays too.

Rather than programming this musical knowledge directly into AI Duet, the team at Google led by Yotam Mann played many musical examples to AI Duet. Over time the program learned about the realtionships between notes and timings and built its own map based on the examples it has heard.

So, you essentially play notes that are fed to the AI Duet neural network which decides on possible repsonses based on the notes and all the information its learned to date.

Interestingly Mann explains some unexpected results, that AI Duet "picks up on key and rhythm, even though I never explicitly programmed in the concepts of key and rhythm".

AI Duet is open source, so you can get into the code and start developing your own machine learning projects easily using Google Magenta and TensorFlow.

Whlie AI Duet isn't going to replace your Jazz musician friend and his/her jamming accumen anytime soon, it's a fascinating look at what is possible with machine learning programs in the music realm.



Rounik is the Executive Editor for Ask.Audio & macProVideo. He's built a crack team of professional musicians and writers to create one of the most visited online resources for news, review, tutorials and interviews for modern musician and producer. As an Apple Certified Trainer for Logic Pro Rounik has taught teachers, professional... Read More


Want to join the discussion?

Create an account or login to get started!