Create Your Own MIDI-Generated Realtime Visuals with openFrameworks

Do you want to create a simple desktop app that generates realtime animations using MIDI data? Then you're in the right place and Liam Lacey is here to show how using openFrameworks.  

As a performing musician, incorporating projected visuals within your performances is a sure-fire way of enhancing the experience for your audience. There are many Video Jockey and Projection Mapping software applications designed for doing this, such as Resolume Arena or ArKaos GrandVJ, which allow artists to trigger and control video, graphics and animations in real time, as well as syncing them to music. Most of these applications can be controlled using MIDI, making them extremely useful to musicians who already use MIDI within their shows. However, the MIDI integration is often quite limited and may not let you control the visuals exactly how you want. One way around this would be to create your own custom software application

In this tutorial, I’m going to demonstrate how to create a simple desktop application that generates real-time animations using MIDI data, using a simple but extensive platform that could be used for almost any MIDI-to-visuals ideas that you may have. 

Resolume Arena 4 projection mapping software 

Resolume Arena 4 projection mapping software.

The platform we will be using here is called openFrameworks (often abbreviated to oF)—a free toolkit for developing interactive and multimedia applications, designed for artists and designers to create software in a simple way using a textual programming language. We will be using oF over other similar platforms, as it is very easy for beginners to use, has great audio/MIDI capabilities, and uses a common programming language which is very useful to learn for software development endeavors.

Live Visuals at Roskilde Festival 2014, Denmark, by Johan Bichel Lindegaard, partly made using openFrameworks

Live Visuals at Roskilde Festival 2014, Denmark, by Johan Bichel Lindegaard, partly made using openFrameworks.

Before we get started, here is a simple example of what we will be making—an application that draws animated circles whenever it receives a MIDI note, using note number and velocity to determine color and size, as well as using MIDI mod wheel messages to control other aspects of the visuals. My particular animation is being triggered by MIDI messages coming out of Ableton Live on OS X, which is also triggering drum samples within Native Instruments Battery.

What you will Need:

  • The openFrameworks toolkit—download this from here, selecting the desktop platform (OS X or Windows) that you will be using. If using Windows, I recommend using oF for Code::Blocks.
  • An IDE (integrated development environment)—this is the software application used for developing your custom application. On OS X you will need Xcode, or on Windows I recommend Code::Blocks. 
  • ofxMidi—this is the MIDI add-on library for oF, and can be downloaded from here.
  • A MIDI controller or applications—you will need one of these for generating MIDI messages to send to your custom application.
  • Download the Source Code for this project HERE.

Step 1—Installing your IDE

First you will need to install your IDE. Installing Xcode on OS X is easy—simply download it from the Mac App Store and follow the installation instructions. For Code::Blocks on Windows the installation process is a bit more complicated, however fortunately oF have provided a setup guide here

Step 2—Installing openFrameworks and ofxMidi

To install oF, simply move the downloaded oF folder into any location on your computer. To install ofxMidi, move the downloaded ofxMidi folder into the addons directory within your oF folder.

Step 3—Setting up the oF Project

Next you will need to set up a project for your IDE so that you can develop your oF application. To do this you will use oF’s projectGenerator application that you can find in the projectGenerator directory within the oF folder, which will automatically generate an Xcode/Code::Blocks project for you. Open up projectGenerator and do the following:

  1. Set ‘Name’ to a meaningful project name (e.g., midi_visualiser).
  2. Make sure ‘Path’ is set to your openFrameworks/apps/myApps directory, which it should be by default.
  3. Make sure ‘Platforms’ is set to the IDE you will be using.
  4. Click on ‘Addons’ and select ofxMidi.
  5. Click ‘Generate Project’.

 

openFrameworks Project Generator application

openFrameworks Project Generator application.

There should now be a new folder in the openFrameworks/apps/myApps directory for your newly created project.

Step 4—Adding the Code

Next you need to open your newly created project and add some code to it:

  1. Open up your IDE application (Xcode or Code::Blocks).
  2. Go to ‘File -> Open…’ and navigate to the newly created project directory.
  3. Select the project file (the .xcodeproj file for Xcode or the .workspace for Code::Blocks) and click ‘Open’.
  4. In the IDE navigate to the left-hand column/pane where there should be an icon next to your projects name. Click on the triangle or ‘+’ icon to the left-side of this to reveal a tree-view of folders, and then do the same on the ‘src’ folder to reveal three source code files—main.cpp, ofApp.cpp, and ofApp.h

 

The Xcode (left image) and Code::Blocks (right image) project navigator where you will find the source code files

The Xcode (left image) and Code::Blocks (right image) project navigator where you will find the source code files.

  1. Download the code for this project from here.
  2. Replace the contents of ofApp.cpp and ofApp.h within your IDE with the contents of the newly downloaded files. To open these files in Xcode just single click on them from the tree-view, however in Code::Blocks you need to double-click on them.

Teaching you how the code for this project works is a bit beyond the scope of this article, so I’ve placed many comments (lines beginning with ‘//’) in the code to describe what each part is doing.

Step 5—Building and Running the Code

You are almost ready to try out the MIDI visualiser application, however before that you need to build the code into a piece of software. To do this complete the following steps:

In Xcode on OS X:

  1. On the top toolbar on the left-hand side next to the play and stop icons there is a dropdown menu—select the ‘[project name] Debug’ option.
  2. From the menu bar select ‘Product -> Build’ to build the application. After a short amount of time it should notify you that that build was successful.
  3. From the menu bar select ‘Product -> Run’ to run the application.
  4. The visualiser application should now have appeared, and an application file should have been created in the ‘bin’ folder of the project directory.

 

The Xcode IDE running the MIDI visualiser application

The Xcode IDE running the MIDI visualiser application.

In Code::Blocks on Windows:

  1. From the menu bar select ‘Build -> Build’ to build the application. After a short amount of time it should notify you in the bottom section of Code::Blocks that the build has finished.
  2. From the menu bar select ‘Build -> Run’ to run the application.
  3. The visualiser application should now have appeared (with a second utility application appearing behind it), and an application file should have been created in the ‘bin’ folder of the project directory.

 

The Code::Blocks IDE running the MIDI visualiser application

The Code::Blocks IDE running the MIDI visualiser application

Congratulations—you have just built your own piece of software! Now it’s time to give it a go.

Step 6—Using the application

Using the MIDI visualiser application is easy. As the instructions state, simply select the MIDI input you want to use with the numbers on your computer keyboard, pressing ‘s’ to hide the instructions. Send some MIDI note data to the application from the selected MIDI input and you’ll notice it draws a circle for each note played, which fades out over time. The size of the circles change depending on note velocity, and colour of the circles change depending on note number. Try sending mod wheel (CC 1) messages—you’ll notice that this changes the background color. The application window can be resized or made full screen.

The MIDI visualiser application

The MIDI visualiser application

Download the Source Code for this project HERE

Conclusion

So that’s how you can build your own software for generating visuals that are controlled via MIDI. Not only have you created something that could enhance your live performances in a unique way, but you’ve also learnt the very basics of a software toolkit that you could use to create an infinite number of diverse multimedia applications and projects; just check out the openFrameworks gallery to see the amazing things that have been done with it. If you want to know more about developing software with oF I recommend checking out the official openFrameworks tutorials for more information. 

 

Liam is the Head Of Development specialising in software development at Modal Electronics, the company behind the 002, 008 and CRAFTsynth synthesisers, and was previously the lead software developer at nu desine, the developers of the AlphaSphere MIDI controllers. He was also an Associate Lecturer at UWE, Bristol, UK where he graduated... Read More

Discussion

cuticulamusic
can you specify for which OS X this should work, cause I guess in 10.9 it is not working. I get tons of debugging errors like "gluBuild2DMipmaps is deprecated" use xxx instead. ???
Liam Lacey
Hey cuticulamusic,
These messages are actually just openFrameworks build warnings that can be ignored in this case, so it should still be possible to build and run the application. Is this true for you, or are they coming up as red errors that prevent you from building? With the latest version of oF (0.8.4) this project should work on OS X 10.6 and above.
cuticulamusic
Hi, I found out that I didn't " On the top toolbar on the left-hand side next to the play and stop icons there is a dropdown menu—select the ‘[project name] Debug’ option." set it right. But after doing that I got like 18 real errors. all of them "Apple Match-O Linker (id) Error". Do you have any idea what that could be? and thanks for your help already and putting up this tutorial (even tough its not working for me, haha).
Liam Lacey
Can you give examples of the exact "Apple Match-O Linker (id) Error" errors?
XacWj
Tried this on OS X 10.11.1 Xcode says build failed.
Liam Lacey
Hey XacWj,
I haven't been able to test this on OS X 10.11 yet. What errors are you getting?
Bonzo
I'm so confused. You linked to a version of Code Blocks, yet the installation guide for Windows you linked to recommends another completely different version. They're nowhere near the same size and the version numbers are completely different. It asks that i 'install' it, but all i find is an install.md file which i can't open.
Bonzo
Thanks for deleting my comment, i figured out where i went wrong anyway. It would have helped if you had explained that oF and code::blocks are two separate programs.
Bonzo
So now i've done all the steps but at the end i just get the grey screen and an empty command prompt box with no instructions. The build had no errors. Any advice?
Liam Lacey
Hey Bonzo,

Code::Blocks is the piece of software used for developing software applications, whereas oF is just a library of files used to make software development a lot easier. Sorry if that wasn't clear in the tutorial.

Since releasing this tutorial there has been a newer release of oF (0.9.0), and unfortunately they no longer suggest Code::Blocks to be the best IDE to use on Windows. Also I have not yet tested this tutorial using version 0.9.0 - just 0.8.4 (which can be downloaded from http://www.openframeworks.cc/download/older.html). Which version of oF did you try this tutorial with?
Stefan Hetzel
@Liam Lacey: Thanks for the great tutorial, it was (almost) no problem to install the app :-) (Win 7 64-bit, using oF 0.8.4)
Eric
This is exactly what I've been looking for. Thank you so much!!!
Alex
Please, what portions of midi_visualizer source code gets copied into CodeBlocks. Think I'm ok with ofx files (h and C++). Does MAMA file go as well. How??

Want to join the discussion?

Create an account or login to get started!