Create Your Own MIDI-Generated Realtime Visuals with openFrameworks

As a performing musician, incorporating projected visuals within your performances is a sure-fire way of enhancing the experience for your audience. There are many Video Jockey and Projection Mapping software applications designed for doing this, such as Resolume Arena or ArKaos GrandVJ, which allow artists to trigger and control video, graphics and animations in real time, as well as syncing them to music. Most of these applications can be controlled using MIDI, making them extremely useful to musicians who already use MIDI within their shows. However, the MIDI integration is often quite limited and may not let you control the visuals exactly how you want. One way around this would be to create your own custom software application

In this tutorial, I’m going to demonstrate how to create a simple desktop application that generates real-time animations using MIDI data, using a simple but extensive platform that could be used for almost any MIDI-to-visuals ideas that you may have. 

Resolume Arena 4 projection mapping software.

The platform we will be using here is called openFrameworks (often abbreviated to oF)—a free toolkit for developing interactive and multimedia applications, designed for artists and designers to create software in a simple way using a textual programming language. We will be using oF over other similar platforms, as it is very easy for beginners to use, has great audio/MIDI capabilities, and uses a common programming language which is very useful to learn for software development endeavors.

Live Visuals at Roskilde Festival 2014, Denmark, by Johan Bichel Lindegaard, partly made using openFrameworks.

Before we get started, here is a simple example of what we will be making—an application that draws animated circles whenever it receives a MIDI note, using note number and velocity to determine color and size, as well as using MIDI mod wheel messages to control other aspects of the visuals. My particular animation is being triggered by MIDI messages coming out of Ableton Live on OS X, which is also triggering drum samples within Native Instruments Battery.

What you will Need:

• The openFrameworks toolkit—download this from here, selecting the desktop platform (OS X or Windows) that you will be using. If using Windows, I recommend using oF for Code::Blocks.
• An IDE (integrated development environment)—this is the software application used for developing your custom application. On OS X you will need Xcode, or on Windows I recommend Code::Blocks. 
• ofxMidi—this is the MIDI add-on library for oF, and can be downloaded from here.
• A MIDI controller or applications—you will need one of these for generating MIDI messages to send to your custom application.
• Download the Source Code for this project HERE.

Step 1—Installing your IDE

First you will need to install your IDE. Installing Xcode on OS X is easy—simply download it from the Mac App Store and follow the installation instructions. For Code::Blocks on Windows the installation process is a bit more complicated, however fortunately oF have provided a setup guide here. 

Step 2—Installing openFrameworks and ofxMidi

To install oF, simply move the downloaded oF folder into any location on your computer. To install ofxMidi, move the downloaded ofxMidi folder into the addons directory within your oF folder.

Step 3—Setting up the oF Project

Next you will need to set up a project for your IDE so that you can develop your oF application. To do this you will use oF’s projectGenerator application that you can find in the projectGenerator directory within the oF folder, which will automatically generate an Xcode/Code::Blocks project for you. Open up projectGenerator and do the following:

1. Set ‘Name’ to a meaningful project name (e.g., midi_visualiser).
2. Make sure ‘Path’ is set to your openFrameworks/apps/myApps directory, which it should be by default.
3. Make sure ‘Platforms’ is set to the IDE you will be using.
4. Click on ‘Addons’ and select ofxMidi.
5. Click ‘Generate Project’.

openFrameworks Project Generator application.

There should now be a new folder in the openFrameworks/apps/myApps directory for your newly created project.

Step 4—Adding the Code

Next you need to open your newly created project and add some code to it:

1. Open up your IDE application (Xcode or Code::Blocks).
2. Go to ‘File -> Open…’ and navigate to the newly created project directory.
3. Select the project file (the .xcodeproj file for Xcode or the .workspace for Code::Blocks) and click ‘Open’.
4. In the IDE navigate to the left-hand column/pane where there should be an icon next to your projects name. Click on the triangle or ‘+’ icon to the left-side of this to reveal a tree-view of folders, and then do the same on the ‘src’ folder to reveal three source code files—main.cpp, ofApp.cpp, and ofApp.h

The Xcode (left image) and Code::Blocks (right image) project navigator where you will find the source code files.

1. Download the code for this project from here.
2. Replace the contents of ofApp.cpp and ofApp.h within your IDE with the contents of the newly downloaded files. To open these files in Xcode just single click on them from the tree-view, however in Code::Blocks you need to double-click on them.

Teaching you how the code for this project works is a bit beyond the scope of this article, so I’ve placed many comments (lines beginning with ‘//’) in the code to describe what each part is doing.

Step 5—Building and Running the Code

You are almost ready to try out the MIDI visualiser application, however before that you need to build the code into a piece of software. To do this complete the following steps:

In Xcode on OS X:

1. On the top toolbar on the left-hand side next to the play and stop icons there is a dropdown menu—select the ‘[project name] Debug’ option.
2. From the menu bar select ‘Product -> Build’ to build the application. After a short amount of time it should notify you that that build was successful.
3. From the menu bar select ‘Product -> Run’ to run the application.
4. The visualiser application should now have appeared, and an application file should have been created in the ‘bin’ folder of the project directory.

The Xcode IDE running the MIDI visualiser application.

In Code::Blocks on Windows:

1. From the menu bar select ‘Build -> Build’ to build the application. After a short amount of time it should notify you in the bottom section of Code::Blocks that the build has finished.
2. From the menu bar select ‘Build -> Run’ to run the application.
3. The visualiser application should now have appeared (with a second utility application appearing behind it), and an application file should have been created in the ‘bin’ folder of the project directory.

The Code::Blocks IDE running the MIDI visualiser application

Congratulations—you have just built your own piece of software! Now it’s time to give it a go.

Step 6—Using the application

Using the MIDI visualiser application is easy. As the instructions state, simply select the MIDI input you want to use with the numbers on your computer keyboard, pressing ‘s’ to hide the instructions. Send some MIDI note data to the application from the selected MIDI input and you’ll notice it draws a circle for each note played, which fades out over time. The size of the circles change depending on note velocity, and colour of the circles change depending on note number. Try sending mod wheel (CC 1) messages—you’ll notice that this changes the background color. The application window can be resized or made full screen.

The MIDI visualiser application

Download the Source Code for this project HERE

Conclusion

So that’s how you can build your own software for generating visuals that are controlled via MIDI. Not only have you created something that could enhance your live performances in a unique way, but you’ve also learnt the very basics of a software toolkit that you could use to create an infinite number of diverse multimedia applications and projects; just check out the openFrameworks gallery to see the amazing things that have been done with it. If you want to know more about developing software with oF I recommend checking out the official openFrameworks tutorials for more information.