Designing Realtime 3D Sound Performance: Sound of MNTN, Ableton Live and Planetary Cymatic Resonance

3D sound is still a relatively new phenomenon and can be complex to work with. Timo Preece investigates how it works, and goes on a journey through space and time...
Photography credit: Pauline Fabry for her awesome photos.

Investigating the space around us or the space within, both the internal and external are compelling trajectories for sonic exploration. When searching for new pathways to artistic, personal, technological and spiritual development, multidimensional space advocates a viable conduit allowing us to reach farther and dive deeper into the expanses of uncharted territories. Space itself is an instrument, a medium we live in and communicate through, an immersive environment in which all material objects are located and all events occur.

Spatialized audio is gaining momentum as we migrate further into the realms of interactive multimedia, virtual reality, 360° videos, multichannel installations and 3D sound performance. Traditionally the technology behind designing and displaying 3D sound has been costly, overly sophisticated and/or technically demanding. Popular opinion dictates that the medium itself has been limited overall by these challenges, rendering it accessible only to those with the right budget and/or knowhow. Most anyone attempting surround setups with Ableton Live can attest to limitations and difficult workflow. The software is simply not designed for multi-channel mixing natively and requires tedious workarounds using auxiliary busses and/or Max for Live.

Using auxiliary sends and return tracks as a multichannel workaround. 

Using auxiliary sends and return tracks as a multichannel workaround. 

The Sound of the Mountain - MNTN

Benjamin Schulz (co-founder & software architect), Johannes Scherzer (co-founder & sound scenographer) and Lukasz Polowczyk (musician & communication and artist relations), a.k.a. The Sound of the Mountain (MNTN), are offering viable solutions with their lightweight, affordable 3D audio software.

MNTN is a spatial sound engine that operates as a standalone application, parallel to your Digital Audio Workstation and routed through Soundflower. It also serves as a global panning interface for positioning and moving sound objects. MNTN uses VST/AU/AAX plug-ins to connect compatible audio software to almost any loudspeaker setup. The plug-ins act as surround panners, are object based and not tied directly to a particular speaker output configuration. Additionally, MNTN is binaural savvy for convenient 3D headphone mixdowns.

 

To demonstrate just how extraordinary their software and concept truly is, the MNTN team was invited to Ableton’s Loop summit last November. At such time, I had the opportunity to run some stems through their multichannel setup and was immediately impressed with the intuitiveness, responsiveness and overall ease of use.

Loud Speaker setup.

The timing of discovering MNTN was synchronistically impeccable. I’m involved in an audio visual project, Planetary Cymatic Resonance (PCR), that focuses on immersive realtime performance, exploring both internal and external space alike. Before discovering MNTN, we were investigating options for 3D sound performance but weren’t quite finding a one-stop dialed, flexible, fluid solution.

Recently PCR was presented the opportunity to put MNTN to the test by integrating their software for a 7.4.2 surround performance that took place in Berlin. Through PCR’s investigations, we’ve come up with some findings, which hopefully prove insightful for those looking to dive into similar multichannel setups.

Note - If you’re unfamiliar with how the MNTN software works in general, start by checking out The Sound of the Mountain website to get a basic idea of its features, specifications and capabilities.

 Transforming 2D to 3D

For Planetary Cymatic Resonances’ performance setup, we use two computers running Ableton Live 9, connected by ethernet, synced via Link. I, Timo Preece, perform the majority of music content (electronic instrumentation, clip playback, effects modulation). My partner Steffen Günther, in addition to triggering various other sonic matter, receives multiple channels from my audio interface, adds effect processing and mixes my outputs according to the vibe, space and task at hand.

Switching from stereo to 3D, we augmented Steffen’s session by creating a track with an audio effect rack housing all of the MNTN plug-in instances that correspond to the channels we want to send dry, without effects. For greater versatility in mixing, we set up a supplementary set of grouped MNTN plug-ins for sending our effect returns to the MNTN engine. This gives us greater control over effect panning, leading to some rather compelling surround imaging.

As a suggestion, route tracks you want to control with one specific MNTN object to the channel declared in the MNTN plug-in you use. With MNTN it does not matter on which track the MNTN plug-ins are instantiated. You can put all associated plug-ins into one track to make writing/recording clip panning automation easier.

 Realtime Surround Automation

Regarding typical multichannel performances, more often these are prearranged playback session meaning the performer plays mixed, automated stems routed through outputs which are sent out to the multi-channel speaker setup. Life gets more difficult when performances are improvised and not mixed ahead of time, as is the case with Planetary Cymatic Resonances’ material. If you’re not equipped with twenty or so limbs or a team of mixers, how does one go about improvising realtime surround automation?

Clip Envelopes

Ableton Live’s session clip envelopes offer a decent solution for defining stationary snapshots for positioning MNTN objects in the surround field. Start by writing or recording automation envelopes for main MNTN object positions. Change several or all object positions by triggering them manually, or by Follow Actions, to reposition them throughout the duration of your set.

While Live’s Clip Envelopes work decently for shifting positions abruptly, getting objects to morph from one setting to another is no easy task, especially if you’re unsure of the sequence that you’d like to trigger them. Without careful planning, panning changes tend to jump from location to location, rather than transition smoothly.

Need to interpolate? ClyphX to the rescue…

ClyphX is a free MIDI Remote Script for Live that provides an extensive list of Actions related to controlling different aspects of Live. These Actions can be accessed via X-Triggers (Session View Clips, Arrange View Locators or MIDI Controls). Each X-Trigger can trigger a single Action or a list of Actions.

One of ClyphX’s most useful features is the ability to create and recall snapshots of device and mixer parameters. PCR was able to smoothly modulate between snapshots and set interpolation times on a clip-by-clip basis via a function called Snap Track. Snap Track will use Smoothing, which will cause the values of Snapped parameters to gradually morph from their current values to the values stored in the X-Snap.

 

X/Y Control

Since MNTN objects work on an X/Y axis, you’ll want a controller that lends itself to such a task. We decided to go with an iPad and the incredibly brilliant Liine Lemur application.

Line Lemur on iPad.

As part of Lemur’s default projects, Studio Combo contains a Bouncy X/Y controller that can be mapped to any MNTN object in Live. Bouncy X/Y instances can be manipulated by Physics controls that set it into motion based on the speed and characteristics you define.

Multichannel Formats and Mixing Down

Capturing your performance is as simple as recording and exporting the multichannel input and output audio stream. Users can export any channel based 2D and 3D schemes, as well as 3D audio for 360°/VR videos and binaural HRTF format for headphone listening. Available file formats are WAV, AIFF, FLAC, and Ogg Vorbis. Users can choose to save as a single multichannel file, or as multiple mono files. Be sure to define all specs and the export format before recording. You won’t be able to change it after doing so.

MNTN Export formats:

  • any current channel based output format
  • binaural HRTF
  • Ambisonics for VR and 360° videos (SN3D, ACN, YouTube)
  • Multichannel input ( = output from your DAW)
  • WAV, AIFF, FLAC, Ogg-Vorbis with up to 32 Bit
  • Split and interleaved multichannel audio files
  • Trim start (export starts with audio signal) 
 

Binaural Listening

I’ll conclude by giving listening examples of how MNTN software based performances can be easily turned into binaural 3D headphone experiences.

Here’s what an excerpt of Planetary Cymatic Resonance’s performance sounded like once we exported our 7.4.2 multichannel mix into binaural HRTF format. This example is an unmediated fold-down with no additional mixing. 


The next selection was taken from a PCR live performance and specifically remixed for binaural headphone listening. 

*Soundcloud link - Planetary Cymatic Resonance - Alautun - HRTF Binaural

 

Additional Resource: MNTN Tutorial - 5 Steps to VR Audio

Note - Students, lecturers, and educational institutes are eligible to request free scholar MNTN licenses. In case you're finishing soon, you can also request a 50% rebate on regular licenses. 

More About MNTN: https://mntn.rocks

Planetary Cymatic Resonance (PCR) is an audio visual project based on tunings calculated from natural frequencies found in our universe, like the movement of planets or the resonances of molecules. PCR extends sound explorations through real time performance, striving to promote mindfulness and inner awareness through improvised cinematic sound journeys.

Timo Preece - Audio Technologist and Sound Designer

Steffen Günther - Sound Designer and Cosmic Frequency Researcher

Soundcloud

Youtube

Homepage

Facebook 

 Photography credit: Pauline Fabry for her awesome photos.
Synthesis 101
The Filter
by Bob Moog Foundation

"Timo Preece (Certified Ableton Trainer, ProTools Expert) is an accomplished audio technologist, multi-media consultant, sound designer and electronic musician. He graduated with a degree in Ethnomusicology from UCLA and a Masters in Sound Design at the University of Edinburgh, UK. From conceptualizing and building cutting-edge setu..." Read More

Discussion

Want to join the discussion?

Create an account or login to get started!