Mixing with Dolby Atmos using Dolby's rendering software for Apple Music’s Spatial Audio

Want to mix in Dolby Atmos for Apple Music's Spatial Audio? Read this!

 

When Apple announced Dolby Atmos and Spatial Audio for Apple Music, I was very excited.

I’ve been a re-recording mixer in TV for almost 30 years, and over 20 years ago, I traveled to Washington DC to attend a surround sound seminar by Tomlinson Holman, who developed THX for Lucasfilm and created the first 10.2 sound system. It was fascinating, to say the least, but what I’ll always remember is hearing The Beatles “Yesterday” as a 5.1 a cappella by Boyz II Men. I wanted surround music to take off, but I had doubts it would considering how few people had a surround system in their home.

 

Relatively few people have true surround in their home still, let alone even more speakers for true Dolby Atmos. Dolby did something exciting with Atmos though. Dolby created a flexible surround format that simply lives in a 3D space, regardless of the number of speakers you have, which could be a typical 7.1.4 or even “stereo” headphones. Got an Atmos decoder connected to your 5.1 system and threw two extra speakers (instead of four) on the ceiling for a 5.1.2? No problem. Have 11.1.8? Definitely no problem. Got an Atmos soundbar that just sits in front of you? Not a problem. How about a 2.1 set up? Still not a problem. Maybe you just have an iPhone or an iPhone with AirPods? Incredibly, that’s not even a problem. It’s kind of crazy when you think about it, but it was brilliant because that’s what it was going to take to bring surround mixes mainstream at home, whether for picture or music.

Apple's Spatial Audio 

I wondered how Apple’s Spatial Audio fit into this, because I knew from my post production career that Dolby had their own binaural mode for Atmos. Apple, though, chose to create their own processing for how to fold down a surround mix to supported stereo headphones rather than using Dolby’s. I say surround mix because some mixes on Apple Music are 5.1 or 7.1, not Atmos. Apple even now allows you to turn on their Spatialize Stereo for stereo tracks.

So being Apple was using proprietary processing, I wanted to know how it differed from Dolby’s. One of the main reasons I wanted to know was because though there are a lot of great sounding Spatial Audio tracks, there are also a lot of really bad ones. I wanted to know why some tracks sounded as if the vocals or percussion were low and distant, where the balance of instruments in the mix just wasn’t right, not only when compared to the stereo version, but even when compared to the surround version while listening on a surround system. Another reason I wanted to know the difference was because of who is listed on the Spatial Audio patent. Tomlinson Holman has been working for Apple for over 10 years and is listed as one of the inventors of the File Format for Spatial Audio.

Mixing for Spatial Audio

The only way I knew to truly hear the answer to that question was to do something I never wanted to do, which was release a couple of songs I had written for myself. About 30 years ago, a friend and I called ourselves Zythem and wrote a handful of thrash metal songs and recorded them on a 4-Track. These songs were really just for me, and I never wanted them out in the world. Eight years ago I decided to build a studio at home and do the majority of my professional mixing there while going into NYC to review the shows with the producers. One of the first things I did with my studio though was re-record two of those old songs. I’m a post mixer, not a music mixer, and those are two different animals, but I figured I’d do the songs enough justice being they were just for me and would always be just for me—so I thought.

I went out on a limb, deciding I’d release those two songs out into the wild, and I wanted this to be a test for two things. One, find out what Spatial Audio sounds like in comparison to Atmos and Dolby’s binaural mode, and two, find out if home musicians can mix actual Atmos relatively easily and cheaply in a 5.1 studio at home.

When I say relatively cheaply, I’m assuming someone creating music already has a home studio, and in this case, hopefully already has a 5.1 studio. Since I’m in post, not music, I realize musicians may not have 5.1 like I do. If you do though, I’ll say upfront that putting out a true Atmos mix can be fairly easy and cheap. All you really need, in addition to what you have, is the Dolby Atmos Production Suite, which is basically an application called Dolby Atmos Renderer. It’s $299, but you can give it a spin for 90 days for free to see if it works for you or not.

But, there’s an even cheaper way as of Oct. 14th, 2021 if you’re a Logic user, or if you want to move over to Logic for your Atmos music needs, Apple released Logic Pro 10.7 with built-in Dolby Atmos and Spatial Audio. Logic Pro is $100 less than just the Dolby Atmos Production Suite.

This article isn’t a tutorial on how to set up and use the Dolby Atmos Renderer with your DAW, but I’ll give some insights that I found along my journey. I’ll start by stating the basics of my setup. I did this in my 5.1 studio on a 2017 3.1 GHz Quad-Core i7 15” MacBook Pro with 16 GB of RAM using Pro Tools Ultimate which was connected to an Avid HD Native Thunderbolt and HD Omni. I also have some Atmos verbs that helped, but they aren’t necessary.

Before I describe my process and thoughts though, first a few notes on Atmos mixing in general for those who are unaware. Mixing in stereo is straightforward enough, and mixing in 5.1, of course, adds a center speaker, left and right surround speakers and a subwoofer. 7.1 basically spreads out the surrounds more, with left and right surrounds in addition to left and right rear surrounds. Dolby Atmos adds overhead speakers, usually either two or four in the home. If you have 5.1.2 or 7.1.2, you will have left and right overhead speakers. If you have 5.1.4 or 7.1.4, you’ll have left and right as well as front and rear overhead speakers.

When mixing in Atmos, you’re not necessarily caring about how many speakers there are...

Remember though that I said Dolby Atmos is a flexible surround format living in a 3D space, which is why it works well in 11.1.8, 7.1.4, 5.1.2 or AirPods. So when mixing in Atmos, you’re not necessarily caring about how many speakers there are, just about where in the space your elements are. So though you may listen with 20, 12, 8, 6 or even 2 speakers, there are 128 channels in the Dolby Renderer and those channels can be set up as a mixture of Beds and Objects.

Beds and Objects

Beds can be as large as 7.1.2, and you can choose to send all the tracks in your DAW to a single 7.1.2 Bed in the Dolby Atmos Renderer and call it a day, using that bed as a master bus in a sense. Or you can have multiple 7.1.2 Beds in the Renderer, using them more as submasters, guitar, percussion, vocals, bass, FX and on and on. You may notice though that if 7.1.2 is the maximum Bed size, then overhead physical speakers are limited to left and right, there are no front to rear overheads.

This is where Objects come into play. You can choose to output individual elements from your mix to an Object in the Renderer rather than to a Bed. By doing this, you can have that Object anywhere in the 3D space, like flying a sound above you from front to rear.

Let’s say you set up four Beds in the Renderer, each of which would be 10 channels wide, and an additional 10 Objects. That would mean you have used 50 of the 128 available channels. The more Beds and Objects you have though, the larger your master file will be, because a master file is a file that actually contains every Bed and Object separately. The master file then plays back your mix, while giving you the opportunity to export Re-renders, which you can think of as the stereo submaster archival stems you may be used to.

Atmos Beds

If your stereo mix file was made like a master Atmos file, there wouldn’t be a stereo mix file at all; there would be a single file containing all your printed submasters that would be combined when played back to give you your stereo mix. That’s why the more Beds and Objects you have, the larger your master Atmos file will be.

Atmos Master Files

There are different types of master Atmos files too, but the only one you’ll really need to be concerned with creating for streaming services is an ADM BWF, or ADM BWAV (Audio Definition Model Broadcast Wave Format). This will be a .wav file and can be created offline directly in Pro Tools as well as imported back into a Pro Tools session.

Another type of master file is a Dolby Audio Master File, or DAMF, which is actually a folder containing a few files for the Dolby Atmos Renderer to read. A DAMF needs to be created in real time through the Renderer and can be punched in and out of, which isn’t possible for an ADM BWF.

Lastly, there is an IMF IAB file (Interoperable Master Format Immersive Audio Bitstream), which is a multichannel .mxf file. You can think of this as the stereo mix file that you’re used to, but you may never really need to make one. It has a smaller file size because it acts as a single file where all your Beds and Objects are already combined. It can be created from the DAMF or the ADM BWF in the Renderer.

With that out of the way, it’s important to mention another key piece of information about Beds and Objects in relation to headphones, specifically Dolby’s binaural mode. You can set each of the 128 channels in Dolby’s metadata to sound in your face (Off) or Near or Mid or Far when listening with headphones in Dolby’s binaural mode. If you set a channel as Far, then in the Dolby binaural mix, that object will sound farther away with the help of built-in reverb and EQ changes, depending on the sounds location, based on how sound moves past your head and ears. If you check your mixes in Dolby’s binaural mode on headphones, and you should, be
aware though that the Spatial Audio version of your mix will not sound like this. Remember, Apple is doing its own thing and that includes not using this metadata, at least not the way Dolby’s binaural mode does.

Atmos Binaural

Prep Work

Before mixing my two songs in Atmos, I prepared my stereo mixes to Apple’s specs so I could streamline the Atmos process, and because Apple requires a stereo mix along with the Atmos mix. We probably all know about the loudness war, and my original mastered songs joined in with levels right around -7 LKFS and a peak of -0.1, though my True Peak wound up being above 0. Apple requires a True Peak of -1 though, so I remastered them. The song “Evermore” wound up at -8.2 LKFS with a -1.1 TP. The song “Behind the Eyes” wound up at -7.8 LKFS with a TP of -1.

My next step was prepping for the Atmos mixes by creating mixed stems of all the elements in my songs, pre-mastering, and using those as a starting point for Atmos because the Dolby Atmos Renderer running alongside Pro Tools can be very CPU intensive.

Pro Tools Zythem Session

The Atmos Renderer

Finally, it was time for Atmos. I started by creating a Pro Tools session that was 24 fps and 24 bit with a sampling rate of 48 kHz while making sure the start and end time of the Atmos file would exactly match the stereo version. After that, I checked Enabled from the Atmos tab under Setup/Peripherals in Pro Tools. I then selected my HD Native Thunderbolt hardware as the output of the Dolby Atmos Renderer and selected Dolby Audio Bridge as my input, while using Dolby Audio Bridge as the Playback Engine in Pro Tools, which linked the two. By doing this, each track in Pro Tools is output to a Bed or Object of my choice in the Dolby Atmos Renderer software which was set up to output to my 5.1 hardware.

Atmos Preferences Panel

 

Playback Engine

I then had to decide how to set up my Beds and Objects. I decided to keep it simple with a smaller master file size so I used only one 7.1.2 Bed with a few Objects for sound effects and some guitar fills. I knew the binaural metadata wouldn’t matter with Spacial Audio, but I still set it up to my liking, mostly by setting surrounds and overheads and some objects to be farther away, if listened to on headphones in Dolby’s binaural mode. I could have used the Renderer for this but chose to use the Dolby plugin called Dolby Atmos Binaural Settings, which allowed me to do this directly in Pro Tools instead.

Dolby Atmos Renderer

I knew I wanted the mix to be mostly straightforward, traditional, in front of you. It was raw thrash metal after all. There are no rules yet about what most people consider right or wrong when mixing music in Atmos, which is awesome, but I have to say that adding Spatial Audio into the equation meant going back and changing things several times. By playing around with the mix then checking the Spatial Audio version, I felt I answered my question as to why some older surround mixes sound subpar to me on Apple Music.

What I found is that Spatial Audio doesn’t seem to like mono sources that are panned front left/ right with no center channel. I’ve loved 5.1 music mixes over the years, but how people mixed those was all over the place, and often the center channel was hardly used. This may sound great in 5.1 or even on an Atmos setup, but in Spatial Audio, that lead vocal that didn’t make use of the center channel may sound low and distant compared to the Atmos mix and even compared to the Dolby Binaural mix. That was the exact issue I had with so many Spatial Audio tracks that I have listened to, and I couldn’t figure out why some songs sounded like that and some didn’t. Now I know why. You can pan left and you can pan right, but if you have that mono element coming from the left and right without using the center channel, you may run into this issue with Spatial Audio.

I found that using the center channel is a must for mono sources that you want to hear directly in front of you, whether hard panned to the center or spread out in some fashion between all three front speakers. By doing this, those elements no longer sounded low and distant in the Spatial Audio version of the mix. I decided that vocals, bass and drums would be varying degrees of left, right and center, with cymbals elevated slightly into the overheads for some airy spread. Guitars were mostly panned left/right with some guitars bleeding into the center and others bleeding into the side surrounds. Some guitar fills had automated pans around the room as Objects. I kept some of the original reverbs, and mostly kept them left/right while accenting or replacing certain reverbs with true 7.1.2 reverbs to fill out the whole space. Some vocal effects and sound effects were spread to the surrounds in a way that felt right to me for the song. An example is that the song Evermore starts with a dark wind and church bells. In stereo, they're both in front of you, but in the Atmos mix the raw wind was mostly placed in the overheads, and the bells were panned mostly to the right rear, then 7.1.2 reverbs were used on each to fill in the rest of the space. Both those effects were output as Objects and set to feel distant (Far) in the metadata, though again this wouldn’t matter for Spatial Audio.

After hearing how Spatial Audio was working, I created a process to check each revision. I would mix in Atmos while monitoring in 5.1, check in stereo from the Dolby Renderer and change the Renderer to its binaural mode to listen to Dolby’s version of a headphone mix. Then I’d bounce an ADM BWF offline from Pro Tools, import it into the Renderer then export an mp4 that I would import to my iPhone. I would then listen to the mp4 with AirPods and Beats in Spatial Audio as well as listen to the Atmos mix in my living room which is set up with overhead speakers.

Bounce mix

Logic Pro 10.7

That process may not even be needed if you’re using the new Logic Pro 10.7 though. What’s yet to be determined is if Logic Pro is processing the Dolby binaural output with Spatial Audio or not. I would think it should, and this would be a game changer, but as of this writing, I can’t find that specific information. In fact, Apple usually capitalizes Spatial Audio, but in the wonderful Logic Pro 10.7 User Guide, it’s referred to as spatial audio mixing with Dolby Atmos. If Apple is not baking in Spatial Audio processing, be aware that the Dolby Atmos binaural listening mode may sound different than what you hear in Apple Music with Spatial Audio.

Logic Pro's 3D Panners and Atmos Plugin

It may be unfortunate, but I believe most consumers will not hear these mixes in Atmos, especially with actual overheads or even surround speakers. Most may only hear the Spatial Audio version or perhaps the Dolby binaural version on other services. Because of this, I felt listening on AirPods was a very important part of the process.

Apple’s Spatial Audio on headphones does not sound like the Dolby Atmos Renderer’s binaural mode. Dolby’s binaural mode is very nice, well-balanced, and you don’t lose those mono elements I talked about, but this version actually sounds narrow compared to the wide soundstage of Spatial Audio, which makes me think Apple named it perfectly. To me, Dolby’s version sounds like a 2.0 fold down of the Atmos mix, while subtly accenting surround elements with slight reverb for depth and slight EQ for location, while Spatial Audio processing goes for an almost over-the-top, wide soundstage to hopefully wow the listener. Neither of these actually sounds like a true surround system though where elements are over your head or behind you, because we’re still talking two speakers. They both have their negatives and benefits, but I think I prefer the experience of Spatial Audio, though perhaps not Apple’s Head Tracking for music. Spatial Audio Head Tracking can be fun when turning your head to hear what is in the surrounds of a mix, but I usually keep it turned off.

Because of all the sonic differences between Atmos, Dolby binaural and Spatial Audio, I had to find a balance between them all. This isn’t unlike what I do in post or what many do in music when checking their mixes on different speaker setups. Some elements in the mix might be a touch hot on one format but a touch low on the other, and in this case, a little less wide here or a little wider there.

Specifications

While finding that balance, I was also dealing with levels of course. I mentioned my loudness war levels above, but Apple Music’s specs for Atmos states that songs cannot be louder than -18 LKFS with a max True Peak of -1. So the final Atmos level of “Evermore” is -18.1 LKFS with a -3.7 TP and “Behind the Eyes” is -18.3 LKFS with a -4.5 TP. When those songs are converted to a binaural format though, it changes. The final Dolby binaural level of “Evermore” is -16.3 LKFS with a -1.7 TP and “Behind the Eyes” is -14.2 LKFS with a -0.1 TP. That’s close to clipping so I’m surprised Apple allows Atmos mixes to have a -1 TP, which makes me wonder if a limiter is built into Spatial Audio. If an Atmos mix reaches a -1 TP, then surely the binaural/ Spatial Audio version is above zero. If I needed to, I could have put my own limiter on, but it wasn’t needed so why mess with the limited dynamic range my thrash metal masterpieces have to begin with. This made me realize though why Apple requires those LKFS levels. Plus it can end the loudness war.

The problem with ending the loudness war though is that songs now vary in level even more with Atmos added to the equation than they did before, which was already not ideal. If you’ve played Spatial Audio tracks before or after stereo tracks, you’ll have noticed that Spatial Audio tracks are comparatively very low in level. Apple’s way of solving this is with Sound Check. I never bothered with Sound Check before Atmos, but now I use it. I also now think that because of Atmos, Sound Check should be on by default.

Sound Check attempts to use the metadata of tracks (or whole albums) to level things out to around -14 LKFS. Sound Check acts like your volume controls so it’s not introducing compression or limiting. It is lowering the overall level of louder tracks. So a -14.2 LKFS Spatial Audio version of one of my songs is about 7 dB lower than my stereo version. That explains why those not using Apple’s Sound Check feel the Atmos mixes (via Spatial Audio) are so much lower than most everything else we’re used to. Plus, it’s known that louder music is often perceived to sound better, so that may even sway some people to instantly think the stereo version is better. In many cases, the stereo version is better, but I think moving forward, that line may get blurred. With Sound Check on though, there’s little to no difference at times between song levels, even from stereo to Spatial Audio, and especially within the same genre of music. I say that because Sound Check does its job as best it can, but I feel it sometimes falls apart when mixing genres.

As I got to the end of my journey and figured out the answers to my initial questions, it was time to export my master Atmos files. I could have done this real time through the Dolby Atmos Renderer, but instead I exported the ADM BWF offline in Pro Tools. To check this file, I opened it in the Renderer and also imported it into a new Pro Tools session.

For those interested in file sizes as a comparison, “Evermore” is 5:30 and the stereo master WAV is 87.7 MB, the Atmos ADM BWF is 761.3 MB and the mp4 of the Atmos file is 43.2 MB. “Behind the Eyes” is 4:27 and the stereo master WAV is 70.9 MB, the Atmos ADM BWF is 615.2 MB and the mp4 of the Atmos file is 34.9 MB.

Sharing the Final Songs Online

Once I felt that my metal masterpieces were acceptable, it was time to choose a distributor that worked with Apple Music and Dolby Atmos. I tried AvidPlay and DistroKid, and ultimately went with DistroKid for the addition of synced lyrics in Apple Music.

Regardless of my final product, because I’m not professing they’re great mixes or recordings to begin with, it was a fun process and a good learning experience for how Spatial Audio differs from Dolby’s binaural mode. My wish is that, unlike a couple decades ago, surround music becomes mainstream thanks to the flexibility of Dolby’s Atmos format along with the help of Apple’s Spatial Audio, both of which impress me for what they’ve accomplished, not only for the listener, but also for the creativity that artists will bring to it. Hopefully artists that want to release in Atmos take advantage of exporting mp4s from the Dolby Atmos Renderer though so they can hear their tracks in Spatial Audio during the mix process. I now wouldn’t be surprised if some day soon, an artist releases an album where the stereo tracks supplied are just a 2.0 fold down of the Atmos mixes. I also wouldn’t be surprised if one day all music is simply released in surround without separate stereo tracks at all, just like when stereo replaced mono, thanks to a flexible format. I can’t wait to hear what everyone comes up with.

Synthesis 101
The Filter
by Bob Moog Foundation

"Billy Gardner started his career in 1992 at The Power Station after graduation from the Institute of Audio Research. There, Billy worked in post and music, earning a platinum album for his work on Meat Loaf’s Bat Out of Hell II: Back into Hell before moving to Betelgeuse in 1994 then joining PostWorks New York in 2005. Billy is a two-t..." Read More

Discussion

Thor
Thank you for this article - very informative and extremely useful to anyone who is interested in this amazing development in audio mixing.
I've been using atmos for music for a while now and it's definitely a brave new world out there. Lots of potential as well as lots of weird limitations. One thing that I've thought of is that seeing 80% of music is consumed on headphones then surely the binaural mix is really the most important thing to consider when finalising a mix? Apple's Spacial Audio format throws another complication into the already confusing mix. To me, it makes sense to focus on binaural headphone aspect of atmos mixing. What are your thoughts on this?

Also, can you think of a solution of how to play an atmos encoded mp4 direct from your computer, that can be monitored on normal headphones?

Want to join the discussion?

Create an account or login to get started!