Pro Tools & Melodyne: Pitch-Shifting, But Better

Pro Tools now has a realtime "elastic audio" bundled in the form of Celemony's industry-leading Melodyne. Joe Albano puts its pitch shifting algorithms to the test.  

Way back when, Pro Tools was one of the first DAWs (after Ableton Live) to incorporate modern realtime time-shifting capabilities—their Elastic Audio feature continues to provide high-quality audio time-shifting for Pro Tools users, including both automatic functions like following Session Tempo and time correction (audio Quantization), and manual Warping for creative applications. Other DAWs quickly followed suit, like Logic, with their Flex Time implementation. But Logic also added realtime pitch-shifting, in the form of Flex Pitch, which presents notes in an audio file as bars in a familiar MIDI-style Piano Roll editor display, allowing for corrective editing like pitch quantization (auto-tuning) and creative editing, like manual editing of melodies or harmony creation. However, for a long time Pro Tools didn’t incorporate a realtime pitch-shifting equivalent to Elastic Audio—until now.

Enter Melodyne

With Pro Tools 2020, Avid began bundling Celemony’s industry-leading tine & pitch-shifting plug-in, Melodyne—or more specifically, Melodyne Essential, the entry-level version. This has finally brought modern realtime pitch-shifting technology to Pro Tools, and it fills a longstanding gap in Pro Tools’ built-in feature set. Of course Pro Tools users have always turned to Melodyne for pitch-shifting, but now they don’t have to spend the extra money purchasing it as a third-party solution for basic pitch-processing needs. So what does Melodyne bring to the party? Let’s take a brief look—since the built-in Elastic Audio feature already has time-shifting well in hand, I’ll focus on Melodyne’s pitch-processing capabilities.

Fig 1

Previous Pitch Processing

Prior to bundling Melodyne (Essential), Pro Tools did, of course, have some pitch-processing capabilities, but they were mostly non-realtime tools based on older pitch-(and time)-shifting algorithms. There’s the realtime Pitch II (AAX) plug-in, but that can only pitch-shift an entire track, and is more suited to small overall tuning applications (like tweaking a track that’s consistently a few cents sharp or flat) or processing instruments and special effects than performing note-by-note pitch-shifting on more demanding audio like a lead vocal—the kind of material Melodyne excels at—where Pitch II tends to produce noticeable artifacts like the infamous Chipmunk Effect.

Fig 2

Non-realtime AudioSuite plug-in have always offered pitch-shifting options. The best option was always X-Form, Pro Tools’ own high-end pitch-shifting algorithm.

Fig 3

This is capable of much better pitch-shifting of demanding audio like a lead vocal, because it includes an option for Formant Correction, to offset the Chipmunk Effect (a.k.a. Munchkinization) that otherwise occurs when pitch-shifting audio (especially vocals) by more than a couple of semitones. In fact, since modern high-quality pitch-shifting algorithms include some degree of formant correction—it’s a key component of their superior pitch-transposing capabilities—this might be the time to drop in a quick sidebar explaining that technical concept.

Formants

A Formant is a fixed resonance in an acoustic instrument or—especially—a voice. A typical voice has several of these fixed formants/resonances, based on the size and shape of the various resonating cavities like the head, nasal passages, chest, throat, larynx, etc. These resonant frequencies are fixed, so they don’t change when different notes (different frequencies) are produced—they provide a consistent, characteristic tone (timbre) for that voice or instrument (in a nutshell, they’re what makes you sound like you).

But when audio is digitally pitch-shifted, not only do the frequencies of the notes change, but the frequencies of those fixed frmants change as well—they’re shifted up or down, which—in the case of a voice—is the equivalent of the person’s head and chest getting smaller when they sing higher notes and larger when they sing lower notes. That’s why pitch-shifting to a higher pitch makes a human voice sound like a small (Chipmunk) voice—it’s a familiar and distinctly unnatural tonal effect.

High-Quality pitch-shifting algorithms compensate for the unwanted shifting of normally-fixed formant frequencies when pitch-shifting, maintaining the fixed formants for a more consistently natural sound. The issue is most noticeable with voice—one of the prime targets for pitch-shifting applications—and while no pitch-shifting tools can transpose a vocal note too far (more than a musical fifth or sixth, say) with complete freedom from artifacts, modern tools like Melodyne do a far better job of compensating for formant shifts and preserving natural tone.

Elastic Pitch

Prior to bundling Melodyne, Pro Tools implemented something they touted as “Elastic Pitch”, but if that title led anyone to believe it did for pitch-shifting what Elastic Audio does for time-shifting, then it was kind of a misnomer. So-called Elastic Pitch does utilize Elastic Audio, but it’s actually just a realtime way of doing what AudioSuite pitch-shifting plug-ins do. While it doesn’t require bouncing to a new audiofile like AudioSuite processing does, if you want to dial up a particular sequence of pitch-shifted notes, you still have to cut up a Clip into pieces—note-by-note—and dial up the desired pitch for each, leaving a messy chopped-up track. And it doesn’t seem to include formant correction, so even shifts of two or three semitones sound noticeably artificial (Chipmunk’d).

That said, Elastic Pitch was probably the easiest way to perform a creative pitch-shifting edit like generating a vocal harmony part. On an Elastic Audio-enabled track, you’d cut a copy of, say, a lead vocal Clip into individual notes, right-click each new Clip, select Elastic Properties, and set the pitch-transposition interval for each note.

Fig 4

It’s easy enough to use, but the sound quality of any pitch transposition that’s more than a few cents of pitch correction is not up the level of Melodyne or other high-end pitch tools.

Melodyne Methodology

Melodyne, of course, solves all those issues. As the industry leader in time & pitch-shifting, its algorithms are possibly the best around, and even though the free bundled version is Melodyne Essential—the entry-level version—the feature set and editing display provide for most pitch-processing needs, with a very nice extra feature, to boot.

Since Melodyne is a plug-in and not a built-in editing feature like Elastic Audio, it’s operation is a bit different. You instantiate the Melodyne plug-in on the track you want to process—ideally in the first (top) insert slot—and editing is performed in a piano-roll display within the Melodyne plug-in window. But to work with audio in Melodyne you have to “Transfer” the audio from the track into a new audiofile that Melodyne creates and places in a dedicated Melodyne folder inside the Session Folder. (Melodyne also offers another way for this to be done, but this is how it’s implemented in Pro Tools, at least for now).

The Transfer is easy to do—you just activate the Transfer button in the plug-in and play the track, and the audio will be analyzed, and appear in the Melodyne editing grid as individual notes in the piano roll. The notes are represented as stylized “blobs” (their term), and you can both apply automatic time/pitch correction and perform manual note-by-note edits.

Fig 5

Typical applications include automatic pitch quantizing (auto-tuning) or time correction (quantization), or manual adjustment (by dragging) of the pitch and timing of individual notes. For manual editing, high-end versions of Melodyne have more tools, but here you can adjust pitch (in semitones or cents), tweak the length of individual notes, and divide longer notes into two or more separate editable notes. Manual editing lends itself to creative applications like altering a bass line or vocal melody, or creating a vocal harmony part by transposing pitches in a copy of a lead vocal track.

Essential Limitations & Quirks

So what are the limitations of the bundled entry-level Melodyne Essential? Well, you can’t edit performance gestures like pitch slides and vibrato. which can be edited in other versions of Melodyne. And, of course, you can only pitch-shift monophonic audio—but to be fair, that’s a limitation of all modern pitch-editors except the top-of-the-line Melodyne Editor, the only one to offer polyphonic pitch-editing (via their specialized DNA feature).

A quirk of working in Melodyne within Pro Tools is that once you’ve transferred the audio from the original Clip in the track into the piano roll editor grid within the Melodyne plug-in window, you’re no linger hearing the audio in the original Clip. That can at times be confusing—though it may look like the audio is playing from the original Clip, it’s actually playing from the rendered Melodyne audiofile, and if you mute the original Clip in the track lane you’ll still hear the Transferred audio play as long as the Melodyne plug-in is active.

Bonus Feature

The bundling of Melodyne with Pro Tools provides an extra feature that’s not included in Melodyne Essential outside of Pro Tools—Audio-to MIDI conversion. This features leverages Melodyne’s pitch-detection capabilites to offer the option of converting an audio track to a MIDI sequence. This could be used on any audio, but I find it especially useful with bass, especially with an audio bass track where you might want to swap out the sound of, say, an electric bass with a different bass (like a fretless or upright bass)—the kind of creative tweak you can’t do with regular audio processing.

It’s very easy to use—you just right-click a Track name or Clip and select “Copy Audio as MIDI”. There are a few simple options (like percussive or melodic algorithms and what to do with the original track), and then Pro Tools creates a new track with the converted MIDI notes—all you have to do is insert an appropriate bass Instrument (and possibly make a few tweaks to the velocity response), and you’re good to go.

Fig 6

This this is implemented in Pro Tools itself, and it’s a nice bonus over and above the regular Melodyne Essential feature set.

Wrap-up

The bundling of Melodyne (Essential) is a long-overdue but very welcome addition to Pro Tools’ set of included features—anyone who’s not already familiar with Melodyne will definitely want to take some time to put it through its paces.

 

Joe is a musician, engineer, and producer in NYC. Over the years, as a small studio operator and freelance engineer, he's made recordings of all types from music & album production to v/o & post. He's also taught all aspects of recording and music technology at several NY audio schools, and has been writing articles for Recording magaz... Read More

Discussion

Want to join the discussion?

Create an account or login to get started!