Phase Vocoder Part 3
Manipulating sound and resynthesising to create effects
This is the final lecture of a three-part series on the phase vocoder, a powerful tool for manipulating audio in the frequency domain. Where Lecture 19 explained how to estimate frequency components from the phase of FFT bins, this lecture now explains how to manipulate and resynthesise those components, producing a pitch shifter and other effects.
Table of contents
- What you’ll learn in this lecture
- What you’ll make in this lecture
- Code examples
- Recommended parts
- Additional references
Lecture 20: Phase Vocoder Part 3
What you’ll learn in this lecture
- Modifying signals in the frequency domain
- Converting frequency to phase
- Analysis and synthesis windows
What you’ll make in this lecture
- Pitch shifting, robotisation and whisperisation effects
Code examples
fft-robotisation: Zeroes the phase of each FFT bin at each hop to produce regularly spaced frequencies, leading to a robot-like effect when applied to a voice. You will modify the code to add analysis and synthesis windows, then make a version which produces a whisperisation effect instead.
fft-pitch-shifter: The capstone project of the phase vocoder lecture series. Convert from phase to frequency and back again, scaling the frequencies in the middle to shift the pitch of the sound without changing its timing.
fft-robotisation-v2: A different and more flexible approach to the robotisation effect, based on the principles used in the pitch shifter. You will implement code for rounding each frequency to the nearest multiple of a desired fundamental.
Recommended parts
Running the examples in this course assumes that you have a Bela Starter Kit or Bela Mini Starter Kit.
No external components are required for this lecture.
Additional references
- A very helpful tutorial on the phase vocoder is Pitch Shifting Using the Fourier Transform by Stephan M. Bernsee. The approach to frequency detection in this lecture is based in part on this resource.
- Table of window function details from VR University
Additionally, here are some textbooks and other resources related to the Fast Fourier Transform and spectral audio processing:
- Alan V. Oppenheim and Ronald W. Shafer, Discrete-Time Signal Processing, 3rd edition. Publisher link. See also the same class on MIT OpenCourseware.
- Julius O. Smith, Mathematics of the Discrete Fourier Transform (DFT) (free online textbook).
- Julius O. Smith, Spectral Audio Signal Processing (free online textbook).
- Joshua D. Reiss and Andrew McPherson, Audio Effects: Theory, Implementation and Application. Publisher link.
- Richard Boulanger and Victor Lazzarini, editors, The Audio Programming Book. Publisher link
- A visual introduction to the Fourier transform on YouTube from 3Blue1Brown
- Divide and Conquer: the FFT algorithm on YouTube from MIT OpenCourseWare (a mathematical explainer)