1.46 Fourier transform: frequencies in sound and signals: study notes

Learn AI by Story

1. The idea in one sentence

Any signal that wiggles over time, such as a sound wave, can be rebuilt by adding simple sine waves of different frequencies, and the Fourier transform is the tool that tells you which frequencies are present and how strong each one is.

2. The story from the video

Rohan is building a guitar tuner app. His phone microphone gives him a long list of numbers: the air pressure measured thousands of times every second. Plotted over 20 milliseconds, the wave looks like a messy squiggle. He cannot read a note name from it by eye.

Rohan's friend explains that the squiggle is really two clean sine waves added together. One wave repeats 220 times per second. The other repeats 330 times per second and is half as tall. In the 20 ms picture, the 220 Hz wave completes 4.4 cycles and the 330 Hz wave completes 6.6 cycles. (The signal is illustrative: x(t) = sin(2π·220t) + 0.5 sin(2π·330t).)

The app now runs a Fourier transform on one second of sound, recorded at 8,000 samples per second. The output is a new graph. Instead of "time" along the bottom, it has "frequency". The graph is flat, apart from two sharp spikes: one at 220 Hz with height 1, and one at 330 Hz with height ½. The tuner simply reads off the tallest spike and says which note the string is closest to.

Rohan learns three more things. One second of audio at 8,000 samples per second gives frequency bins that are 1 Hz apart. The highest frequency the app can see is half the sampling rate, 4,000 Hz (the Nyquist frequency). CD audio, at 44,100 samples per second, can represent up to 22,050 Hz. A clever shortcut called the Fast Fourier Transform (FFT) makes all this about 50,000 times cheaper than the direct method for a million samples.

Finally, the video shows that if you slide a short window along a recording and take a Fourier transform in each window, you get a spectrogram, a picture of frequency against time. Speech recognisers such as Whisper start from exactly that picture. The same maths also lets computers filter sound and speed up convolutions.

3. Key definitions (glossary)

Signal: a quantity that changes over time (or space), such as air pressure at a microphone. Plain English: a list of measurements that wiggles.
Sine wave: the smooth repeating curve sin(2πft), with frequency f and some amplitude. Plain English: the purest possible tone.
Frequency (f): the number of complete cycles per second, measured in hertz (Hz). Plain English: how fast the wave wiggles, which our ears hear as pitch.
Amplitude: the height of a wave from the middle to the peak. Plain English: how loud that one tone is.
Phase: how far along its cycle a wave is at time zero, measured as an angle. Plain English: where the wave starts.
Time domain: describing a signal by its value at each moment. Plain English: the wiggly line you see on an oscilloscope.
Frequency domain: describing the same signal by how much of each frequency it contains. Plain English: the list of ingredients instead of the finished dish.
Fourier analysis: expressing a function as a sum of periodic components (NumPy docs). Plain English: breaking a sound into pure tones.
Continuous Fourier transform: X(f) = ∫ x(t) e^(−2πift) dt. Plain English: for each frequency f, measure how well the signal lines up with a spinning arrow at that frequency.
Discrete Fourier transform (DFT): for n samples a_m, A_k = Σ a_m e^(−2πi mk/n), summed over m = 0 to n−1. Plain English: the same idea for a finite list of numbers.
Fast Fourier Transform (FFT): an algorithm that computes the DFT in about N log₂N steps instead of N². Plain English: the same answer, hugely faster.
Spectrum: the magnitudes (and phases) of the Fourier coefficients, plotted against frequency. Plain English: the "spikes graph".
Magnitude and phase: for a complex number A_k, the magnitude is its length and the phase is its angle. Plain English: how much, and where in the cycle.
Sampling rate (fs): the number of measurements per second. Plain English: how often the microphone is read.
Frequency resolution: fs/N, the gap between neighbouring frequency bins for N samples. Plain English: how finely you can tell two pitches apart.
Nyquist frequency: fs/2, the highest frequency a sampled signal can represent. Plain English: the ceiling set by how fast you sample.
Aliasing (background): a frequency above Nyquist showing up as a false lower one. Plain English: fast wiggles disguised as slow ones.
Spectrogram: a short-time Fourier transform shown as frequency against time. Plain English: a picture of sound where colour shows loudness at each pitch and moment.
Mel scale: a pitch scale that spaces frequencies the way human hearing does, giving finer detail at low pitches. Plain English: a ruler tuned to ears (background).
Convolution theorem: convolution in time equals point-wise multiplication in frequency. Plain English: a slow sliding-window operation becomes simple multiplication.

4. Detailed explanation, from basics to the full picture

Step 1: A signal is a list of numbers

A microphone measures air pressure many times per second. Each measurement is called a sample. If you take fs = 8,000 samples per second for one second, you have N = 8,000 numbers. This list is the time-domain view of the sound.

Step 2: Pure tones are the building blocks

A sine wave sin(2πft) repeats f times each second. A 220 Hz tone repeats 220 times per second. Two tones added together make a more complex wave. Three tones make a more complex one still. A real guitar string makes a main tone plus overtones at higher frequencies. Fourier's big idea is that the building blocks go the other way too: any reasonable signal can be written as a sum of such waves. NumPy's documentation states this as expressing a function as a sum of periodic components.

Step 3: Asking "how much of frequency f is in here?"

To test for a frequency f, imagine an arrow that spins around a circle f times per second. Multiply the signal by that spinning arrow, and add up the result over time. If the signal contains a wave at frequency f, the products line up and the total grows large. If it does not, the positive and negative pieces cancel and the total is near zero. This is exactly what the formula does:

X(f) = ∫ x(t) e^(−2πift) dt

The symbol ∫ means "add up over all time". The term e^(−2πift) is the spinning arrow. It is a complex number: e^(−iθ) = cos θ − i sin θ, so it carries both a cosine and a sine part. That is why the answer X(f) is complex.

Step 4: Magnitude and phase

The complex number X(f) has a length, called its magnitude, and an angle, called its phase. The magnitude tells you how much of frequency f the signal contains. The angle tells you where in its cycle that wave starts. A tuner only needs magnitude. To rebuild the original sound exactly, you need both.

Step 5: The discrete version for computers

A computer has a finite list a₀, a₁, ..., a_{n−1}, not a continuous curve. The discrete Fourier transform replaces the integral with a sum:

A_k = Σ a_m e^(−2πi mk/n), with m = 0, 1, ..., n−1

Each output A_k answers the question "how much of frequency k (counted in cycles per whole window) is in this list?" The NumPy documentation gives exactly this definition for numpy.fft.

Step 6: Turning bin numbers into hertz

If the window lasts T seconds and holds N samples, bin k is the frequency k/T hertz. Since T = N/fs, bin k is k × fs/N Hz. So the gap between bins is fs/N. For fs = 8,000 and N = 8,000, the gap is 1 Hz, and bin 220 is exactly 220 Hz. The 220 Hz and 330 Hz tones both fit a whole number of cycles in the one-second window, so each lands cleanly on a bin and the picture shows razor-sharp spikes.

Step 7: Scaling the result

The raw FFT numbers are large. For a real signal, multiplying the magnitudes by 2/N turns each spike's height into the amplitude of the original sine wave. That is how the app reads 1.0 at 220 Hz and 0.5 at 330 Hz (verified with numpy).

Step 8: The Nyquist limit

You cannot see frequencies above fs/2. With fs = 8,000, the ceiling is 4,000 Hz. With CD audio at 44,100 samples per second, it is 22,050 Hz, which just covers the range of human hearing (the CD format is two-channel 16-bit PCM at 44.1 kHz, per Wikipedia).

Step 9: Why the FFT matters

Computing the DFT directly means N outputs, each a sum of N terms, so about N² multiplications. The FFT organises the work to need only about N log₂N. The idea is old: NumPy's documentation notes it was known to Gauss in 1805 and was brought to light in its current form by Cooley and Tukey in 1965.

Step 10: Windows and spectrograms

A single Fourier transform of a whole song tells you which pitches occur somewhere in the song, but not when. To keep timing, cut the audio into short overlapping windows, run an FFT on each, and stack the results side by side. That picture is a spectrogram, also called a short-time Fourier transform. It is the standard way to feed sound to neural networks.

Step 11: The convolution theorem

Convolution slides a small pattern (a kernel) along a signal, multiplying and adding at every position. The convolution theorem says you can get the same result by taking the Fourier transform of both the input and the kernel, multiplying them point by point, and taking the inverse transform. Goodfellow, Bengio and Courville describe this in the section on efficient convolution algorithms and note it can be faster for some problem sizes.

5. Everything the video skipped

Variants

Inverse transform: the Fourier transform can be run backwards to rebuild the signal from its frequencies, with nothing lost.
Real-signal FFT: for real-valued signals such as sound, the second half of the spectrum mirrors the first, so libraries such as numpy offer rfft, which returns only the useful half (background).
Discrete cosine transform (background): a close relative that uses only cosines. JPEG image compression uses it.
Short-time Fourier transform: the windowed version behind every spectrogram.

Formulas in simple words

Frequency resolution: Δf = fs/N. More seconds of audio means finer detail.
Nyquist frequency: fs/2.
Window length in samples: N = (window in seconds) × fs.
Euler's formula: e^(iθ) = cos θ + i sin θ, which is why the spinning arrow contains both cosine and sine.

Limitations and pitfalls (background)

Aliasing: a tone above the Nyquist frequency is not lost; it shows up as a false lower tone. Recording systems filter out very high frequencies before sampling to avoid this.
Spectral leakage: if a tone does not fit a whole number of cycles into the window, its energy smears across neighbouring bins. Tapering the window edges with a window function such as a Hann window reduces the smear.
Time versus frequency trade-off: a short window tells you when something happens but gives coarse frequency detail; a long window gives fine frequency detail but blurs timing. You cannot have both.

Background and history

Fourier analysis expresses a function as a sum of periodic components. The fast algorithm was known to Gauss in 1805 and brought to light in its current form by Cooley and Tukey in 1965 (NumPy docs). Concert pitch A4 is 440 Hz (background), so a tuner app is mostly asking "is the strongest spike near 440 Hz, and is it a little above or below?"

Links on the roadmap

This video follows 1.45 on mutual information and Jensen's inequality. The next video, 1.47, leaves signals behind and starts graph theory with nodes, edges and adjacency. Spectrograms return later when we meet speech models, and convolutions connect back to the convolutional networks idea.

6. Worked examples with real numbers

Example A: Finding 220 Hz and 330 Hz in a signal

Setup: x(t) = sin(2π·220t) + 0.5 sin(2π·330t), sampled at fs = 8,000 samples per second for 1 second, so N = 8,000. (Illustrative signal; checked with numpy.)

Count cycles in the 20 ms picture: 220 × 0.020 = 4.4 cycles for the first tone.
Count cycles for the second tone: 330 × 0.020 = 6.6 cycles.
Compute the frequency resolution: fs/N = 8,000 ÷ 8,000 = 1 Hz, so bin k is k Hz.
Run the FFT and scale the magnitudes by 2/N.
Read bin 220: height 1.000 (numpy gives 0.9999999999999993, which is 1 up to rounding). Read bin 330: height 0.500.
Check a neighbour: bin 221 has height about 5 × 10⁻¹⁵, which is zero up to rounding.

Result: two spikes, 220 Hz with amplitude 1 and 330 Hz with amplitude ½, and nothing else. The app recovers exactly the recipe that made the signal.

Example B: Resolution and the Nyquist limit

Setup: how finely and how high can we see, for different recordings?

One second at 8,000 samples per second: N = 8,000, resolution = 8,000 ÷ 8,000 = 1 Hz.
Nyquist frequency for that recording: 8,000 ÷ 2 = 4,000 Hz.
Now use only 0.1 second: N = 0.1 × 8,000 = 800 samples. Resolution = 8,000 ÷ 800 = 10 Hz. The 220 Hz and 330 Hz tones are still 110 Hz apart, so they remain easy to separate, but two tones 5 Hz apart would blur into one.
CD audio: 44,100 ÷ 2 = 22,050 Hz.
Whisper audio at 16,000 Hz: Nyquist is 16,000 ÷ 2 = 8,000 Hz (arithmetic from the stated sampling rate).

Result: longer windows give finer frequency detail (1 Hz for one second versus 10 Hz for a tenth of a second), and the highest visible frequency is always half the sampling rate: 4,000 Hz, 22,050 Hz and 8,000 Hz in the three cases above.

Example C: Why the FFT is fast

Setup: a clip with N = 2²⁰ = 1,048,576 samples. Compare the direct DFT cost, about N², with the FFT cost, about N log₂N.

Direct cost: N² = (2²⁰)² = 2⁴⁰ ≈ 1.1 × 10¹² operations.
Take the logarithm: log₂N = log₂(2²⁰) = 20.
FFT cost: N log₂N = 1,048,576 × 20 = 20,971,520 ≈ 2.1 × 10⁷ operations.
Ratio: 1.1 × 10¹² ÷ 2.1 × 10⁷ = N ÷ 20 = 1,048,576 ÷ 20 ≈ 52,429.

Result: the FFT needs roughly 50,000 times fewer operations for a million samples. At a billion simple operations per second, 10¹² operations would take about 17 minutes, while 2 × 10⁷ takes about 0.02 seconds (background illustration of the ratio).

Example D: Whisper window sizes in samples

Setup: Whisper re-samples all audio to 16,000 Hz and computes an 80-channel log-magnitude Mel spectrogram on 25-millisecond windows with a stride of 10 milliseconds (Radford et al., 2022).

Window length in samples: 0.025 s × 16,000 = 400 samples.
Stride in samples: 0.010 s × 16,000 = 160 samples.
Frames per second: 1 ÷ 0.010 = 100 spectrogram columns every second.
Overlap between neighbouring windows: 400 − 160 = 240 samples.
Frequency resolution of one window: 16,000 ÷ 400 = 40 Hz (arithmetic, background reasoning from fs/N).

Result: each Whisper window holds 400 samples, slides forward 160 samples at a time, and produces 100 columns of 80 numbers per second of audio.

Example E: A tiny 4-point DFT you can do by hand

Setup: take a = (1, 2, 3, 4) and n = 4. The spinning arrow for bin k at sample m is e^(−2πi mk/4), which equals (−i)^(mk) because e^(−2πi/4) = e^(−iπ/2) = −i.

Bin 0 has all arrows equal to 1: A₀ = 1 + 2 + 3 + 4 = 10. This is the sum, the "zero frequency" or average part.
Bin 1 uses the arrows 1, −i, −1, i: A₁ = 1 + 2(−i) + 3(−1) + 4(i) = 1 − 2i − 3 + 4i = −2 + 2i.
Bin 2 uses the arrows 1, −1, 1, −1: A₂ = 1 − 2 + 3 − 4 = −2.
Bin 3 uses the arrows 1, i, −1, −i: A₃ = 1 + 2i − 3 − 4i = −2 − 2i.
Check with numpy: np.fft.fft([1, 2, 3, 4]) returns [10, −2+2i, −2, −2−2i], matching all four values.
Check the magnitude of bin 1: √((−2)² + 2²) = √8 ≈ 2.83.

Result: A = (10, −2+2i, −2, −2−2i), confirmed by numpy. For a second check, a = (1, 0, −1, 0) gives A = (0, 2, 0, 2): a wave that completes one cycle in the window shows up in bins 1 and 3 only (bin 3 is the mirror image of bin 1 for real signals).

7. Comparison tables

Time domain versus frequency domain

Question	Time domain	Frequency domain	Tool
What does it show?	Value at each moment	Strength of each frequency	FFT converts between them
Best for	When something happens	Which pitches are present	Spectrogram has both
Example	Wiggly line over 20 ms	Spikes at 220 Hz and 330 Hz	Same information, two views

Direct DFT versus FFT for N = 2²⁰

Method	Cost	Operations	Relative
Direct DFT	N²	about 1.1 × 10¹²	1×
FFT	N log₂N	about 2.1 × 10⁷	about 50,000× fewer

Sampling rates and their Nyquist limits

Setting	fs	Nyquist fs/2	Source
Rohan's tuner	8,000 Hz	4,000 Hz	Video example
CD audio	44,100 Hz	22,050 Hz	Wikipedia
Whisper input	16,000 Hz	8,000 Hz	Radford et al.

Window length trade-off (fs = 8,000 Hz)

Window	Samples N	Resolution fs/N	Timing detail
1 s	8,000	1 Hz	Coarse
0.1 s	800	10 Hz	Medium
25 ms	200	40 Hz	Fine

8. Common mistakes and myths

Myth: the Fourier transform only works on perfect sine waves. It works on any reasonable signal; sine waves are just the building blocks.
Myth: the spectrum tells you when each tone happened. A single transform of a whole recording loses timing. That is why we use windows and spectrograms.
Mistake: forgetting that bins are numbers, not hertz. Bin k is k × fs/N Hz. Always convert.
Mistake: expecting frequencies above fs/2. They cannot be represented, and if they sneak in they alias into false lower tones (background).
Mistake: using a window too short for the job. With a 25 ms window at 8,000 samples per second the bins are 40 Hz apart, too coarse to tell a tuner that a string is 3 Hz flat.
Mistake: ignoring leakage. Tones that do not fit the window exactly smear across bins; the clean spikes in the example only happen because 220 and 330 are whole numbers of cycles per second (background).
Myth: the FFT is a different transform. It gives exactly the same answer as the DFT, only faster.
Mistake: reading the raw FFT height as the amplitude. Scale by 2/N first for a real signal.
Myth: phase does not matter. For a tuner it does not; to rebuild the sound it does.

9. Real-life and AI uses

Guitar tuners and music apps: find the strongest frequency and compare it to the nearest note, for example A4 = 440 Hz (background).
Audio formats: CD audio stores two-channel 16-bit PCM at 44,100 samples per second, so frequencies up to 22,050 Hz are representable.
Speech recognition: Whisper converts audio to 16 kHz, then to an 80-channel log-magnitude Mel spectrogram on 25 ms windows with a 10 ms stride, and feeds that picture to a neural network.
Neural networks on audio: the deep learning textbook lists audio preprocessed with a Fourier transform into a two-dimensional tensor of frequency × time as a standard input form for convolutional networks.
Fast convolution and filtering: transform input and kernel, multiply point by point, transform back. Filtering is the same idea with a "keep these frequencies, remove those" multiplier (for example, cut hiss above some frequency).
Image compression (background): JPEG uses the related discrete cosine transform.
Signal processing in general: radio, radar, medical scanners and vibration monitoring all start by looking at spectra (background).

10. Self-check questions (with answers)

What does the Fourier transform tell you about a signal? Which frequencies it contains and how strong each is (and their phases).
In the video, what are the two spikes in Rohan's spectrum? 220 Hz with amplitude 1 and 330 Hz with amplitude ½.
What is the frequency resolution for N = 8,000 samples at fs = 8,000 samples per second? fs/N = 8,000 ÷ 8,000 = 1 Hz.
What is the Nyquist frequency at 8,000 samples per second? fs/2 = 4,000 Hz.
What is the Nyquist frequency of CD audio at 44,100 samples per second? 44,100 ÷ 2 = 22,050 Hz.
Write the DFT formula. A_k = Σ a_m e^(−2πi mk/n), summed over m = 0 to n−1.
How does the FFT cost compare with the direct DFT for N = 2²⁰? About 2.1 × 10⁷ against 1.1 × 10¹² operations, roughly 50,000 times fewer.
What is a spectrogram? A short-time Fourier transform: an FFT on each short window, shown as frequency against time.
How many samples are in Whisper's 25 ms window and 10 ms stride at 16 kHz? 400 samples and 160 samples.
What does the convolution theorem say? Convolution in time equals point-wise multiplication in frequency, so convolution can be done by transforming, multiplying and transforming back.

11. Key takeaways, go deeper, next

Key takeaways

A signal can be written as a sum of sine waves; the Fourier transform finds the recipe.
X(f) = ∫ x(t) e^(−2πift) dt in continuous form, and A_k = Σ a_m e^(−2πi mk/n) for lists of samples.
Bin spacing is fs/N, and the top frequency is fs/2.
The FFT turns N² work into about N log₂N, roughly 50,000 times less for N = 2²⁰.
A spectrogram stacks many short FFTs and is the usual input for audio models such as Whisper (16 kHz, 80 mel channels, 25 ms windows, 10 ms stride).
Convolution in time is multiplication in frequency, which gives fast convolution and filtering.

Go deeper

Watch 3Blue1Brown, "But what is the Fourier Transform? A visual introduction" (2018), https://www.youtube.com/watch?v=spUNpyF58BY. This is a video for intuition, not a source of the facts above.
Run numpy.fft.rfft on a signal you make yourself, and change the window length to see the resolution change.
Try a tone that does not fit the window (for example 220.5 Hz for one second) and watch the leakage; then apply a Hann window (background).
Read the efficient convolution section of the deep learning book's convolutional networks chapter.

Next on the roadmap: 1.47 Graph theory basics: nodes, edges, adjacency.

12. Sources and further reading

NumPy developers, Discrete Fourier Transform (numpy.fft) (NumPy documentation), Discrete Fourier Transform (numpy.fft), definition of the DFT, FFT history and time versus frequency domain
Radford, Kim, Xu, Brockman, McLeavey, Sutskever, Robust Speech Recognition via Large-Scale Weak Supervision (Whisper) (2022), Whisper paper, audio re-sampling to 16,000 Hz and 80-channel log-magnitude Mel spectrogram
Goodfellow, Bengio, Courville, Deep Learning (2016), Chapter 9 Convolutional Networks, Table 9.1 and section 9.8 Efficient convolution algorithms
Wikipedia, Compact disc, Compact disc, two-channel 16-bit PCM at 44.1 kHz
3Blue1Brown, But what is the Fourier Transform? A visual introduction (2018), Video for intuition (go deeper only, not used as a source of facts)

Accuracy note: the signal, the guitar tuner story and Rohan are illustrative. The FFT values (220 Hz amplitude 1.0, 330 Hz amplitude 0.5, the 4-point DFT, the cost ratio of about 52,429 and the window sizes) were checked in Python with numpy. Facts about the DFT definition, the history of the FFT, Whisper's settings and CD audio come from the listed sources. Items marked "(background)", such as aliasing, spectral leakage, Hann windows, the discrete cosine transform in JPEG and A4 = 440 Hz, are general knowledge not taken from the listed sources. The 3Blue1Brown video is recommended for intuition only.

Fourier transform: frequencies in sound and signals: study notes