Posts

Fourier transform: frequencies in sound and signals: study notes

-
  1.46 Fourier transform: frequencies in sound and signals: study notes Learn AI by Story 1. The idea in one sentence Any signal that wiggles over time, such as a sound wave, can be rebuilt by adding simple sine waves of different frequencies, and the Fourier transform is the tool that tells you which frequencies are present and how strong each one is. 2. The story from the video Rohan is building a guitar tuner app. His phone microphone gives him a long list of numbers: the air pressure measured thousands of times every second. Plotted over 20 milliseconds, the wave looks like a messy squiggle. He cannot read a note name from it by eye. Rohan's friend explains that the squiggle is really two clean sine waves added together. One wave repeats 220 times per second. The other repeats 330 times per second and is half as tall. In the 20 ms picture, the 220 Hz wave completes 4.4 cycles and the 330 Hz wave completes 6.6 cycles. (The signal is illustrative: x(t) = sin(2π·220t) + 0.5 sin(2π...
Post a Comment
Read more

Mutual information and Jensen's inequality: study notes

-
  1.45 Mutual information and Jensen's inequality: study notes Learn AI by Story 1. The idea in one sentence Mutual information I(X; Y) measures how many bits of uncertainty about X disappear when you learn Y (it is zero exactly when X and Y are independent and never negative), and the reason it can never be negative is Jensen's inequality: for a convex function f, f(E[x]) ≤ E[f(x)]. 2. The story from the video Zoya runs an umbrella stall. Over 100 mornings (illustrative data) she wrote down the sky and whether it rained: Cloudy and rain: 20 mornings Cloudy and dry: 15 mornings Clear and rain: 5 mornings Clear and dry: 60 mornings So P(rain) = (20 + 5) ÷ 100 = 0.25 and P(cloudy) = (20 + 15) ÷ 100 = 0.35. Before looking at the sky, her uncertainty about rain is H(rain) = H(0.25) ≈ 0.81 bits. If the sky is cloudy, P(rain) = 20 ÷ 35 ≈ 0.571 and the uncertainty is about 0.985 bits (nearly a coin flip). If the sky is clear, P(rain) = 5 ÷ 65 ≈ 0.077 and the uncertainty is only about ...
Post a Comment
Read more

MCMC: Metropolis–Hastings and Gibbs: study notes

-
  1.44 MCMC: Metropolis–Hastings and Gibbs: study notes Learn AI by Story 1. The idea in one sentence Markov chain Monte Carlo (MCMC) draws samples from a probability distribution p(x) that is too awkward to sample directly, by taking a random walk that is built so that, in the long run, it visits each place in proportion to p(x), and it only ever needs ratios of p, so the unknown normalising constant Z cancels out. 2. The story from the video Kavya visits five river villages. The villages have populations in the ratio 1 : 2 : 3 : 4 : 5, so a "fair" random visitor should spend 1/15 ≈ 6.7%, 2/15 ≈ 13.3%, 3/15 = 20.0%, 4/15 ≈ 26.7% and 5/15 ≈ 33.3% of her time in villages 1 to 5. (All numbers in this story are illustrative.) The catch is that Kavya does not know the total population, so she cannot write down the exact percentages. She only knows how big each village is compared with its neighbour. So she makes up a simple rule: Flip a coin to propose the left or right neighbour...
Post a Comment
Read more

Monte Carlo sampling: study notes

-
  1.43 Monte Carlo sampling: study notes Learn AI by Story 1. The idea in one sentence Monte Carlo sampling estimates a hard average, area or integral by drawing random samples and averaging them, so that the error shrinks like 1/√n no matter how many dimensions the problem has. 2. The story from the video Sam has a square tile of side 1 and a bag of rice. He imagines a quarter circle of radius 1 drawn in the corner of the tile. The quarter circle covers π/4 ≈ 0.7854 of the square's area. So if he throws grains at random onto the square, about 78.54% of them should land inside the quarter circle, and π ≈ 4 × (grains inside) ÷ (grains thrown). He tries it with 40 grains (a run from a computer simulation with seed 7, illustrative of one run): 32 land inside, giving 4 × 32/40 = 3.2. Not bad, but not great. With a computer he throws more. In one run (seed 42): 100 grains give 3.36; 1,000 grains give 3.156; 10,000 grains give 3.1348; and 1,000,000 grains give 3.142568. The true value is...
Post a Comment
Read more