1.10 Distributions: uniform, Bernoulli, normal: study notes

Learn AI by Story

1. The idea in one sentence

A probability distribution is a complete description of how likely each possible outcome of an uncertain quantity is, and three shapes (uniform, Bernoulli and normal) cover a huge share of what you meet in everyday life and in AI.

2. The story from the video

Kiran waits for a bus that could arrive at any moment in the next 10 minutes, with no moment more likely than another: a flat, uniform spread. Then he flips a coin, a single trial with two outcomes, and checks whether it will rain (a 30% chance), which is a Bernoulli trial with bars of 0.7 and 0.3. Finally he measures the heights of many people and finds a bell curve centred on 170 cm, with most people within 10 cm of the middle. Three situations, three shapes, and one idea: a distribution shows how likely each outcome is.

3. Key definitions (glossary)

Random variable. A quantity whose value is determined by a random process, usually written X. Plain English: a number that you do not know yet, such as tomorrow's wait time.

Outcome and sample space. An outcome is one possible result; the sample space is the set of all possible results. Plain English: everything that could happen.

Distribution. A rule that tells how probability is spread across the possible values of a random variable. Plain English: the shape of "how likely is each value".

Discrete random variable. One that takes separate, countable values (0, 1, 2, ...). Plain English: you can list the options, such as the faces of a die.

Continuous random variable. One that can take any value in a range, including decimals. Plain English: measured quantities like time, height and weight.

Probability mass function (PMF). For a discrete variable, the function giving P(X = x) for each value x. The values are real probabilities and they add up to 1. Plain English: a bar chart of probabilities.

Probability density function (PDF). For a continuous variable, a curve f(x) such that the probability of landing in an interval equals the area under the curve over that interval. The total area is 1. Plain English: the height of the curve is not a probability; the area under it is.

Density. The height of a PDF at a point, measured in probability per unit of x. Plain English: how crowded the probability is near x. It can even exceed 1 when the x-axis is narrow.

Cumulative distribution function (CDF). F(x) = P(X ≤ x), the probability of being at or below x. It rises from 0 to 1. Plain English: the running total of probability up to x.

Uniform distribution. All values in a range are equally likely. Continuous version on [a, b]: density 1/(b − a). Discrete version: each of n values has probability 1/n. Plain English: a flat shape.

Bernoulli distribution. One trial with two outcomes, 1 (success) with probability p and 0 (failure) with probability 1 − p. Plain English: a single yes/no, like one coin flip.

Binomial distribution. The number of successes in n independent Bernoulli trials, each with success probability p. Plain English: how many heads in n flips.

Normal (Gaussian) distribution. A symmetric bell-shaped continuous distribution fully described by its mean μ and standard deviation σ. Plain English: the bell curve.

Mean (μ). The centre or average of the distribution. Plain English: where the bell sits.

Standard deviation (σ). A measure of the typical distance of values from the mean. Plain English: how wide the bell is.

Variance (σ²). The standard deviation squared. Plain English: the spread, measured in squared units.

z-score. z = (x − μ) / σ, the number of standard deviations a value is above (positive) or below (negative) the mean. Plain English: "how unusual is this value", on a common scale.

Standard normal. The normal distribution with mean 0 and standard deviation 1. Plain English: the bell curve every other normal can be converted into.

Central limit theorem (CLT). The sum or average of many independent random pushes tends toward a normal distribution, whatever the shape of the individual pushes (under mild conditions). Plain English: lots of small random effects add up to a bell.

Heavy-tailed distribution. One where extreme values are much more common than a normal distribution would predict. Plain English: rare giant events happen more often than the bell curve says.

Categorical distribution. One trial with k possible outcomes, each with its own probability, summing to 1. Plain English: a Bernoulli with more than two options, such as a die or a choice among many words.

Poisson distribution. Counts of events in a fixed time or space when events occur independently at a steady average rate λ. Plain English: how many buses, calls or typos in an hour.

Exponential distribution. The waiting time between such events, with mean 1/λ. Plain English: how long until the next one arrives.

4. Step-by-step explanation

Step 1: From one outcome to all outcomes

A single measurement tells you one value. A distribution tells you the whole pattern of possibilities and their probabilities. Think of repeating an experiment thousands of times and drawing a histogram of the results: as the number of repeats grows, the histogram settles into a smooth shape. That shape is the distribution. The shape matters because it answers questions like "how likely is a wait of under 4 minutes?" without running the experiment.

Step 2: The random variable and the two kinds

First we give the uncertain thing a name, X, such as "Kiran's wait in minutes" or "whether it rains (1) or not (0)". If X takes separate values, it is discrete and we describe it with a probability mass function (bars). If X is a measurement that can be any value in a range, it is continuous and we describe it with a probability density function (a curve).

Step 3: Why a continuous variable needs area

This is the subtle part. For a continuous variable, the chance that the wait is exactly 3.000000... minutes is zero, because there are infinitely many possible values and the probability is spread across all of them. So we ask about intervals: "between 2 and 4 minutes". The probability of an interval is the area under the density curve over it. That is why the curve's height is called density, not probability: it is probability per minute, like a population density is people per square kilometre.

Step 4: Uniform, the flat shape

On a bus arriving anywhere in 0 to 10 minutes with equal chance, the density is constant. For the total area to be 1, a flat curve over a width of 10 must have height 0.1 per minute (10 × 0.1 = 1). Then the probability of waiting less than 4 minutes is the area of a rectangle: 4 × 0.1 = 0.4. General formulas for a continuous uniform on [a, b]: density 1/(b − a), mean (a + b)/2, variance (b − a)² / 12. For 0 to 10: mean 5, variance 100/12 ≈ 8.33, standard deviation ≈ 2.89 minutes.

Step 5: Bernoulli, the single yes/no

With P(rain) = 0.3, the distribution has two bars: 0.3 for rain and 0.7 for no rain, summing to 1. Written in a general form, P(X = 1) = p and P(X = 0) = 1 − p. The mean is p (the average of many 0/1 results is the fraction of ones) and the variance is p × (1 − p). For p = 0.3: mean 0.3, variance 0.3 × 0.7 = 0.21, standard deviation ≈ 0.458. Variance is biggest at p = 0.5 (a fair coin, 0.25), and zero when the outcome is certain (p = 0 or p = 1).

Step 6: Adding Bernoullis gives the binomial

Flip a fair coin 10 times and count heads. Each flip is Bernoulli with p = 0.5, and the total number of heads follows a binomial distribution. The probability of exactly k successes in n trials is C(n, k) × p^k × (1 − p)^(n − k), where C(n, k) counts the number of ways to choose which k trials succeed. For exactly 5 heads in 10 flips: C(10, 5) = 252, and each particular sequence has probability (1/2)^10 = 1/1024, so the answer is 252/1024 ≈ 0.246. The binomial has mean n × p and variance n × p × (1 − p).

Step 7: The bell curve appears

Many things are the sum of many small random pushes: height depends on many genes plus many nutrition and health effects, and each adds or subtracts a little. The central limit idea says such sums pile up in the middle and thin out toward the sides, forming the bell. You can see it with the binomial itself: for n = 10, 20, 50 flips, the bars get closer and closer to a bell shape.

Step 8: The two knobs, mean and standard deviation

A normal distribution is fully set by two numbers. The mean μ moves the bell left or right. The standard deviation σ makes it wider or narrower while the total area stays at 1, so a wider bell is also a lower bell. The peak density is 1/(σ × √(2π)), about 0.399/σ. For σ = 10 cm, the peak is about 0.0399 per cm.

The formula in simple words: take how far x is from the mean, square it, divide by twice the variance, and put a minus sign on it; then e to that power gives a bump that is tallest at the mean and falls off symmetrically. The constant in front just makes the area equal 1. In symbols, f(x) = (1 / (σ √(2π))) × e^(−(x − μ)² / (2σ²)).

Step 9: The 68-95-99.7 rule

For any normal distribution, about 68% of values fall within 1 standard deviation of the mean, about 95% within 2, and about 99.7% within 3. (More precisely 68.27%, 95.45% and 99.73%; the exact 95% point is at 1.96 standard deviations.) For heights with mean 170 and standard deviation 10: 160 to 180 cm holds about 68%, 150 to 190 cm about 95%, and 140 to 200 cm about 99.7%.

Step 10: z-scores put everything on one ruler

To compare values from different bells, convert to z-scores: z = (x − μ) / σ. A height of 185 cm gives z = (185 − 170) / 10 = 1.5, meaning 1.5 standard deviations above average. The standard normal distribution (mean 0, standard deviation 1) is the bell for z, and standard tables or software give its probabilities. Because every normal can be converted to it, one table is enough.

5. Everything the video skipped

The CDF, a different view of the same information

The CDF F(x) = P(X ≤ x) is the running total of probability. For the bus wait, F(x) = x/10 for x between 0 and 10, a straight ramp from 0 to 1. For the normal, the CDF is an S-shaped curve. A great practical use: P(a < X ≤ b) = F(b) − F(a). This is how normal probabilities are actually computed, since the normal's area has no simple closed formula.

Discrete versus continuous uniform

A fair die is a discrete uniform: each face 1 to 6 has probability 1/6, and the mean is 3.5. The bus wait is a continuous uniform. The idea is the same (nothing is favoured), but one uses bars that sum to 1 and the other a flat density whose area is 1.

Other distributions worth knowing

Distribution	Type	Describes	Parameters	Example
Uniform	continuous or discrete	all values equally likely	a, b (or n)	wait time, die
Bernoulli	discrete	one yes/no	p	one coin flip, rain or not
Binomial	discrete	successes in n trials	n, p	heads in 10 flips
Categorical	discrete	one pick among k options	k probabilities	which word comes next
Poisson	discrete	events per interval	λ	calls per hour
Exponential	continuous	time until next event	λ	time between buses that arrive randomly
Normal	continuous	sums of many small effects	μ, σ	heights, measurement errors

Heavy tails: why not everything is normal

The normal curve gives astronomically small probabilities to values far from the mean. Real life disagrees in many areas. Incomes, city populations, word frequencies, earthquake sizes and stock market crashes have heavy tails: huge values occur much more often than the bell predicts, and one extreme value can dominate an average. A normal model for stock returns would call a 5-standard-deviation day nearly impossible, yet such days happen. The CLT also needs independent, reasonably well-behaved pieces; sums of heavy-tailed pieces may not become normal. Always look at your data before assuming a bell.

Truncated and bounded things

Heights cannot be negative, yet the normal curve technically has tails extending below zero. For heights with mean 170 and standard deviation 10, the probability of a negative height is astronomically small, so the model is fine in practice. But for quantities close to zero with a large spread, a normal model can be nonsense.

History

Abraham de Moivre found the bell shape as an approximation for the binomial in the 1730s. Carl Friedrich Gauss used it to describe measurement errors in astronomy (which is why it is called Gaussian), and Pierre-Simon Laplace proved an early version of the central limit theorem. Francis Galton later showed it in human traits such as height. The Bernoulli distribution is named for Jacob Bernoulli, whose work on repeated trials in the early 1700s led to the law of large numbers (see 1.12).

Roadmap connections

Back: 1.08 Probability basics provides the rules that distributions obey (probabilities are between 0 and 1 and the total is 1). 1.09 Conditional probability and Bayes' theorem explains how a distribution (a prior) turns into a new distribution (a posterior) when evidence arrives. Forward: 1.11 Mean, variance, standard deviation and correlation formalises the two knobs of the normal and the summary numbers for any distribution. 1.12 Sampling and the law of large numbers shows why averages of samples settle down and why the bell appears. 1.13 Entropy measures how spread out or surprising a distribution is: the uniform distribution has the highest entropy among distributions with a fixed range.

Limitations and trade-offs

Choosing a distribution is a modelling decision, and a wrong choice can mislead.
The normal is convenient (two numbers describe everything) but understates extremes.
The uniform says "I know nothing except the range", which is honest but rarely exactly true.
Density values are easy to misread; always convert to area before calling something a probability.

6. Worked examples

Worked example 1: The bus wait between 2 and 7 minutes

The wait X is uniform on 0 to 10 minutes. Find P(2 < X < 7).

Density = 1/(10 − 0) = 0.1 per minute.
Width of the interval = 7 − 2 = 5 minutes.
Area of the rectangle = 5 × 0.1 = 0.5.

So P(wait between 2 and 7 minutes) = 0.5. Check with the CDF: F(7) − F(2) = 0.7 − 0.2 = 0.5. Related checks: P(wait < 4) = 4 × 0.1 = 0.4, and P(wait > 8) = 2 × 0.1 = 0.2. The mean wait is 5 minutes and the standard deviation is about 2.89 minutes.

Worked example 2: Heights above 180 cm

Heights are normal with mean 170 cm and standard deviation 10 cm. What fraction of people are taller than 180 cm?

z = (180 − 170) / 10 = 1.
About 68% of people lie within 1 standard deviation (160 to 180), so about 32% lie outside.
By symmetry, half of that, about 16%, is above 180 and about 16% below 160.
More precisely, the standard normal gives P(z > 1) ≈ 0.1587.

So about 16% of people are taller than 180 cm.

Worked example 3: A z-score of 1.5

A person is 185 cm tall.

z = (185 − 170) / 10 = 1.5.
From the standard normal, P(z < 1.5) ≈ 0.9332.
So about 93.3% of people are shorter, and P(taller than 185) ≈ 0.0668, about 6.7%.

Another question: what fraction are between 160 and 190 cm? That is z from −1 to 1.5, so 0.9332 − 0.1587 = 0.7745, about 77.5%.

Worked example 4: Counting heads with a binomial

Flip a fair coin 10 times. What is the probability of exactly 5 heads, and of 9 or more?

Total sequences: 2^10 = 1,024, all equally likely.
Exactly 5 heads: C(10, 5) = 10! / (5! × 5!) = 252. So P = 252/1,024 ≈ 0.246.
Exactly 9 heads: C(10, 9) = 10. Exactly 10 heads: 1.
9 or more: (10 + 1)/1,024 = 11/1,024 ≈ 0.0107, about 1%.
The mean number of heads is 10 × 0.5 = 5 and the variance is 10 × 0.5 × 0.5 = 2.5, giving a standard deviation of about 1.58.

Notice that even the single most likely result (5 heads) happens only about a quarter of the time. Getting exactly the average is not typical; getting near it is.

7. Comparison tables and diagrams

Feature	Uniform	Bernoulli	Normal
Kind	continuous (or discrete)	discrete	continuous
Shape	flat	two bars	bell
Parameters	range a to b	p	mean μ and SD σ
Mean	(a + b)/2	p	μ
Variance	(b − a)² / 12	p(1 − p)	σ²
Story example	bus wait 0 to 10	rain with P = 0.3	heights around 170 cm
AI use	random starting values	yes/no labels, dropout	noise, weight initialisation

Text sketches of the three shapes:

Uniform (0 to 10 min)      Bernoulli (rain 0.3)      Normal (mean 170, SD 10)
density 0.1                bar heights               68% between 160 and 180
 ______________            0.7  ##                         .-"""-.
|              |                ##                       .'       '.
|______________|           0.3  ##  ##                _.'           '._
0              10               no   rain             140 150 160 170 180 190 200

The 68-95-99.7 rule at a glance:

Range	Heights (cm)	Share of people
mean ± 1 SD	160 to 180	about 68%
mean ± 2 SD	150 to 190	about 95%
mean ± 3 SD	140 to 200	about 99.7%

Probability mass versus probability density:

	PMF (discrete)	PDF (continuous)
What the height means	an actual probability	density, probability per unit
How to get a probability	read the bar or add bars	find the area under the curve
Total	bars add to 1	area equals 1
Value at one exact point	can be above 0	probability is 0

8. Common mistakes and myths

"The height of the curve is the probability." For continuous variables it is density. Probability is area. A density can exceed 1 for a narrow range (a uniform on 0 to 0.5 has density 2).
"Everything is normal." Many real quantities are skewed or heavy-tailed. Incomes are the classic counterexample.
"A fair coin must give exactly 5 heads in 10 flips." That happens only about 24.6% of the time. Fairness describes the long run.
"After several heads, tails is due." Each flip is independent; the coin has no memory (the gambler's fallacy).
"A larger standard deviation means a larger mean." They are separate knobs: one controls position, the other width.
"95% of data lie within 2 standard deviations for any distribution." That rule is for normal-ish data. For other shapes the share can differ.
Forgetting that probabilities (or the total area) must equal 1. A flat density over a wide range must be low.
Confusing the Bernoulli with the binomial. Bernoulli is one trial; binomial counts successes in many trials.
Treating "random" as "uniform". Random only means uncertain; the outcomes may be very unequal.

9. Real-life and AI/ML uses

Uniform in AI. Random initial values, random choice of a data batch, random sampling for simulations, and random search over settings (for example a learning rate picked uniformly in a range) all use the uniform. Computers build most other random numbers from uniform ones.
Bernoulli in AI. Binary labels (spam or not, click or not) are Bernoulli outcomes, and a classifier that outputs a probability of 0.8 is predicting the p of a Bernoulli. Dropout uses a Bernoulli mask: each neuron is kept with probability, say, 0.8, and dropped with 0.2, which reduces overfitting. With 10 neurons and keep probability 0.8, the expected number kept is 8.
Categorical in AI. A language model's next-word choice is a categorical distribution across the vocabulary; the softmax function creates the probabilities.
Normal in AI. Gaussian noise is added to images or inputs for robustness and in diffusion models. Weight initialisation often draws from a normal: Xavier (Glorot) initialisation uses a variance of 2 / (fan-in + fan-out), and He initialisation uses a variance of 2 / fan-in, which suits ReLU layers. For example, a layer with 256 inputs has a He standard deviation of √(2/256) ≈ 0.088. These choices keep signals from shrinking or exploding as they pass through layers. Many models also assume errors are normal, which leads to the squared-error loss.
Binomial and Poisson. A/B tests count conversions (binomial); call-centre and server-load models count arrivals (Poisson).
Everyday uses. Quality control (normal measurement errors), insurance (claim counts), weather forecasts (Bernoulli rain chance), exam scores (often roughly normal), and waiting times (exponential or uniform).

10. Self-check questions

What is a probability distribution? A description of how likely each possible outcome of a random variable is, the shape of its probabilities.
What is the difference between discrete and continuous random variables? Discrete variables take countable separate values (die faces); continuous variables take any value in a range (time, height).
For a uniform wait on 0 to 10 minutes, what is the density and P(wait < 4)? Density = 1/10 = 0.1 per minute; P = 4 × 0.1 = 0.4.
Why is the probability of exactly 3.5 minutes zero for a continuous variable? A single point has no width, so the area under the density over it is zero; probabilities come from intervals.
Give the mean and variance of a Bernoulli with p = 0.3. Mean 0.3; variance 0.3 × 0.7 = 0.21.
What is P(exactly 5 heads in 10 fair flips)? C(10, 5)/2^10 = 252/1,024 ≈ 0.246.
What two numbers define a normal distribution, and what does each control? The mean μ sets the centre; the standard deviation σ sets the width.
State the 68-95-99.7 rule for heights with mean 170 cm and SD 10 cm. About 68% lie in 160 to 180, 95% in 150 to 190, and 99.7% in 140 to 200.
What is the z-score of a 185 cm height, and what does it mean? z = (185 − 170)/10 = 1.5, meaning 1.5 standard deviations above the mean.
Name one reason not to assume everything is normal. Heavy-tailed data such as incomes or crash sizes have far more extreme values than a bell curve allows.

11. Key takeaways, go deeper, and what is next

Key takeaways

A distribution shows how likely each outcome is; discrete ones use bars (PMF), continuous ones use a curve (PDF), and for the curve, area equals probability.
Uniform: everything equally likely; density 1/(b − a); bus example P(wait < 4) = 0.4.
Bernoulli: one yes/no with probability p; mean p, variance p(1 − p); binomial adds many together.
Normal: bell curve set by mean and standard deviation; 68-95-99.7 rule; z = (x − μ)/σ.
Many small random pushes add up to a bell (central limit idea), but not everything is normal.
AI uses uniform for random starts, Bernoulli for labels and dropout, normal for noise and weight initialisation.

Go deeper

Simulate 10,000 sets of 10 coin flips and plot a histogram of the number of heads.
Look up the standard normal table and practise converting heights to z-scores.
Read about Poisson and exponential distributions and how they connect.
Explore heavy-tailed distributions such as the power law and why averages can fail there.
Read about Xavier and He initialisation and see why the variance formulas look the way they do.

Next roadmap ID: 1.11 Mean, variance, standard deviation, correlation.

Learn AI by Story: one concept, one story, every day.

Distributions: uniform, Bernoulli, normal: study notes