1.01 Vectors: study notes

Learn AI by Story

1. The idea in one sentence

A vector is an ordered list of numbers that you can picture as an arrow: the numbers tell you which way the arrow points and how long it is, and in AI the same list of numbers can also describe a thing, such as a house, a movie or a word.

2. The story from the video

Arjun is lost in an unfamiliar town and phones a friend. The friend says, "Walk 3 blocks east, then 2 blocks north." Those two numbers, written [3, 2], are a vector. They give a direction (east and north) and a length (how far). Swap them to [2, 3] and Arjun ends up in a different place, so order matters. The video then shows that the same trick describes anything: a house is [120, 3, 2] (size, bedrooms, bathrooms), a movie is [0.9, 0.1] (action, romance), and a word becomes roughly 300 numbers, called an embedding. Remember: a list of numbers is an arrow with a direction and a length.

3. Key definitions (glossary)

Each entry gives a precise definition, then a plain-English line.

Term	Precise definition	Plain English
Scalar	A single number, such as 5 or -0.3.	One number with no direction.
Vector	An ordered list of numbers, written like [3, 2].	A list of numbers that is also an arrow.
Component (or entry)	One number inside a vector. In [3, 2], 3 is the first component and 2 is the second.	One slot in the list.
Dimension	The number of components in a vector. [3, 2] has dimension 2; [120, 3, 2] has dimension 3.	How many numbers are in the list.
2D, 3D, n-D	Vectors with 2, 3, or n components.	Flat map, real space, or "any number of directions".
Column vector	A vector written vertically, as a stack of numbers.	Numbers standing in a column.
Row vector	A vector written horizontally, as a row of numbers.	Numbers lying in a row.
Zero vector	The vector whose components are all 0, such as [0, 0].	"Do not move at all."
Vector addition	Adding matching components: [a, b] + [c, d] = [a + c, b + d].	Do one walk, then the other.
Scalar multiplication (scaling)	Multiplying every component by one number: k x [a, b] = [ka, kb].	Stretch, shrink or flip the arrow.
Negative vector	The vector with every component's sign flipped: -[3, 2] = [-3, -2].	The same walk backwards.
Subtraction	a - b means a + (-b), done component by component.	The walk that gets you from b to a.
Norm (L2 norm, length, magnitude)	The square root of the sum of squared components: length of v = sqrt(v1^2 + v2^2 + ...).	The straight-line length of the arrow.
L1 norm	The sum of the absolute values of the components.	Length measured by walking only along the grid.
Unit vector	A vector whose length is exactly 1.	An arrow of length one that only carries direction.
Normalising	Dividing a vector by its own length to get a unit vector.	Shrink or stretch to length 1, keep the direction.
Linear combination	A sum of scaled vectors, such as 2a + 3b.	A recipe: some of this, some of that.
Feature vector	A vector whose components are measured properties (features) of one example.	A thing described as a row of facts.
Embedding	A learned vector, often hundreds of numbers, that represents an item such as a word so that similar items get similar vectors.	A word turned into a list of numbers that captures its meaning.
Tip-to-tail	The geometric rule for adding arrows: place the tail of the second at the tip of the first.	Walk one arrow, then start the next where you stopped.
Origin	The point [0, 0, ...], where all components are zero.	The starting corner of the map.
Dot product (preview)	Multiply matching components and add: [a, b] . [c, d] = ac + bd.	A single number that says how much two arrows agree.

4. Step-by-step explanation

Step 1: Start with a single number

A single number such as 3 is a scalar. It can say "3 blocks" but not which way. Alone it is missing direction. That gap is why we need something bigger.

Step 2: Put numbers in a list and keep the order

Write two numbers in square brackets: [3, 2]. By agreement, the first slot means "east" and the second means "north". The list is ordered, so each position has a fixed meaning. This is why [3, 2] and [2, 3] are different vectors. They start at the same place but end at different places.

Why this matters: a computer has no common sense. If the first slot is always size and the second always bedrooms, it can compare thousands of houses reliably. The meaning lives in the position.

Step 3: See the list as an arrow

Draw a flat map. The origin is where Arjun stands. Go 3 to the right (east), then 2 up (north). Draw an arrow from the origin to that point. The arrow has a direction (up and to the right) and a length. The list of numbers and the arrow are two views of one object.

Step 4: Count the dimensions

Two numbers give a flat map (2D). Add a third, height, and you get 3D space: [3, 2, 1] means 3 east, 2 north, 1 up. We cannot draw more than three directions, but nothing stops us writing four, 300 or 4,000 numbers. A vector with n numbers lives in n-dimensional space. The rules are exactly the same; only the drawing is impossible.

Step 5: Add vectors

Arjun walks [3, 2] and then his friend says "now go 1 east, 2 north": [1, 2]. The total is [3, 2] + [1, 2] = [3 + 1, 2 + 2] = [4, 4].

Geometrically, this is the tip-to-tail rule: draw the first arrow, start the second where the first ended, and the sum goes from the very start to the very end. The algebra (add matching slots) and the picture (chain the arrows) always agree. Order does not matter for addition: [1, 2] + [3, 2] is also [4, 4].

Step 6: Scale a vector

Multiply every component by one number. 2 x [3, 2] = [6, 4]: the same direction, double the length. Multiplying by 0.5 gives [1.5, 1], half the length. Multiplying by -1 gives [-3, -2], the same line but pointing the opposite way. Scaling never turns an arrow sideways; it only stretches, shrinks or flips it.

Step 7: Measure length

The length of [3, 4] is found using Pythagoras. The arrow is the long side of a right-angled triangle with sides 3 and 4, so length = sqrt(3^2 + 4^2) = sqrt(9 + 16) = sqrt(25) = 5. In general, square each component, add them up, take the square root. This works in any number of dimensions.

Step 8: Make a unit vector (normalise)

Sometimes you only care about direction. Divide the vector by its length: [3, 4] / 5 = [0.6, 0.8]. Check: 0.6^2 + 0.8^2 = 0.36 + 0.64 = 1, so the length is 1. Every non-zero vector has exactly one unit vector pointing the same way.

Step 9: Subtract and reverse

The negative of [3, 2] is [-3, -2], which brings Arjun home. To subtract, add the negative: [5, 4] - [3, 2] = [2, 2]. Read this as "the walk that takes you from the point [3, 2] to the point [5, 4]". Subtraction therefore measures the gap between two things, which is how AI measures how different two examples are.

Step 10: Combine vectors (linear combinations)

A linear combination scales some vectors and adds them. Take a = [3, 2] and b = [1, 2]. Then 2a - b = [6, 4] + [-1, -2] = [5, 2]. Almost everything in later maths for AI is built from this single move: scale, then add.

Step 11: Use vectors to describe things

Once you accept that a vector is just an ordered list, you can make the slots mean anything. A house is [120, 3, 2]: size in square metres, bedrooms, bathrooms. A movie is [0.9, 0.1]: how much action, how much romance. A word, after training, is about 300 numbers. This is a feature vector. Closeness of two arrows then becomes closeness of two things.

5. Everything the video skipped

Row versus column vectors

A column vector stands upright; a row vector lies flat. For plain addition or length it makes no difference. It matters later when we multiply by matrices (roadmaps 1.02 and 1.03), where the convention decides which way the multiplication is written. Most maths-for-AI texts use column vectors for a single example.

Points versus arrows

The pair [3, 2] can be seen as a point on the map or as an arrow from the origin to that point. A vector can also be thought of as a free arrow that you can slide around: "3 east, 2 north" is the same instruction whatever your starting spot. Both views are useful. Points are good for data; arrows are good for movement.

Two ways to see the same thing

The geometric view is arrows and pictures. The algebraic view is lists of numbers and arithmetic. Geometry gives intuition; algebra scales to 300 dimensions. A good habit is to find the picture in 2D first, then trust the algebra for higher dimensions.

The norm family

The "length" we computed is the L2 norm (also called the Euclidean norm), the ordinary straight-line distance. A cousin is the L1 norm: add up the absolute values of the components. For [3, 4] the L1 norm is 3 + 4 = 7, as if you could only walk along streets. Machine learning uses both. L2 is common in distance and in penalties that discourage big weights; L1 is used when you want many weights to become exactly zero.

Distance between two vectors

The distance between a and b is the length of a - b. For the movies [0.9, 0.1] and [0.8, 0.2], the difference is [0.1, -0.1], and the length is sqrt(0.01 + 0.01) = sqrt(0.02), about 0.14. Small distance means similar movies.

The dot product (a preview)

Multiply matching slots and add: [3, 2] . [1, 2] = 3 + 4 = 7. A big positive value means the two arrows point roughly the same way, zero means they are at right angles, and negative means roughly opposite. It is the engine behind similarity search and attention in language models. It appears in full in later roadmap items.

Why high dimensions feel strange

In 3D we can picture everything. At 300 dimensions, intuition fails in a few ways. First, there are vastly more directions, so there is room for many concepts to have their own direction. Second, in very high dimensions most random vectors are nearly at right angles to each other, so truly similar items stand out. Third, data becomes sparse, because even a million examples cannot fill a space of 300 directions. This is often called the "curse of dimensionality". The lesson: do the picture in 2D, and trust the algebra beyond.

Units, scaling and the order of features

Features can live on very different scales. House size is around 120, bedrooms around 3. If you measure the distance between houses without adjusting, size dominates and bedrooms barely count. The usual fix is to scale features to a comparable range (for example divide the size by 100, or standardise each feature). Also remember that slot order is a contract: if one house lists [size, bedrooms, baths] and another lists [bedrooms, size, baths], adding or comparing them is meaningless.

A little history

The word "vector" comes from a Latin word meaning "carrier". In the 1800s, mathematicians and physicists such as Hamilton, Grassmann and Gibbs developed the arrows-in-space language to describe forces and velocities. Computers later made the "list of numbers" view the everyday one. Embeddings, which turn words into vectors, became popular in the 2010s with methods such as word2vec.

Limitations

A vector is only as good as its features. A bad choice of slots (leaving out something important) gives a bad description. Vectors also cannot, by themselves, say what each slot means; humans or training must define that. And in embeddings the individual slots usually have no human-readable meaning; only the overall pattern does.

Where this fits in the roadmap

1.01 Vectors is the first building block of the Maths for AI module. Next comes 1.02 Matrices, which are grids of numbers that act on vectors, then 1.03 Matrix multiplication. The idea of vectors comes back in every later area: neural-network inputs, weights, embeddings, similarity, gradients (a vector of slopes) and more.

6. Worked examples

Example 1: Arjun's two-leg journey

Leg one: a = [3, 2]. Leg two: b = [1, 2].

Total = a + b = [3 + 1, 2 + 2] = [4, 4]. Arjun is 4 blocks east and 4 blocks north of the start.
Length of leg one: sqrt(3^2 + 2^2) = sqrt(13), about 3.61 blocks in a straight line, even though he walked 3 + 2 = 5 blocks along streets (the L1 norm).
Straight-line distance from start to finish: sqrt(4^2 + 4^2) = sqrt(32), about 5.66 blocks.
Return trip: -[4, 4] = [-4, -4], which is 4 west and 4 south. Check: [4, 4] + [-4, -4] = [0, 0], the zero vector.

Example 2: Unit vector and a 3D length

Take v = [2, 3, 6].

Length = sqrt(4 + 9 + 36) = sqrt(49) = 7.
Unit vector = [2/7, 3/7, 6/7], about [0.286, 0.429, 0.857].
Check: (2/7)^2 + (3/7)^2 + (6/7)^2 = 4/49 + 9/49 + 36/49 = 49/49 = 1.

Example 3: Comparing houses, with and without scaling

House A = [120, 3, 2] and House B = [100, 4, 2].

Difference A - B = [20, -1, 0].
Raw distance = sqrt(400 + 1 + 0) = sqrt(401), about 20.02. Almost all of it comes from the size slot.
Divide the size by 100 so features are comparable: A = [1.2, 3, 2], B = [1.0, 4, 2]. Difference = [0.2, -1, 0].
New distance = sqrt(0.04 + 1) = sqrt(1.04), about 1.02. Now the bedroom difference matters as much as it should.

Example 4: Movie similarity

Movies: M1 = [0.9, 0.1] (action film), M2 = [0.8, 0.2] (action film), M3 = [0.1, 0.9] (romance).

M1 - M2 = [0.1, -0.1]; distance = sqrt(0.01 + 0.01), about 0.14.
M1 - M3 = [0.8, -0.8]; distance = sqrt(0.64 + 0.64) = sqrt(1.28), about 1.13.
M1 is much closer to M2, so a viewer who liked M1 is a better match for M2 than for M3.

Example 5: A linear combination

With a = [3, 2] and b = [1, 2]: 2a - b = 2 x [3, 2] + (-1) x [1, 2] = [6, 4] + [-1, -2] = [5, 2].

7. Comparison table or diagram

Scalar versus vector versus (preview) matrix

	Scalar	Vector	Matrix (see 1.02)
What it is	One number	Ordered list of numbers	Grid of numbers (rows and columns)
Example	5	[3, 2]	[[3, 5], [4, 2]]
Picture	A point on a number line	An arrow	A rule that moves arrows
Directions	None	Has direction and length	Describes many vectors or a transformation

The four moves on vectors

Operation	Rule	Example	Picture
Add	Add matching slots	[3, 2] + [1, 2] = [4, 4]	Chain arrows tip to tail
Scale	Multiply every slot	2 x [3, 2] = [6, 4]	Stretch the arrow
Negate	Flip every sign	-[3, 2] = [-3, -2]	Turn it around
Length	sqrt of sum of squares	length of [3, 4] = 5	Hypotenuse of a right triangle

L1 versus L2 for [3, 4]

Norm	Formula	Value	Intuition
L1	abs(3) + abs(4)	7	Walk along the streets
L2	sqrt(3^2 + 4^2)	5	Fly straight, as the crow flies

Text picture of tip-to-tail

start ----[3,2]----> A ----[1,2]----> end
start ------------[4,4]-------------> end   (the sum)

8. Common mistakes and myths

Myth: "[3, 2] and [2, 3] are the same, they are just two numbers." Order matters. Each slot has a meaning.
Mistake: adding vectors of different dimensions, such as [3, 2] + [1, 2, 5]. This is undefined; the sizes must match.
Mistake: forgetting the square root in the length. 3^2 + 4^2 = 25 is the squared length; the length is 5.
Mistake: adding lengths instead of vectors. The length of [3, 2] + [1, 2] is not the length of [3, 2] plus the length of [1, 2]. (About 5.66 versus 3.61 + 2.24 = 5.84.) Straight-line distance is never longer than the sum of the legs.
Myth: "Vectors are only for geometry." In AI they are mostly lists of features, where "direction" is a useful picture, not a physical place.
Mistake: comparing raw features on different scales and trusting the answer.
Myth: "More dimensions is always better." Extra, useless features add noise and make data sparse.
Mistake: reading each embedding slot as a human-meaningful property such as "royalty". Usually only the overall pattern has meaning.
Mistake: thinking scaling can rotate a vector. Multiplying by a number only stretches, shrinks or flips it.

9. Real-life uses with concrete examples

Navigation and games: a character's velocity is a vector; add it to the position every frame to move.
Physics and engineering: forces, wind and speed are vectors, and adding two forces is vector addition.
House-price prediction: each house is a feature vector like [120, 3, 2], and a model turns it into a predicted price.
Recommendations: a movie such as [0.9, 0.1] is compared with a viewer's taste vector; close arrows lead to a recommendation.
Search engines and chatbots: a sentence is converted to an embedding vector, and the system finds stored passages whose vectors are closest.
Language models: each word or token starts as an embedding, often hundreds or thousands of numbers (illustrative: about 300 in the video).
Image and audio: a photo, a face or a voice clip can be summarised as one vector, so that two photos of the same person land close together.
Optimisation: the gradient, which tells a model how to improve, is a vector with one slope per parameter.

10. Self-check questions

What is a scalar, and how is it different from a vector?
In the video, why do [3, 2] and [2, 3] lead Arjun to different places?
Compute [4, 1] + [2, 5].
Compute 3 x [2, -1].
Find the length of [6, 8].
Turn [6, 8] into a unit vector and show its length is 1.
What is the dimension of [120, 3, 2], and what might each slot mean?
Why can you not add [1, 2] and [1, 2, 3]?
Why might comparing houses by raw distance be misleading, and how can you fix it?
What is the L1 norm of [3, -4], and how does it differ from the L2 norm?

Answers

A scalar is a single number with no direction; a vector is an ordered list of numbers with a direction and a length.
Because the slots have fixed meanings (east, then north). [3, 2] is 3 east, 2 north; [2, 3] is 2 east, 3 north, a different endpoint.
[4 + 2, 1 + 5] = [6, 6].
[6, -3].
sqrt(36 + 64) = sqrt(100) = 10.
[6, 8] / 10 = [0.6, 0.8]; 0.36 + 0.64 = 1, so the length is 1.
Dimension 3. In the video: size, bedrooms, bathrooms.
They have different numbers of components, so there are no matching slots to add.
Features with large numbers (size) swamp small ones (bedrooms). Fix by scaling or standardising features first.
L1 = abs(3) + abs(-4) = 7. L2 = sqrt(9 + 16) = 5. L1 is the sum of absolute values; L2 is the straight-line length.

11. Key takeaways, go deeper, and what is next

Key takeaways

A vector is an ordered list of numbers; it is also an arrow with direction and length.
Order matters: each slot has a meaning.
The core moves are add, scale, negate and measure length.
Length is sqrt(sum of squares); a unit vector has length 1.
In AI, vectors describe things: houses, movies, words (embeddings).
Scale your features, or big numbers will dominate small ones.

Go deeper

Practise the four moves on paper in 2D until the pictures feel natural.
Look up the dot product and cosine similarity (they appear in later roadmap items).
Read about word embeddings (word2vec) to see 300-number words in action.
Try the L1 and L2 norms on a few vectors and notice when they differ.
Explore feature scaling (min-max scaling and standardisation).

Next on the roadmap

Next: 1.02 Matrices.

Vectors: study notes