1.02 Matrices: study notes

Learn AI by Story

1. The idea in one sentence

A matrix is a rectangular grid of numbers, arranged in rows and columns, that can store a table of information or act as a machine that transforms vectors, and it is the main way AI stores data and learned knowledge.

2. The story from the video

Maya runs a small shop and keeps a notebook of sales. Each row is a day and each column is a product. On Monday she sold 3 apples and 5 loaves of bread, so the row is [3, 5]. On Tuesday she sold 4 apples and 2 loaves, so the row is [4, 2]. Together the notebook is a 2 x 2 matrix. The video then shows that a matrix can also transform a vector: [[2, 0], [0, 2]] doubles [2, 1] into [4, 2], and a 90-degree rotation matrix turns [3, 1] into [-1, 3]. In AI, a photo is a grid of pixel numbers, a dataset is a table of examples, and each neural-network layer stores its weights as a matrix. Big models hold billions of numbers (70 billion is an illustrative figure), and training adjusts them.

3. Key definitions (glossary)

Term	Precise definition	Plain English
Matrix	A rectangular array of numbers arranged in rows and columns.	A spreadsheet of numbers.
Row	A horizontal line of entries in a matrix.	One line across, like one day in Maya's notebook.
Column	A vertical line of entries in a matrix.	One line down, like one product.
Entry (element)	A single number inside a matrix, written a_ij for row i, column j.	One cell of the grid.
Size (shape, order)	rows x columns. A matrix with 2 rows and 3 columns is 2 x 3.	How tall and how wide the grid is.
Square matrix	A matrix with the same number of rows and columns.	A grid shaped like a square.
Identity matrix (I)	A square matrix with 1s on the main diagonal and 0s elsewhere. Multiplying a vector by it leaves the vector unchanged.	The "do nothing" machine.
Diagonal matrix	A square matrix whose non-zero entries are only on the main diagonal.	Stretches each direction independently.
Zero matrix	A matrix whose entries are all 0.	A grid of zeros; it sends every vector to zero.
Transpose	The matrix flipped across its main diagonal, so rows become columns. Written A^T.	Turn the grid on its side: rows become columns.
Matrix addition	Adding two matrices of the same size entry by entry.	Add the matching cells.
Scalar multiplication	Multiplying every entry by one number.	Scale every cell.
Matrix-vector product	The vector obtained by taking the dot product of each row of the matrix with the vector.	Feed a vector into the machine and get a new vector.
Linear transformation	A rule that sends vectors to vectors, keeping lines straight and the origin fixed. Every such rule in 2D or 3D can be written as a matrix.	A reshaping of space such as stretching or rotating.
Rotation matrix	A matrix that turns every vector by a fixed angle around the origin.	Spin the arrow.
Scaling matrix	A diagonal matrix that stretches or shrinks along the axes.	Make bigger or smaller.
Shear matrix	A matrix that slides one coordinate in proportion to another.	Push the top of a stack sideways.
Dot product	Multiply matching entries of two vectors and add them up.	One number that says how well two lists line up.
Tensor	A grid of numbers with any number of dimensions: a scalar (0D), vector (1D), matrix (2D), or higher.	A matrix with more directions.
Weights	Learned numbers that say how strongly each input affects each output.	The knobs a model tunes.
Bias	A learned vector added after the matrix step.	A built-in nudge.
Layer	One stage of a neural network that turns an input vector into an output vector, usually with a weight matrix plus a bias.	One processing step.
Dataset matrix	A matrix with one row per example and one column per feature.	The table AI learns from.

4. Step-by-step explanation

Step 1: Start from a table

Maya's notebook is a table. Monday: 3 apples, 5 bread. Tuesday: 4 apples, 2 bread. Strip away the labels and you have a matrix:

	Apples	Bread
Monday	3	5
Tuesday	4	2

Written as a matrix: [[3, 5], [4, 2]]. Each inner bracket is one row. A matrix is nothing more mysterious than a table of numbers where position tells you what each number means.

Step 2: Read the size

Size is rows x columns, always in that order. Maya's matrix has 2 rows and 2 columns, so it is 2 x 2. If she adds cookies, so that Monday is [3, 5, 1] and Tuesday is [4, 2, 8], the matrix [[3, 5, 1], [4, 2, 8]] is 2 x 3: two rows, three columns. A 3 x 2 matrix is a different shape, with three rows and two columns.

Why this matters: shapes decide what operations are allowed, and mismatched shapes are the most common error when working with matrices.

Step 3: Find entries by position

The entry in row i, column j is written a_ij. In Maya's matrix, a_11 = 3 (Monday apples), a_12 = 5 (Monday bread), a_21 = 4 and a_22 = 2. Row first, column second. Remember: "rows then columns" like reading a table.

Step 4: Do simple arithmetic

Addition works cell by cell, and needs the same size. If the second week of sales was [[1, 2], [0, 3]], the two-week total is [[3 + 1, 5 + 2], [4 + 0, 2 + 3]] = [[4, 7], [4, 5]].

Scalar multiplication scales every cell. Doubling Maya's matrix gives 2 x [[3, 5], [4, 2]] = [[6, 10], [8, 4]].

Step 5: Transpose

Flip the grid so rows become columns. For [[3, 5, 1], [4, 2, 8]] (2 x 3), the transpose is [[3, 4], [5, 2], [1, 8]] (3 x 2). The first row [3, 5, 1] became the first column. A 2 x 3 matrix always becomes 3 x 2.

Step 6: See a matrix as a machine

Here is the big shift. A matrix can act on a vector. Give it a vector in, get a vector out. Take the doubling matrix [[2, 0], [0, 2]] and the vector [2, 1]: out comes [4, 2]. The arrow was stretched to twice its length. A different matrix can do something else, such as rotate.

Step 7: Apply a matrix to a vector: each row meets the vector

The rule: take each row of the matrix, multiply matching entries with the vector, and add. Each row produces one number in the output.

For [[2, 0], [0, 2]] applied to [3, 1]:

Row 1 is [2, 0]: 2 x 3 + 0 x 1 = 6.
Row 2 is [0, 2]: 0 x 3 + 2 x 1 = 2.
Result: [6, 2].

So the vector doubled, as expected. Why does the rule work this way? Each output number is a weighted mix of the inputs, and the row holds the weights. That is exactly what a neural-network layer needs.

Step 8: Rotation

The 90-degree (counter-clockwise) rotation matrix is [[0, -1], [1, 0]]. Apply it to [3, 1]:

Row 1 is [0, -1]: 0 x 3 + (-1) x 1 = -1.
Row 2 is [1, 0]: 1 x 3 + 0 x 1 = 3.
Result: [-1, 3].

The arrow has turned a quarter-turn but kept its length: the length of [3, 1] is sqrt(10) and so is the length of [-1, 3].

Step 9: Matrices in AI

A photo is a grid of brightness numbers, so it is a matrix.
A dataset is a table with one row per example and one column per feature, so it is a matrix.
A neural-network layer holds a weight matrix. The input vector goes in, the matrix multiplies it, and the output vector comes out.
Training means nudging those numbers, millions or billions of them, so predictions improve. Big models hold billions of numbers (70 billion is an illustrative figure).

5. Everything the video skipped

Special matrices

The identity matrix has 1s down the diagonal. In 2 x 2 it is [[1, 0], [0, 1]], and multiplying any vector by it returns that vector: [[1, 0], [0, 1]] x [3, 1] = [3, 1]. A diagonal matrix such as [[2, 0], [0, 3]] stretches the first direction by 2 and the second by 3. The zero matrix flattens everything to [0, 0]. A square matrix has the same number of rows and columns; only square matrices can be identity or diagonal in the usual sense.

Two ways to read the matrix-vector product

The row view (used in the video): each row takes a dot product with the vector. The column view: the output is a mix of the matrix's columns, with the vector's entries as the amounts. Try [[1, 2], [3, 4]] x [5, 6].

Row view: 1 x 5 + 2 x 6 = 17, and 3 x 5 + 4 x 6 = 39. Result: [17, 39].
Column view: 5 x [1, 3] + 6 x [2, 4] = [5, 15] + [12, 24] = [17, 39].

Same answer. The column view is powerful: it says the output is a linear combination (see 1.01) of the columns.

Where the columns of a matrix come from

The first column of a 2D matrix is where the east arrow [1, 0] lands, and the second column is where the north arrow [0, 1] lands. Check with the rotation matrix [[0, -1], [1, 0]]: [1, 0] goes to [0, 1] (the first column) and [0, 1] goes to [-1, 0] (the second column). Quarter-turn, as promised. So you can build any transformation by deciding where the two basic arrows go.

The general rotation matrix

To rotate by an angle theta, use [[cos theta, -sin theta], [sin theta, cos theta]]. For 90 degrees, cos is 0 and sin is 1, giving [[0, -1], [1, 0]]. For 180 degrees, cos is -1 and sin is 0, giving [[-1, 0], [0, -1]], which flips every vector to its negative.

Scaling and shear

A scaling matrix [[a, 0], [0, b]] stretches x by a and y by b. For [[2, 0], [0, 3]] x [3, 1] = [6, 3]. A shear matrix such as [[1, 1], [0, 1]] adds the second coordinate to the first: [[1, 1], [0, 1]] x [3, 1] = [1 x 3 + 1 x 1, 0 x 3 + 1 x 1] = [4, 1]. Shapes get slanted like a pushed deck of cards.

Tensors

A scalar is a single number (0 dimensions of grid), a vector is a list (1D), a matrix is a table (2D). A tensor is the general idea: a grid of any number of dimensions. A colour photo is naturally 3D: height x width x 3 colour channels. A batch of photos is 4D. The tensors in frameworks such as PyTorch and TensorFlow are exactly this, which is where those names come from.

Images as matrices

A grayscale picture 28 x 28 pixels has 28 x 28 = 784 numbers, each one a brightness (often 0 for black up to 255 for white). The classic handwritten-digit dataset is of this kind. A colour picture of the same size has three layers (red, green, blue), so 28 x 28 x 3 = 2,352 numbers. Sometimes the 28 x 28 grid is unrolled into a single vector of length 784 so that a layer can process it.

Datasets as n_samples x n_features

In machine learning a dataset is normally a matrix X with n_samples rows (one per example) and n_features columns (one per measurement). Using house features [size, bedrooms, baths], 1,000 houses would form a 1,000 x 3 matrix (illustrative). Each row is a feature vector from 1.01.

Weights and biases in a layer

A neural-network layer computes output = W x input + b. W is the weight matrix, b is the bias vector. If W is 2 x 3, the layer takes a 3-number input and gives a 2-number output. The shape rule: the number of columns of W must equal the length of the input; the number of rows of W is the length of the output. For a layer from 784 inputs to 128 outputs, W is 128 x 784, which has 128 x 784 = 100,352 weights, plus 128 biases, giving 100,480 learned numbers (a worked count, not a recommendation).

Why GPUs love matrices

A matrix-vector product is lots of small multiply-and-add jobs that do not depend on each other. Graphics processors (GPUs) were built to do thousands of such jobs at once, first for pictures and games, which also rely on matrices. That match is a big reason modern AI took off.

Limits and caveats

A matrix can only represent linear transformations: straight lines stay straight, and the origin stays put. Real-world patterns are rarely that simple, so neural networks add a non-linear step between layers. Also, not every matrix can be undone (inverted). Matrices of billions of entries are expensive to store and run.

A little history

Arrays of numbers were used in China more than two thousand years ago to solve equations. The word "matrix" was introduced in the 1850s by James Joseph Sylvester, and Arthur Cayley developed the algebra of matrices soon after. Modern AI inherits this tool almost unchanged.

Where this fits in the roadmap

Before this: 1.01 Vectors. A matrix is a stack of row vectors (or column vectors). After this: 1.03 Matrix multiplication, which multiplies two matrices together (the video only multiplied a matrix by a vector). Later topics build on both.

6. Worked examples

Example 1: Maya's two-week sales

Week 1 = [[3, 5], [4, 2]]. Week 2 = [[1, 2], [0, 3]].

Both are 2 x 2, so adding is allowed.
Total = [[3 + 1, 5 + 2], [4 + 0, 2 + 3]] = [[4, 7], [4, 5]].
Reading it back: over two weeks, Monday sold 4 apples and 7 loaves; Tuesday sold 4 apples and 5 loaves.
Scaling check: 2 x Week 1 = [[6, 10], [8, 4]].

Example 2: Three transformations of the same vector [3, 1]

Doubling matrix [[2, 0], [0, 2]]: [2 x 3 + 0 x 1, 0 x 3 + 2 x 1] = [6, 2].
Rotation by 90 degrees [[0, -1], [1, 0]]: [0 x 3 + (-1) x 1, 1 x 3 + 0 x 1] = [-1, 3].
Shear [[1, 1], [0, 1]]: [1 x 3 + 1 x 1, 0 x 3 + 1 x 1] = [4, 1].

Lengths: the doubled vector has length sqrt(36 + 4) = sqrt(40), about 6.32, which is twice sqrt(10), about 3.16. The rotated vector has length sqrt(1 + 9) = sqrt(10), unchanged. The sheared vector has length sqrt(17), about 4.12, so shear changes the length.

Example 3: A 2 x 3 matrix on a 3-number vector

W = [[1, 0, 2], [0, 1, 1]] and x = [3, 1, 2].

Shapes: W is 2 x 3 and x has 3 entries, so it fits. The output has 2 entries.
Row 1: 1 x 3 + 0 x 1 + 2 x 2 = 3 + 0 + 4 = 7.
Row 2: 0 x 3 + 1 x 1 + 1 x 2 = 0 + 1 + 2 = 3.
W x = [7, 3].
Add a bias b = [1, -1]: output = [7 + 1, 3 - 1] = [8, 2].

This is one tiny neural-network layer before any non-linear step.

Example 4: Transpose and size

A = [[3, 5, 1], [4, 2, 8]] is 2 x 3. Its transpose is [[3, 4], [5, 2], [1, 8]], which is 3 x 2. Check a_12 of A is 5, and entry (2,1) of the transpose is 5. Transposing swaps the indices.

Example 5: Counting the numbers in an image batch (illustrative)

A batch of 32 grayscale images of 28 x 28 has 32 x 28 x 28 = 25,088 numbers. Flatten each image into a vector of 784 and stack them: a 32 x 784 matrix, and 32 x 784 = 25,088. The count agrees.

7. Comparison table or diagram

Vector versus matrix versus tensor

	Vector	Matrix	Tensor
Dimensions of the grid	1	2	3 or more (general)
Written as	[3, 2]	[[3, 5], [4, 2]]	e.g. 28 x 28 x 3 block
Size	length n	rows x columns	one size per dimension
AI example	One house, one word	A dataset, a weight layer	A colour photo, a batch of images

Special matrices at a glance

Matrix	Looks like (2 x 2)	What it does to [3, 1]
Identity	[[1, 0], [0, 1]]	[3, 1] (nothing)
Doubling	[[2, 0], [0, 2]]	[6, 2]
Scaling	[[2, 0], [0, 3]]	[6, 3]
Rotation 90 degrees	[[0, -1], [1, 0]]	[-1, 3]
Shear	[[1, 1], [0, 1]]	[4, 1]
Zero	[[0, 0], [0, 0]]	[0, 0]

Text picture of a layer

input vector  (3 numbers)  --->  [ weight matrix W, 2 x 3 ]  --->  output vector (2 numbers)
                                          + bias (2 numbers)

8. Common mistakes and myths

Mistake: reading size as columns x rows. It is rows x columns.
Mistake: mixing up a_ij order. Row first, column second.
Mistake: adding matrices of different sizes. They must match.
Mistake: shapes that do not fit in a matrix-vector product. A 2 x 3 matrix needs a vector of length 3.
Myth: "A matrix is just a number grid and nothing more." It is also a transformation of space.
Myth: "Every matrix is a rotation." Rotations are one special family; there are stretches, shears, projections and more.
Mistake: swapping rows and columns when applying the matrix. Each row meets the vector.
Myth: "AI stores facts in a database." A model's learned knowledge sits mostly in the numbers of its weight matrices.
Myth: "More weights always means smarter." Size helps but data, training and design matter too.
Mistake: assuming order does not matter. Later, in 1.03, you will see that A times B and B times A usually differ.

9. Real-life uses with concrete examples

Spreadsheets and databases: sales by day and product, like Maya's notebook, are matrices.
Photos and screens: each image is a grid of pixel values; filters like blur or sharpen are matrix operations.
Games and animation: rotating, scaling and moving 3D objects are done with matrices, many thousands of times per second.
Recommendation systems: a matrix of users by movies, with ratings as entries, is used to predict what you will like.
Language models: embeddings are rows of a big matrix, and every layer multiplies by weight matrices.
Self-driving and robotics: camera images, sensor readings and robot-arm movements all use matrices.
Machine learning in general: the dataset X is an n_samples x n_features matrix, and training adjusts the weights.
Large AI models: billions of weights (illustrative: 70 billion) are stored as many matrices.

10. Self-check questions

What is a matrix, and what does "2 x 3" mean?
In Maya's notebook [[3, 5], [4, 2]], what is the entry a_21 and what does it mean?
Compute [[1, 2], [3, 4]] + [[5, 6], [7, 8]].
Compute 3 x [[1, 0], [2, 5]].
What is the transpose of [[1, 2, 3], [4, 5, 6]]?
Apply [[2, 0], [0, 2]] to [2, 1].
Apply [[0, -1], [1, 0]] to [3, 1]. What happened to the vector?
Compute [[1, 2], [3, 4]] x [1, 1].
A layer has a 4 x 10 weight matrix. How long must the input be, how long is the output, and how many weights are there?
Why do GPUs suit matrix work?

Answers

A matrix is a rectangular grid of numbers in rows and columns; 2 x 3 means 2 rows and 3 columns.
a_21 = 4: row 2 (Tuesday), column 1 (apples), so 4 apples on Tuesday.
[[1 + 5, 2 + 6], [3 + 7, 4 + 8]] = [[6, 8], [10, 12]].
[[3, 0], [6, 15]].
[[1, 4], [2, 5], [3, 6]], a 3 x 2 matrix.
[2 x 2 + 0 x 1, 0 x 2 + 2 x 1] = [4, 2].
[0 x 3 + (-1) x 1, 1 x 3 + 0 x 1] = [-1, 3]. The vector was rotated 90 degrees counter-clockwise and its length stayed sqrt(10).
[1 x 1 + 2 x 1, 3 x 1 + 4 x 1] = [3, 7].
Input length 10 (the number of columns), output length 4 (the number of rows), and 4 x 10 = 40 weights.
A matrix product is made of many independent multiply-and-add jobs, and GPUs run thousands of such jobs at once.

11. Key takeaways, go deeper, and what is next

Key takeaways

A matrix is a grid of numbers; size is rows x columns.
Entry a_ij is row i, column j.
Add cell by cell (same size); scale every cell; transpose swaps rows and columns.
Applying a matrix to a vector: each row meets the vector (a dot product), giving one output number per row.
Matrices can stretch, rotate and shear space.
In AI: images, datasets and layer weights are all matrices (or tensors), and training tunes their numbers.

Go deeper

Draw the effect of the doubling, rotation and shear matrices on a small square.
Practise the row view and the column view on small matrices until both give the same answer.
Read about tensors and shapes in a deep-learning library.
Look at how a 28 x 28 digit becomes a vector of 784 numbers.
Preview the idea of multiplying two matrices together.

Next on the roadmap

Next: 1.03 Matrix multiplication.

Matrices: study notes