1.03 Matrix multiplication: study notes

Learn AI by Story

1. The idea in one sentence

Matrix multiplication combines two tables of numbers into a new table by taking each row of the first table and each column of the second, multiplying the pairs that line up, and adding them into a single number. It is the one operation that sits at the heart of almost every neural network layer.

2. The story from the video

Dev has two tables of numbers and wants to combine them. He learns one rule: pick a row from the first table and a column from the second, multiply the pairs that line up, and add everything up to get ONE number. Repeat for every row and column pairing and the new table fills in. He also learns that the sizes must fit (the inner sizes match), that swapping the order gives a different answer, and that a neural network does this at every layer: inputs times a table of weights.

3. Key definitions (glossary)

Each entry has a precise definition and a one-line plain-English meaning.

Term	Precise definition	Plain English
Scalar	A single number.	One number on its own.
Vector	An ordered list of numbers, seen as a 1-column or 1-row matrix.	A list of numbers.
Matrix	A rectangular grid of numbers arranged in m rows and n columns.	A table of numbers.
Shape (size)	The pair m x n: number of rows by number of columns.	How tall and how wide the table is.
Entry (element)	The number at row i, column j, written a_ij.	One cell of the table.
Row / column	A horizontal / vertical line of entries.	Across versus down.
Matrix product AB	For A of shape m x n and B of shape n x p, the m x p matrix whose entry c_ij is the sum over k of a_ik times b_kj.	The "row times column, then add" table.
Inner dimension	The shared size n between the columns of A and the rows of B.	The sizes that must match.
Outer dimensions	The m and p that survive into the result.	The sizes you keep.
Dot product	The sum of products of matching entries of two equal-length vectors.	Multiply pairs, add. This is one cell of a matrix product.
Element-wise (Hadamard) product	Multiplying two same-shaped matrices cell by cell.	Cell times same cell. NOT matrix multiplication.
Identity matrix I	A square matrix with 1s on the diagonal and 0s elsewhere, so that AI = IA = A.	The "multiply by 1" of matrices.
Transpose A^T	The matrix whose rows are the columns of A.	Flip the table over its diagonal.
Matrix-vector product	A matrix times a column vector, giving a column vector.	A table applied to a list.
Linear transformation	A map that keeps straight lines straight and the origin fixed; every one can be written as a matrix.	A stretch, turn or shear of space.
Weight matrix	A learned table of numbers in a layer.	The knobs the network learns.
Bias	A learned vector added after the multiplication.	A learned shift.
Batch	Several input examples processed together, stacked as rows.	Many examples at once.
Broadcasting	Automatically repeating a smaller array across a bigger one so shapes fit.	Copy the bias to every row.
FLOP	One floating-point operation (add or multiply).	One small arithmetic step.
Multiply-add (MAC)	One multiplication plus one addition.	The basic unit of the work.

4. Step-by-step explanation

Step 1: Start with the smallest piece, the dot product

Take a row [1, 2] and a column [5, 7]. Multiply matching positions: 1 times 5 and 2 times 7. Add: 5 + 14 = 19. That single number is a dot product. Why: it is the natural way to ask "how much does this row respond to this column?" Each input is weighted and the weighted pieces are pooled.

Step 2: Do it for every row and column pairing

A matrix product is just many dot products arranged in a table. The cell in row i and column j of the answer uses row i of the first matrix and column j of the second. Why: this makes every row of the first matrix get tested against every column of the second, so one operation handles all pairings at once.

Step 3: Check the shapes first

If A is m x n and B is n x p, the product exists and is m x p. Write the shapes side by side: (m x n)(n x p). The two middle numbers must be equal, and they disappear. The outer numbers remain. Why: a row of A has n entries, so it can only be paired with a column that also has n entries. If the numbers differ, some entries would have no partner.

Step 4: Fill the answer cell by cell

Use the formula c_ij = a_i1 b_1j + a_i2 b_2j + ... + a_in b_nj. Read it as "row i of A, column j of B, multiply pairs, add." The index k that runs from 1 to n is the shared inner size. Why: the formula is simply the dot product written with indices, so it works for any size.

Step 5: See matrix-vector as a special case

If B has only one column, it is a vector, and the answer is also a single column. Example: [[2, 0], [1, 3]] times [4, 5]. Top entry: 2×4 + 0×5 = 8. Bottom entry: 1×4 + 3×5 = 19. Result [8, 19]. Why this matters: one neuron's output is a dot product, a whole layer for one input is a matrix-vector product, and a layer for many inputs is a full matrix-matrix product. It is one idea at three scales.

Step 6: Understand what the product means

The columns of a matrix describe where the basic directions land. Multiplying a matrix by a vector means "send this point somewhere new." Multiplying two matrices means "do one transformation, then another, in a single step." Why: this is the reason order matters. Rotating then stretching is not the same as stretching then rotating.

Step 7: Scale up to a neural-network layer

A layer computes Y = X W + b. X holds the inputs (one row per example), W the learned weights, and b the bias. The product X W does all the weighted sums for all examples in one go. Why: one big multiply is far faster on modern hardware than thousands of tiny separate ones.

5. Everything the video skipped

The general formula in simple words

c_ij = sum over k of a_ik times b_kj. In words: "the entry at row i, column j is the dot product of row i of A with column j of B." Nothing more.

Three ways to see the same product

Cell by cell (what the video did): each cell is a row-times-column dot product.
Row view: each row of AB is a combination of the rows of B, weighted by the numbers in that row of A.
Column view: each column of AB is a combination of the columns of A, weighted by the numbers in that column of B.

All three give identical numbers. Different views help in different situations.

Matrix-vector and other special cases

Matrix times column vector gives a column vector (shape m x n times n x 1 gives m x 1).
Row vector times matrix gives a row vector (1 x n times n x p gives 1 x p).
Row vector times column vector (1 x n times n x 1) gives a 1 x 1 result: the dot product, the topic of 1.04.
Column vector times row vector (n x 1 times 1 x p) gives a full n x p matrix called an outer product.

The identity matrix

For a 2 x 2 case, I = [[1, 0], [0, 1]]. Multiplying any compatible matrix by I leaves it unchanged. Check: [[1, 2], [3, 4]] times I: top-left 1×1 + 2×0 = 1, top-right 1×0 + 2×1 = 2, bottom-left 3×1 + 4×0 = 3, bottom-right 3×0 + 4×1 = 4. The matrix is the same.

Rules that hold, and the one that does not

Property	Holds?	Statement
Associativity	Yes	(AB)C = A(BC)
Distributivity	Yes	A(B + C) = AB + AC
Scalar pulling	Yes	(kA)B = k(AB)
Identity	Yes	AI = IA = A
Commutativity	NO	AB is usually not equal to BA

Associativity means you may regroup a long chain, and the grouping you choose can change the cost enormously even though the answer is the same.

Transpose of a product

(AB)^T = B^T A^T. The order reverses. This is like taking off socks and shoes: you put on socks then shoes, but you remove shoes then socks. It is used constantly when deriving how networks learn.

Composition of transformations

If B transforms a point first and A transforms the result second, the single combined matrix is AB. Notice it is written right to left: the matrix nearest the vector acts first. This is why AB and BA differ: they describe different orders of operations.

Batches and the neural layer in detail

Imagine 32 images, each flattened to 784 numbers (a 28 x 28 picture). Stack them into X of shape 32 x 784. A layer with 128 neurons has weights W of shape 784 x 128. The shapes (32 x 784)(784 x 128) share the inner 784, so the product is 32 x 128: for every example, 128 outputs. Then the bias b, a vector with 128 entries, is added to every row. That copying of b down all 32 rows is broadcasting. The whole layer is Y = X W + b, and then an activation function (a later topic) is applied.

Parameter count for this layer: 784 x 128 = 100,352 weights plus 128 biases = 100,480 learnable numbers.

Cost

For two n x n matrices, each of the n squared answer cells needs n multiplications and about n additions, so about n cubed multiply-adds (roughly 2 n cubed FLOPs). For shapes (m x n)(n x p) the count is m × n × p multiply-adds. For the layer above: 32 × 784 × 128 = 3,211,264 multiply-adds, about 6.4 million FLOPs for a single small layer and a single batch. Real models do this billions of times.

Computational notes

GPUs: Each answer cell is independent of the others, so thousands can be computed at the same moment. GPUs have thousands of simple cores and are built for exactly this pattern. Modern chips also carry units specialised for small matrix blocks.
Libraries: Never write the three loops yourself in real work. Libraries such as BLAS, NumPy and PyTorch use heavily tuned code.
Strassen: In 1969 Volker Strassen showed that two 2 x 2 matrices can be multiplied with 7 multiplications instead of 8. Applied recursively this gives a cost near n to the power 2.81 instead of n cubed. Better theoretical algorithms exist, but in practice the plain method, tuned for hardware, usually wins at neural-network sizes.
Memory: Often the bottleneck is moving numbers, not multiplying them.

History in two lines

Arthur Cayley formalised matrix multiplication in the 1850s while studying combined linear transformations. In the 1980s neural networks revived it as the core of learning, and from the 2010s GPUs made it fast enough to train huge models.

Limitations and cautions

Matrix multiplication only expresses linear relationships. Stacking several matrices without anything in between collapses into one matrix, because (AB)x = A(Bx). That is why networks insert non-linear activation functions between layers.
Dimensions grow quickly, so memory and compute can become limits.
Numbers can overflow or lose precision (very large or tiny values), a concern in real training.

Where it sits on the roadmap

Builds on 1.01 Vectors (lists of numbers) and 1.02 Matrices (tables, shapes).
Feeds directly into 1.04 Dot product and cosine similarity, which is the single-cell version of what you did here.
Later it powers layers in neural networks, attention in transformers and the backward pass of learning, where transposes appear everywhere.

6. Worked examples

Example 1: the video's 2 x 2 case

A = [[1, 2], [3, 4]], B = [[5, 6], [7, 8]]. Shapes (2 x 2)(2 x 2), inner sizes match, result 2 x 2.

Cell (1,1): row [1, 2] with column [5, 7]: 1×5 + 2×7 = 5 + 14 = 19
Cell (1,2): row [1, 2] with column [6, 8]: 1×6 + 2×8 = 6 + 16 = 22
Cell (2,1): row [3, 4] with column [5, 7]: 3×5 + 4×7 = 15 + 28 = 43
Cell (2,2): row [3, 4] with column [6, 8]: 3×6 + 4×8 = 18 + 32 = 50

AB = [[19, 22], [43, 50]].

Now the other order, BA, with B = [[5, 6], [7, 8]] first:

(1,1): 5×1 + 6×3 = 5 + 18 = 23
(1,2): 5×2 + 6×4 = 10 + 24 = 34
(2,1): 7×1 + 8×3 = 7 + 24 = 31
(2,2): 7×2 + 8×4 = 14 + 32 = 46

BA = [[23, 34], [31, 46]], which differs from AB. Order matters.

Check with the transpose rule: (AB)^T = [[19, 43], [22, 50]]. And B^T A^T with B^T = [[5, 7], [6, 8]] and A^T = [[1, 3], [2, 4]]: (1,1) 5×1 + 7×2 = 19; (1,2) 5×3 + 7×4 = 43; (2,1) 6×1 + 8×2 = 22; (2,2) 6×3 + 8×4 = 50. Same result, so the rule holds.

Check with composition: take the point x = [1, 1]. Bx = [5+6, 7+8] = [11, 15]. Then A(Bx) = [1×11 + 2×15, 3×11 + 4×15] = [41, 93]. And (AB)x = [19+22, 43+50] = [41, 93]. Identical: one matrix does both steps.

Example 2: a 2 x 3 times 3 x 2 case

A = [[1, 2, 3], [4, 5, 6]] (2 x 3), B = [[7, 8], [9, 10], [11, 12]] (3 x 2). Inner sizes 3 and 3 match, so the result is 2 x 2.

(1,1): 1×7 + 2×9 + 3×11 = 7 + 18 + 33 = 58
(1,2): 1×8 + 2×10 + 3×12 = 8 + 20 + 36 = 64
(2,1): 4×7 + 5×9 + 6×11 = 28 + 45 + 66 = 139
(2,2): 4×8 + 5×10 + 6×12 = 32 + 50 + 72 = 154

AB = [[58, 64], [139, 154]].

Interesting: BA also exists here, but it is 3 x 3 (shapes (3 x 2)(2 x 3)). So AB and BA are not even the same size.

A failing case: A (2 x 3) times another 2 x 3. The shapes (2 x 3)(2 x 3) have inner sizes 3 and 2. They disagree, so the product does not exist.

Example 3: a mini neural layer (numbers are illustrative)

Two examples with three features each: X = [[1, 0, 2], [3, 1, 0]] (2 x 3). Weights W = [[1, 2], [0, 1], [4, 0]] (3 x 2). Bias b = [1, -1].

XW:

(1,1): 1×1 + 0×0 + 2×4 = 9
(1,2): 1×2 + 0×1 + 2×0 = 2
(2,1): 3×1 + 1×0 + 0×4 = 3
(2,2): 3×2 + 1×1 + 0×0 = 7

XW = [[9, 2], [3, 7]]. Add b to each row (broadcasting): [[9+1, 2-1], [3+1, 7-1]] = [[10, 1], [4, 6]]. Output shape 2 x 2: two examples, two outputs each.

7. Comparison table

Feature	Matrix product AB	Element-wise product A × B	Dot product a . b
Inputs	m x n and n x p	Same shape both	Two vectors, same length
Output	m x p matrix	Same shape matrix	One number
Rule	Row times column, add	Same cell times same cell	Multiply pairs, add
Shapes must	Match on inner size	Be identical	Be equal length
Commutative?	No	Yes	Yes
Example with [[1,2],[3,4]] and [[5,6],[7,8]]	[[19,22],[43,50]]	[[5,12],[21,32]]	n/a

Shape cheat sheet:

Left	Right	Result	Name
2 x 3	3 x 2	2 x 2	works
3 x 2	2 x 3	3 x 3	works
2 x 3	2 x 3	none	inner mismatch
m x n	n x 1	m x 1	matrix-vector
32 x 784	784 x 128	32 x 128	neural layer

8. Common mistakes and myths

Myth: "It is just multiplying matching cells." That is the element-wise product, a different operation. Matrix multiplication uses whole rows and columns. Using the wrong one gives [[5,12],[21,32]] instead of [[19,22],[43,50]].
Mistake: ignoring shapes. Always write the shapes before computing. A mismatch means the operation is not defined.
Mistake: assuming AB = BA. It is usually false, and sometimes BA does not even exist.
Mistake: mixing up which outer sizes remain. The result is (rows of A) x (columns of B).
Mistake: forgetting the reversal in (AB)^T. It is B^T A^T, not A^T B^T.
Mistake: reading the cost as n squared. Each of the n squared cells needs n steps, so the total is about n cubed.
Mistake: using columns as rows in the second matrix. Run down a column of B, not across a row.
Myth: "Stacking many linear layers makes a network deeper in power." Without non-linear activations, it collapses to a single matrix.
Myth: "GPUs are fast because they are faster at everything." They are fast because this specific job splits into many independent pieces.

9. Real-life uses with concrete examples

Image classifiers: A 28 x 28 picture becomes 784 numbers and is multiplied by a weight matrix to give scores for ten digits.
Language models: Words become vectors; every layer multiplies them by learned matrices. A large share of the compute in training and running a chatbot is matrix multiplication.
Recommendations: A table of users by tastes multiplied by a table of tastes by movies predicts how much each user will like each movie.
Graphics and games: Rotating, scaling and moving 3D objects is done with matrices, and chained transformations are multiplied together once, then applied to millions of points.
Economics and networks: Multiplying a connection table by itself counts two-step routes between cities, web pages or people.
Hardware: GPU and AI-accelerator performance is advertised largely by how many matrix operations per second they can do.

10. Self-check questions

In your own words, what does one cell of a matrix product represent?
A is 4 x 3 and B is 3 x 5. Does AB exist? What shape is it? What about BA?
Compute [[2, 1], [0, 3]] times [[1, 4], [2, 0]].
Why must the inner sizes match?
Is matrix multiplication commutative? Give a one-line reason or example.
What is (AB)^T equal to in terms of A^T and B^T?
A layer takes a batch of 64 inputs with 100 features each and has 20 neurons. What are the shapes of X, W and the output? How many multiply-adds does X W need?
What is the difference between A × B (element-wise) and AB?
Why do networks put a non-linear function between layers?
Why are GPUs a good fit for matrix multiplication?

Answers

A dot product of one row of the first matrix with one column of the second: the weighted total of one row against one column.
AB exists and is 4 x 5 (inner 3 and 3 match). BA does not exist, since (3 x 5)(4 x 3) has inner sizes 5 and 4.
(1,1): 2×1 + 1×2 = 4; (1,2): 2×4 + 1×0 = 8; (2,1): 0×1 + 3×2 = 6; (2,2): 0×4 + 3×0 = 0. Result [[4, 8], [6, 0]].
A row of the first has as many entries as it has columns; a column of the second has as many entries as it has rows. Pairs only line up if those counts are equal.
No. In the video AB = [[19,22],[43,50]] but BA = [[23,34],[31,46]].
B^T A^T (order reversed).
X is 64 x 100, W is 100 x 20, output is 64 x 20. Multiply-adds: 64 × 100 × 20 = 128,000.
Element-wise multiplies same-position cells and needs equal shapes. AB uses row-times-column sums and needs matching inner sizes.
Without it, many linear layers collapse into one matrix and the network could only learn straight-line relationships.
Each output cell can be computed independently, so thousands of cores can work at the same moment.

11. Key takeaways, go-deeper list, and what comes next

Key takeaways

Matrix product cell = dot product of a row and a column.
Shapes: (m x n)(n x p) gives m x p. Inner sizes match and vanish.
Order matters. AB is usually not BA. (AB)^T = B^T A^T.
Matrix times vector is the special case; a neural layer is Y = X W + b.
Cost is m × n × p multiply-adds, which GPUs perform in parallel.

Go deeper

Try the row view and column view on the video's example and confirm you get the same table.
Look at how NumPy writes it (the @ operator) and how it differs from × for arrays.
Read about linear transformations as rotations, scalings and shears in 2D.
Explore Strassen's algorithm and why hardware-tuned simple loops still win in practice.
Preview: think about why 3 stacked layers without activations equal one matrix.

Next roadmap ID: 1.04 Dot product and cosine similarity. You just used dot products as the building block of each cell; next you will see what a single dot product tells you about how alike two vectors are.

Matrix multiplication: study notes