1.18 Linear transformations: matrices as motion: study notes

Learn AI by Story

1. The idea in one sentence

A linear transformation is a way of moving every point of the plane (or space) so that straight lines stay straight and the origin stays put, and a matrix is simply the list of where the two basic arrows land after the move, which is enough to tell you where everything else goes.

2. The story from the video

Mira the map maker has a grid-paper map and a magic lens. When she looks through it, the map is stretched, tilted or turned, yet the grid lines stay straight, parallel and evenly spaced, and the town at the centre never moves. Mira discovers she does not need to redraw the whole map. She only needs to note where two arrows, one pointing east and one pointing north, end up. Write those two landing spots as the columns of a small table and she has captured the entire lens. Every other point follows by simple scaling and adding.

3. Key definitions

Each term has a precise meaning followed by a plain meaning.

Vector: an ordered list of numbers, drawn as an arrow from the origin. Plain meaning: an arrow, or a point reached by that arrow.
Transformation: a function that takes a vector in and gives a vector out. Plain meaning: a rule that moves every point somewhere.
Linear transformation: a transformation that keeps grid lines parallel and evenly spaced and leaves the origin fixed. Equivalently, L(a v + b w) = a L(v) + b L(w) for all numbers a, b and vectors v, w. Plain meaning: the lens that never bends lines and never moves the centre.
Origin: the point (0, 0), where the axes cross. Plain meaning: home.
Basis vectors i-hat and j-hat: the arrows i-hat = (1, 0) and j-hat = (0, 1) that define the usual grid. Plain meaning: one step east and one step north.
Linear combination: a sum of vectors, each multiplied by a number, such as 3 v + 2 w. Plain meaning: mix of scaled arrows.
Span: all vectors reachable as linear combinations of a given set. Two non-parallel 2D vectors span the whole plane. Plain meaning: everything you can build from these arrows.
Matrix: a rectangular table of numbers. For a 2D linear transformation, the columns are the images of i-hat and j-hat. Plain meaning: the transformation written as a table.
Column: one vertical line of a matrix. Plain meaning: where one basis arrow lands.
Matrix-vector product: scale each column by the matching entry of the vector and add. Equivalently, each output entry is the dot product of one row with the vector. Plain meaning: applying the move to a point.
Composition: doing one transformation and then another. Plain meaning: two lenses in a row.
Matrix multiplication: the matrix of the composition. The right-hand matrix acts first. Plain meaning: the table for "do this, then that".
Identity matrix: the matrix that moves nothing; its columns are (1, 0) and (0, 1). Plain meaning: the do-nothing lens.
Rotation: a transformation that turns the plane about the origin. Plain meaning: spin the map.
Shear: a transformation that slides layers sideways, keeping one axis fixed. Plain meaning: push the top of a deck of cards sideways.
Stretch (scaling): a transformation that multiplies each axis direction by its own factor. Plain meaning: enlarge by different amounts in different directions.
Weight matrix: the matrix of learned numbers in a dense neural-network layer. Plain meaning: the layer's lens.
Affine transformation: a linear transformation followed by a shift (adding a vector). Plain meaning: a lens plus a slide, so the origin may move.
Activation function: a non-linear function (such as ReLU, which replaces negatives with 0) applied after a layer. Plain meaning: the part that adds bends.

4. Step-by-step explanation

Step 1: Transformations as movement

A transformation takes an input vector and produces an output vector. The 3Blue1Brown lesson on linear transformations suggests picturing it as movement: every point slides to a new place. Think of the whole plane as a rubber sheet being moved.

Step 2: The two rules of "linear"

A transformation is linear when two things hold: all lines remain lines (grid lines stay parallel and evenly spaced), and the origin stays in place. Angles between grid lines may change, so a shear is allowed, but curving is not. A shift of the whole plane is not allowed because it moves the origin.

Step 3: Why two arrows are enough

Every vector in the plane is a mix of i-hat and j-hat. The vector (x, y) equals x times i-hat plus y times j-hat. Because the transformation keeps grid lines parallel and evenly spaced, the mix survives the move: the output is x times the new i-hat plus y times the new j-hat. So knowing where just two arrows land fixes the fate of every point. This is the central insight of the video and of the 3Blue1Brown lesson.

Step 4: Writing the landing spots as a matrix

Put where i-hat lands in the first column and where j-hat lands in the second. If i-hat goes to (1, -2) and j-hat goes to (3, 0), the matrix is written with rows [1, 3] and [-2, 0]. For an input (x, y) the output is x(1, -2) + y(3, 0) = (x + 3y, -2x). The lesson uses exactly these landing spots as its example.

Step 5: Applying the matrix to a vector

Take v = (-1, 2). That is -1 times i-hat plus 2 times j-hat. After the move it is -1 times (1, -2) plus 2 times (3, 0). First part: (-1, 2). Second part: (6, 0). Sum: (5, 2). The lesson reaches the same (5, 2).

Step 6: Reading a matrix at a glance

Look at the columns. They tell the story. Columns (1, 0) and (0, 1) mean nothing moves. Columns (0, 1) and (-1, 0) mean a quarter turn counterclockwise. Columns (1, 0) and (1, 1) mean the east arrow stays but the north arrow leans to the east. Columns (2, 0) and (0, 3) mean double along east and triple along north.

Step 7: Combining moves, composition

Doing one move and then another is a single new linear move. To find its matrix, track where i-hat and j-hat end up after both. The matrix multiplication lesson stresses that composition is read right to left: in the product "second times first", the right matrix acts first. The lesson's example is a 90 degree rotation followed by a shear, giving a combined matrix with rows [1, -1] and [1, 0].

Step 8: Why order matters

Rotating then stretching is not the same as stretching then rotating. The lesson notes that matrix multiplication is not commutative, and shows that a shear followed by a rotation differs greatly from a rotation followed by a shear. The grouping, however, never matters: (AB)C = A(BC), because both apply C first, then B, then A.

Step 9: Matrices in neural networks

A dense layer takes an input vector x and computes W x, a matrix-vector product, where W is the weight matrix. The d2l textbook says this same operation computes each layer's outputs from the previous layer's outputs. Each output entry is the dot product of one row of W with x. If the layer has a bias b, it computes W x + b, which is affine, not strictly linear, because adding b moves the origin.

Step 10: Why activations are needed

Stack two linear layers and the result is just one linear transformation: W2 (W1 x) equals (W2 W1) x. Two matrices collapse into one product matrix. So a deep stack of purely linear layers is no more expressive than a single layer. A non-linear activation between layers, such as ReLU, supplies the bends that make depth worthwhile.

5. Everything the video skipped

Non-square matrices

A matrix with m rows and n columns takes an n-dimensional input and gives an m-dimensional output. The d2l textbook states that an m by n matrix times an n-dimensional vector gives an m-dimensional vector. A 3 by 2 matrix lifts the plane into 3D space; a 1 by 3 matrix crushes space onto a single number line.

Three dimensions

The idea is identical with three basis arrows, so the matrix has three columns of three numbers each. Many 3D graphics operations (turning an object, scaling it) are such matrices.

The Span lesson and basis

The 3Blue1Brown lesson on span explains that the span of a set of vectors is everything you can reach with linear combinations, and defines a basis as a set of linearly independent vectors that span the space. The standard basis i-hat, j-hat is only one choice; any two non-parallel 2D vectors work. This reappears in 1.21 Basis and change of basis.

Affine maps and homogeneous thinking

Real transformations often include a shift. The shift is not linear, but a linear transformation plus a shift is affine, the form W x + b used in neural networks. Because b moves the origin, an affine layer does not satisfy the two rules of "linear" strictly.

Inverses, rank and determinants

Some transformations can be undone, and some cannot. A move that squashes the plane onto a line destroys information. The next lessons, 1.19 Determinant and matrix inverse and 1.20 Rank and linear independence, quantify how much area a transformation scales and whether it can be undone. As a preview, the lesson-style matrix with columns (1, -2) and (3, 0) turns the unit square into a parallelogram with corners (0, 0), (1, -2), (4, -2) and (3, 0), and its area is 6, found by the formula 1 times 0 minus 3 times (-2) (computed here for the example; the formula itself is taught in 1.19).

Special motions

Rotation, shear and stretch are the three basic building blocks. Reflection (flipping across a line) is another, for instance the matrix with columns (0, 1) and (1, 0) swaps the axes. Many matrices can be thought of as a rotation, a stretch and another rotation in sequence, which is the idea behind the singular value decomposition in 1.15 Eigenvectors and SVD.

Eigenvectors link

An eigenvector is a direction that a transformation only stretches, without turning. If you picked those directions as your grid, the matrix would be a pure stretch. See 1.15.

Limits and trade-offs

Linear maps cannot bend, curve or create new features, so they cannot model most real-world patterns on their own.
Linear maps are very easy to compute and analyse, which is why they are the workhorse inside neural networks.
Matrix multiplication is cheap to run on modern hardware, but matrix size grows quickly with the number of inputs and outputs.
Linear thinking is a model, not a law: a real lens or a real tissue rarely stays perfectly linear.

Roadmap links

Earlier: 1.15 Eigenvectors and SVD, 1.16 Loss landscapes, and 1.17 Vector norms (norms tell us how much a transformation stretches a vector). Later: 1.19 Determinant and matrix inverse, 1.20 Rank and linear independence, 1.21 Basis and change of basis, then 1.22 Orthogonality and projections.

6. Worked examples

Example A: Mira's transformation on a vector and a second vector

Landing spots: i-hat goes to (1, -2), j-hat goes to (3, 0). So the matrix has columns (1, -2) and (3, 0), written by rows as [1, 3] and [-2, 0], and the rule is L(x, y) = (x + 3y, -2x).

Vector (-1, 2):

Split into parts: -1 times i-hat and 2 times j-hat.
Scale the columns: -1 times (1, -2) = (-1, 2), and 2 times (3, 0) = (6, 0).
Add: (-1 + 6, 2 + 0) = (5, 2).
Check with the rule: x + 3y = -1 + 6 = 5, and -2x = 2. Same answer.

A second vector, (2, 1):

2 times (1, -2) = (2, -4), and 1 times (3, 0) = (3, 0).
Add: (5, -4).
Check: x + 3y = 2 + 3 = 5; -2x = -4. Correct.

A third check from the lesson-style numbers: the matrix with rows [1, -6] and [4, 2] applied to (3, -4) gives 3 times (1, 4) plus (-4) times (-6, 2) = (3, 12) + (24, -8) = (27, 4), matching the lesson.

Example B: composition and order

Let R be the 90 degree counterclockwise rotation: columns (0, 1) and (-1, 0), rows [0, -1] and [1, 0]. Let S be the stretch by 2 along east and 3 along north: columns (2, 0) and (0, 3), rows [2, 0] and [0, 3].

Rotate first, then stretch (the product S times R):

R sends i-hat to (0, 1). Then S sends (0, 1) to (0, 3). So the first column is (0, 3).
R sends j-hat to (-1, 0). Then S sends (-1, 0) to (-2, 0). So the second column is (-2, 0).
Matrix: rows [0, -2] and [3, 0].

Stretch first, then rotate (the product R times S):

S sends i-hat to (2, 0). Then R sends (2, 0) to (0, 2). First column (0, 2).
S sends j-hat to (0, 3). Then R sends (0, 3) to (-3, 0). Second column (-3, 0).
Matrix: rows [0, -3] and [2, 0].

The two matrices differ, so order matters. Test with the point (1, 2). Rotate then stretch: R sends it to (-2, 1), then S gives (-4, 3). Stretch then rotate: S sends it to (2, 6), then R gives (-6, 2). Different destinations, confirming the matrices.

A related check from the matrix multiplication lesson: rotation followed by shear (shear columns (1, 0), (1, 1)) gives rows [1, -1] and [1, 0]. Shear followed by rotation gives rows [0, -1] and [1, 1]. Again, different.

Example C: two linear layers collapse into one (illustrative)

Let the first layer have weight matrix W1 with rows [1, 2] and [0, 1], and the second W2 with rows [2, 0] and [1, 1]. Take the input x = (1, 1). All numbers are illustrative.

First layer: W1 x = (1 + 2, 0 + 1) = (3, 1).
Second layer: W2 (3, 1), computed row by row: row 1 is 2 times 3 + 0 times 1 = 6; row 2 is 1 times 3 + 1 times 1 = 4. Output (6, 4).
Now multiply the matrices first: W2 W1 has rows [2, 4] and [1, 3].
Apply to x: row 1 gives 2 + 4 = 6; row 2 gives 1 + 3 = 4. Output (6, 4). Identical.

So the two layers are exactly one layer with the matrix W2 W1. With ReLU between them, that is no longer true. For example the input (-3, 1) gives W1 x = (-3 + 2, 1) = (-1, 1), and ReLU turns that into (0, 1). The negative entry was clipped, a bend no matrix can mimic everywhere: ReLU(-1) + ReLU(1) = 1, but ReLU(-1 + 1) = ReLU(0) = 0, so ReLU breaks the linearity rule L(v + w) = L(v) + L(w).

Example D: turning twice (a quick sanity check)

Rotating by 90 degrees twice should give a half turn. R times R has rows [-1, 0] and [0, -1], which sends every point (x, y) to (-x, -y). Test: R sends (3, 1) to (-1, 3), and R again sends (-1, 3) to (-3, -1). That is the half turn of (3, 1). The numbers agree.

7. Comparison table

Motion	Where i-hat lands	Where j-hat lands	Matrix rows	What it does
Identity	(1, 0)	(0, 1)	[1, 0], [0, 1]	nothing
90 degree counterclockwise rotation	(0, 1)	(-1, 0)	[0, -1], [1, 0]	quarter turn
Shear	(1, 0)	(1, 1)	[1, 1], [0, 1]	tilts north arrow east
Stretch by 2 and 3	(2, 0)	(0, 3)	[2, 0], [0, 3]	widens and lengthens
Mira's example	(1, -2)	(3, 0)	[1, 3], [-2, 0]	general mix
Swap axes (reflection)	(0, 1)	(1, 0)	[0, 1], [1, 0]	flips across the diagonal
Rotate 90 then stretch (2, 3)	(0, 3)	(-2, 0)	[0, -2], [3, 0]	S times R
Stretch then rotate 90	(0, 2)	(-3, 0)	[0, -3], [2, 0]	R times S

8. Common mistakes and myths

Putting landing spots in rows instead of columns. The columns of the matrix are the images of i-hat and j-hat. Swapping rows and columns gives a different (transposed) transformation.
Thinking the order of multiplication is left to right. The right-hand matrix acts first.
Assuming matrix multiplication commutes. Rotating then stretching differs from stretching then rotating, as Example B shows.
Believing a shift is linear. Moving the origin breaks the rules; a shift is affine.
Thinking a bias is part of the linear map. A layer with a bias is affine, not strictly linear.
Believing deeper always means more powerful. Stacked linear layers collapse into one matrix; only the activations add bends.
Confusing "linear" with "line-shaped data". Linearity here is a property of the rule, not the shape of the data.
Forgetting grid lines may tilt. Angles can change under a linear map (a shear), but lines stay straight and parallel lines stay parallel.
Thinking a transformation moves vectors "one at a time". A single matrix moves the whole plane at once.

9. Real-life uses

Neural network layers: every dense layer multiplies by a weight matrix (the d2l textbook describes this matrix-vector view of layers).
Computer graphics: rotating, scaling and shearing images and 3D models.
Robotics and animation: composing several rotations into a single matrix.
Data processing: rescaling and mixing features before training.
Image editing: tilt, rotate and resize operations on pictures.
Physics and engineering: describing small deformations and rotations as matrices.
Compression and analysis: SVD (1.15) breaks a matrix into rotations and stretches.

10. Self-check questions

What two rules make a transformation linear? Answer: grid lines stay parallel and evenly spaced (lines stay straight), and the origin stays fixed.
Where in a matrix do the landing spots of i-hat and j-hat go? Answer: they form the columns.
If i-hat lands on (1, -2) and j-hat on (3, 0), where does (-1, 2) land? Answer: (5, 2).
What are the columns of the 90 degree counterclockwise rotation? Answer: (0, 1) and (-1, 0).
What are the columns of the shear in this lesson? Answer: (1, 0) and (1, 1).
What matrix stretches by 2 along east and 3 along north? Answer: columns (2, 0) and (0, 3).
Rotate 90 degrees then stretch by (2, 3): what is the matrix, and what does the product look like when order is reversed? Answer: rows [0, -2], [3, 0] for rotate-then-stretch; rows [0, -3], [2, 0] for stretch-then-rotate. They differ.
In the product S times R, which acts first? Answer: R, the right-hand matrix.
Why can't a stack of purely linear layers learn more than one layer? Answer: the product of matrices is one matrix, so W2 W1 x equals a single layer's output.
Why is a layer with a bias not strictly linear? Answer: adding a bias vector moves the origin, so it is affine.

11. Key takeaways, go deeper, next

Key takeaways

A linear transformation keeps lines straight and the origin fixed.
Two arrows decide everything: the columns of the matrix are where i-hat and j-hat land.
Applying a matrix means scaling each column by the matching coordinate and adding.
Matrix multiplication is composition, read right to left, and order matters.
Stacked linear layers collapse into one matrix, so neural networks need activations to bend space.

Go deeper

Use NumPy to apply np.matmul (or the @ operator) to the matrices above and reproduce Examples A to D.
Watch the 3Blue1Brown lessons on linear transformations, span and matrix multiplication.
Study MIT 18.06 lectures for a full course treatment.
Try drawing the unit square before and after each matrix in the comparison table.

Next roadmap ID: 1.19 Determinant and matrix inverse.

12. Sources and further reading

Accuracy note: The definition of linearity, the columns-as-landing-spots rule, the (1, -2), (3, 0) example, the rotation and shear matrices, composition read right to left, non-commutativity and the dense-layer description were confirmed in the sources above, and all arithmetic was checked by computation. The two-layer weights in Example C and the ReLU numbers are illustrative; the remark that a bias makes a layer affine is a standard definition rather than a claim from these pages. The graphics, robotics and engineering uses, the eigenvector and SVD remarks and the 1 by 3 and 3 by 2 examples are standard background, not statements taken from the pages above.

Linear transformations: matrices as motion: study notes