1.19 Determinant and matrix inverse: study notes

Learn AI by Story

1. The idea in one sentence

The determinant is a single number that tells you how much a matrix stretches or shrinks area (and whether it flips space over), and the inverse is the matrix that plays the motion backwards, which exists exactly when that number is not zero.

2. The story from the video

Theo is a baker who kneads dough. Each press of his hands is a transformation: the flat square of dough gets stretched, sheared and turned. Theo notices that some presses make the dough cover more table, some less, and one press squashes it into a thin strip. The number that says "how much more table" is the determinant. When a press flattens the dough into a line, no amount of kneading can bring the original shape back, so that press has no inverse. When a press only stretches or tilts the dough, Theo can always knead it back by doing the reverse press.

3. Key definitions

Matrix: a rectangular grid of numbers. In this topic it stands for a linear transformation, a motion of space where grid lines stay straight and parallel. Plain meaning: a recipe for moving every point.
Linear transformation: a rule that keeps the origin fixed and keeps straight, evenly spaced grid lines straight and evenly spaced (see 1.18). Plain meaning: a stretch, turn or shear with no bending.
Unit square: the square with corners (0,0), (1,0), (0,1), (1,1), with area 1. Plain meaning: our ruler for measuring area change.
Determinant: the factor by which a linear transformation scales areas (volumes in 3D), with a sign that records orientation. Written det(A). Plain meaning: the "area multiplier".
2x2 determinant formula: for the matrix with rows (a, b) and (c, d), det = ad - bc. Plain meaning: multiply the main diagonal, subtract the other diagonal.
Orientation: whether the basis vectors i-hat and j-hat keep their left-to-right order. A negative determinant means the plane has been flipped over like a sheet of paper. Plain meaning: mirror-image or not.
Singular matrix: a square matrix with determinant 0. Plain meaning: a motion that squashes space to something thinner and cannot be undone.
Invertible (non-singular) matrix: a square matrix with nonzero determinant. Plain meaning: a motion that can be reversed.
Inverse matrix, written A^-1: the matrix such that applying A and then A^-1 leaves every vector unchanged, so A times A^-1 equals the identity. Plain meaning: the undo button.
Identity matrix, I: the matrix that changes nothing, with 1s on the diagonal and 0s elsewhere. Plain meaning: the "do nothing" motion.
Linear system: equations of the form A x = v, where A and v are known and x is unknown. Plain meaning: "which input lands on this output?"
Column space: the set of all possible outputs of A, which is the span of its columns. Plain meaning: everywhere the motion can reach.
Null space (kernel): the set of vectors that A sends to the zero vector. Plain meaning: what gets crushed to the origin.
Rank: the number of dimensions in the output of a transformation (the next topic, 1.20). Plain meaning: how many directions survive.
Adjugate (adjoint) formula: the general recipe for an inverse as one over det times a matrix of cofactors. The video did not use this word. Plain meaning: the grown-up version of "swap, negate, divide".
Jacobian determinant: the determinant of the matrix of partial derivatives of a smooth function, giving the local area or volume scaling of a possibly curved map. The video did not use this term. Plain meaning: the determinant for bendy motions, measured one tiny patch at a time.

4. Step-by-step explanation

Step 1: a matrix is a motion, and the unit square is the test shape. From 1.18 we know a matrix is fixed by where i-hat and j-hat land, and its columns are those landing spots. Because grid lines stay parallel and evenly spaced, every square of the grid is changed in exactly the same way. So if the unit square's area becomes 6, every region's area is multiplied by 6. One number describes the whole motion. That number is the determinant.

Step 2: read the determinant off a picture. The matrix with rows (3, 0) and (0, 2) stretches i-hat to (3, 0) and j-hat to (0, 2). The unit square becomes a 3-by-2 rectangle with area 6, so the determinant is 6. In general the transformed unit square is a parallelogram spanned by the two columns, and its area is the determinant (up to sign).

Step 3: the formula ad - bc. For rows (a, b) and (c, d), the columns are (a, c) and (b, d). The parallelogram they span has area ad - bc. Check with the stretch: a = 3, d = 2, b = c = 0, so 3 x 2 - 0 = 6. Matches the picture.

Step 4: what different values mean. A determinant of 3 triples areas. A determinant of 1/2 halves them. A determinant of 1 keeps area the same (a rotation or a shear does this). A determinant of 0 squashes the plane onto a line or a point, so every region ends up with zero area. A negative determinant flips orientation, and its absolute value is still the area factor.

Step 5: why zero kills the inverse. If a motion squashes the whole plane onto a line, many different inputs land on the same output point. Given only an output, you cannot tell which input it came from, so no reverse motion can exist. That is why the inverse exists exactly when the determinant is not zero.

Step 6: building the inverse. For the 2x2 matrix with rows (a, b) and (c, d), the inverse is one over det times the matrix with rows (d, -b) and (-c, a). In words: swap the diagonal entries, negate the off-diagonal entries, and divide everything by the determinant. The division is the reason a zero determinant breaks the recipe. For bigger matrices you normally let a computer do it.

Step 7: using the inverse to solve a system. If A x = v and A is invertible, apply A^-1 to both sides: x = A^-1 v. Picture it as playing the motion backwards from the target v to find where x started. The solution is unique.

Step 8: when there is no inverse. If det(A) = 0, the plane is squashed, so there are two possibilities. If v is not inside the squashed output (the column space) there is no solution at all. If v is inside it, there are infinitely many solutions, because the whole null space of crushed directions can be added to any solution. Either way, no single A^-1 v exists.

Step 9: products and sizes. Doing one motion and then another multiplies their area factors: det(AB) = det(A) x det(B). We check this numerically in section 6. It also explains why an invertible matrix's inverse has determinant 1/det(A): undoing a x5 stretch must shrink areas by a factor of 5.

5. Everything the video skipped

3x3 and bigger. In 3D the determinant is the volume scaling factor, and the sign is checked with the right-hand rule (a left hand fits the flipped version). A zero determinant means the unit cube flattens into a sheet, a line or a point. The 3x3 formula has six terms. In general, determinants are computed by elimination (LU factorization) instead of by the formula.

Cofactors and the adjugate. The general inverse formula is A^-1 = (1/det A) times the adjugate. It is elegant but slow for large matrices, so it is rarely used for computation.

Numerical trouble. Real computers use rounding. A matrix with a tiny but nonzero determinant is "nearly singular", and its inverse has huge entries that amplify errors. The Dive into Deep Learning text cautions that explicit inversion is often avoided in practice because it is numerically unstable, especially for matrices close to low rank. The usual fix is to solve A x = v directly with a solver instead of forming A^-1. Also, the size of a determinant is not a good measure of near-singularity: scaling a whole matrix by 3 multiplies a 2x2 determinant by 9 without making the matrix any better behaved.

Overflow and logs. For large matrices the raw determinant can overflow or underflow. The NumPy documentation recommends slogdet, which returns the sign and the logarithm of the determinant. This is exactly how a Gaussian log-density is computed in practice.

Properties worth knowing. det(AB) = det(A) det(B). det of the identity is 1. det of the transpose equals the determinant. Swapping two columns flips the sign. The inverse of a product reverses the order: (AB)^-1 = B^-1 A^-1, just like taking off shoes and socks in the opposite order from putting them on.

Pseudo-inverse and least squares. When a matrix is not square or not invertible, a generalised inverse (the Moore-Penrose pseudo-inverse, usually computed with SVD from 1.15) gives the best compromise solution. This links to regression and to later topics.

Links across the roadmap. Earlier: 1.15 Eigenvectors and SVD (the determinant equals the product of the eigenvalues; the singular values show how close a matrix is to singular); 1.18 Linear transformations (matrices as motion). Later: 1.20 Rank and linear independence (rank is the exact test of "how much got squashed"), 1.21 Basis and change of basis (the 2x2 inverse rule is used to convert coordinates), and 1.22 Orthogonality and projections (rotations have determinant 1 and their inverse is simply their transpose).

History note. We keep to what the sources say: we did not confirm any dates for who first defined determinants, so we make no historical claims here.

6. Worked examples

All numbers below were checked in Python.

Example 1: determinant, inverse, and solving a system

Take A with rows (2, 1) and (1, 3).

Determinant: 2 x 3 - 1 x 1 = 6 - 1 = 5. Areas are multiplied by 5.
Inverse: swap the diagonal entries (2 and 3 trade places), negate the off-diagonal entries (1 becomes -1), divide by 5. So A^-1 has rows (3/5, -1/5) and (-1/5, 2/5), which is (1/5) times the matrix with rows (3, -1) and (-1, 2).
Check A times A^-1: first row times first column is (2 x 3 + 1 x -1)/5 = 5/5 = 1. First row times second column is (2 x -1 + 1 x 2)/5 = 0. Second row times first column is (1 x 3 + 3 x -1)/5 = 0. Second row times second column is (1 x -1 + 3 x 2)/5 = 5/5 = 1. We get the identity.
Solve A x = (4, 7): x = A^-1 (4, 7) = (1/5)(3 x 4 - 1 x 7, -1 x 4 + 2 x 7) = (1/5)(5, 10) = (1, 2).
Check: A (1, 2) = (2 x 1 + 1 x 2, 1 x 1 + 3 x 2) = (4, 7). Correct.

Example 2: a singular matrix

Take B with rows (2, 4) and (1, 2).

Determinant: 2 x 2 - 4 x 1 = 4 - 4 = 0.
The columns are (2, 1) and (4, 2). The second is exactly twice the first, so they point along the same line.
Every output is a x (2, 1) + b x (4, 2) = (a + 2b) x (2, 1), which lies on the line through (2, 1). The whole plane is squashed onto that line.
No inverse exists: the formula would divide by 0.
Solving B x = v: a solution exists only if v is on that line. For v = (4, 2) there are infinitely many solutions, for example x = (2, 0) or x = (0, 1). For v = (1, 0) there is none.

Example 3: orientation flip, and areas multiply

The swap matrix with rows (0, 1) and (1, 0) exchanges the two axes. Determinant: 0 x 0 - 1 x 1 = -1. Area is unchanged but orientation is flipped.
The matrix with rows (1, 2) and (3, 4) has determinant 1 x 4 - 2 x 3 = -2. It flips orientation and doubles areas. Its inverse has rows (-2, 1) and (3/2, -1/2). To solve this matrix times x = (5, 6) we compute x = (-4, 9/2), and the check gives 1 x -4 + 2 x 4.5 = 5 and 3 x -4 + 4 x 4.5 = 6.
Product rule: let M be the matrix from Example 1 (determinant 5) and N the shear with rows (1, 2) and (0, 1) (determinant 1). The product MN has determinant 5, which equals 5 x 1, as the rule predicts.

7. Comparison table

Determinant	Geometric meaning	Area factor	Orientation	Inverse?	Example
Greater than 1	Stretches space	More than 1	Kept	Yes	Rows (3, 0), (0, 2): det 6
Between 0 and 1	Shrinks space	Less than 1	Kept	Yes	Rows (1, 0), (0, 0.5): det 0.5
Exactly 1	Keeps area	1	Kept	Yes	Rotation, shear
Negative	Flips the plane	Absolute value	Flipped	Yes (if nonzero)	Swap matrix: det -1
Exactly 0	Squashes onto a line or point	0	Not defined	No	Rows (2, 4), (1, 2)

8. Common mistakes and myths

Myth: the determinant is the area of the matrix. It is the area scale factor of the motion, not the area of anything fixed.
Mistake: forgetting to divide by the determinant when building the inverse. Swapping and negating alone gives A times the result equal to det times the identity, not the identity.
Mistake: negating the wrong entries. Swap the diagonal, negate the off-diagonal.
Myth: a negative determinant means a negative area. The area is the absolute value; the sign records the flip.
Mistake: assuming a small determinant is harmless or a big one is safe. Scaling a matrix changes its determinant without changing how invertible it is in practice.
Mistake: thinking det(A + B) = det(A) + det(B). The determinant multiplies over products, not sums.
Mistake: inverting a matrix to solve a system on a computer. Use a solver instead; it is faster and more stable.
Myth: a system with a singular matrix always has no solution. It has none or infinitely many, depending on whether v is in the column space.

9. Real-life uses

Linear regression. The normal equation (X^T X)^-1 X^T y fails when two features are duplicates or one is a copy scaled by a constant, because X^T X becomes singular and has no inverse. This is the multicollinearity problem.
Gaussian distributions. The multivariate normal density uses the determinant of the covariance matrix; the determinant measures the volume of the spread.
Normalizing flows. These models turn a simple density into a complex one by a series of invertible transformations (the abstract of Rezende and Mohamed describes this). To track how probability density changes, one uses the Jacobian determinant, which is the change-of-variables volume factor. The paper's abstract itself does not mention the determinant, so that last link comes from the standard change-of-variables rule, not from that abstract.
Graphics and games. A negative determinant reveals a mirrored transform; a zero determinant reveals a flattened one.
Physics and engineering. Solving linear systems appears in circuits, structures and simulations.

10. Self-check questions

What does a determinant of 6 tell you? Areas are multiplied by 6, orientation is kept.
Compute det of the matrix with rows (2, 1), (1, 3). 2 x 3 - 1 x 1 = 5.
What does det = 0 mean geometrically? Space is squashed onto a line or a point, and every region has zero area.
Why can a singular matrix not have an inverse? Many inputs land on the same output, so the outputs cannot be traced back to one input.
Write the 2x2 inverse recipe. Swap the diagonal, negate the off-diagonal, divide by the determinant.
Find the inverse of the matrix with rows (2, 1), (1, 3) and use it to solve A x = (4, 7). A^-1 = (1/5) times rows (3, -1), (-1, 2); x = (1, 2).
What does a negative determinant mean? Orientation is flipped; the area factor is the absolute value.
If det(A) = 5 and det(B) = 1, what is det(AB)? 5.
For the singular matrix with rows (2, 4), (1, 2), when does A x = v have a solution? Only when v lies on the line through (2, 1), the column space.
Why do duplicate features break the normal equation? X^T X becomes singular, so its inverse does not exist.

11. Key takeaways, go-deeper list, next roadmap ID

Key takeaways

Determinant = area (volume) scale factor with a sign for orientation.
ad - bc for 2x2; zero means squashed, so no inverse.
Inverse = undo button; x = A^-1 v solves A x = v.
2x2 inverse: swap, negate, divide by det.
det(AB) = det(A) det(B).
In practice, solve systems with a solver, not with an explicit inverse.

Go deeper

Cofactor expansion and the adjugate formula.
Eigenvalues and the determinant (1.15).
Pseudo-inverse and least squares.
The Jacobian determinant and change of variables.
Conditioning and numerical stability.

Next roadmap ID: 1.20 Rank and linear independence.

12. Sources and further reading

Accuracy note: every figure here was computed in Python or taken from the pages above, and the figures of the form "illustrative" are teaching examples, not measurements.

Search This Blog