M3.5 · LESSON 5 OF 6

Eigenvectors

Multiply matrices, track dimensions and interpret transformations, inverses, eigenvectors and SVD.

THE IDEA

Directions that a matrix does not turn

Most vectors are turned by a matrix: A v points a different way from v. A few special directions are not. For those, A v lies on the same line as v, so A just stretches (or flips) the vector: A v = λ v. Such a v is an eigenvector and the stretch factor λ is its eigenvalue. The zero vector is excluded: A·0 = λ·0 for every λ, so it would carry no information.

A =

direction of v: 60°

v = [0.5, 0.866]

A v = [2.366, 1.732]

angle between v and A v: 23.794°

cross(v, A v) = -1.183

A v points a different way from v, so this direction is turned by the matrix. Keep turning.

show the eigen directions (purple dashed)

Turn the test vector v with the slider. Watch the angle between v (blue) and A v (green). The curve shows cross(v, A v) for every direction; eigen directions are where it touches zero.

Predict first. What happens to [1, 0] under diag(2, 3), the matrix [[2, 0], [0, 3]]?

Finding them

A v = λ v means (A − λI) v = 0 has a nonzero solution, which needs det(A − λI) = 0. For a 2 by 2 matrix this is λ² − (trace)·λ + det = 0. For A = [[3, 1], [0, 2]]: trace 5, determinant 6, so λ² − 5λ + 6 = 0, giving λ = 3 and λ = 2. The eigenvector for 3 is [1, 0]; for 2 it is [-1, 1] (check: A[-1, 1] = [-3 + 1, 2] = [-2, 2] = 2·[-1, 1]). Any nonzero multiple of an eigenvector is an eigenvector too: it is the direction that matters, not the length or sign.

Why care? Repeated application of A stretches along eigenvectors by λ each time: λ above 1 explodes, below 1 fades. That is how people reason about whether a recurrent system or a repeated update is stable, and the directions of a data cloud with the most spread are eigenvectors of its covariance matrix.

Common mistakes

Rotations have none (in the real plane). A 90 degree turn moves every direction. Pick the "rotate 90" preset: the curve never reaches zero and the discriminant trace² − 4·det = −4 is negative.
Shear has only one line. [[1, 1], [0, 1]] has eigenvalue 1 and keeps only the x-axis direction.
Eigenvalues can be negative. The mirror preset has eigenvalue -1: the vector is flipped but stays on its line.
Eigenvalues are not singular values. They coincide only for special matrices; M3.6 explains the other concept.

Chapter context

A matrix represents a linear map once input and output bases are chosen. Multiplication composes maps. Tracking shapes is an algebraic requirement and also a practical way to detect implementation mistakes before running code.

TRY IT YOURSELF

Part 1. Do it by hand

The slider moves in steps of 5 degrees. A direction counts as an eigenvector when cross(v, A v) is zero up to rounding.

A =

direction of v: 60°

v = [0.5, 0.866]

A v = [2.366, 1.732]

angle between v and A v: 23.794°

cross(v, A v) = -1.183

A v points a different way from v, so this direction is turned by the matrix. Keep turning.

show the eigen directions (purple dashed)

To do: Turn the test vector v until A v points along v with stretch factor 3
To do: Find the second direction that A keeps: parallel again, but with stretch factor 2
To do: Edit the matrix so that no direction survives: no real eigenvectors at all

Part 2. Do it in code

Write is_eigenvector(A, v, tol=1e-9): True when v is nonzero and A v is a multiple of v. Compare with a tolerance.
Write eigenvalue_of(A, v): the stretch factor, using (v . Av) / (v . v). Raise ValueError if v is the zero vector or not an eigenvector.
Write real_eigenvalues(A): the sorted real eigenvalues of a square matrix, or an empty list when they are complex (use np.linalg.eig and check the imaginary parts).

import numpy as np

def is_eigenvector(A, v, tol=1e-9):
    # nonzero v, and A v is parallel to v
    pass

def eigenvalue_of(A, v):
    # the stretch factor lambda with A v = lambda v
    pass

def real_eigenvalues(A):
    # sorted real eigenvalues, [] if they are complex
    pass

print(is_eigenvector([[2, 0], [0, 3]], [1, 0]))

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

What happens to [1,0] under diag(2,3)?

AIt becomes [2,0], so the eigenvalue is 2.BIt becomes [3,0], so the eigenvalue is 3.CIt is rotated and becomes [0,2], so it has no eigenvalue.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

C_{ij}=\sum_k A_{ik}B_{kj},\quad (AB)^T=B^TA^T.

\det\begin{bmatrix}a&b\\c&d\end{bmatrix}=ad-bc,\quad Av=\lambda v.

A=U\Sigma V^T.

U and V contain orthonormal singular directions; diagonal entries of Sigma are singular values. Eigenvalues and singular values are different concepts.

For an invertible 2x2 matrix, the inverse is

\begin{bmatrix}a&b\\c&d\end{bmatrix}^{-1}=\frac1{ad-bc}\begin{bmatrix}d&-b\\-c&a\end{bmatrix}.

This formula requires a nonzero determinant. A near-zero determinant can signal numerical sensitivity, so exact invertibility alone does not guarantee a well-conditioned computation.

Worked example

Multiply A=[[1,2],[3,4]] by B=[[2,0],[1,2]]. Row-column products give C=[[4,4],[10,8]]. The determinant of A is 1×4−2×3=−2, so A is invertible. In general AB is not equal to BA; their shapes may even differ.

Reference code

The following example uses the Python standard library and does not call a remote service.

A, B = [[1,2],[3,4]], [[2,0],[1,2]]
C = [[sum(A[i][k]*B[k][j] for k in range(2)) for j in range(2)] for i in range(2)]
assert C == [[4,4],[10,8]]
assert A[0][0]*A[1][1] - A[0][1]*A[1][0] == -2

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

AB usually differs from BA. A transpose is not generally an inverse. A negative eigenvalue is possible, while singular values are nonnegative.