M3.2 · LESSON 2 OF 6

Matrix multiplication

Multiply matrices, track dimensions and interpret transformations, inverses, eigenvectors and SVD.

THE IDEA

Doing one transformation after another

If A and B are both transformations, then "do B, then do A" is a transformation too. Its matrix is the product AB. The order reads right to left because the vector sits on the right: (AB) x = A (B x), so B touches x first. Without a product matrix you would have to push every vector through two machines; with it, one matrix does both jobs.

A =

B =

AB = [[2, 0], [2, 1]] (B first, then A)

BA = [[2, 0], [1, 1]] (A first, then B)

test: [1, 1] → B → [2, 1] → A → [2, 3]

AB and BA are different: the green F (AB) and the dashed purple F (BA) land in different places.

Orange outline: the unit square after B. Green outline: after B and then A, which is the single matrix AB. The grid is deformed by AB.

Edit A and B. Orange: the unit square after B. Green: after B and then A, which is the single matrix AB. The dashed purple F is what you get with the order swapped (BA).

Computing the entries

Entry (i, j) of AB is the dot product of row i of A with column j of B: C[i][j] = Σ A[i][k]·B[k][j]. Step through A = [[1, 2], [3, 4]] and B = [[2, 0], [1, 2]].

Entry 1 of 4

AB[1][1] = row 1 of A · column 1 of B

= [1, 2] · [2, 1] = 1·2 + 2·1 = 4

Four dot products fill the four entries of AB.

All four together: AB = [[4, 4], [10, 8]]. Now swap the order: BA = [[2, 4], [7, 10]]. They differ, so order matters: rotating then shearing is not the same as shearing then rotating.

Predict first. For A = [[1, 2], [3, 4]] and B = [[2, 0], [1, 2]], what is the top-left entry of BA (not AB)?

The shape rule

A product only exists when the inner dimensions match: an (m, n) matrix times an (n, p) matrix gives an (m, p) matrix. The inner n is consumed by the dot products; the outer m and p remain. This is also a free bug detector: if the shapes you expect do not line up on paper, the code is wrong before you run it. Pick shapes below, and try the (2, 3) times (2, 4) button.

A has shape (rows , columns )B has shape (rows , columns )

A B is defined and has shape (2, 4): the outer numbers survive, the inner pair cancels. B A does not exist at all.

Inner pair matches: defined, outer pair is the result. Otherwise: an error. Notice that B A may exist when A B does not, or both may exist with different shapes.

Common mistakes

* is not matrix multiplication

import numpy as np
A = np.array([[1, 2], [3, 4]])
B = np.array([[2, 0], [1, 2]])
A * B     # elementwise: [[2, 0], [3, 8]]   (NOT the product)
A @ B     # matrix product: [[4, 4], [10, 8]]
B @ A     # [[2, 4], [7, 10]]   a different matrix

Two more traps. First, AB = 0 does not mean A = 0 or B = 0: you will build such a pair in the practice. Second, A @ B == B @ A is false for most pairs. It holds for special ones, such as two diagonal matrices or any matrix with the identity.

Chapter context

A matrix represents a linear map once input and output bases are chosen. Multiplication composes maps. Tracking shapes is an algebraic requirement and also a practical way to detect implementation mistakes before running code.

TRY IT YOURSELF

Part 1. Do it by hand

Both matrices start as the identity. The readout shows AB and BA after every change, and the picture shows where the F lands in each order.

A =

B =

AB = [[1, 0], [0, 1]] (B first, then A)

BA = [[1, 0], [0, 1]] (A first, then B)

test: [1, 1] → B → [1, 1] → A → [1, 1]

AB equals BA for these two (it happens, but only for special pairs).

Orange outline: the unit square after B. Green outline: after B and then A, which is the single matrix AB. The grid is deformed by AB.

To do: Make AB the quarter-turn rotation [[0, -1], [1, 0]] with neither A nor B equal to the identity
To do: Make AB different from BA while both A and B can be undone (determinant not 0)
To do: Make AB the zero matrix even though neither A nor B is the zero matrix

Part 2. Do it in code

Write matmul(A, B) with three loops (no @ or np.dot). Raise ValueError whose message includes both shapes when the inner dimensions differ.
Write commutes(A, B): True when AB and BA are both defined and equal. Use np.allclose for the comparison, and return False when either product is undefined.

import numpy as np

def matmul(A, B):
    # C[i][j] = sum over k of A[i][k] * B[k][j]
    pass

def commutes(A, B):
    # True if A @ B equals B @ A (and both exist)
    pass

print(matmul([[1, 2], [3, 4]], [[2, 0], [1, 2]]))

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Can shapes (2,3) and (2,4) multiply in that order?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

C_{ij}=\sum_k A_{ik}B_{kj},\quad (AB)^T=B^TA^T.

\det\begin{bmatrix}a&b\\c&d\end{bmatrix}=ad-bc,\quad Av=\lambda v.

A=U\Sigma V^T.

U and V contain orthonormal singular directions; diagonal entries of Sigma are singular values. Eigenvalues and singular values are different concepts.

For an invertible 2x2 matrix, the inverse is

\begin{bmatrix}a&b\\c&d\end{bmatrix}^{-1}=\frac1{ad-bc}\begin{bmatrix}d&-b\\-c&a\end{bmatrix}.

This formula requires a nonzero determinant. A near-zero determinant can signal numerical sensitivity, so exact invertibility alone does not guarantee a well-conditioned computation.

Worked example

Multiply A=[[1,2],[3,4]] by B=[[2,0],[1,2]]. Row-column products give C=[[4,4],[10,8]]. The determinant of A is 1×4−2×3=−2, so A is invertible. In general AB is not equal to BA; their shapes may even differ.

Reference code

The following example uses the Python standard library and does not call a remote service.

A, B = [[1,2],[3,4]], [[2,0],[1,2]]
C = [[sum(A[i][k]*B[k][j] for k in range(2)) for j in range(2)] for i in range(2)]
assert C == [[4,4],[10,8]]
assert A[0][0]*A[1][1] - A[0][1]*A[1][0] == -2

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

AB usually differs from BA. A transpose is not generally an inverse. A negative eigenvalue is possible, while singular values are nonnegative.