M2.5 · LESSON 5 OF 6

Projection

Compute vector operations, lengths, angles and projections and explain what each measures.

THE IDEA

The part of a that lies along b

Shine a light straight down onto the line through b. The shadow that a casts on that line is the projection of a onto b, written proj_b a. What is left over, r = a − p, sticks out at a right angle. So every vector splits into a piece along b plus a piece perpendicular to b.

a·b = 10 b·b = 5

c = a·b / b·b = 2

p = c·b = [4, 2]

r = a − p = [-1, 2]

r·b = 0 (always 0: r is perpendicular to b)

‖a‖² = 25 = ‖p‖² + ‖r‖² = 20 + 5

The green arrow p is the part of a that lies along b. The purple dashed arrow r is what is left over, and it meets b at a right angle.

Drag a and b. The green arrow is the projection and always lies on the line through b; the purple dashed arrow is the residual, with r·b = 0.

The recipe

The formula is proj_b a = (a·b / b·b) b. Step through it for a = [2, 3] and b = [4, 1].

Step 1 of 4

Dot product a·b. a·b = 2·4 + 3·1 = 11. How much of a runs along b, in units of b's length.

Four small steps, every number computed.

Predict first. What is the projection of [3, 4] onto [0, 2]?

Common mistakes

Mixing up the order. The result is a multiple of the vector you project onto. proj_b a for a = [3, 4], b = [1, 0] is [3, 0], a multiple of b. Swapping the roles, proj_a b = [0.36, 0.48], a multiple of a, and a different vector.

Dividing by ‖b‖ instead of b·b. The numerator a·b already contains one factor of b's length, and the vector b we multiply by contains another. Dividing by b·b = ‖b‖² cancels both. Dividing only by ‖b‖ leaves the answer too long by a factor ‖b‖. And b must not be the zero vector: then b·b = 0 and there is no direction to project onto.

Chapter context

A vector is an ordered set of coordinates in a specified space. It can describe a displacement, a data record or a learned representation. Operations are meaningful only when the coordinates refer to compatible dimensions.

TRY IT YOURSELF

Part 1. Do it on the plane

Drag a and b. The formula panel recomputes c, p and r from scratch on every move.

a·b = 10 b·b = 5

c = a·b / b·b = 2

p = c·b = [4, 2]

r = a − p = [-1, 2]

r·b = 0 (always 0: r is perpendicular to b)

‖a‖² = 25 = ‖p‖² + ‖r‖² = 20 + 5

The green arrow p is the part of a that lies along b. The purple dashed arrow r is what is left over, and it meets b at a right angle.

To do: Make the projection of a onto b equal [3, 0] while a has a nonzero second coordinate
To do: Make the residual a - proj_b(a) the zero vector with both vectors nonzero and a different from b
To do: Make the projection point opposite to b, with length at least 2

Part 2. Do it in code

Write project(a, b): the projection of a onto b, using (a . b) / (b . b) times b. Raise ValueError if b is the zero vector.
Write residual(a, b) = a - project(a, b). It must be perpendicular to b.

import numpy as np

def project(a, b):
    # the vector (a.b / b.b) * b
    pass

def residual(a, b):
    # what is left of a after removing its component along b
    pass

print(project([3, 4], [1, 0]))

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Project [3,4] onto [1,0].

A[1,0], because the projection is always the unit vector along b.B[3,0], because [1,0] points along the x-axis, so only the first coordinate of [3,4] survives.C[0,4], because the projection onto [1,0] keeps the second coordinate.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

a\cdot b=\sum_i a_i b_i,\quad \|a\|_2=\sqrt{\sum_i a_i^2}.

\cos(a,b)=\frac{a\cdot b}{\|a\|_2\|b\|_2},\quad \operatorname{proj}_b a=\frac{a\cdot b}{b\cdot b}b.

Both vectors must have the same dimension; cosine and projection need the denominator to be nonzero.

Worked example

For a=[2,3] and b=[4,1], matching products are 8 and 3, so the dot product is 11. Squared norms are 13 and 17. The cosine is 11/sqrt(221), approximately 0.73994. The projection of [3,4] onto [1,0] keeps only the first coordinate.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import sqrt, isclose
from operator import mul
a, b = [2, 3], [4, 1]
dot = sum(map(mul, a, b))
assert dot == 11
cosine = dot / (sqrt(sum(x*x for x in a))*sqrt(sum(x*x for x in b)))
assert isclose(cosine, 11/sqrt(221))
assert isclose(cosine, 0.7399400734, abs_tol=1e-9)

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Dot product is not elementwise multiplication alone. Similarity and distance have opposite ordering conventions. Zero vectors cannot be normalized by dividing by their norm.