M2.3 · LESSON 3 OF 6

Norm and distance

Compute vector operations, lengths, angles and projections and explain what each measures.

THE IDEA

Length comes from Pythagoras

To measure the straight-line distance between two points, move along the axes to make a right triangle: across by dx, up by dy. The straight line is the hypotenuse, so distance = √(dx² + dy²). The coordinates of the difference vector Q − P are exactly dx and dy.

	offset	squared
across (Qx − Px)	3	9
up (Qy − Py)	4	16
sum of squares		25

‖Q − P‖ = √((3)² + (4)²)

= √25 = 5

‖P − Q‖ = 5 (swapping the points changes nothing)

Drag P and Q. The dashed legs are the offsets; the table squares and adds them; the green line is the distance.

Norm: the length of one vector

The Euclidean norm of a vector is its length: ‖a‖ = √(a₁² + a₂² + …). It is the distance from the origin to the tip. For [3, 4] that is √(9 + 16) = √25 = 5, the famous 3-4-5 triangle. The distance between two points is the norm of their difference: dist(P, Q) = ‖Q − P‖. For P = [1, 2] and Q = [4, 6] the difference is [3, 4], so the distance is 5.

A neat link to the last lesson: a·a = a₁² + a₂² + …, so ‖a‖ = √(a·a). The dot product of a vector with itself is its squared length.

Predict first. What is the distance between [1, 2] and [4, 6]?

Any number of coordinates

Nothing in the formula depends on there being two coordinates. For four, add two more squared offsets. You cannot draw four dimensions, but you can compute them, and that is what machine learning does all the time.

i	uᵢ	vᵢ	(uᵢ − vᵢ)²
1	1	2	1
2	0	-1	1
3	2	0	4
4	3	3	0
sum			6

‖u − v‖ = √6 = 2.449

‖u‖ = √(u₁² + u₂² + u₃² + u₄²) = 3.742

Two vectors with four coordinates each. Move the sliders and watch the squared offsets.

Common mistakes

Distance is the norm of the difference, not the difference of the norms. Take a = [3, -3] and b = [-3, 3]: both have norm 4.243, so the difference of norms is 0, yet the points are 8.485 apart (they sit on opposite sides of the origin). Also easy to forget: the final square root. The sum of squares is the squared distance. It is a legitimate shortcut when you only need to rank which point is nearest, because squaring keeps the order, but it is not the distance.

Chapter context

A vector is an ordered set of coordinates in a specified space. It can describe a displacement, a data record or a learned representation. Operations are meaningful only when the coordinates refer to compatible dimensions.

TRY IT YOURSELF

Part 1. Do it on the plane

Both points are draggable. Read the table to see which offsets you need.

	offset	squared
across (Qx − Px)	3	9
up (Qy − Py)	2	4
sum of squares		13

‖Q − P‖ = √((3)² + (2)²)

= √13 = 3.606

‖P − Q‖ = 3.606 (swapping the points changes nothing)

To do: Put P and Q exactly 5 apart with neither a horizontal nor a vertical offset of 0
To do: Put P and Q exactly 2 times the square root of 2 apart (the diagonal of a 2 by 2 square)
To do: Make the distance between P and Q exactly 0 while P is not at the origin

Part 2. Do it in code

Write norm(a) and dist(a, b) with math.sqrt. They must work for any number of coordinates.
Write squared_dist(a, b) as well: it skips the square root, which keeps the same ordering of nearest points and is cheaper.

from math import sqrt

def norm(a):
    # square root of the sum of squared coordinates
    pass

def dist(a, b):
    # norm of the difference a - b; ValueError if the lengths differ
    pass

def squared_dist(a, b):
    # like dist but without the square root
    pass

print(norm([3, 4]))

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Find the norm of [3,4].

A25, because the sum of squares is the norm without the square root.B7, because the coordinates 3 and 4 are added.C5, because the square root of 3^2 + 4^2 = 25 is 5 (the 3-4-5 triangle).

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

a\cdot b=\sum_i a_i b_i,\quad \|a\|_2=\sqrt{\sum_i a_i^2}.

\cos(a,b)=\frac{a\cdot b}{\|a\|_2\|b\|_2},\quad \operatorname{proj}_b a=\frac{a\cdot b}{b\cdot b}b.

Both vectors must have the same dimension; cosine and projection need the denominator to be nonzero.

Worked example

For a=[2,3] and b=[4,1], matching products are 8 and 3, so the dot product is 11. Squared norms are 13 and 17. The cosine is 11/sqrt(221), approximately 0.73994. The projection of [3,4] onto [1,0] keeps only the first coordinate.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import sqrt, isclose
from operator import mul
a, b = [2, 3], [4, 1]
dot = sum(map(mul, a, b))
assert dot == 11
cosine = dot / (sqrt(sum(x*x for x in a))*sqrt(sum(x*x for x in b)))
assert isclose(cosine, 11/sqrt(221))
assert isclose(cosine, 0.7399400734, abs_tol=1e-9)

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Dot product is not elementwise multiplication alone. Similarity and distance have opposite ordering conventions. Zero vectors cannot be normalized by dividing by their norm.