M2.2 · LESSON 2 OF 6

Dot products

Compute vector operations, lengths, angles and projections and explain what each measures.

THE IDEA

Multiply matching coordinates, then add

The dot product turns two vectors into one number: multiply the coordinates that sit in the same position, then add those products. a·b = a₁b₁ + a₂b₂ + …. Step through it for a = [2, 3] and b = [4, 1].

Product 1 of 2

a1·b1 = 2 · 4 = 8

running total = 8 = 8

Each step adds one more product to the running total.

What the number means

Algebra gives one formula; geometry gives another that always agrees: a·b = ‖a‖‖b‖cos θ, where θ is the angle between the arrows. Read it as the length of a times the length of b's shadow on a. The shadow is the part of b that lies along a.

Positive: the arrows point broadly the same way (θ below 90 degrees).
Zero: they are perpendicular. b has no shadow on a.
Negative: they point broadly opposite ways (θ above 90 degrees).

i	aᵢ	bᵢ	aᵢ·bᵢ
1	2	4	8
2	3	1	3
a · b = sum			11

a·b = ‖a‖ · (shadow of b on a)

= 3.606 · 3.051 = 11

a·b = ‖a‖‖b‖cos θ = 3.606 · 4.123 · 0.74 (θ = 42.274°)

Positive: the vectors point broadly the same way (angle under 90 degrees).

The green bar is the signed shadow of b on a: it is drawn pointing the opposite way when the dot product is negative.

Drag the tips. The table is the algebra, the green bar is the geometry, and both give the same number.

Predict first. What is [1, 0]·[0, 5]?

Common mistakes

Elementwise product is not the dot product

>>> import numpy as np
>>> a, b = np.array([2, 3]), np.array([4, 1])
>>> a * b            # elementwise: still two numbers
array([8, 3])
>>> np.dot(a, b)     # elementwise products, then summed
11

The second trap: the dot product mixes alignment and size. Compare [1, 1]·[1, 1] = 2 (angle 0 degrees, perfectly aligned) with [10, 0]·[1, 1] = 10 (angle 45 degrees). The less aligned pair scores higher because one vector is long. A large dot product does not by itself mean "similar direction". Removing the length effect is the job of lesson M2.4.

Chapter context

A vector is an ordered set of coordinates in a specified space. It can describe a displacement, a data record or a learned representation. Operations are meaningful only when the coordinates refer to compatible dimensions.

TRY IT YOURSELF

Part 1. Do it on the plane

Watch the table and the green shadow while you move the dots. The dot product only moves in steps because the dots snap to a half-unit grid.

i	aᵢ	bᵢ	aᵢ·bᵢ
1	2	4	8
2	3	1	3
a · b = sum			11

a·b = ‖a‖ · (shadow of b on a)

= 3.606 · 3.051 = 11

a·b = ‖a‖‖b‖cos θ = 3.606 · 4.123 · 0.74 (θ = 42.274°)

Positive: the vectors point broadly the same way (angle under 90 degrees).

The green bar is the signed shadow of b on a: it is drawn pointing the opposite way when the dot product is negative.

To do: Make a · b equal exactly 0 with both vectors nonzero
To do: Make a · b less than or equal to -10
To do: Make the vectors point the same way (angle at most 10 degrees) and still have a · b at most 3

Part 2. Do it in code

Write dot(a, b) from the definition (multiply matching coordinates, then add). Raise ValueError for different lengths.
Write elementwise(a, b) returning the list of products, and alignment(a, b) returning "same", "opposite" or "perpendicular" from the sign of the dot product.

def elementwise(a, b):
    # the list of products a[i] * b[i]
    pass

def dot(a, b):
    # sum of the elementwise products; ValueError if the lengths differ
    pass

def alignment(a, b):
    # "same" if dot > 0, "opposite" if dot < 0, "perpendicular" if dot == 0
    pass

print(dot([2, 3], [4, 1]))

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Compute [2,3] dot [4,1].

A11, because the products of matching coordinates are 2*4 = 8 and 3*1 = 3, and they add to one number.B[8,3], because multiplying matching coordinates already gives the dot product.C14, because the first coordinates are added and the second ones are multiplied.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

a\cdot b=\sum_i a_i b_i,\quad \|a\|_2=\sqrt{\sum_i a_i^2}.

\cos(a,b)=\frac{a\cdot b}{\|a\|_2\|b\|_2},\quad \operatorname{proj}_b a=\frac{a\cdot b}{b\cdot b}b.

Both vectors must have the same dimension; cosine and projection need the denominator to be nonzero.

Worked example

For a=[2,3] and b=[4,1], matching products are 8 and 3, so the dot product is 11. Squared norms are 13 and 17. The cosine is 11/sqrt(221), approximately 0.73994. The projection of [3,4] onto [1,0] keeps only the first coordinate.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import sqrt, isclose
from operator import mul
a, b = [2, 3], [4, 1]
dot = sum(map(mul, a, b))
assert dot == 11
cosine = dot / (sqrt(sum(x*x for x in a))*sqrt(sum(x*x for x in b)))
assert isclose(cosine, 11/sqrt(221))
assert isclose(cosine, 0.7399400734, abs_tol=1e-9)

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Dot product is not elementwise multiplication alone. Similarity and distance have opposite ordering conventions. Zero vectors cannot be normalized by dividing by their norm.