M1.5 · LESSON 5 OF 6

Summation notation

Evaluate functions, compare growth rates and read a loss curve without confusing inputs and outputs.

THE IDEA

Sigma is a loop in one symbol

The Greek capital sigma, Σ, means "add these up". Under it sits the starting value of the index (here i), above it the last value, and to the right the expression to evaluate for each i. So Σ from i = 1 to 3 of i² is 1² + 2² + 3² = 14. Both ends are included. The index is just a counter; its name does not matter.

lower bound = 1upper bound = 4

Σ (i from 1 to 4) of i²

= 1 + 4 + 9 + 16

= 30

nothing added yet

total = 0 for i in range(1, 5): # i = 1 ... 4, both ends included total += i ** 2 print(total) # 30

Python's range stops one before its end, which is why the loop uses 5 for an upper bound of 4. Using range(1, 4) by mistake would give 14 and silently drop the last term.

Choose an expression and bounds. Press Next term to add one term at a time and compare it with the Python loop.

Expand a small case first

Before you manipulate a large sum, write out three or four terms. It exposes the pattern and the number of terms: from i = lo to hi there are hi − lo + 1 terms, so the sum from 1 to 3 has 3, and the sum from 0 to 3 has 4. If hi is below lo, there are no terms and the sum is 0 (an empty sum).

Predict first. How many terms are in the sum of 2i from i = 3 to i = 7?

Predict first. In Python, which loop computes the sum of i from 1 to 5?

Where sums show up in learning

Almost every training objective is a sum over examples. The mean squared error over n predictions is (1/n) · Σ (yᵢ − ŷᵢ)²: square each miss (so over and under cannot cancel), add them up, divide by how many there are. For y = [3, −0.5, 2, 7] and predictions [2.5, 0, 2, 8] the squared misses are 0.25, 0.25, 0 and 1, so the total is 1.5 and the mean is 0.375.

Chapter context

A function maps an input to an output under a stated rule. Graphs let us see the same rule at many inputs; formulas let us calculate a particular output exactly. Both views should describe the same quantity and units.

TRY IT YOURSELF

Part 1. Do it on the plot

The expansion shows every term of the sum with its value, and the total is recomputed live.

lower bound = 1upper bound = 3

Σ (i from 1 to 3) of i

= 1 + 2 + 3

= 6

To do: Make the sum of i squared, starting at i = 1, equal 385
To do: Make the sum of the odd numbers 2i − 1, starting at i = 1, equal 169
To do: For 1/i starting at i = 1, find the smallest upper bound where the sum reaches 3 or more
To do: Make an empty sum (no terms at all) and see what total it gives

Part 2. Do it in code

Write sigma(f, lo, hi): the sum of f(i) for i = lo, lo + 1, ..., hi, with BOTH ends included. An empty range (hi below lo) gives 0.
Write sum_squares(n) using sigma: the sum of i squared for i = 1 to n.
Write mean_squared_error(y, y_hat) = (1/n) times the sum of (y_i - y_hat_i) squared. Raise ValueError if the lists differ in length or are empty.

def sigma(f, lo, hi):
    # sum of f(i) for i from lo to hi, both included
    pass

def sum_squares(n):
    # 1*1 + 2*2 + ... + n*n, built with sigma
    pass

def mean_squared_error(y, y_hat):
    # average of the squared differences
    pass

print(sigma(lambda i: i ** 2, 1, 3))

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Expand sum of i squared for i=1 through 3.

A36, because the sum is squared after adding 1 + 2 + 3.B6, because only 1 + 2 + 3 is added and the squares are left out.C1 + 4 + 9 = 14.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

f:X\to Y,\quad y=f(x),\quad \log_b(b^x)=x.

For the illustrative loss:

L(\theta)=(\theta-1.2)^2\ge0.

Theta is a tunable scalar, 1.2 is the target, and L is a nonnegative output. An exponent and its logarithm undo each other only on their valid domains.

Worked example

Evaluate L(0)=1.44, L(2)=0.64 and L(1.2)=0. The parameter 2 gives a larger input than 0 but a smaller loss. Squaring makes positive and negative distances contribute the same penalty. This is a synthetic landscape, not a claim that real models have one simple valley.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import isclose, log2
loss = lambda theta: (theta - 1.2) ** 2
assert isclose(loss(0), 1.44)
assert isclose(loss(2), 0.64)
assert loss(1.2) == 0
assert log2(8) == 3
assert sum(i*i for i in range(1, 4)) == 14

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

The output of a loss function is not its input parameter. Larger inputs do not always produce larger outputs. Logarithms require an explicit base when comparing information quantities.