M1.3 · LESSON 3 OF 6

Powers and exponentials

Evaluate functions, compare growth rates and read a loss curve without confusing inputs and outputs.

THE IDEA

Three ways to grow

Linear growth adds the same amount each step: 50·n goes 50, 100, 150. Polynomial growth uses a fixed power of n: n² goes 1, 4, 9, 16. Exponential growth multiplies by the same factor each step: 2ⁿ goes 1, 2, 4, 8, 16. The difference between a power and an exponential is where n sits. In n² the variable is the base, with a fixed exponent. In 2ⁿ the variable is the exponent, with a fixed base.

n	1	2	3	4	5	6	8	10	20
50·n	50	100	150	200	250	300	400	500	1000
n²	1	4	9	16	25	36	64	100	400
2ⁿ	2	4	8	16	32	64	256	1024	1048576

Early on, the exponential looks harmless: at n = 5 it is 32, less than 50·n = 250. By n = 20 it is 1048576, thousands of times bigger than the line.

The race, and the exact moment of overtaking

Change the shapes and watch who leads. The dotted vertical lines mark the moment the exponential gets ahead for good: after that point it never loses the lead again. Then try the log-scale box. On a log axis, each step up is a multiplication by 10, so a curve that multiplies by the same factor each step becomes a straight line, while a polynomial bends flat.

line slope k = 50degree p = 2coefficient c = 1growth factor g = 2

log scale on the vertical axisview up to n = 20

line 50·n polynomial 1·n^2 exponential 2^n

exponential is ahead of the line for good from n = 9

exponential is ahead of the polynomial for good from n = 5

inspect n = 10

at n = 10	line	polynomial	exponential
value	500	100	1024
next step multiplies by	1.1	1.21	2

Leader at n = 10: exponential. Dotted vertical lines mark the moment the exponential gets ahead for good.

Blue line, orange polynomial, green exponential. Turn on the log scale, then compare how the three curves bend.

Predict first. For 2ⁿ against n², the exponential ties at n = 4 (both are 16). Is it ahead for good from n = 5?

Why this matters, and the common mistake

Exponentials are how the number of possibilities explodes: 30 yes/no choices make 1073741824 combinations, and 300 make a number with 21 digits. This is why brute-force search over everything is hopeless, and why a model with a tiny extra factor per layer can blow up.

For a small exponential factor the win just comes later: 1.2ⁿ is still below n² at n = 20 (38.3376 against 400) but overtakes for good at n = 41.

The line 250·n is overtaken at n = 12; the polynomial 100·n⁵ only at n = 32.

Chapter context

A function maps an input to an output under a stated rule. Graphs let us see the same rule at many inputs; formulas let us calculate a particular output exactly. Both views should describe the same quantity and units.

TRY IT YOURSELF

Part 1. Do it on the plot

The panel computes the exact overtaking point from the numbers you set (checked up to n = 2000). Use it to hit each target.

line slope k = 50degree p = 2coefficient c = 1growth factor g = 2

log scale on the vertical axisview up to n = 20

line 50·n polynomial 1·n^2 exponential 2^n

exponential is ahead of the line for good from n = 9

exponential is ahead of the polynomial for good from n = 5

inspect n = 10

at n = 10	line	polynomial	exponential
value	500	100	1024
next step multiplies by	1.1	1.21	2

Leader at n = 10: exponential. Dotted vertical lines mark the moment the exponential gets ahead for good.

To do: Set the line's slope k so that the exponential overtakes it for good at exactly n = 12
To do: Pick a degree p and coefficient c so the polynomial c*n^p holds off the exponential until n = 30 or later
To do: Find a growth factor g so g^n is still below n squared at n = 20 but ahead of it by n = 60
To do: Switch the vertical axis to a log scale and look at the exponential

Part 2. Do it in code

Write first_overtake(f, g, n_max): the smallest n of 1 or more such that g(n) > f(n) for every n from there up to n_max. Return None if g is not ahead at n_max.
Write growth_ratios(f, n_max): the list of f(n+1) / f(n) for n = 1 to n_max - 1. For an exponential, the ratio stays constant. For a polynomial, it shrinks towards 1.
Python integers never overflow, so 2 ** 200 is computed exactly. No special tricks are needed.

def first_overtake(f, g, n_max):
    # smallest n >= 1 with g(m) > f(m) for all m in n..n_max, else None
    pass

def growth_ratios(f, n_max):
    # [f(2)/f(1), f(3)/f(2), ..., f(n_max)/f(n_max - 1)]
    pass

print(first_overtake(lambda n: n * n, lambda n: 2 ** n, 100))

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Compare 2 to the fifth power with five squared.

A10 and 32, because the base and the exponent are swapped in the second one.BThey are 32 and 25, respectively.C25 and 32, because 2 to the fifth means 2 times 5 and five squared means 5 to the fifth.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

f:X\to Y,\quad y=f(x),\quad \log_b(b^x)=x.

For the illustrative loss:

L(\theta)=(\theta-1.2)^2\ge0.

Theta is a tunable scalar, 1.2 is the target, and L is a nonnegative output. An exponent and its logarithm undo each other only on their valid domains.

Worked example

Evaluate L(0)=1.44, L(2)=0.64 and L(1.2)=0. The parameter 2 gives a larger input than 0 but a smaller loss. Squaring makes positive and negative distances contribute the same penalty. This is a synthetic landscape, not a claim that real models have one simple valley.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import isclose, log2
loss = lambda theta: (theta - 1.2) ** 2
assert isclose(loss(0), 1.44)
assert isclose(loss(2), 0.64)
assert loss(1.2) == 0
assert log2(8) == 3
assert sum(i*i for i in range(1, 4)) == 14

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

The output of a loss function is not its input parameter. Larger inputs do not always produce larger outputs. Logarithms require an explicit base when comparing information quantities.