M6.1 · LESSON 1 OF 6

Events and distributions

Calculate conditional probabilities, Bayes updates, expectations, variance and softmax probabilities.

THE IDEA

A model of what can happen

Probability starts with a list of everything that could happen: the sample space. For one die it is six faces. For two dice it is 36 pairs, drawn below as a grid. An event is any set of outcomes you care about, such as "the sum is 7" or "the dice match". The probability of an event is the total probability of the outcomes inside it. With fair dice every cell has probability 1/36, so P(event) = (cells in the event) / 36.

cells in the event: 0 of 36

P(event) = 0/36 = 0 = 0%

P(not event) = 36/36 = 100%

Roll the pair of dice:

rolls so far: 0 (seed 2024)

frequency of the event: no rolls yet

exact probability: 0

Click cells to build an event and read its probability. Then roll with a seeded generator and watch the frequency settle toward the exact value. Same seed, same rolls, every time.

Predict first. There are 11 possible sums (2 to 12). So P(sum = 7) = 1/11, right? What is it really?

Frequencies approach probabilities

A probability is not a promise about the next roll. It is the long-run share. Roll the grid above ten times and the frequency of an event can be far from its probability. After a thousand it is much closer, and after ten thousand closer still. This is the law of large numbers, and it is why simulation is a legitimate way to estimate probabilities you cannot easily count.

The two rules every distribution obeys

A list of probabilities over mutually exclusive, exhaustive outcomes is a distribution when (1) none is negative and (2) they sum to exactly 1. Everything that can happen has been given some share of one unit of probability, no more, no less.

face 120%face 220%face 320%face 420%face 520%face 620%

total = 0.2 + 0.2 + 0.2 + 0.2 + 0.2 + 0.2 = 1.2 Not valid: 0.2 too much probability.

Six sliders for the faces of a die. They start at the case from the quiz: 0.2 on every face.

Predict first. A die gives each of its six faces probability 0.2. Is that a valid distribution?

Common mistakes

Adding overlapping events. "At least one six" is not 1/6 + 1/6 = 12/36, because the cell (6,6) sits in both the first-die-six row and the second-die-six column. Count it once: 11/36. In general, P(A or B) = P(A) + P(B) - P(A and B). Doubles (6 cells) or a sum of at least 10 (6 cells) share (5,5) and (6,6), so the union has 10 cells: 5/18, not 12/36.

Density is not probability. For a continuous quantity, probability is area under a curve. The height at a single point is a density and can exceed 1: a narrow bell curve with standard deviation 0.2 is about 1.99 tall at its centre, yet the total area is still 1.

Chapter context

Probability quantifies uncertainty relative to a model of possible outcomes. Conditioning changes the relevant population. Before inserting numbers into a formula, say which event is given and which event is being predicted.

TRY IT YOURSELF

Part 1. Do it hands-on

Click cells to build each event exactly. The task ticks only when your cells are the right set, and stays ticked once earned.

cells in the event: 0 of 36

P(event) = 0/36 = 0 = 0%

P(not event) = 36/36 = 100%

Roll the pair of dice:

rolls so far: 0 (seed 2024)

frequency of the event: no rolls yet

exact probability: 0

To do: Select exactly the event “the sum of the two dice is 7”
To do: Select exactly the event “at least one die shows a 6”
To do: Select exactly the event “the dice match, or the sum is at least 10”

Part 2. Spot the wrong statements

Six claims about probabilities. Tick the ones that are wrong. Explanations appear once you flag exactly the wrong ones.

For two fair dice, P(sum = 7) = 6/36 = 1/6.
A die with face probabilities 0.1, 0.1, 0.1, 0.1, 0.1 and 0.3 is a valid probability distribution.
If P(rain) = 0.3 then P(no rain) = 0.7.
For two fair dice, P(at least one six) = 1/6 + 1/6 = 1/3.
A probability density can exceed 1 at a point: a normal curve with standard deviation 0.2 is about 1.99 high at its mean.
P(A or B) = P(A) + P(B) whenever A and B can both happen.

Tick every claim you think is wrong.

Part 3. Do it in code

Write is_valid_distribution(probs): True only if every probability is at least 0 and the total is 1 (compare with a tolerance of 1e-9, because 0.1 added ten times is not exactly 1.0 in floating point). An empty list is not a distribution.
Write event_probability(dist, event). dist maps outcomes to probabilities, event is a set of outcomes. Outcomes that are not in dist have probability 0.
Write two_dice_sums(): a dict from each possible sum (2 to 12) to its probability for two fair dice.

def is_valid_distribution(probs):
    # nonnegative and sums to 1 (within 1e-9). Return a bool.
    pass

def event_probability(dist, event):
    # add up dist[outcome] for the outcomes in the event
    pass

def two_dice_sums():
    # {2: 1/36, 3: 2/36, ..., 12: 1/36}
    pass

print(is_valid_distribution([0.2] * 6))

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

A die assigns probability 0.2 to each of six faces; is this valid?

ANoBYes

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)},\quad E[X]=\sum_x xP(X=x).

\operatorname{Var}(X)=E[(X-E[X])^2],\quad p_i=\frac{e^{z_i-m}}{\sum_j e^{z_j-m}},\;m=\max_jz_j.

Bayes' denominator must be nonzero. The softmax normalization runs over the candidate classes.

Worked example

In a fictional population, illness prevalence is 1%, sensitivity is 90% and the false-positive rate is 10%. True positives account for 0.01×0.9=0.009 of the population; false positives account for 0.99×0.1=0.099. Among positives, the illness fraction is 0.009/0.108=1/12≈8.33%. This is an arithmetic example, not medical guidance.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import exp, log, isclose
posterior = .01*.9 / (.01*.9 + .99*.1)
assert isclose(posterior, 1/12)
scores = [0, log(2), log(3)]
weights = [exp(z-max(scores)) for z in scores]
probs = [w/sum(weights) for w in weights]
assert all(isclose(a,b) for a,b in zip(probs, [1/6,2/6,3/6]))

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

P(A given B) and P(B given A) are different quantities. A high uncalibrated model confidence is not evidence of a correct probability model.