M6.2 · LESSON 2 OF 6

Conditional probability

Calculate conditional probabilities, Bayes updates, expectations, variance and softmax probabilities.

THE IDEA

Conditioning shrinks the sample space

Suppose you learn that the first die shows a 6. Of the 36 cells only the 6 in that row are still possible, and they are still equally likely. Learning something does not change the dice. It changes which outcomes you are looking at. That is conditioning: P(A given B) is the probability of A inside the world where B happened.

In area terms: draw the whole outcome space as a unit square, with B as a column of width P(B). The part of that column where A also holds has area P(A and B). Conditioning on B throws away the rest of the square and rescales B to fill it, so P(A given B) = P(A and B) / P(B). This needs P(B) > 0. You cannot rescale a world that never happens.

P(B)30%P(A given B)70%P(A given not B)20%

Condition on B (throw away everything outside B and rescale)

P(A and B) = P(B) x P(A given B) = 0.3 x 0.7 = 0.21

P(A) = 0.21 + 0.14 = 0.35

P(A given B) = 0.7

P(B given A) = P(A and B) / P(A) = 0.6

Draw people (seed 11):

people drawn: 0, in B: 0, with A: 0, both: 0

among the 0 in B, share with A: n/a (target 0.7)

among the 0 with A, share in B: n/a (target 0.6)

Tick the checkbox to zoom into B. The blue share of the zoomed column is P(A given B). Below, the bar shows who holds A: the blue part is how much of A lies inside B.

Order matters

Take A = "the sum is at least 11" (3 cells) and B = "the first die is 6" (6 cells). They share 2 cells: (6,5) and (6,6). Then P(A given B) = 2/6 = 1/3, but P(B given A) = 2/3. Same overlap, different denominators: the first divides by the size of B, the second by the size of A.

Predict first. Among the outcomes with a first die of 6, what fraction have a sum of at least 11?

Filtering is conditioning

Simulation makes this concrete. To estimate P(A given B) by drawing random people, throw away every draw where B did not happen, then count how often A happened among the survivors. In the widget above, draw a few thousand people and compare the shares with the targets. Dividing by all the draws instead of only the B draws computes something else, P(A and B).

If knowing B changes nothing, P(A given B) = P(A), we call A and B independent. Then P(A and B) = P(A) P(B).

The common mistake: swapping the direction

Set P(B) = 10%, P(A given B) = 80% and P(A given not B) = 50% in the widget. A is common outside B, so most people with A are not in B: P(B given A) is only 8/53 = 15.1%. "80% of B has A" and "80% of A is in B" are different statements about different groups. Next lesson uses exactly this to explain why a positive test is not the same as being ill.

Chapter context

Probability quantifies uncertainty relative to a model of possible outcomes. Conditioning changes the relevant population. Before inserting numbers into a formula, say which event is given and which event is being predicted.

TRY IT YOURSELF

Part 1. Do it hands-on

Use the three sliders. Each task is checked from the exact numbers, so values like 60% must be hit exactly.

P(B)30%P(A given B)70%P(A given not B)20%

Condition on B (throw away everything outside B and rescale)

P(A and B) = P(B) x P(A given B) = 0.3 x 0.7 = 0.21

P(A) = 0.21 + 0.14 = 0.35

P(A given B) = 0.7

P(B given A) = P(A and B) / P(A) = 0.6

Draw people (seed 11):

people drawn: 0, in B: 0, with A: 0, both: 0

among the 0 in B, share with A: n/a (target 0.7)

among the 0 with A, share in B: n/a (target 0.6)

To do: Set P(A given B) = 80% while P(B given A) is at most 20%
To do: Make A independent of B with P(A) = 30%, while P(B) is strictly between 0% and 100%
To do: Make P(A given B) = 60% and P(B given A) = 60% at the same time

Part 2. Spot the wrong statements

Which of these conditional probability statements are wrong?

P(A given B) = P(A and B) / P(B), as long as P(B) is not zero.
If 90% of sick people test positive, then 90% of people who test positive are sick.
P(A given B) + P(not A given B) = 1.
P(A given B) + P(A given not B) = 1.
If P(A given B) = P(A), then learning that B happened does not change how likely A is.
20 of 100 people wear glasses and 5 of those 20 are left-handed, so P(glasses given left-handed) = 5/20.

Tick every claim you think is wrong.

Part 3. Do it in code

Write conditional(joint, event, given). joint maps outcomes to probabilities; event and given are functions that take an outcome and return True or False. Return P(event and given) / P(given). Raise ValueError if P(given) is 0.
Write estimate_conditional(event, given, trials, seed): roll two dice with random.Random(seed) (use rng.randint(1, 6) for each die), keep only the trials where given((d1, d2)) is True, and return the fraction of those where event((d1, d2)) is also True. Raise ValueError if no kept trial exists.

import random

def conditional(joint, event, given):
    # P(event and given) / P(given); ValueError when P(given) == 0
    pass

def estimate_conditional(event, given, trials, seed):
    # simulate two dice; the denominator counts only the trials where 'given' happened
    pass

joint = {(a, b): 1 / 36 for a in range(1, 7) for b in range(1, 7)}
print(conditional(joint, lambda o: o[0] + o[1] >= 11, lambda o: o[0] == 6))

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Does a high P(test positive given illness) alone imply a high P(illness given positive)?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)},\quad E[X]=\sum_x xP(X=x).

\operatorname{Var}(X)=E[(X-E[X])^2],\quad p_i=\frac{e^{z_i-m}}{\sum_j e^{z_j-m}},\;m=\max_jz_j.

Bayes' denominator must be nonzero. The softmax normalization runs over the candidate classes.

Worked example

In a fictional population, illness prevalence is 1%, sensitivity is 90% and the false-positive rate is 10%. True positives account for 0.01×0.9=0.009 of the population; false positives account for 0.99×0.1=0.099. Among positives, the illness fraction is 0.009/0.108=1/12≈8.33%. This is an arithmetic example, not medical guidance.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import exp, log, isclose
posterior = .01*.9 / (.01*.9 + .99*.1)
assert isclose(posterior, 1/12)
scores = [0, log(2), log(3)]
weights = [exp(z-max(scores)) for z in scores]
probs = [w/sum(weights) for w in weights]
assert all(isclose(a,b) for a,b in zip(probs, [1/6,2/6,3/6]))

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

P(A given B) and P(B given A) are different quantities. A high uncalibrated model confidence is not evidence of a correct probability model.