M6.3 · LESSON 3 OF 6

Bayes updates

Calculate conditional probabilities, Bayes updates, expectations, variance and softmax probabilities.

THE IDEA

Count people instead of multiplying fractions

A fictional disease affects 1% of a population. A fictional test is positive for 90% of sick people (its sensitivity) and also positive for 10% of healthy people (its false-positive rate). You test positive. How likely are you to be sick? Most people say about 90%. Count instead. Take 1000 people: 10 are sick and 990 are healthy. Of the sick, 9 test positive. Of the healthy, 99 also test positive. So there are 108 positives, and only 9 of them are sick: 9/108 = 1/12, about 8.3%. (This is an arithmetic example, not medical guidance.)

● sick, positive: 9● sick, negative: 1● healthy, positive: 99● healthy, negative: 891

prevalence1%sensitivity90%false-positive rate10%

10 sick of 1000. 9 of them test positive (sensitivity 90%).

990 healthy. 99 of them also test positive (false-positive rate 10%).

positives in total: 9 + 99 = 108

P(sick given positive) = 9/108 = 8.3% (exact Bayes: 8.33%; icons are rounded to whole people)

Of 108 people with a positive test, 9 are really sick: about 8 in 100.

Each dot is a person. Change the prevalence and the error rates and watch which colours make up the positives. Use the presets for the cases in the text.

Predict first. With the 1% / 90% / 10% setting, about how many positives out of 100 are really sick?

The same thing as prior times likelihood, normalised

Bayes' rule is that counting argument in formula form. For each hypothesis, multiply how plausible it was before (the prior) by how well it explains the evidence (the likelihood). The products are not probabilities yet; divide them by their total.

Stage 1 of 3

	sick	healthy	total
prior	0.01	0.99	1

Step through the update for the two hypotheses, sick and healthy.

Prior: how common is each case before the test?. Multiply by the likelihood of the evidence (a positive test). Normalise: scale so the cases sum to 1. The denominator is just the probability of seeing a positive result at all: true positives plus false positives.

Evidence accumulates

The posterior from one test becomes the prior for the next. If a second, independent test is also positive, the posterior moves from 8.33% to 45%, and a third gives 88%. Each positive multiplies the odds of being sick by 9 (that is 0.9 / 0.1). Odds of 1:99 become 9:99, then 81:99.

Predict first. Two independent positive tests with the numbers above: where does the probability of being sick end up?

The base-rate trap

A test with 99% sensitivity and a 1% false-positive rate sounds nearly perfect. At 1% prevalence a positive means exactly 50%. At 0.1% prevalence it falls to 9%: of 1000 people, 1 is sick and tests positive, while 10 healthy people also test positive. Ignoring the prior (the base rate) and reading P(positive given sick) as P(sick given positive) is one of the most common reasoning errors with probabilities. Always ask: how many people are in each group before the test?

Chapter context

Probability quantifies uncertainty relative to a model of possible outcomes. Conditioning changes the relevant population. Before inserting numbers into a formula, say which event is given and which event is being predicted.

TRY IT YOURSELF

Part 1. Do it hands-on

The array starts at the worked example. Each task asks you to move the posterior with the sliders; targets are checked with exact arithmetic on per-mille values.

● sick, positive: 9● sick, negative: 1● healthy, positive: 99● healthy, negative: 891

prevalence1%sensitivity90%false-positive rate10%

10 sick of 1000. 9 of them test positive (sensitivity 90%).

990 healthy. 99 of them also test positive (false-positive rate 10%).

positives in total: 9 + 99 = 108

P(sick given positive) = 9/108 = 8.3% (exact Bayes: 8.33%; icons are rounded to whole people)

Of 108 people with a positive test, 9 are really sick: about 8 in 100.

To do: Keep prevalence at 1%. Lower only the test error rates until P(sick given positive) reaches at least 50%
To do: Fix sensitivity at 99% and the false-positive rate at 5%. Then raise the prevalence until P(sick given positive) reaches 50%
To do: Base-rate trap: with sensitivity 99% and false-positive rate 1%, find a prevalence (at least 0.1%) where a positive test means P(sick) below 20%

Part 2. Do it in code

Write posterior(prior, sensitivity, false_positive_rate): P(sick given positive). Raise ValueError if an input is outside [0, 1] or if nobody can test positive (the denominator is 0).
Write update(priors, likelihoods): the general Bayes update over several hypotheses. Multiply, then normalize. Raise ValueError if the lengths differ or if the evidence is impossible under every hypothesis.
Write after_k_positives(prior, sensitivity, false_positive_rate, k): feed each posterior in as the next prior, for k independent positive tests.
Write natural_frequencies(prior, sensitivity, false_positive_rate, population=1000): a dict with true_pos, false_neg, false_pos and true_neg counts (floats are fine).

def posterior(prior, sensitivity, false_positive_rate):
    pass

def update(priors, likelihoods):
    pass

def after_k_positives(prior, sensitivity, false_positive_rate, k):
    pass

def natural_frequencies(prior, sensitivity, false_positive_rate, population=1000):
    pass

print(posterior(0.01, 0.9, 0.1))   # should be 1/12

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

What else is needed besides sensitivity to interpret a positive test?

AThe prior prevalence and false-positive rate.BOnly the specificity, because the prior is already contained in the test result.CThe total number of people tested, since a larger sample removes the need for a prior.

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)},\quad E[X]=\sum_x xP(X=x).

\operatorname{Var}(X)=E[(X-E[X])^2],\quad p_i=\frac{e^{z_i-m}}{\sum_j e^{z_j-m}},\;m=\max_jz_j.

Bayes' denominator must be nonzero. The softmax normalization runs over the candidate classes.

Worked example

In a fictional population, illness prevalence is 1%, sensitivity is 90% and the false-positive rate is 10%. True positives account for 0.01×0.9=0.009 of the population; false positives account for 0.99×0.1=0.099. Among positives, the illness fraction is 0.009/0.108=1/12≈8.33%. This is an arithmetic example, not medical guidance.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import exp, log, isclose
posterior = .01*.9 / (.01*.9 + .99*.1)
assert isclose(posterior, 1/12)
scores = [0, log(2), log(3)]
weights = [exp(z-max(scores)) for z in scores]
probs = [w/sum(weights) for w in weights]
assert all(isclose(a,b) for a,b in zip(probs, [1/6,2/6,3/6]))

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

P(A given B) and P(B given A) are different quantities. A high uncalibrated model confidence is not evidence of a correct probability model.