Bayes' Theorem
The rule for updating the probability of a hypothesis when new evidence arrives, relating prior and conditional probabilities.
_(attributed_to)_-_Equestrian_Portrait_of_Sir_Ralph_Gore_(1725%E2%80%931802%5E),_6th_Bt,_Later_1st_Earl_of_Rosse,_on_His_Bay_Hunt_-_631087_-_National_Trust.jpg)
Bayes' theorem describes how the probability of a hypothesis should change when new evidence is observed. It follows directly from the definition of conditional probability and is one of the most consequential formulas in applied mathematics: it is the mathematical basis of diagnostic testing, spam filters, machine-learning classifiers, and much of modern scientific inference.
The theorem states that the probability of a hypothesis A given evidence B is:
A standard example makes the logic concrete. Suppose a condition affects 1% of a population (prior P(A) = 0.01) and a test for it is 95% sensitive (detects 95% of cases) and 95% specific (correctly negative for 95% of healthy people). A positive result seems strong evidence, but:
| Quantity | Value | |
|---|---|---|
| P(A) | 0.01 | |
| P(B | A) | 0.95 |
| P(B | not A) | 0.05 |
| P(A | B) | 0.95 · 0.01 / (0.95 · 0.01 + 0.05 · 0.99) ≈ 0.161 |
The posterior is only about 16%, not 95%. The base rate of 1% dominates: among 10,000 people there are 100 cases (95 detected) and 9,900 healthy people, of whom 495 test positive falsely, so most positive results are false positives. Ignoring the prior is the "base rate fallacy."
Bayes' theorem also provides a learning rule: the posterior from one observation becomes the prior for the next, which is how sequential evidence is accumulated. Naive Bayes classifiers apply the theorem with an independence assumption and remain competitive in text classification and spam filtering. In science, the framework makes the strength of evidence explicit: a surprising result consistent with a hypothesis raises its probability more than a result the hypothesis already predicted.
Tags
probability reasoning statistics