Probability

What a probability is

Kestrel Energy's load balancer sends each new customer session to one of 4 identical portal servers, chosen at random. How likely is it that a session lands on server 2? There are 4 equally likely outcomes and 1 of them is "server 2", so the chance is 1 in 4.

A probability is a number between 0 and 1 that measures how likely an event is: 0 means it cannot happen, 1 means it is certain. When all outcomes are equally likely, it is a fraction:

Here P(\text{server 2}) = \frac{1}{4} = 0.25, or 25%. You will see all three forms; they mean the same thing. Counting the outcomes is often the hard part, and Counting and Combinatorics gives the tools for that.

A second view of probability is the long-run share: if you repeat an experiment many times, the fraction of times the event happens settles near its probability. A simulation shows this. The random generator is given a fixed seed so the output is the same every run (see seeding for repeatable results):

import random

random.seed(2026)
for sessions in (100, 10_000, 100_000):
    on_server_2 = sum(1 for _ in range(sessions) if random.random() < 0.25)
    print(f"{sessions:>7} sessions: {on_server_2 / sessions:.4f} on server 2")

    100 sessions: 0.1600 on server 2
  10000 sessions: 0.2526 on server 2
 100000 sessions: 0.2505 on server 2

With 100 sessions the share is far from 0.25; with 100,000 it is very close. Small samples are noisy, which matters whenever you judge a change from a handful of incidents.

The basic rules

The complement rule

Something either happens or it does not, and those two probabilities add up to 1. So the probability that event A does not happen is

If a backup job succeeds with probability 0.97, it fails with probability 1 - 0.97 = 0.03. This rule looks trivial but it is the key to the "at least one" questions further down.

The addition rule: A or B

Last month Kestrel's service desk closed 200 incidents. Of these, 80 were network-related, 50 had high priority, and 30 were both. Pick one incident at random: how likely is it to be network-related or high priority?

1. P(\text{network}) = 80/200 = 0.40 and P(\text{high}) = 50/200 = 0.25.
2. Simply adding gives 0.65, but the 30 incidents that are both have been counted twice: once among the network incidents and once among the high-priority ones.
3. Subtract the overlap once: P(\text{both}) = 30/200 = 0.15, so the answer is 0.40 + 0.25 - 0.15 = 0.50.

In general ("or" means one, the other, or both):

The symbols come from set theory: \cup (union) is "or" and \cap (intersection) is "and" (see Sets and Relations). When two events cannot happen together, they are mutually exclusive, the overlap is 0, and you may simply add.

Independent events and "at least one"

Two events are independent when knowing that one happened does not change the probability of the other. For independent events the chance that both happen is the product:

Worked example: an API call to a supplier fails 10% of the time, and suppose each attempt fails independently of the others. A client retries up to 3 times in total. How likely is it that the request eventually succeeds?

1. All 3 attempts fail: 0.1 \cdot 0.1 \cdot 0.1 = 0.001.
2. "Succeeds at least once" is the complement of "all fail": 1 - 0.001 = 0.999.

The same trick answers "how likely is at least one failure?". Suppose a disk has a 2% chance of failing within a year. For a server with n disks, assuming independence:

p_fail = 0.02               # one disk fails within a year
for n in (1, 4, 12):
    print(n, f"{1 - (1 - p_fail) ** n:.4f}")

1 0.0200
4 0.0776
12 0.2153

With 12 disks there is more than a 1 in 5 chance that at least one fails within the year, which is why storage arrays use RAID and hot spares. The same multiplication rule gives the availability of components in series (both must work: multiply the availabilities) and in parallel (all must fail: multiply the unavailabilities); the cloud page works this out in components in series and in parallel. The "at least one" rule also explains false alarms in data analysis: test 20 independent metrics at the 5% level and the chance that at least one looks significant by luck alone is 1 - 0.95^{20} \approx 64\%; see peeking and multiple comparisons.

Conditional probability

Often you know something already and want the probability given that knowledge. The conditional probability of A given B, written P(A \mid B), is the probability of A when you only look at the cases where B happened:

Read the vertical bar as "given". Back to the 200 incidents: what is the chance that an incident is high priority, given that it is network-related? Only look at the 80 network incidents; 30 of them are high priority, so P(\text{high} \mid \text{network}) = 30/80 = 0.375. With the formula: 0.15 / 0.40 = 0.375. The same answer, because dividing by P(B) is exactly "only look at the cases where B happened".

Note that P(\text{high}) = 0.25 overall but 0.375 among network incidents: knowing the category changed the probability, so these two events are dependent. Independence can be stated exactly this way: P(A \mid B) = P(A).

A phishing filter, counted out

Kestrel's mail filter catches 95% of phishing mails, and wrongly flags 2% of legitimate mails. About 1% of incoming mail is phishing. A mail has just been flagged. How likely is it that it really is phishing? Many people guess 95%. Count it out for 10,000 mails:

1. 1% is phishing: 100 phishing mails and 9,900 legitimate ones.
2. The filter flags 95% of the phishing: 95 mails.
3. It also flags 2% of the legitimate mails: 0.02 × 9,900 = 198 mails.
4. In total 95 + 198 = 293 mails are flagged, and 95 of them are phishing.
5. P(\text{phishing} \mid \text{flagged}) = 95/293 \approx 0.324.

Only about 1 in 3 flagged mails is phishing, even though the filter is "95% accurate". The reason is the base rate: legitimate mail is so much more common that its small 2% error rate still produces twice as many flags as the real phishing does.

Bayes' theorem

The counting above follows a general rule. You knew P(\text{flagged} \mid \text{phishing}), how the filter behaves on phishing, but you wanted the reverse, P(\text{phishing} \mid \text{flagged}). Bayes' theorem turns a conditional probability around:

In words: the probability of the cause given the evidence equals how likely the evidence is under that cause, times how common the cause was to begin with, divided by how common the evidence is overall. The bottom line, P(B), usually has to be built from the branches of the tree: every way in which the evidence can appear.

1. P(\text{flagged}) = 0.95 \cdot 0.01 + 0.02 \cdot 0.99 = 0.0095 + 0.0198 = 0.0293.
2. P(\text{phishing} \mid \text{flagged}) = \frac{0.95 \cdot 0.01}{0.0293} = \frac{0.0095}{0.0293} \approx 0.324.

# Bayes: 1% of mails are phishing; the filter flags 95% of phishing
# and 2% of legitimate mail. What share of flagged mail is phishing?
p_phish = 0.01
p_flag_given_phish = 0.95
p_flag_given_ok = 0.02
p_flag = p_flag_given_phish * p_phish + p_flag_given_ok * (1 - p_phish)
print(f"P(flagged) = {p_flag:.4f}")
print(f"P(phishing | flagged) = {p_flag_given_phish * p_phish / p_flag:.3f}")

P(flagged) = 0.0293
P(phishing | flagged) = 0.324

In machine-learning terms, P(\text{phishing} \mid \text{flagged}) is the filter's precision and P(\text{flagged} \mid \text{phishing}) its recall; see accuracy, precision, recall and F1. The same effect explains alert fatigue in monitoring: a rule that fires on a rare problem must have a very low false-alarm rate, or most of its alerts will be noise.

Why a p-value is not the probability that a hypothesis is true, the reverse of this question, is on Statistics for Analysts.

Expected value

Kestrel's out-of-hours service desk tracked how many tickets arrive in an hour. In half of the hours none arrive, in 30% one, in 15% two, and in 5% of the hours (storm nights) five. What is the average number per hour, in the long run?

The expected value is that long-run average: multiply each possible value by its probability and add the results. For a quantity X with values x_i:

1. 0 \cdot 0.5 = 0
2. 1 \cdot 0.3 = 0.3
3. 2 \cdot 0.15 = 0.3
4. 5 \cdot 0.05 = 0.25
5. Total: E[X] = 0 + 0.3 + 0.3 + 0.25 = 0.85 tickets per hour.

The expected value does not have to be a possible outcome: no single hour has 0.85 tickets. It is useful for planning: over a 10-hour night shift you should expect about 8.5 tickets. Exact fractions avoid any rounding:

from fractions import Fraction

# Expected value: tickets per hour with their probabilities
dist = {0: Fraction(1, 2), 1: Fraction(3, 10), 2: Fraction(3, 20), 5: Fraction(1, 20)}
print(sum(dist.values()))                     # the probabilities must add up to 1
expected = sum(k * p for k, p in dist.items())
print(expected, float(expected))
print(float(10 * expected), "tickets in a 10-hour shift")

1
17/20 0.85
8.5 tickets in a 10-hour shift

Security risk analysis uses the same idea: the annualized loss expectancy is the expected yearly cost of an incident, its cost times how often it is expected per year (see quantitative risk). Forecasts that simulate many possible futures, such as a Monte Carlo forecast, report probabilities instead of a single expected value, because the spread matters too.

The birthday paradox and collisions

How many people must be in a room before there is a better than even chance that two of them share a birthday? Many people guess somewhere near half of 365. The answer is 23. This surprising result is the birthday paradox: in a group, the chance that some pair shares a value rises much faster than intuition expects, because the number of pairs grows quickly. 23 people form \binom{23}{2} = 253 pairs, and each pair is a chance for a match.

Compute it with the complement rule (assume 365 equally likely birthdays):

1. The first person's birthday is free: \frac{365}{365}.
2. The second must avoid it: \frac{364}{365}. The third must avoid both: \frac{363}{365}, and so on.
3. Multiply these for n people to get P(\text{all different}).
4. P(\text{some pair shares}) = 1 - P(\text{all different}).

import math

def p_shared_birthday(n, days=365):
    p_unique = 1.0
    for i in range(n):
        p_unique *= (days - i) / days
    return 1 - p_unique

print(f"{p_shared_birthday(23):.4f}")
print(f"{p_shared_birthday(50):.4f}")
# Collisions in a 32-bit hash: number of items for about a 50% chance
print(round(math.sqrt(2 * 2 ** 32 * math.log(2))))

0.5073
0.9704
77163

For IT, replace birthdays by hash values or random IDs. With N equally likely values, a collision becomes about 50% likely after roughly

items, so for a 32-bit hash (N = 2^{32}, over 4 billion values) only about 77,000 items are needed, not billions. That is why short IDs and 32-bit checksums are fine for detecting accidental errors but useless as unique identifiers for millions of records, and why a cryptographic hash needs twice as many bits as the collision resistance it aims for. The cryptography page treats this as the birthday bound; see The Math Behind Cryptography.