Sample Space & Basic Probability: Building the Foundation

⚡ Key Takeaways

A sample space is the set of all possible outcomes of a random experiment; probability assigns each outcome a number between 0 and 1 measuring how likely it is. For equally likely outcomes, the probability of an event equals (favourable outcomes) ÷ (total outcomes). This framework underlies all of statistics, gambling, genetics, and risk analysis, and was formalised by Pascal, Fermat, and Laplace between the 1650s and 1810s.

💡 The Core Insight

Probability is built in two layers. First, list every possible outcome — the sample space. Second, attach weights (probabilities) to outcomes so the weights are non-negative and sum to 1. Once these two steps are done, the probability of any event is simply the sum of the weights of the outcomes inside it. Everything else — addition rules, complements, conditional probability — is bookkeeping on this foundation.

DefinitionRandom Experiment, Sample Space, Event

A random experiment is a repeatable procedure with a well-defined set of possible results. The sample space $S$ (or $\Omega$ ) is the set of all distinct possible outcomes. Each individual outcome $\omega \in S$ is called a sample point or elementary event. An event $A$ is any subset of the sample space, $A \subseteq S$ . We say "event $A$ occurs" if the actual outcome $\omega$ satisfies $\omega \in A$ .

The certain event is $S$ itself.
The impossible event is the empty set $\varnothing$ .
An elementary event is a single-outcome subset $\{\omega\}$ .

DefinitionProbability Measure — Kolmogorov's Axioms

A probability measure $P$ assigns to each event $A \subseteq S$ a real number $P(A)$ satisfying:

1

\quad P(A) \ge 0 \quad \text{for every event } A

Derivation hint: This is an axiom (non-negativity); probabilities cannot be negative because they measure a proportion of likelihood.

2

\quad P(S) = 1

Derivation hint: An axiom (normalisation); something in the sample space must happen, so the total probability is exactly 1.

3

\quad P\!\left(\bigcup_{i} A_i\right) = \sum_{i} P(A_i) \quad \text{if } A_i \cap A_j = \varnothing \text{ for } i \ne j

Derivation hint: The additivity axiom; for mutually exclusive (disjoint) events, the probability of "at least one occurring" is the sum of their separate probabilities. Here $A_i \cap A_j = \varnothing$ means the events are mutually exclusive — they cannot both occur.

TheoremClassical (Laplace) Probability for Equally Likely Outcomes

If $S$ is finite with $n = |S|$ outcomes that are all equally likely, then for any event $A$ containing $|A|$ outcomes:

4

\quad P(A) = \frac{|A|}{|S|} = \frac{\text{number of favourable outcomes}}{\text{total number of outcomes}}

Derivation hint: Assign each of $n$ outcomes equal weight $1/n$ (forced by [2] and [3]); summing the weights of the $|A|$ outcomes in $A$ gives $|A|/n$ .

TheoremBasic Consequences

From axioms [1]–[3], the following hold for all events $A, B \subseteq S$ :

5

\quad P(A^c) = 1 - P(A)

Derivation hint: $A$ and its complement $A^c$ are disjoint and their union is $S$ ; apply [3] then [2].

6

\quad P(\varnothing) = 0

Derivation hint: Take $A = S$ in [5], since $S^c = \varnothing$ and $P(S)=1$ .

7

\quad 0 \le P(A) \le 1

Derivation hint: Lower bound from [1]; upper bound because $A \subseteq S$ forces $P(A) \le P(S) = 1$ via monotonicity.

8

\quad P(A \cup B) = P(A) + P(B) - P(A \cap B)

Derivation hint: The inclusion–exclusion principle; outcomes in the overlap $A \cap B$ get counted twice when adding $P(A)+P(B)$ , so subtract them once.

mathematician · Andrey Kolmogorov | 1903–1987

The Soviet mathematician Andrey Nikolaevich Kolmogorov gave probability its modern rigorous footing in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of Probability"). He recognised that probability is a special case of measure theory: a sample space with a "measure" that assigns total mass 1. Before him, probability was a collection of clever but informal techniques; after him, it became a fully axiomatic branch of pure mathematics. Kolmogorov also made deep contributions to turbulence, dynamical systems, and algorithmic complexity.

🧠 Intuition

Think of the sample space as a pie and probability as how the pie is sliced. The whole pie has area 1 (axiom [2]). Each outcome takes a slice whose size is its probability, and no slice can have negative area (axiom [1]). An event is a group of slices, and its probability is just the combined area (axiom [3]). For a fair die, the pie is cut into 6 equal slices, each $\tfrac{1}{6}$ . The event "even number" $= \{2,4,6\}$ is three slices, area $\tfrac{3}{6} = \tfrac12$ . The crucial mental move is separating the listing from the weighting. First ask: what can happen? Then ask: how likely is each? Many errors come from skipping the first question — for instance, forgetting that rolling two dice has 36 outcomes, not 11.

Figure

The sample space of two dice as a 6×6 grid; the diagonal band marks outcomes summing to 7.

ExampleDrawing a Card from a Standard Deck

Given. A well-shuffled standard deck of 52 cards. Draw one card at random. Find the probability the card is either a king or a heart.

Step 1 — Sample space. All 52 cards are equally likely, so $|S| = 52$ .

Step 2 — Define events.

$K$ = "card is a king", $|K| = 4$ .
$H$ = "card is a heart", $|H| = 13$ .
$K \cap H$ = "king of hearts", $|K \cap H| = 1$ .

Step 3 — Individual probabilities using [4]:

P(K) = \frac{4}{52}, \quad P(H) = \frac{13}{52}, \quad P(K\cap H) = \frac{1}{52}.

Step 4 — Addition rule [8] (overlapping events, so subtract the intersection):

P(K \cup H) = \frac{4}{52} + \frac{13}{52} - \frac{1}{52} = \frac{16}{52}.

Step 5 — Simplify. $\dfrac{16}{52} = \dfrac{4}{13}$ .

\boxed{P(\text{king or heart}) = \frac{4}{13} \approx 0.3077}

ExampleSum of Two Fair Dice

Given. Roll two fair six-sided dice. Find (a) $P(\text{sum} = 7)$ and (b) $P(\text{sum} \ge 10)$ .

Step 1 — Sample space. Ordered pairs $(d_1, d_2)$ , so $|S| = 6 \times 6 = 36$ equally likely outcomes.

Step 2 (a) — Event sum = 7. Favourable pairs: $(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)$ → $|A| = 6$ .

P(\text{sum}=7) = \frac{6}{36} = \frac{1}{6} \approx 0.1667.

Step 3 (b) — Event sum ≥ 10. Means sum = 10, 11, or 12 (mutually exclusive).

Sum 10: $(4,6),(5,5),(6,4)$ → 3 outcomes
Sum 11: $(5,6),(6,5)$ → 2 outcomes
Sum 12: $(6,6)$ → 1 outcome

Total favourable $= 3 + 2 + 1 = 6$ (by additivity [3]).

P(\text{sum} \ge 10) = \frac{6}{36} = \frac{1}{6} \approx 0.1667.

\boxed{P(\text{sum}=7) = \tfrac{1}{6}, \qquad P(\text{sum}\ge 10) = \tfrac{1}{6}}

⚠️ Outcomes Are Not Always Equally Likely

Formula [4] only applies when every outcome in $S$ is equally likely. A common blunder is to list outcomes by result rather than by equally likely cases. For two dice, the sums $\{2,3,\dots,12\}$ form a valid sample space of 11 outcomes — but they are not equally likely, so $P(\text{sum}=7) \ne \tfrac{1}{11}$ . The correct equally likely sample space is the 36 ordered pairs. Always choose your sample space so the counting formula is justified, or weight the outcomes explicitly.

⚠️ The Gambler's Fallacy

Independent random trials have no memory. After five consecutive heads, the probability of heads on the next fair toss is still $\tfrac12$ , not lower. The coin does not "owe" you a tail. The mistaken belief that past outcomes influence future independent ones is the gambler's fallacy, and it has cost countless people their savings. See Independent Events.

CounterexampleWhy "Equally Likely" Can't Be Assumed

Consider tossing two coins and recording the number of heads: the sample space is $\{0, 1, 2\}$ . A naïve student might say each has probability $\tfrac13$ . But list the equally likely raw outcomes: $\{HH, HT, TH, TT\}$ , four cases.

$0$ heads: only $TT$ → $\tfrac14$
$1$ head: $HT, TH$ → $\tfrac24 = \tfrac12$
$2$ heads: only $HH$ → $\tfrac14$

So $P(1\text{ head}) = \tfrac12$ , double the naïve guess. The lesson: a valid sample space need not have equally likely members — you must verify it.

connection

Sample spaces and basic probability are the gateway to the whole subject. The addition rule [8] generalises to the Inclusion-Exclusion Principle for many events. Restricting the sample space to a known event leads directly to Conditional Probability and Bayes' Theorem. When outcomes carry numerical values, the weighting becomes a Random Variable and its Probability Distribution. Counting the size of events efficiently requires Combinatorics (permutations and combinations). Finally, the frequentist justification that observed relative frequencies approach true probabilities is made precise by the Law of Large Numbers.

🔬 Where this appears

⏳ History

- 1564 (pub. 1663). Gerolamo Cardano writes Liber de Ludo Aleae, the first systematic treatment of dice odds, anticipating equally likely outcomes.

- 1654. Blaise Pascal and Pierre de Fermat exchange letters solving the problem of points (how to divide stakes in an interrupted game), founding mathematical probability.

- 1657. Christiaan Huygens publishes De Ratiociniis in Ludo Aleae, the first printed probability textbook, introducing expected value.

- 1713. Jacob Bernoulli's posthumous Ars Conjectandi proves the first Law of Large Numbers, linking probability to long-run frequency.

- 1733–1738. Abraham de Moivre derives the normal approximation to the binomial, foreshadowing the bell curve.

- 1812. Pierre-Simon Laplace's Théorie analytique des probabilités gives the classical definition [4] and treats probability as a tool for reasoning under uncertainty.

- 1900s. The frequentist (von Mises) and subjective/Bayesian (Ramsey, de Finetti) interpretations are debated.

- 1933. Andrey Kolmogorov publishes the axioms [1]–[3], unifying probability with measure theory and giving the modern foundation.

open-problem

The neat picture of a finite, equally likely sample space breaks down in subtle cases. When $S$ is uncountably infinite (e.g. picking a random real number in $[0,1]$ ), individual outcomes have probability zero, yet events still have positive probability — handled by Measure Theory and probability densities, not sums. Moreover, on such spaces one cannot consistently assign a probability to every subset (non-measurable sets exist, as shown by the Vitali and Banach–Tarski constructions), so we restrict to a $\sigma$ -algebra of "nice" events. A deeper, still-debated question is the interpretation of probability itself: is $P(A)$ an objective long-run frequency, a degree of rational belief (Bayesian), or a propensity of the physical system? This is a live issue in the foundations of statistics and quantum mechanics, where probabilities appear irreducibly fundamental.

Sample Space & Basic Probability: Building the Foundation

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also