Basic probability theory

The mathematical model for a class of measurements is called the probability space and is composed of a mathematical triple of a sample space $\Omega$, $\sigma$-algebra $\mathcal{F}$, and probability measure $P$, typically denoted $(\Omega,\mathcal{F},P)$, each of which we will consider in turn (Wikipedia 2019).

The sample space $\Omega$ of an experiment is the set representing all possible outcomes of the experiment. If a coin is flipped, the sample space is $\Omega = \{H,T\}$, where $H$ is heads and $T$ is tails. If a coin is flipped twice, the sample space could be \[\begin{aligned} \Omega = \{ HH,HT,TH,TT \}.\end{aligned}\]

However, the same experiment can have different sample spaces. For instance, for two coin flips, we could also choose \[\begin{aligned} \Omega = \{ \text{the flips are the same}, \text{the flips are different} \}.\end{aligned}\] We base our choice of $\Omega$ on the problem at hand.

An event is a subset of the sample space. That is, an event corresponds to a yes-or-no question about the experiment. For instance, event $A$ (remember: $A\subseteq\Omega$) in the coin flipping experiment (two flips) might be $A = \{HT,TH\}$. $A$ is an event that corresponds to the question, “Is the second flip different than the first?” $A$ is the event for which the answer is “yes.”

Algebra of events

Because events are sets, we can perform the usual set operations with them.

Example 3.1

Consider a toss of a single die. We choose the sample space to be Ω = {1, 2, 3, 4, 5, 6}. Let the following define events. $$\begin{aligned} A &\equiv \{\text{the result is even}\} = \{2,4,6\} \\ B &\equiv \{\text{the result is greater than } 2\} = \{3,4,5,6\}. \end{aligned}$$ Find the following event combinations: $$\begin{aligned} A\cup B && A\cap B && A\setminus B && B\setminus A && \overline{A}\setminus B. \end{aligned}$$

$$\begin{aligned} A\cup B &= \{2,3,4,5,6\} \quad\text{(even or greater than 2)} \\ A\cap B &= \{4,6\} \quad\text{(even and greater than 2)} \\ A\setminus B &= \{2\} \quad\text{(even but not greater than 2)} \\ B\setminus A &= \{3,5\} \quad\text{(greater than two and odd)} \\ \overline{A}\setminus B &= \{1,3,5\} \setminus \{3,4,5,6\} \quad\text{(not even and not greater than 2)}. \end{aligned}$$

The $\sigma$-algebra $\mathcal{F}$ is the collection of events of interest. Often, $\mathcal{F}$ is the set of all possible events given a sample space $\Omega$, which is just the power set of $\Omega$ (Wikipedia 2019). When referring to an event, we often state that it is an element of $\mathcal{F}$. For instance, we might say an event $A\in\mathcal{F}$.

We’re finally ready to assign probabilities to events. We define the probability measure $P:\mathcal{F}\rightarrow [0,1]$ to be a function satisfying the following conditions.

For every event $A\in\mathcal{F}$, the probability measure of $A$ is greater than or equal to zero—i.e. $P(A)\ge 0$.
If an event is the entire sample space, its probability measure is unity—i.e. if $A=\Omega$, $P(A) = 1$.
If events $A_1, A_2,\cdots$ are disjoint sets (no elements in common), then $P(A_1\cup A_2\cup\cdots) = P(A_1) + P(A_2) + \cdots$.

We conclude the basics by observing four facts that can be proven from the definitions above.

$P(\emptyset) = 0$.
$P(A\cup B) = P(A) + P(B) - P(A\cap B)$.
If $B\subset A$, then $P(B) < P(A)$. In fact, $P(A\setminus B) = P(A) - P(B)$.
$P(A_1 \cup A_2 \cup \cdots) \le P(A_1) + P(A_2) + \cdots$.

Wikipedia. 2019. “Probability space — Wikipedia, the Free Encyclopedia.” http://en.wikipedia.org/w/index.php?title=Probability%20space&oldid=914939789.

Online Resources for Section 3.2

No online resources.