Probability Theory (Series)
This series is an introduction to Probability Theory, It closely follows the book “Probability Essentials” by Jean Jacod and Philip Protter.
Day 0 : Philosphical Introduction to Probability Theory
Lets Start with a Random Experiment, A random experiment is an experiment whose outcome is not predictable with certainty. For example, tossing a coin, rolling a die, etc.
Now a random experiment consist of three things:
- State Space: The set of all possible outcomes of a random experiment is called the state space, denoted by $\Omega$. For example, in the case of tossing a coin, the state space is $\Omega = {H, T}$, where $H$ denotes head and $T$ denotes tail.
- Event: An event is a question about random experiment outcome whose answer is either true or false. For example, in the case of tossing a coin, the event “Head” is true if the outcome is head and false otherwise. In mathematical terms, an event is a subset of the state space $\Omega$. And since It is a set, It must adhere to set properties like union, intersection, complement, etc. From this suppose we have two event $A$ and $B$, then following are the set operations:
- Conmpliment: The compliment of an event $A$ is denoted by $A^c$ and is defined as $A^c = \Omega - A$. In the case of tossing a coin, the compliment of event “Head” is “Tail”.Hence if A is the event “Head”, then $A^c = {T}$.
- Union: The union of two events $A$ and $B$ is denoted by $A \cup B$ and is defined as $A \cup B = {x \in \Omega : x \in A \text{ or } x \in B}$. In the case of tossing a coin, the union of event “Head” and “Tail” is the entire state space $\Omega$.
- Intersection: The intersection of two events $A$ and $B$ is denoted by $A \cap B$ and is defined as $A \cap B = {x \in \Omega : x \in A \text{ and } x \in B}$. In the case of tossing a coin, the intersection of event “Head” and “Tail” is an empty set $\emptyset$.
- Sure Event: The sure event is the event that is always true, It is denoted by $\Omega$.
- Impossible Event: The impossible event is the event that is always false, It is denoted by $\emptyset$.
- Elementary Event: The elementary event is the event that contains only one outcome, A singleton i.e a subset ${\omega}$ of the state space $\Omega$.
- Conmpliment: The compliment of an event $A$ is denoted by $A^c$ and is defined as $A^c = \Omega - A$. In the case of tossing a coin, the compliment of event “Head” is “Tail”.Hence if A is the event “Head”, then $A^c = {T}$.
The family of all events is called the $\sigma$-algebra denoted by $\mathcal{A}$. The $\sigma$-algebra must satisfy the following properties:
- $\Omega \in \mathcal{A}$
- If $A \in \mathcal{A}$, then $A^c \in \mathcal{A}$
- If $A_1, A_2, \ldots \in \mathcal{A}$, then $\cup_{i=1}^{\infty} A_i \in \mathcal{A}$ These condition ensures the point mentioned above named as compliment, union, intersection, sure event, impossible event, and elementary event.
- Probability: The probability is a function that assigns a number between 0 and 1 to each event in the $\sigma$-algebra $\mathcal{A}$. Going by the conventional approach probability can be seen as limits of the freequency of occurrence of an event in a large number of trials under Identitcal Conditions. For example, the probability of getting a head in a coin toss is 0.5, which means in a large number of coin tosses, the number of heads will be half of the total number of tosses. The probability function must satisfy the following properties:
- Non-Negativity: The probability of any event is non-negative, i.e $P(A) \geq 0$.
- Normalization: The probability of the sure event is 1, i.e $P(\Omega) = 1$.
- Additivity: The probability of the union of two disjoint events is the sum of the probability of the individual events, i.e $P(A \cup B) = P(A) + P(B)$ if $A \cap B = \emptyset$. We will discuss the Proability in detail in the upcoming posts. A fourth notion that is closely related to probability is the random variable, which we will discuss in the next post.
Leave a comment