Probability Models and Axioms

Goals

Probability as a mathematical framework for reasoning about uncertainty

Sample space
Axioms of probability
Simple Examples

Symbols:

P(A) The probability of the subset/event A
∨ This is the symbol for the OR operation
∧ This is the symbol for the AND operation
Ω This is the symbol for the entire set in the sample space
A'c Complement of A, The set of probability that are not A

What Is probability?

In the most narrow view probability is just a branch of mathematics, with some axiom we consider models that satisfy these axioms and we establish some consequence, which are the theorems of this theory.

Probability can be also thought of as frequencies, if i have a fair coin, and i toss it infinitely many times then the fraction that i will observe will be $1/2$ this is not universal for example let’s consider the probability that the current president will be elected it doesn’t make sense to think of it as a frequency.

In cases like these it’s better to think of probability as just some way to describe our beliefs, but aren’t beliefs subjective?

Well Probability, at the minimum gives us some rules for thinking systematically about uncertain situations, and if it happens that our probability model, our subjective beliefs, have some relation with the real world, than probability theory is a useful tool to make predictions/decisions about the real world, whether your predictions and decisions will be of any good will depend on whether you have chosen a good model. Have you chosen a model that provides a good enough representation of the real world? and How do you make sure that’s the case? That’s a whole other field, the field of statistics whose purpose is to complement probability theory by using data to come up with good models.

Sample Space

What is a Sample Space?

”List” of possible outcomes
List must be:
- Mutually exclusive
- Collectively exhaustive
- At the right granularity

Example of sample space: flipping a coin, rolling a die etc..

Here we have an the sample space (the list of all possibilities) of 3 coins each with two possibility of H or T. We can calculate the number of all combinations with 2^3.

So let’s get into details:

Mutually exclusive:

If an outcome happens for example (HHT) that’s the only outcome happening, no other outcome is happening with it.

Collectively exhaustive:

We have all the possible outcomes in our sample space, therefore it is Exhaustive.

At the right granularity

Imagine we have the sample space of flipping a coin we have two possible outcomes: H for heads or T for tails, we can actually express an infinite list of outcomes depending for example H + bad weather, H + good weather, T + bad weather, T + good weather if we thing that the quality of weather has a hand into determining our outcome, This what we mean by granularity, for the problem of coin tossing the first sample space of H and T will be better for the problem of spinning a coin, but this is not a hard requirement it just mean that the second information (raining and not raining) is irrelevant.

Examples of Sample space:

Suppose we have the set

Ω = {◂...▸ Heads and it is raining, Heads and it is not raining, Tails}

This is a valid set since it’s mutually exclusive and and collectively exhaustive.

Ω = {◂...▸ Heads and it is raining, Tails and it is not raining, Tails}

This is not a valid as it is not mutually exclusive we have Tails and it is not raining and Tails which have the same outcome Tails.

Sample Space: discrete/finite

For Example if we took the sample space of flipping two coins we find that it is discrete: it has a possible set of values {HT, TH, HH, TT} (For our problem order matters we can also deal with problems where the order doesn’t matter) and finite as that set is made of only 4 possibilities. one way to represent the space is a matrix:

H	T
HH	HT
TH	TT

where have the possibilities of the first coin as columns and second coin as lines.

Sample Space: Continuous

For example let’s say we are playing throwing darts we can have a sample space of the (x,y) coordinates of our throw in the board, given that we have a range of possible values we say that our Sample space is coninuous.

Probability axioms

Let’s take the example of the last sample space the dart throwing case, the probability of hitting any point there is 0 because each point is infinitely small so we don’t assign probability to a single element in the sample space rather we choose a subset of the sample space that’s called an Event.

Event:

A subset of the sample space

Probability is assigned to events in the following notation: P(A) A being the event.

Probability:

notation: P(A)

By convention probabilities are always given in range between 0 and 1, 0 means that we believe something cannot happen and 1 we are certain that the event is going to happen.

Axioms:

Nonnegativity: P(A) >= 0 A probability can never be negative
Normalization: P(Ω) = 1 The probability of the entire sample space is always 1
(Finite) Additivity: If A ∧ B = None, then P(A ∨ B) = P(A) + P(B)

Example:

Let A and B be events on the same sample space, with P(A)=0.6 and P(B)=0.7. Can these two events be disjoint?

The answer is: NO

If the two events were disjoint, the additivity axiom would imply that P(A∪B)=P(A)+P(B)=1.3>1, which would contradict the normalization axiom.

These are consequences of the axioms.

Rules of probability:

Rules are the results of the axioms

if we have two sets A and B, A is part of B so set A smaller than B, So if B is larger than A then naturally the probability that the outcome falls inside B should be at least as big as the probability that the outcome falls inside A.

Formally: if A inside B, then P(A) ⇐ P(B)

B = A U (B ∩ A’c) P(B) = P(A) + P(B ∩ A’c) >= P(A) Since a probability can never be negative

let’s say a=P(A and B’c) b=P(A and B) c=P(B and A’c)

P(A U B) = a + b + c
P(A) + P(B) - P(A and B) = (a+b) + (b+c) - b
						 = a + b + c

As a consequence we have:

More Examples: Discrete

We Can see that every possible outcome has a probability of 1/16 Let P(x = 1) = 4 * 1/16 = 1/4 this the probability that the first roll is 1

Let Z = min(x,y) is a function that gives us the minimum of x and y P(z == 4) = 1/16 this is because if x==4 then y==4 vice versa.

Let P(Z==2) = 5 * 1/16 this is because we take all x, y that are x, y >= 2 which are 5

Discreet uniform law

More Examples: Continuous

![[Mathematics/Porbability/Pasted image 20220927145207.png]]

let’s take the example of the dart board from earlier

the Area of a subset is the probability of that area of that subset.

for the first probability P({(x, y) | x + y ⇐ 1/2})) we took the area under that triangle and got 1/8, this the probability of x equals or under half and y equals or under 1/2.

for the second probability we have P({(0.5, 0.3)}) it’s 0 because the area of a single point it zero.

Probability calculation steps

We have four steps

Specify the sample space: we first write down the sample space
Specify a probability law: Choose a probability that suits the phenomena that you are studying for example for the last example we chose the event in which the area of the subset is the probability of the event
Identify an event of interest We need to know the event to get the probability of
Calculate…

Example: Discrete but infinite sample space

An example of such an experiment is the number of coin tosses until we observe heads for the first time it’s properties are following:

Sample space: {1, 2, …} because it can take any arbitrary positive integer.
then we specify the probability law we want to use: P(n) = 1/2^n, n = 1, 2, 3,… This probability law works because if we add all of the probabilities we get 1 $\sum_{n = 1}^{\infty} = 1/2 * 1/ (1 - (1/2)) = 1$
for an event we will take all the event when the outcome is even P(outcome is even)
Let’s calculate $P (e v e n) = P (2, 4, 6, ...)$ $= 1/ 2^{2} + 1/ 2^{4} + 1/ 2^{6} + ... = 1/4 (1 + 1/4 + 1/ 4^{2} + ...) = 1/4 * 1/ (1 - 1/4) = 1/3$

in our calculations we introduce another axiom which is

Countable additivity axiom if $A_{1}, A_{2}, A_{3}, ...$ is an infinite sequence of disjoint events then $P (A_{1} \cup A_{2} \cup A_{3} \cup A_{n}) = P (A_{1}) + P (A_{2}) + P (A_{3}) + ...$

Notes:

PS: i just discovered you can mathjax in obsidian so i can output math by latex inside $...$ like so: $P (A) + P (A^{c}) + P (B) = P (A \cup A^{c} \cup B)$

PS: Don’t be afraid to calculate and get into the nitty gritty of math

🪐 Cryptolake

Explorer

Unit 1 Probability Models and Axioms