Probability axioms

Template:Probability fundamentals The standard probability axioms are the foundations of probability theory introduced by Russian mathematician Andrey Kolmogorov in 1933.^[1] Like all axiomatic systems, they outline the basic assumptions underlying the application of probability to fields such as pure mathematics and the physical sciences, while avoiding logical paradoxes.^[2]

The probability axioms do not specify or assume any particular interpretation of probability, but may be motivated by starting from a philosophical definition of probability and arguing that the axioms are satisfied by this definition. For example,

Cox's theorem derives the laws of probability based on a "logical" definition of probability as the likelihood or credibility of arbitrary logical propositions.^[3]^[4]
The Dutch book arguments show that rational agents must make bets which are in proportion with a subjective measure of the probability of events.

The third axiom, σ-additivity, is relatively modern, and originates with Lebesgue's measure theory. Some authors replace this with the strictly weaker axiom of finite additivity, which is sufficient to deal with some applications.^[5]

Kolmogorov axioms

In order to state the Kolmogorov axioms, the following pieces of data must be specified:

The sample space, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\textstyle \Omega} , which is the set of all possible outcomes or elementary events.
The space of all events, which are each taken to be sets of outcomes (i.e. subsets of ${\textstyle \Omega }$ ). The event space, ${\textstyle F}$ , must be a $σ$ -algebra on ${\textstyle \Omega }$ .
The probability measure ${\textstyle P}$ which assigns to each event $E\in F$ its probability, $P(E)$ .

Taken together, these assumptions mean that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (\Omega, F, P)} is a measure space. It is additionally assumed that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\Omega)=1} , making this triple a probability space.^[1]

First axiom

The probability of an event is a non-negative real number. This assumption is implied by the fact that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P} is a measure on Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle F} .

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(E)\geq 0 \qquad \forall E \in F}

Theories which assign negative probability relax the first axiom.

Second axiom

This is the assumption of unit measure: that the probability that one of the elementary events in the entire sample space will occur is 1.Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\Omega) = 1} From this axiom it follows that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(E)} is always finite, in contrast with more general measure theory.

Third axiom

This is the assumption of σ-additivity: Any countable sequence of disjoint sets (synonymous with mutually exclusive events) Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E_1, E_2, \ldots} satisfies

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P\left(\bigcup_{i = 1}^\infty E_i\right) = \sum_{i=1}^\infty P(E_i).}

This property again is implied by the fact that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P} is a measure. Note that, by taking Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E_1 = \Omega} and Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E_i = \emptyset} for all Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle i>1} , one deduces that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\emptyset) = 0} . This in turn shows that σ-additivity implies finite additivity.

Some authors consider merely finitely additive probability spaces, in which case one just needs an algebra of sets, rather than a σ-algebra.^[6] Quasiprobability distributions in general relax the third axiom.

Elementary consequences

In order to demonstrate that the theory generated by the Kolmogorov axioms corresponds with classical probability, some elementary consequences are typically derived.^[7]

Since Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P} is finitely additive, we have Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(A) + P(A^c) = P(A\cup A^c)= P(\Omega) = 1} , so Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(A^c) = 1-P(A)} .
In particular, it follows that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\emptyset) = 0} . The empty set is interpreted as the event that "no outcome occurs", which is impossible.
Similarly, if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A \subseteq B} , then Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(B) = P(A \cup (B\setminus A)) = P(A) + P(B\setminus A) \ge P(A)} . In other words, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P} is monotone.^[8]
Since Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \emptyset \subseteq E \subseteq \Omega} for any event Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E} , it follows that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 0 \le P(E) \le 1} .

By dividing Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A \cup B } into the disjoint sets Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A \setminus (A \cap B) } , Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle B \setminus (A \cap B)} and Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A \cap B} , one arrives at a probabilistic version of the inclusion-exclusion principle^[9]Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(A \cup B) = P(A) + P(B) - P(A \cap B).} In the case where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Omega} is finite, the two identities are equivalent.

In order to actually do calculations when Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Omega} is an infinite set, it is sometimes useful to generalize from a finite sample space. For example, if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Omega} consists of all infinite sequences of tosses of a fair coin, it is not obvious how to compute the probability of any particular set of sequences (i.e. an event). If the event is "every flip is heads", then it is intuitive that the probability can be computed as:Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\text{infinite sequence of heads}) = \lim_{n \to \infty} P(\text{sequence of n heads}) = \lim_{n \to \infty} 2^{-n} = 0.} In order to make this rigorous, one has to prove that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P} is continuous, in the following sense. If Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_j,\,\, j = 1, 2, \ldots} is a sequence of events increasing (or decreasing) to another event Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A} , then^[10]Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \lim_{n \to \infty} P(A_n) = P(A).}

Simple example: Coin toss

Consider a single coin-toss, and assume that the coin will either land heads (H) or tails (T) (but not both). No assumption is made as to whether the coin is fair.^[11]

We may define:

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Omega = \{H,T\}}

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle F = \{\varnothing, \{H\}, \{T\}, \{H,T\}\}}

Kolmogorov's axioms imply that:

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\varnothing) = 0}

The probability of neither heads nor tails, is 0.

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\{H,T\}^c) = 0}

The probability of either heads or tails, is 1.

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\{H\}) + P(\{T\}) = 1}

The sum of the probability of heads and the probability of tails, is 1.

References

↑ ^1.0 ^1.1 Kolmogorov, Andrey (1950) [1933]. Foundations of the theory of probability. New York, US: Chelsea Publishing Company.
↑ Aldous, David. "What is the significance of the Kolmogorov axioms?". David Aldous. Retrieved November 19, 2019.
↑ Cox, R. T. (1946). "Probability, Frequency and Reasonable Expectation". American Journal of Physics. 14 (1): 1–10. Bibcode:1946AmJPh..14....1C. doi:10.1119/1.1990764.
↑ Cox, R. T. (1961). The Algebra of Probable Inference. Baltimore, MD: Johns Hopkins University Press.
↑ Bingham, N.H. (2010). "Finite Additivity Versus Countable Additivity: de Finetti and Savage" (PDF). Electronic J. History of Probability and Statistics. 6 (1): 1–6.
↑ Hájek, Alan (August 28, 2019). "Interpretations of Probability". Stanford Encyclopedia of Philosophy. Retrieved November 17, 2019.
↑ Gerard, David (December 9, 2017). "Proofs from axioms" (PDF). Retrieved November 20, 2019.
↑ Ross, Sheldon M. (2014). A first course in probability (Ninth ed.). Upper Saddle River, New Jersey. pp. 27, 28. ISBN 978-0-321-79477-2. OCLC 827003384.
↑ Jackson, Bill (2010). "Probability (Lecture Notes - Week 3)" (PDF). School of Mathematics, Queen Mary University of London. Retrieved November 20, 2019.
↑ Evans, Michael; Rosenthal, Jeffrey (25 July 2003). Probability and Statistics: The Science of Uncertainty. W. H. Freeman and Company. pp. 27–29. ISBN 978-0716747420.
↑ Diaconis, Persi; Holmes, Susan; Montgomery, Richard (2007). "Dynamical Bias in the Coin Toss" (PDF). SIAM Review. 49 (211–235): 211–235. Bibcode:2007SIAMR..49..211D. doi:10.1137/S0036144504446436. Retrieved 5 January 2024.

Probability axioms

Contents

Kolmogorov axioms

First axiom

Second axiom

Third axiom

Elementary consequences

Simple example: Coin toss

See also

References

Further reading

Navigation menu

Probability axioms

Kolmogorov axioms

First axiom

Second axiom

Third axiom

Elementary consequences

Simple example: Coin toss

See also

References

Further reading

Navigation menu

Search