probability theory



Probability theory is concerned with mathematical models of phenomena that exhibit randomness , or more generally phenomena about which one has incomplete information.

Its central mathematical model is based mostly on measure theory. So from a pure mathematical viewpoint probability theory today could be characterized as the study of measurable spaces with a finite volume normalized to 11.

Broader perspectives may stress the relevance of other pure mathematical concepts for probability theory, or include aspects of the interpretation of mathematical results to phenomenology, the latter part making naturally contact with the field of statistics.

Notice that in this respect probability theory has a similar status as (other(?!)) theories of physics: there is a mathematical model (measure theory here as the model for probability theory, or for instance symplectic geometry as a model for classical mechanics) which can be studied all in itself, and then there is in addition a more or less concrete idea of how from that model one may deduce statements about the observable world (the average outcome of a dice role using probability theory, or the observability of the next solar eclipse using Hamiltonian mechanics). The step from the mathematical model to its use as a tool for making statements about the observable world is subtle, maybe a subject of philosophy, but in any case outside of the realm of mathematics. In probability theory the meaning of this step is traditionally a cause of debate, with two antagonistic main schools of thought being the frequentist interpretation and the Bayesian perspective on the nature of the relation of probability theory to the observable world.


Basic theory

Random variables are defined typically in terms of probability spaces, cf. the basic entries on measure space, probability space, conditional probability. The modern point of view emphasises that many facts about random variables do not depend much on the choice of the probability spaces; the random variables are also often identified with their distributions.

Some argue that in the study of measure and probability, one should start not only with sigma algebra of measurable sets but also another of null sets. Somehow this is abstractly captured by the approach of commutative von Neumann algebras.

Stochastic processes


ergodic process


Statistical Manifolds

Families of probability distributions often form statistical models, that is, submanifolds of the space of all probability measures on a sample space. Techniques from differential geometry may be applied in a theory known as information geometry.

Probability theory from the nPOV

We describe here some perspectives on (parts of) probability theory from the categorical point of view (see nPOV). This perspective mainly applies to the study of situations involving Markov kernels and Chapman-Kolmogorov property.

Prakash Panangaden in Probabilistic Relations defines the category SRelSRel (stochastic relations) to have as objects sets equipped with a σ\sigma-field. Morphisms are conditional probability densities or stochastic kernels. So, a morphism from (X,Σ X)( X, \Sigma_X) to (Y,Σ Y)( Y, \Sigma_Y) is a function h:X×Σ Y[0,1]h: X \times \Sigma_Y \to [0, 1] such that

  1. BΣ Y.λxX.h(x,B)\forall B \in \Sigma_Y . \lambda x \in X . h(x, B) is a bounded measurable function,
  2. xX.λBΣ Y.h(x,B)\forall x \in X . \lambda B \in \Sigma_Y . h(x, B) is a subprobability measure on Σ Y\Sigma_Y.

If kk is a morphism from YY to ZZ, then khk \cdot h from XX to ZZ is defined as (kh)(x,C)= Yk(y,C)h(x,dy)(k \cdot h)(x, C) = \int_Y k(y, C)h(x, d y).

This is based on earlier work by Michele Giry, see Giry's monad.

  • Michèle Giry, A categorical approach to probability theory Categorical aspects of topology and analysis (Ottawa, Ont., 1980), pp. 68–85, Lecture Notes in Math., 915, Springer.

Panangaden’s definition differs from Giry’s in the second clause where subprobability measures, rather than ordinary probability measures, are allowed.

Panangaden emphasises that the mechanism is similar to the way that the category of relations can be constructed from the power set functor. Just as the category of relations is the Kleisli category of the powerset functor over the category of sets Set, SRelSRel is the Kleisli category of the functor over the category of measurable spaces and measurable functions which sends a measurable space, XX, to the measurable space of subprobability measures on XX. This functor gives rise to a monad.

What is gained by the move from probability measures to subprobability measures? One motivation seems to be to model probabilistic processes from XX to a coproduct X+YX + Y. This you can iterate to form a process which looks to see where in YY you eventually end up. This relates to SRelSRel being traced.

There is a monad on MeasureSpacesMeasureSpaces, 1+:MeasMeas1 + -: Meas \to Meas. A probability measure on 1+X1 + X is a subprobability measure on XX. Panangaden’s monad is a composite of Giry’s and 1+1 + -.

The opposite of the Kleisli category of Giry's monad has as morphisms XYX \to Y, linear maps from bounded functions on XX to bounded functions on YY, which send the characteristic function on XX to the characteristic function on YY.

For more details on Giry’s monad and its variants see Giry's monad.




The modern formalization of probability theory in measure theory originates around

  • Andrey Kolmogorov, Grundbegriffe der Wahrscheinlichkeitsrechnung, Ergebnisse der Mathematik und Ihrer Grenzgebiete, Springer Berlin Heidelberg, 1933

Lecture notes include

  • Alexander Grigoryan, Measure theory and probability, 2008 pdf

  • Terence Tao, A review of probabiltiy theory, 2010 (web)

just as the natural numbers can be defined abstractly without reference to any numeral system (e.g. by the Peano axioms), core concepts of probability theory, such as random variables, can also be defined abstractly, without explicit mention of a measure space; we will return to this point when we discuss free probability later in this course.

For references related to Giry's monad and variants see there.

For a setting of probability theory within category theory see

For a more convenient setting for ‘higher-order’ probability theory, that is, one which admits higher-order functions, the following article uses the cartesian closed category of quasi-Borel spaces rather than the category of measurable spaces:

For big picture in probability theory see answers to

An instance of a “categorical thinking” (in a generalized sense) in solving probability problems is a solution to Buffon’s noodle problem (wikipedia) discussed by Tom Leinster at nCafe here.

  • Klain, Gian-Carlo Rota, Introduction to geometric probability

  • John C. Baez, Jacob D. Biamonte, A course on quantum techniques for stochastic mechanics, pdf

Discussion from a perspective of formal logic/type theory is in

  • Neil Toronto, Useful Languages for Probabilistic Modeling and Inference, PhD Thesis, 2014 (pdf, slides)

Mikhail Gromov on possible generalizations/modifications of probability theory (especially probability theory seen as, fundamentally, a “”functor“ from a ”complex category“ to a ”simple category“”), as well as applications of probability within and without pure mathematics:

  • Probability, symmetry, linearity. (six lectures). IHES, Nov 2014. (videos) (pdf).

category: probability

Revised on April 17, 2017 14:50:04 by Toby Bartels (