This entry (apart from a few remarks) will be mainly about general statistics and mathematical statistics. For more about physical applications in statistical mechanics and probabilistic interpretation of quantum mechanics, see there. There is also a technical notion of statistic (singular).

Idea

Statistics studies the analysis of collections of random or sample data, and the probabilistic likelihood of various inferences on the basis of these data, as well as the mathematical regularities in large ensembles of occurrences of such data.

In physics, statistics also pertains to the behaviour of large ensembles of particles. For identical particles, this is the subject of particle statistics and for general systems the subject of statistical mechanics.

Mathematical statistics is based on probability theory. Most of the standard formalism uses measure theory as used in probability. Statistical mechanics in addition heavily uses ergodic theory.

Peter McCullagh, What is a statistical model?, Ann. Statist. 30:5 (2002), 1225-1310 euclidMR1936320doi – on applying category theory to describe statistical models.

Lior Pachter, Bernd Sturmfels, Tropical geometry of statistical models, PNAS 101 no. 46, 16132–16137, doi