Having grown weary of constantly having the same discussion, I am posting this to clearly articulate the two potential mathematical definitions of “impossible” in the context of probability and to present the most accessible explanation I can think of of why I feel that the word impossible is misused in undergrad probability texts (most graduate texts simply don’t use the word at all).

I am not looking to start an(other) argument; I’m simply posting the definitions and my reasoning so I can just link to it in the future when this inevitably comes up. I am aware of the fact that much of what I am about to say flies in the face of most introductory probability textbooks; judge what I say with appropriate skepticism.

Very little knowledge of measure theory is needed in what follows; an undergrad probability course and some point-set topology should be all that’s required.

**The Fundamental Premise**

Fundamental Premise of Probability: The mathematical field of Probability Theory is the study of random variables, particularly sequences of them, and probability theory is concerned solely with the distribution of said variables.

I submit that almost every probabilist would agree with the above. Theorems such as the Strong Law of Large Numbers and the Central Limit Theorem would seem to be adequate justification.

**Definitions**

I will deliberately work in the naive concrete setup as probability is usually first presented. Specifically, I will use the setup of most introductory textbooks where probability spaces are point spaces and random variables are pointwise defined functions (using parentheticals to indicate how we understand them in the purely measurable setup).

A (topological model of a) **probability space** is a topological space K, a sigma-algebra — usually the Borel or Lebesgue sets — of subsets of K and a measure Prob with Prob(K) = 1. Elements of the sigma-algebra are called **events**.

A (representative of a) **random variable** is a function X : K –> R which is measurable: the preimage of every measurable subset of R is in the sigma-algebra of K. Throughout, R denotes the real numbers.

Two random variables X and Y are **independent** when for every x,y in R, Prob(x >= X and y >= Y) = Prob(x >= X) Prob(y >= Y).

Two variables X and Y are **identically distributed** when for every x in R, Prob(x >= X) = Prob(x >= Y).

A sequence of random variables X_n is **iid** when the variables are independent and identically distributed.

A **null set** or **null event** is any element N of the sigma-algebra with Prob(N) = 0. The empty set is a null set.

The **support** of the measure Prob is the smallest closed subset K_0 of K such that Prob(K_0) = 1. Equivalently, K_0 is the intersection of all the closed sets L in K with Prob(L) = 1. Any subset of the complement of the support is a null set. The support will be written supp(Prob).

If you are unfamiliar with topology, just think of K as being the real numbers and K_0 being the smallest closed interval where the probability measure “lives”. So, for example, if the probability is supposed to represent picking a random number between 0 and 1 then K_0 is [0,1].

**The Question**

The question is what should be referred to as an *impossible* event?

The at first glance “obvious” answer is that any event outside the support of Prob should be deemed impossible (an indisputable statement) and that any event inside the support should be deemed possible. For example, if we pick a number uniformly at random from [0,1] then this is the claim that it is impossible we picked 2 (indisputable) but possible we picked specifically 1. I shall refer to this as topological impossibility: an event E is **topologically impossible** when E intersect supp(Prob) is empty and correspondingly an event F is **topologically possible** when F intersect supp(Prob) is nonempty.

The alternative answer is that any event with probability zero should be deemed impossible. I shall refer to this as measurable impossibility: an event E is **measurably impossible** when Prob(E) = 0, i.e. when E is a null set, and an event F is **measurably possible** when Prob(F) > 0. This is a more subtle notion than topological impossibility.

It is immediate that every topologically impossible event is measurably impossible and that any measurably possible event is topologically possible (since positive measure sets are nonempty), so our discussion should focus entirely sets which are measurably impossible yet topologically possible.

**The Math**

Since sets in the complement of supp(Prob) are impossible in both senses, we will from here on assume that supp(Prob) = K. This is not an issue, we may simply replace K by K_0. Having made this modification, the only topologically impossible set is now the empty set.

Let N be a nonempty null set, aka N is topologically possible but measurably impossible. Consider the random variable X : K –> R which is the characteristic function of N: X(k) = 1 for k in N and X(k) = 0 otherwise; and the random variable Z : K –> R given by Z(k) = 0, i.e. Z is the constant zero function.

For x >= 0, the set of points { k : x >= X(k) } contains the complement of N because X(k) = 0 for k not in N. So Prob(x >= X) >= 1 – Prob(N) = 1 – 0 = 1 for x >= 0. For x < 0, { x >= X } is the empty set so Prob(x >= X) = 0 for x < 0. Likewise, Prob(x >= Z) = 1 for x >= 0 and Prob(x >= Z) = 0 for x < 0. Thus X and Z are identically distributed.

For x,z >= 0, Prob(x >= X and z >= Z) = 1 = Prob(x >= X) Prob(z >= Z). For x,z in R with at least one less than zero, Prob(x >= X and z >= Z) = 0 = Prob(x >= X) Prob(z >= Z). So X and Z are independent. Note that Prob(x >= X and z >= X) behaves the same way so that in fact X is independent from itself (something about that should bother you; we will address it later).

The fundamental premise says that probability is concerned only with the distribution of a random variable: a random variable identically distributed to the zero distribution should always take on the value zero. That is, if we repeatedly sample from the constantly zero distribution, we only ever get zeroes.

Here is the kicker: if our event N is “possible” then it must follow that it is “possible” for X to equal 1; this violates our premise.

On the other hand, if we say that “possible” should mean measurably possible then indeed we get what we expect: it is impossible to get a 1 by sampling from the zero distribution.

**The First Potential Objection**

The most obvious objection to what I just wrote is that it’s some sort of trickery and that the X_n are not actually an iid sequence of zero functions. But this is not the case, I proved they were identically distributed to zero.

A more reasonable objection would be that perhaps identically distributed is not defined properly and we should demand more, perhaps such as that the functions be pointwise equal. Equivalently, the objection would be that my Fundamental Premise is faulty.

The problem with that is that two of the most fundamental theorems of probability — the Strong Law of Large Numbers and the Central Limit Theorem — apply only to iid sequences with that definition, not a stronger one. This is the basis of the Fundamental Premise.

If we use topological possibility then we are stuck saying that a sequence of trials of the zero event could possibly yield a 1 as an outcome. This violates our fundamental premise, so the notion of topological impossibility is the wrong one; measurable impossibility is the only notion which makes sense in the context of probability theory.

A far more interesting objection would be that even though probability theory cannot distinguish topologically possible null sets from topologically impossible events, we should still “keep the model around” since it contains information relevant to what we are modeling. This objection is best addressed after some further mathematics (and will be).

**Measure Algebras, aka the Abstract Setup**

We want to consider the space of all random variables but we want to identify two variables which are identically distributed. The good news is that being identically distributed is an equivalence relation. So we can quotient out by it and consider equivalence classes of functions which are id to one another. Our X and Z above are now *the same*, as well they should be. The “space of random variables” then should not be the collection of all measurable functions on K but should instead be the collection of all equivalence classes of them (we should not be able to distinguish X from Z).

What have we done at the level of the space though? We have declared that a null set is *equivalent* to the empty set. More generally, we have declared that any set E is *equivalent* to any other set F where Prob(E symmetric difference F) = 0. The collection of equivalence classes of our sigma-algebra is what should properly be thought of as the “space of events” but we can no longer think of this algebra as being subsets of some space K. Instead, we are forced to consider just this measure algebra and the measure. There is no underlying space anymore since we can no longer speak of “points”: any set consisting of a single point has been declared equivalent to the empty set.

In fact, the correct definition of event is not that it is a measurable set but instead: an **event** is an equivalence class of measurable sets modulo null sets. The collection of all events is the **measure algebra**. Writing [] to denote equivalence classes, we can now define the **impossible event** [emptyset] = { null sets } which is unique precisely because our probability space has no way of distinguishing null events (note the parallel to what happened in the naive setup: we restricted to the support of the measure and there was a unique topologically impossible event, the empty set).

This explains the parentheticals: a topological space with a sigma-algebra is a model for a probability space when the sigma-algebra mod the ideal of null sets is the measure algebra of the probability space. A representative of a random variable is a pointwise defined function on the model which is in the equivalence class that is the random variable.

For those who know category theory this should be easy to summarize: the category of probability spaces is not concrete as there is no natural map from it to Set. See this link for a category theory approach to this type of idea.

**Functions as Vectors (but not quite)**

It turns out this same idea of quotienting out by null sets arises for a completely different (well, imo not really different but at first glance seems to be different) reason.

Anyone who’s taken linear algebra knows that the “magic” is the dot product. So it’s natural to ask whether or not we can come up with some sort of dot product for functions and make them into a nice inner product space (we can add functions and multiply them by scalars so they are already a vector space).

In the context of a measure space (M,Sigma,mu), there is an obvious candidate for the inner product and norm: we’d like to say that <f,g> = Int f(x) g(x) dmu(x) and ||f|| = sqrt(Int |f(x)|^{2} dmu(x)). If we then look at the set of functions { f : ||f|| < infty }, we should have a nice inner product space.

But not quite. The problem is that if f is the characteristic function of a null set then for every g we would get <f,g> = 0 and ||f|| = 0. If you remember the definition of an inner product space, we need that to only happen if f is the zero function. Seems like we’re stuck, but…

Quotienting to the rescue: say that f ~ g when they are equal **almost everywhere**: when { m : f(m) ≠ g(m) } is a null set. Then define **L ^{2}(M,Sigma,mu)** to be the space of

*equivalence classes*of functions with ||f|| < infty. We will write [f] for the equivalence class of a function f. Now we have an inner product (and a norm) and since there is only one element [f] of L

^{2}with ||f|| = 0, namely the equivalence class of the zero function. Without quotienting out by null sets, we have none of that structure. L

^{2}is the canonical example of an

**infinite-dimensional Hilbert space**: a vector space with an inner product that is complete with respect to the norm (completeness meaning that if ||[f_n] – [f_m]|| –> 0 then [f_n] –> [f] for some [f] in L

^{2}).

More generally, we can define ||f||_p = (Int |f(x)|^{p} dmu(x))^{1/p} and ask about the functions with ||f||_p < infty. This is also a vector space but it suffers the same issue: ||f||_p = 0 for functions that are characteristic of null sets. Quotienting: **L ^{p}(M,Sigma,mu)** is the set of equivalence classes of functions with ||f||_p < infty. This makes ||f||_p a norm and so we have a

**Banach space**(complete normed vector space). If you’ve seen any functional analysis, you know that Banach spaces are where all the theorems are proved; so in essence to even begin bringing functional analysis into the game, we

*have to*quotient out by the null sets.

In analysis textbooks, it is common to “perform the standard abuse of notation and simply write f to mean [f]”. This is perfectly fine as long as one is aware of it, but the conflation of f and [f] is exactly what leads to the mistaken idea that empty is somehow different than null: the null event [null] = the impossible event [emptyset].

**The Usual Counterargument**

The most common argument in favor of topological impossibility is that null events happen in the real world all the time so they are necessarily possible.

The usual setup for this discussion is throwing a dart at an interval; the claim then is that after the dart is thrown it must have landed somewhere and so the set consisting of just that point, a null set, must somehow have been possible. Alternatively, one can invoke sequences of coin flips and argue that it is possible to flip a coin infinitely many times and get all heads.

The claim usually boils down to the idea that, based on some sort of “real-world intuition”, there is a natural topological space which models the scenario and therefore we should work in that specific topological model of our probability space and, in particular, think of “possible” as meaning topologically possible. For the case of throwing a dart, this model is usually taken to be [0,1].

My first objection to this is that we’ve already seen that it is irrelevant in probability whether or not a particular null set is empty; the mathematics naturally leads us to the conclusion of measure algebras. So this counterargument becomes the claim that a probability space alone does not fully model our scenario. That’s fine, but from a purely mathematical perspective, if you’re defining something and then never using it, you’re just wasting your time.

My second, and more substantive, objection is that this appeal to reality is misinformed. I very much want my mathematics to model reality as accurately and completely as it can so if keeping the particular model around made sense, I would do so. The problems is that in actual reality, there is no such thing as an ideal dart which hits a single point nor is it possible to ever actually flip a coin an infinite number of times. Measuring a real number to infinite precision is the same as flipping a coin an infinite number of times; they do not make sense in physical reality.

The usual response would be that physics still models reality using real numbers: we represent the position of an object on a line by a real number. The problem is that this is simply false. Physics does not do that and hasn’t in over a hundred years. Because it doesn’t actually work. The experiments that led to quantum mechanics demonstrate that modeling reality as a set of distinguishable points is simply wrong.

Quantum mechanics explicitly describes objects using wavefunctions. Wavefunction is a fancy way of saying element of Hilbert space: a wavefunction is an equivalence class of functions modulo null sets. So if the appeal is going to be to how physics models reality then the answer is simple: according to our best theory of physics, a measurably impossible event simply cannot happen.

Whether or not one accepts quantum mechanics, thinking of physical reality as being made up of distinguishable points is a convenient fiction but an ultimately misleading one. Same goes for probability spaces: topological models are a useful fiction but one needs to avoid mistaking the fiction for reality.

**So Why Does “Everyone” Define Probability Spaces as Sets of Points Then?**

Simple answer: because in our current mathematics, it is far easier to describe sets of distinguishable points than it is to talk about measure algebras. Working in a material set theory, objects like measure algebras and L^{2} require far more work to define and far more care to work with.

Undergraduate textbooks prefer to avoid the complications and simply define topological models of probability spaces and work only with those. I have no objection to that. The problem comes when they tell the “white lie” that properties of the specific model are relevant, for instance when they define impossible using the topology.

More complex answer: despite the name, probability theory is *not* the study of probability spaces; it is the study of (sequences of) random variables. Up to isomorphism, there is a unique nonatomic standard Borel probability space so probabilists almost never actually talk about the space. The study of probability spaces is really a part of ergodic theory, functional analysis, and operator algebras.

**When Topological Models Are Important**

Before concluding, I should point out that there are certainly times when it does make sense to work with a specific topological model: specifically and only when you are trying to prove something *about that topological space*.

When proving that almost every real number is normal, of course we need to keep the topological space in mind since we are trying to prove things about it. The mistake would be to turn around and try to define what it means for an “element of a probability space” to be normal when this only makes sense for that particular model.

Of course, this leaves open the possibility of claiming that when we say “throw a dart at a line””, what we mean is look the topological space [0,1] with the Lebesgue measure. My answer would be that that is not even wrong.

**Conclusion**

My view is that it doesn’t even make sense to speak of which specific point a dart lands on; the only meaningful questions are whether or not it landed in some positive measure region (the probability of this happening, of course, is the probability of the region).

This may sound counterintuitive, but it’s actually far more intuitive than the alternative: the measure algebra view says that we can never measure something to infinite precision, we can only measure it up to some error, which is exactly what real-world experience suggests.

The mathematics and the physics both lead us to measure algebras. This is a very good thing: the mathematics models reality as closely as possible. Anyone who has studied physics knows that at some point, you give up on the intuition and have to just trust the math. Because the results match up with experiment.

Counterintuitive as it may seem, trust the math: there are no points in a probability space and null events never happen.

submitted by Nevin Manimala Nevin Manimala /u/sleeps_with_crazy

[link] [comments]