Thoughts on statistics

After reading some Good Math Bad Math (which should of course be Good Maths Bad Maths) on statistics, and while doing quite a bit of stats in my current research, I decided to try and write up a few interesting thoughts I have had on the subject. As ever my thoughts are a little disorganised so I will probably come back and edit this a few times...

It seems to me that statistics is all about the partitioning of sets of outcomes based on some property they have.For example take the tossing of a coin. By saying that a coin has a 50/50 chance of coming up heads or tails, what we are saying is that there are a large number of outcomes to the action of flipping a coin (with each "outcome" here meaning a possible trajectory for the coin) which we can partition in to two sets, one set having the property that the coin will end up heads up and the other that it will end up tails up.

So the Frequentist / Bayesian argument then becomes one of whether you approach this situation in a top down or a bottom up manner.

The Frequentist then is bottom up says that the partition between sets does actually exist and therefore we can determine it's position by exploring the space (in this case by tossing the coin repeatedly).

The Bayesian on the other hand says that the partition only exists because we lack the information to fully describe the path of the coin and so in the absence of further knowledge the best we can do is say that all paths are equal and partition the space in two.

Fundamentally the two are the same. Maybe I can make myself clearer by looking at the biassed coin question. The Frequentist will say that the chance of the coin coming up heads is not 50/50 and the bayesian will say that it is. The problem is not in the model however, the problem is that they are talking about different experiments!

The Frequentist with his bottom up partitioning approach starts with the biassed coin, looks at the possible paths and looks where the partition should be drawn. The only thing he can say is that the partition is not equal. Repeated tosses of the coin would show this to be true.

The Bayesian on the other hand has a top down approach and so he looks first at the possible number of biassed coins he could get and then at the possible number of biasses. He sees the total number of outcomes (coin bias + coin trajectory) as being equal that end in heads and that end in tails.

So if a Frequentist considers the experiment to be "You are given a biassed coin and toss it" rather than "you toss this particular biassed coin" he too comes to the same conclusion as the Bayesian. Whereas if the Bayesian were given a specific biassed coin and asked to come up with a statistical model of how it come up heads or tails, he would toss it a load of times, determine the bias and then put that in to his model as a prior

"Ah" you say. "That's all well and good, so long as you consider a totally mechanistic view of the universe where true randomness does not exist. Once you allow for true randomness the two views deviate again."

Well, that's only true if you allow for true randomness. And in some future post I will show that we have no reason to believe in true randomness at all.