Skip to main content

Fun with the Reverend Bayes

A recent review of Bayes' Rule by James V. Stone for review, has reminded me of the delightful case of the mathematician's coloured balls. (Mathematicians often have cases of coloured balls. Don't ask me why.)

This is a thought experiment that helps illustrate why we have problems dealing with uncertainty and probability.

Imagine I've got a jar with 50 white balls and 50 black balls in it. I take out a ball but don't look at it. What's the chance that this ball is black?

I hope you said 50% or 50:50 or 1/2 or 0.5 - all ways of saying that it has equal chances of being either white or black. With no further information that's the only sensible assumption.

Now keep that ball to one side, still not looking at it. You pull out another ball and you do look at this one. (Mathematicians know how to have a good time.) It's white.

Now what's the chance that the first ball was black?

You might be very sensibly drawn to suggest that it's still 50:50. After all, how could the probability change just because I took another ball out afterwards? But the branch of probability and statistics known as Bayesian tells us that probabilities are not set in stone or absolute - they are only as good as the information we have, and gaining extra information can change the probability.

Initially you had no information about the balls other than that there were 50 of each colour in the pot. Now, however, you also know that a ball drawn from the remainder was white. If that first ball had been black, you would be slightly more likely to draw a white ball next time. So drawing a white makes it's slightly more likely that the first ball was black than it was white - you've got extra information. Not a lot of information, it's true. Yet it does shift the probability, even though the information comes in after the first ball was drawn.

If you find that hard to believe, imagine taking the example to the extreme. I've got a similar pot with just two balls in, one black, one white. I draw one out but don't look at it. What's the chance that this ball is black? Again it's 50%. Now lets take another ball out of the pot and look at. It's white. Do you still think that looking at another ball doesn't change the chances of the other ball being black? If so let's place a bet - because I now know that the other ball is definitely black.

So even though it appears that there's a 0.5 chance of the ball being black initially, what is really the case is that 0.5 is our best bet given the information we had. It's not an absolute fact, it's our best guess given what we know. In reality the ball was either definitely white or definitely black, not it some quantum indeterminate state. But we didn't know which it was, so that 0.5 gave us a best guess.

One final example to show how information can change apparently fixed probabilities.

We'll go back to the first example to show another way that information can change probability. Again I've got a pot, then with 50 black and 50 white balls. I draw one out. What's the probability it's black? You very reasonably say 50%.  So far this is exactly the same situation as the first time round.

I, however, have extra information. I now share that information with you - and you change your mind and say that the probability is 100% black, even though nothing has changed about the actual pot or ball drawn. Why? Because I have told you that all the balls at the bottom of the pot are white and all the balls at the top are black. My extra information changes the probabilities.

Comments

Popular posts from this blog

The AI Delusion - Gary Smith *****

This is a very important little book ('little' isn't derogatory - it's just quite short and in a small format) - it gets to the heart of the problem with applying artificial intelligence techniques to large amounts of data and thinking that somehow this will result in wisdom.

Gary Smith as an economics professor who teaches statistics, understands numbers and, despite being a self-confessed computer addict, is well aware of the limitations of computer algorithms and big data. What he makes clear here is that we forget at our peril that computers do not understand the data that they process, and as a result are very susceptible to GIGO - garbage in, garbage out. Yet we are increasingly dependent on computer-made decisions coming out of black box algorithms which mine vast quantities of data to find correlations and use these to make predictions. What's wrong with this? We don't know how the algorithms are making their predictions - and the algorithms don't kn…

Infinity in the Palm of your Hand - Marcus Chown *****

A new Marcus Chown book is always a treat - and this is like a box of chocolates: a collection of bite-sized delights as Chown presents us with 50 science facts that are strange and wonderful.

The title is a quote from William Blake's Auguries of Innocence: 'To see a World in a Grain of Sand, / And a Heaven in a Wild Flower, / Hold Infinity in the palm of your hand, / And Eternity in an hour.' It would seem particularly appropriate if this book were read on a mobile phone (so it would be literally in the palm), which could well be true for ebook users, as the short essays make excellent reading for a commute, or at bedtime. I found them distinctly moreish - making it difficult to put the book down as I read just one more. And perhaps another. Oh, and that next one looks really interesting...

Each of the 50 pieces has a title and a short introductory heading, which mostly give a feel for the topic. The very first of these, however, briefly baffled me: 'You are a third mus…

How to Invent Everything - Ryan North ****

Occasionally you read a book and think 'I wish I'd thought of that.' This was my immediate reaction to Ryan North's How to Invent Everything. The central conceit manages to be both funny and inspiring as a framework for writing an 'everything you ever wanted to know about everything (and particularly science)' book.

What How to Invent Everything claims to be is a manual for users of a time machine (from some point in the future). Specifically it's a manual for dealing with the situation of the time machine going wrong and stranding the user in the past. At first it appears that it's going to tell you how to fix the broken time machine - but then admits this is impossible. Since you're stuck in the past, you might as well make the best of your surroundings, so the aim of the rest of the book is to give you the knowledge you need to build your own civilisation from scratch.

We start with a fun flow chart for working out just how far back in time you are…