Goodwin, a maths professor at the University of Bath, starts with a series of chapters telling us what's wrong with many of the statistics we see everyday. And he makes good points. We discover the dangers of rankings and trying to summarise a complex distinction in a single measure. We see why proxies are poor (essentially, if you can't actually measure what you want to, using something else that might be an appropriate indicator, but often isn't). We explore why polls are problematic. And there's a bit on Bayesian statistics and how it still tends to be disregarded by some, including the courts.
This is almost all negative, which is fine. Books like The Tiger that Isn't, one of my favourite titles on dodgy numbers and statistics take just such an approach. But they do so with lots of interesting stories and a plethora of examples. Although Goodwin does use some specifics, they feel more like case studies - they just don't engage in the way they should and there are too many generalities.
The other side of the book is we're promised a toolkit to help us cut through dodgy statistics. This is a good idea, but I'm really not sure how to use much of it in practice. For example, one instruction is 'If a questionnaire was used to obtain the number, was it biased?' With specifics such as checking whether, for example, it's based on leading questions, or questions which unrealistically limit people's response options. But I don't see how this can be used. This is supposed to be a toolkit to help ordinary people deal with statistics in the media (social and mainstream) - but how often does an article include details of the questionnaire used, or even the sample size? How are we supposed to answer these questions?
One last observation - the author proved at one point to be, perhaps surprisingly, honest. He tells us of an experiment he did showing students information on different tech products, asking which they would prefer, then repeating the exercise twice over four months, finding their choices were not set in stone, but changed. Goodwin points out limitations: that there may have been changes in technology over that period, news and reviews could have changed opinions, or as they weren't actually buying the technology, the students might not have cared much about the choice. 'But some of the inconsistency may have arisen simply because the respondents didn't really know what their true preferences were.' This is true, but equally it may not - in effect, he's telling us it wasn't a very useful study. (It would be interesting to ask, for example, why products that stay the same for decades and aren't likely to be reviewed, such as chocolate bars, weren't used, rather than tech?) Admitting this is quite brave.
I didn't dislike the book, and although it inevitably wheels out a lot of familiar examples, there were some new ones I hadn't come across before. But there was something about the presentation that just didn't do it for me. Even so, if, like me, you collect titles on dodgy statistics and how to deal with them, it's definitely one to add to the collection.
Hardback:
Kindle:
Using these links earns us commission at no cost to you
Review by Brian Clegg
Comments
Post a Comment