Skip to main content

Something Doesn't Add Up - Paul Goodwin ***

If there's one thing that's better than a juicy statistic, it's enjoying the process of pulling apart a dodgy one. It's why the radio programme More or Less is so excellent - so Paul Goodwin's book, subtitled 'surviving statistics in a post-truth world' was something I was really looking forward to - but for reasons I find it hard to put my finger on, it doesn't quite hit the spot.

Goodwin, a maths professor at the University of Bath, starts with a series of chapters telling us what's wrong with many of the statistics we see everyday. And he makes good points. We discover the dangers of rankings and trying to summarise a complex distinction in a single measure. We see why proxies are poor (essentially, if you can't actually measure what you want to, using something else that might be an appropriate indicator, but often isn't). We explore why polls are problematic. And there's a bit on Bayesian statistics and how it still tends to be disregarded by some, including the courts.

This is almost all negative, which is fine. Books like The Tiger that Isn't, one of my favourite titles on dodgy numbers and statistics take just such an approach. But they do so with lots of interesting stories and a plethora of examples. Although Goodwin does use some specifics, they feel more like case studies - they just don't engage in the way they should and there are too many generalities.

The other side of the book is we're promised a toolkit to help us cut through dodgy statistics. This is a good idea, but I'm really not sure how to use much of it in practice. For example, one instruction is 'If a questionnaire was used to obtain the number, was it biased?' With specifics such as checking whether, for example, it's based on leading questions, or questions which unrealistically limit people's response options. But I don't see how this can be used. This is supposed to be a toolkit to help ordinary people deal with statistics in the media (social and mainstream) - but how often does an article include details of the questionnaire used, or even the sample size? How are we supposed to answer these questions?

One last observation - the author proved at one point to be, perhaps surprisingly, honest. He tells us of an experiment he did showing students information on different tech products, asking which they would prefer, then repeating the exercise twice over four months, finding their choices were not set in stone, but changed. Goodwin points out limitations: that there may have been changes in technology over that period, news and reviews could have changed opinions, or as they weren't actually buying the technology, the students might not have cared much about the choice. 'But some of the inconsistency may have arisen simply because the respondents didn't really know what their true preferences were.' This is true, but equally it may not - in effect, he's telling us it wasn't a very useful study. (It would be interesting to ask, for example, why products that stay the same for decades and aren't likely to be reviewed, such as chocolate bars, weren't used, rather than tech?) Admitting this is quite brave.

I didn't dislike the book, and although it inevitably wheels out a lot of familiar examples, there were some new ones I hadn't come across before. But there was something about the presentation that just didn't do it for me. Even so, if, like me, you collect titles on dodgy statistics and how to deal with them, it's definitely one to add to the collection.

Using these links earns us commission at no cost to you
Review by Brian Clegg


Popular posts from this blog

On the Fringe - Michael Gordin *****

This little book is a pleasant surprise. That word 'little', by the way, is not intended as an insult, but a compliment. Kudos to OUP for realising that a book doesn't have to be three inches thick to be interesting. It's just 101 pages before you get to the notes - and that's plenty. The topic is fringe science or pseudoscience: it could be heavy going in a condensed form, but in fact Michael Gordin keeps the tone light and readable. In some ways, the most interesting bit is when Gordin plunges into just what pseudoscience actually is. As he points out, there are elements of subjectivity to this. For example, some would say that string theory is pseudoscience, even though many real scientists have dedicated their careers to it. Gordin also points out that, outside of denial (more on this a moment), many supporters of what most of us label pseudoscience do use the scientific method and see themselves as doing actual science. Gordin breaks pseudoscience down into a n

A (Very) Short History of Life on Earth - Henry Gee *****

In writing this book, Henry Gee had a lot to live up to. His earlier title  The Accidental Species was a superbly readable and fascinating description of the evolutionary process leading to Homo sapiens . It seemed hard to beat - but he has succeeded with what is inevitably going to be described as a tour-de-force. As is promised on the cover, we are taken through nearly 4.6 billion years of life on Earth (actually rather more, as I'll cover below). It's a mark of Gee's skill that what could have ended up feeling like an interminable list of different organisms comes across instead as something of a pager turner. This is helped by the structuring - within those promised twelve chapters everything is divided up into handy bite-sized chunks. And although there certainly are very many species mentioned as we pass through the years, rather than feeling overwhelming, Gee's friendly prose and careful timing made the approach come across as natural and organic.  There was a w

Michael D. Gordin - Four Way Interview

Michael D. Gordin is a historian of modern science and a professor at Princeton University, with particular interests in the physical sciences and in science in Russia and the Soviet Union. He is the author of six books, ranging from the periodic table to early nuclear weapons to the history of scientific languages. His most recent book is On the Fringe: Where Science Meets Pseudoscience (Oxford University Press). Why history of science? The history of science grabbed me long before I knew that there were actual historians of science out there. I entered college committed to becoming a physicist, drawn in by the deep intellectual puzzles of entropy, quantum theory, and relativity. When I started taking courses, I came to understand that what really interested me about those puzzles were not so much their solutions — still replete with paradoxes — but rather the rich debates and even the dead-ends that scientists had taken to trying to resolve them. At first, I thought this fell under