Skip to main content

Something Doesn't Add Up - Paul Goodwin ***

If there's one thing that's better than a juicy statistic, it's enjoying the process of pulling apart a dodgy one. It's why the radio programme More or Less is so excellent - so Paul Goodwin's book, subtitled 'surviving statistics in a post-truth world' was something I was really looking forward to - but for reasons I find it hard to put my finger on, it doesn't quite hit the spot.

Goodwin, a maths professor at the University of Bath, starts with a series of chapters telling us what's wrong with many of the statistics we see everyday. And he makes good points. We discover the dangers of rankings and trying to summarise a complex distinction in a single measure. We see why proxies are poor (essentially, if you can't actually measure what you want to, using something else that might be an appropriate indicator, but often isn't). We explore why polls are problematic. And there's a bit on Bayesian statistics and how it still tends to be disregarded by some, including the courts.

This is almost all negative, which is fine. Books like The Tiger that Isn't, one of my favourite titles on dodgy numbers and statistics take just such an approach. But they do so with lots of interesting stories and a plethora of examples. Although Goodwin does use some specifics, they feel more like case studies - they just don't engage in the way they should and there are too many generalities.

The other side of the book is we're promised a toolkit to help us cut through dodgy statistics. This is a good idea, but I'm really not sure how to use much of it in practice. For example, one instruction is 'If a questionnaire was used to obtain the number, was it biased?' With specifics such as checking whether, for example, it's based on leading questions, or questions which unrealistically limit people's response options. But I don't see how this can be used. This is supposed to be a toolkit to help ordinary people deal with statistics in the media (social and mainstream) - but how often does an article include details of the questionnaire used, or even the sample size? How are we supposed to answer these questions?

One last observation - the author proved at one point to be, perhaps surprisingly, honest. He tells us of an experiment he did showing students information on different tech products, asking which they would prefer, then repeating the exercise twice over four months, finding their choices were not set in stone, but changed. Goodwin points out limitations: that there may have been changes in technology over that period, news and reviews could have changed opinions, or as they weren't actually buying the technology, the students might not have cared much about the choice. 'But some of the inconsistency may have arisen simply because the respondents didn't really know what their true preferences were.' This is true, but equally it may not - in effect, he's telling us it wasn't a very useful study. (It would be interesting to ask, for example, why products that stay the same for decades and aren't likely to be reviewed, such as chocolate bars, weren't used, rather than tech?) Admitting this is quite brave.

I didn't dislike the book, and although it inevitably wheels out a lot of familiar examples, there were some new ones I hadn't come across before. But there was something about the presentation that just didn't do it for me. Even so, if, like me, you collect titles on dodgy statistics and how to deal with them, it's definitely one to add to the collection.

Using these links earns us commission at no cost to you
Review by Brian Clegg


Popular posts from this blog

Patricia Fara - Four Way Interview

Patricia Fara lectures in the history of science at Cambridge University, where she is a Fellow of Clare College. She was the President of the British Society for the History of Science (2016-18) and her prize-winning book, Science: A Four Thousand Year History (OUP, 2009), has been translated into nine languages. An experienced public lecturer, Patricia Fara appears regularly in TV documentaries and radio programmes. She also contributes articles and reviews to many popular magazines and journals, including History Today, BBC History, New Scientist, Nature and the Times Literary SupplementHer new book is Erasmus Darwin.

Why history of science?
I read physics at university, but half-way through the course I realised that had been a big mistake. Although I relished the intellectual challenge, I was bored by the long hours spent lining up recalcitrant instruments in dusty laboratories. Why was nobody encouraging us to think about the big questions – What is gravity? Does quantum mechani…

The Idea of the Brain: Matthew Cobb *****

Matthew Cobb is one of those people that you can’t help but admire but also secretly hate just a little bit for being so awesome. He is professor for zoology at the University of Manchester with a sizable teaching load that he apparently masters with consummate skill. He’s a scientific researcher, who researches the sense of smell of fruit fly maggots; I kid you not!  He’s also an attentive and loving family father but he still finds time and energy to write brilliant history of science books, three to date. His first, The Egg and Sperm Race, describes the search for the secret of human reproduction in the seventeenth and eighteenth centuries and is one of my favourite history of science books, on the period. His second, Life’s Greatest Secret is a monster, both in scope and detail, description of the hunt to decipher the structure and function of DNA that along the way demolishes a whole boatload of modern history of science myths. The most recent, and the subject of this review, is

The Search for Life on Mars - Elizabeth Howell and Nicholas Booth ***

From the book’s enticing subtitle, ‘The Greatest Scientific Detective Story of All Time’, I was expecting something rather different. I thought the authors would kick off by introducing the suspects (the various forms life might take on Mars, either now or in the past) and the kind of telltale traces they might leave, followed by a chronological account of the detectives (i.e. scientists) searching for those traces, ruling out certain suspects and focusing on others, turning up unexpected new clues, and so on. But the book is nothing like that. Continuing with the fiction analogy, this isn’t a novel so much as a collection of short stories – eleven self-contained chapters, each with its own set of protagonists, suspects and clues.

Some of the chapters work better than others. I found the first three – which despite their early placement cover NASA’s most recent Mars missions – the most irritating. For one thing, they unfold in a way that’s at odds with the cerebral ‘detective story’ na…