Skip to main content

Something Doesn't Add Up - Paul Goodwin ***

If there's one thing that's better than a juicy statistic, it's enjoying the process of pulling apart a dodgy one. It's why the radio programme More or Less is so excellent - so Paul Goodwin's book, subtitled 'surviving statistics in a post-truth world' was something I was really looking forward to - but for reasons I find it hard to put my finger on, it doesn't quite hit the spot.

 Goodwin, a maths professor at the University of Bath, starts with a series of chapters telling us what's wrong with many of the statistics we see everyday. And he makes good points. We discover the dangers of rankings and trying to summarise a complex distinction in a single measure. We see why proxies are poor (essentially, if you can't actually measure what you want to, using something else that might be an appropriate indicator, but often isn't). We explore why polls are problematic. And there's a bit on Bayesian statistics and how it still tends to be disregarded by some, including the courts.

 This is almost all negative, which is fine. Books like The Tiger that Isn't, one of my favourite titles on dodgy numbers and statistics take just such an approach. But they do so with lots of interesting stories and a plethora of examples. Although Goodwin does use some specifics, they feel more like case studies - they just don't engage in the way they should and there are too many generalities.

 The other side of the book is we're promised a toolkit to help us cut through dodgy statistics. This is a good idea, but I'm really not sure how to use much of it in practice. For example, one instruction is 'If a questionnaire was used to obtain the number, was it biased?' With specifics such as checking whether, for example, it's based on leading questions, or questions which unrealistically limit people's response options. But I don't see how this can be used. This is supposed to be a toolkit to help ordinary people deal with statistics in the media (social and mainstream) - but how often does an article include details of the questionnaire used, or even the sample size? How are we supposed to answer these questions?

 One last observation - the author proved at one point to be, perhaps surprisingly, honest. He tells us of an experiment he did showing students information on different tech products, asking which they would prefer, then repeating the exercise twice over four months, finding their choices were not set in stone, but changed. Goodwin points out limitations: that there may have been changes in technology over that period, news and reviews could have changed opinions, or as they weren't actually buying the technology, the students might not have cared much about the choice. 'But some of the inconsistency may have arisen simply because the respondents didn't really know what their true preferences were.' This is true, but equally it may not - in effect, he's telling us it wasn't a very useful study. (It would be interesting to ask, for example, why products that stay the same for decades and aren't likely to be reviewed, such as chocolate bars, weren't used, rather than tech?) Admitting this is quite brave.

 I didn't dislike the book, and although it inevitably wheels out a lot of familiar examples, there were some new ones I hadn't come across before. But there was something about the presentation that just didn't do it for me. Even so, if, like me, you collect titles on dodgy statistics and how to deal with them, it's definitely one to add to the collection.

Hardback:   
Kindle:  
Using these links earns us commission at no cost to you
Review by Brian Clegg
Labels:

Comments

Post a Comment

Popular posts from this blog

The Laws of Thought - Tom Griffiths *****

In giving us a history of attempts to explain our thinking abilities, Tom Griffiths demonstrates an excellent ability to pitch information just right for the informed general reader.  We begin with Aristotelian logic and the way Boole and others transformed it into a kind of arithmetic before a first introduction of computing and theories of language. Griffiths covers a surprising amount of ground - we don't just get, for instance, the obvious figures of Turing, von Neumann and Shannon, but the interaction between the computing pioneers and those concerned with trying to understand the way we think - for example in the work of Jerome Bruner, of whom I confess I'd never heard.  This would prove to be the case with a whole host of people who have made interesting contributions to the understanding of human thought processes. Sometimes their theories were contradictory - this isn't an easy field to successfully observe - but always they were interesting. But for me, at least, ...
Post a Comment
Read more

The AI Paradox - Virginia Dignum ****

This is a really important book in the way that Virginia Dignum highlights various ways we can misunderstand AI and its abilities using a series of paradoxes. However, I need to say up front that I'm giving it four stars for the ideas: unfortunately the writing is not great. It reads more like a government report than anything vaguely readable - it really should have co-authored with a professional writer to make it accessible. Even so, I'm recommending it: like some government reports it's significant enough to make it necessary to wade through the bureaucrat speak. Why paradoxes? Dignum identifies two ways we can think about paradoxes (oddly I wrote about paradoxes recently , but with three definitions): a logical paradox such as 'this statement is false', or a paradoxical truth such as 'less is more' - the second of which seems a better to fit to the use here.  We are then presented with eight paradoxes, each of which gives some insights into aspects of t...
Post a Comment
Read more

Einstein's Fridge - Paul Sen ****

In Einstein's Fridge (interesting factoid: this is at least the third popular science book to be named after Einstein's not particularly exciting refrigerator), Paul Sen has taken on a scary challenge. As Jim Al-Khalili made clear in his excellent The World According to Physics , our physical understanding of reality rests on three pillars: relativity, quantum theory and thermodynamics. But there is no doubt that the third of these, the topic of Sen's book, is a hard sell. While it's true that these are the three pillars of physics, from the point of view of making interesting popular science, the first two might be considered pillars of gold and platinum, while the third is a pillar of salt. Relativity and quantum theory are very much of the twentieth century. They are exciting and sometimes downright weird and wonderful. Thermodynamics, by contrast, has a very Victorian feel and, well, is uninspiring. Luckily, though, thermodynamics is important enough, lying behind ...
Post a Comment
Read more