Skip to main content

How to Expect the Unexpected - Kit Yates ****

The topic here is one everyone is interested in - getting a better handle on the future, and it's an interesting read. Arguably Kit Yates' title is a touch misleading. This isn't a 'how to' book - after reading it, you won't be any better at doing anything, but you may be less likely to make some popular errors.

My background is in Operational Research, which includes a lot on forecasting and mathematical prediction, so I was slightly disappointed that this isn't really covered here. Instead it gives us mostly ways that we instinctively get predictions wrong, so it's arguably more a psychology book that a mathematical one. There have been quite a few others that tread the path of uncovering our biases, for example with a mathematical approach in Jordan Ellenberg's How Not to be Wrong and with a more psychological twist in Richard Nesbitt's Mindware. But Yates has a particular focus on our tendency to assume linearity - that things will broadly continue the way they always have. By bringing in plenty of examples where this isn't the case - it's very often true in reality - including chaotic systems, he gives us a fresh viewpoint.

For me, the best chapter was 'reading between the lines', where Yates focuses most directly on non-linearity and really unpacks what's happening in some real world examples. And there were plenty of others with interesting examples and observations in other chapters - but I did have a few issues.

Occasionally Yates makes a statement that is hard to back up. Some of this, as is often the case with academics dipping a toe into popular science, was on historical matters - we are told 'It was will into the Middle Ages before the spherical view of the world became the predominant theory.' This just isn't true. I think he is also wrong about the millennium bug, calling it a self-defeating impact from predictions. The idea is that because of all the effort that was put in, there were few big problems, so people thought it was overhyped. I was consulting for the IT department of a global company at the time, and the reality was far more nuanced - the analysis was that it genuinely was overhyped, in that far too much was spent on checking non-critical systems that can have failed relatively painlessly, where a more effective approach would have been only to check mission- or safety-critical systems and leave the rest to fail and be fixed if necessary. 

On other occasions, Yates provides a lack of explanation. For example, he introduces Benford's law, without telling us why it occurs. Some of the material was a little dull - I was particularly disappointed with the chapter on game theory, which failed to capture the intriguing nature of the subject and didn't explain enough for the reader to get their head around what was going on. Bearing in mind a lot of the book is based on psychological research, I was really surprised there was no mention of the replication crisis (surely in itself demonstrating a glaring lack of ability to predict the future) - I would be surprised if some of the studies he cites haven't failed to be capable of reproduction, or weren't based on far too small a sample to be meaningful. At the very least, this should be discussed in a book based on such studies.

The linearity bias isn't the only one that Yates covers, though most of the ones mentioned tie into it. As is always the case with books like this, it proved very interesting to read about, but I very rapidly forgot what all the biases are (again), and found it difficult to think of practical applications of what I've read. It's fine if you are a business or government wanting to deal with uncertainty (though even there, the book isn't a practical guide), but I think it's very unlikely to make much difference to the way we go about making predictions about the future in our everyday lives, beyond 'don't bother'.

Overall, this is an interesting topic and Yates presents a novel approach and does a good job of getting the reader to appreciate the dangers of relying on linearity. The book does have a few issues, but is still well worth a read.

Hardback:   
Kindle 
Using these links earns us commission at no cost to you
Review by Brian Clegg - See all Brian's online articles or subscribe to a weekly email free here

Comments

Popular posts from this blog

Rakhat-Bi Abdyssagin Five Way Interview

Rakhat-Bi Abdyssagin (born in 1999) is a distinguished composer, concert pianist, music theorist and researcher. Three of his piano CDs have been released in Germany. He started his undergraduate degree at the age of 13 in Kazakhstan, and having completed three musical doctorates in prominent Italian music institutions at the age of 20, he has mastered advanced composition techniques. In 2024 he completed a PhD in music at the University of St Andrews / Royal Conservatoire of Scotland (researching timbre-texture co-ordinate in avant- garde music), and was awarded The Silver Medal of The Worshipful Company of Musicians, London. He has held visiting affiliations at the Universities of Oxford, Cambridge and UCL, and has been lecturing and giving talks internationally since the age of 13. His latest book is Quantum Mechanics and Avant Garde Music . What links quantum physics and avant-garde music? The entire book is devoted to this question. To put it briefly, there are many different link...

Should we question science?

I was surprised recently by something Simon Singh put on X about Sabine Hossenfelder. I have huge admiration for Simon, but I also have a lot of respect for Sabine. She has written two excellent books and has been helpful to me with a number of physics queries - she also had a really interesting blog, and has now become particularly successful with her science videos. This is where I'm afraid she lost me as audience, as I find video a very unsatisfactory medium to take in information - but I know it has mass appeal. This meant I was concerned by Simon's tweet (or whatever we are supposed to call posts on X) saying 'The Problem With Sabine Hossenfelder: if you are a fan of SH... then this is worth watching.' He was referencing a video from 'Professor Dave Explains' - I'm not familiar with Professor Dave (aka Dave Farina, who apparently isn't a professor, which is perhaps a bit unfortunate for someone calling out fakes), but his videos are popular and he...

Everything is Predictable - Tom Chivers *****

There's a stereotype of computer users: Mac users are creative and cool, while PC users are businesslike and unimaginative. Less well-known is that the world of statistics has an equivalent division. Bayesians are the Mac users of the stats world, where frequentists are the PC people. This book sets out to show why Bayesians are not just cool, but also mostly right. Tom Chivers does an excellent job of giving us some historical background, then dives into two key aspects of the use of statistics. These are in science, where the standard approach is frequentist and Bayes only creeps into a few specific applications, such as the accuracy of medical tests, and in decision theory where Bayes is dominant. If this all sounds very dry and unexciting, it's quite the reverse. I admit, I love probability and statistics, and I am something of a closet Bayesian*), but Chivers' light and entertaining style means that what could have been the mathematical equivalent of debating angels on...