Why statistics?
Because I want to believe true things, and I want to achieve my goals.
There are a lot of questions we can answer without statistics, and goals we can achieve without statistics – if I want to know whether the shop has milk I can go and check, and if I want to cook my children dinner I can just do it. But there are also lots of situations where, if we want to do a good job, we need statistics, and to use them carefully.
That’s true both at a policy level and at a personal one. Should the government spend billions of pounds on cancer screening? We can’t answer that without asking questions about the specificity and sensitivity of the screening, about the prevalence of the cancer, and about the impact of early diagnosis on outcomes. Should I worry about drinking a second beer on a Friday evening? We can’t answer that without asking questions about the correlations between alcohol intake and health outcomes, and whether those correlations are causal.
These questions are subtle, and using statistics to answer them is hard, and you can easily be led astray. But, as the saying goes, while it is easy to lie with statistics, it is even easier to lie without them.
Why this book?
I wish I had a noble reason, but I started writing it out of sheer spite. In 2021 my second book was published, and it had a chapter on Bayes. I rewrote that chapter into a feature for the Observer, and an editor put in the subheading the phrase “an obscure theorem”. People went absolutely crazy and I had a three-day Twitterstorm of professors of biostatistics and stuff yelling at me, saying “how can this be obscure, I, a professor of biostatistics, use it all the time,” and so on.
So in a fit of pique I immediately went and pitched a book about Bayes to my publishers.
But I do think it’s the most important one-line equation in the world. It describes all of decision theory; it describes how we decide what is true or not in science (even if not all scientists want to use it); it dissolves vast philosophical conundrums; it contains within it all of propositional logic, all the “all men are mortal, Socrates is a man, ergo Socrates is mortal” stuff; and it describes how our brains work! For something made entirely out of mathematical operations my eight-year-old daughter could do easily, that’s pretty impressive.
Is there any realistic possibility that scientific studies will move away from frequentist statistics?
Not entirely, and probably that’s fine.You don’t need to use Bayes if you’re looking for the Higgs boson with the Large Hadron Collider — you have so much data that any prior would be washed out anyway. And sometimes finding a prior in a principled way might be tricky.
But the world is increasingly Bayesian — the Pfizer COVID vaccine trials were run in a very Bayesian way, for instance. And the growth of probabilistic programming languages makes Bayesian reasoning easier and easier for scientists to do. So I think Bayesianism will continue to make headway, because it feels less bolted together.
What’s next?
My friend Stuart and I run a podcast, called The Studies Show, in which we look at controversial or interesting scientific topics and try to work out the evidence behind them. We hope relatively soon to turn that into a radio documentary. And there is an idea for another book. Plus there’s the day job — Semafor’s Flagship newsletter, a vital daily roundup of the most important news in the world, which I thoroughly commend to the readers.
What’s exciting you at the moment?
Tough question! I hate to be predictable (as it were) but I am absolutely blown away by the speed of progress in AI, and I find people who profess to cynicism about it (“It’s just a stochastic parrot” or “we were told it would get better quickly but it’s been six months and it’s still not only passing the bar exam at the 50th percentile”) absolutely baffling. My first book was about AI. I wrote it in 2017 and it was published in 2019. That’s really not very long ago but the stuff we see now is just unrecognisable from what I was writing about then. It was still a big deal that AI image classifiers could tell the difference between a cat and a dog!
I could understand being scared of AI — it’s a huge deal and could change the world in frightening, even dangerous ways (the topic of that book, The Rationalist’s Guide to the Galaxy, was whether it will kill us all; I concluded that that’s not a crazy thing to worry about). But the idea of being unimpressed or unexcited is just bizarre to me.
Interview by Brian Clegg - See all of Brian's online articles or subscribe to a digest free here
Comments
Post a Comment