h2g2 The Hitchhiker's Guide to the Galaxy: Earth Edition

Here's the fiver

Well I'm sure you've got it covered, but when I learned Experimental Design and Statistics, ooh, mumblemumble years ago, and it came to Type I and Type II Errors and Statistical Power, the example I remember concerned cancer and cocoa.

See...when people do psychology or medical experiments, they often lazily default to a 'standard' measure of significance. Obviously if you get results at significance p=0.01 that's better than p=0.1, right? Except that that's not what significance means (and Tim Harford has been talking a lot abut this on R4's 'More or Less'.)

Cancer and Cocoa. Say you have two treatments for cancer. One is a nasty drug with horrible side effects that might even kill you. The other is cocoa. Even if the nasty drug seems to work very well...you'd want to do make sure you achieved a high level of significance. Unfortunately you need a lot of subjects to be able to say that. Cocoa...tests on a handful of people suggest that it might to something, but at a low level of significance. So what? It's only cocoa.

I enjoyed Stats. The lecturer used to tell stories about how his granddad carried a piano on his back from one pub to another down Wakefield high street. 'He won the bet but it crippled the daft bugger'.

Multivariate correlation was ice cream, chips and beer sales.