Hypotheses are tentative explanations of scientific data that describe pinciples thought to be operating in nature. When these hypotheses are seen to prove true for every measurement and observation they are often developed into theories. For example: ‘Mendel's genetic theory apparently concerns all inherited characteristics in all plants and animals’[1] even though only a few of these have been observed—a general conclusion inferred from particular instances, an induction. This means that our observations do not conclusively prove our theory, or strong hypothesis, true in all cases everywhere; there is always the possibility that some data could appear to prove our theory false.

More often than not our theories fit our facts only where they touch.

Sometimes we can close the gap between fact and theory by intuition that takes us direct to the underlying principles at work in natural philosophy, determined by synthesis rather than analysis of the nature of causes and effects upon the subject. Our problem remains how to select the correct cause of an effect from an infinite number of possible causes; any choice we make is fraught with uncertainty, for we have no proof, only a gut-feeling.

Theories and the experimental data gathered in their support are often put to statistical test in an effort to determine significance of the results. We read and hear of scientific experiments that produce results found to be statistically significant, the imprimatur of successful science.

Well, yes and no. To be statistically significant requires quite a few educated guesses on the part of the researchers. Data has been rejected as unreasonable. Some scientific approaches have been discarded as unlikely to produce meaningful results. And then there is much quibbling in statistical circles about the impartial rigour of the statistical tools used to bludgeon the data. As Mark Twain famously declared: There are lies, damn lies, and statistics.

Statistics is an attempt to measure the probability of an event in an uncertain world. Reality is a question of probability, probability governed by an infinite number of events from the very large to the very small. Human beings love certainty, yet our only certainty in life is our eventual death.

We are actually better equipped to cope with uncertainty, as long as we take a keen interest in what is happening in the world around us. When well-informed, we can make good decisions after weighing the pros and cons of different courses of action in light of historical evidence. We calculate our actions upon the probable outcome with no guarantee of success, just a degree of probability. It is highly probable that we will be older tomorrow, but there is also a corresponding very small probability that we will be younger—it is just highly improbable that any of us will be younger tomorrow. This is an extreme example. Nothing is certain, perhaps not even death.

Science too works in probability, particularly if you are a follower of Bayes' Theorem. ‘According to the Bayesian view, scientific and indeed much of everyday reasoning is conducted in probabilistic terms. In other words, when evaluating an uncertain claim, one does so by calculating the probability of the claim in the light of given information’[1]. A good practical example of this technique can be found in the spambayes spam filter, which works remarkably well by applying Bayesian analysis of new email against historical receipts maintained in two corpora: one for spam, the other for ham. Spambayes is one of several Bayesian anti-spam utilities, this one is available free from the SourceForge.

References

1. Scientific Reasoning: The Bayesian Approach, by Colin Howson & Peter Urbach (1989, Open Court)