LS: Speaking of Science Column December 08, 2008
Scientific method exposes snake oil
Scientists are trained to think about things in a different way than most people do. Their way of thinking is influenced by something called the “scientific method.”
The scientific method is attributed to a scientist named Ibn al-Haytham (965–1039) who was born in Basra, Persia, (now Iraq) and educated in that city and in Baghdad.
The steps of the scientific method may vary somewhat depending on who describes it, but they usually go something like this:
• Propose a hypothesis.
• Design and conduct an experiment capable of testing the hypothesis.
• Collect and evaluate data from the experiment.
• Accept or reject the hypothesis based on results of the experiment.
This process is then repeated until patterns emerge upon which scientific laws are based. It sounds easy, but designing an experiment that is actually capable of testing the hypothesis is often more difficult than it sounds. Why?
For one thing, if the process being evaluated is poorly understood, how do you know if an experiment will be effective in testing the hypothesis? (Of course, if you knew in advance what was going to happen, you wouldn’t need to perform the test, would you?) The better your understanding, the more likely you are to design effective experiments.
But even if the experiments disprove your hypothesis, that information still helps put you on the right track.
First guesses are not always right, and learning to change your thinking after a setback is one of the great benefits of using the scientific method.
In addition, it may also be difficult to decipher and evaluate the data obtained from an experiment, because complicated mathematics or subtle measurements may be involved.
Experiments often involve measurements at the limits of what can be detected, so that even if
the hypothesis is right, the experiment may not be sensitive enough to show it conclusively.
The process being tested may also be complex, involving numerous interacting factors that may confound the results (the issue of man’s contribution to global warming falls into this category). The use of statistics may help untangle the results and lead to the right answer, but the mathematics involved can themselves be complicated and subject to error or
And if developing a well-designed experiment wasn’t difficult enough, a researcher may introduce bias into the experiment design or the interpretation of results.
This is often unintentional and happens when you pay attention to some results more than others. But there is the occasional “bad apple” whose motivation to advance a personal point of view outweighs the willingness to perform a fair evaluation of the data.
Fortunately, good science also employs a method to reduce the effect of bias — publishing the results — which allows others to reproduce an experiment on their own to see if they get the same answers.
This allows the hypothesis to be evaluated by a larger group than just the original experimenter and helps to ensure that the conclusion is honest and accurate.
When the results can be reproduced by an impartial audience, it helps ensure that scientific knowledge is based on factual information rather than wishful thinking, a flawed hypothesis or somebody’s personal agenda.
So the next time you hear of a “miracle” diet or drug, or hear that some industry is harming the public or the environment, or see a reality television show with teams carrying technical-looking equipment chasing ghosts or UFOs, try thinking like a scientist. Is there enough evidence, confirmed by impartial analysis and results that can be duplicated, for you to believe what you are being told or is someone trying to sell you snake oil?
If we all start to think more like scientists, we can make the most of our efforts to improve our health, environment and way of living, instead of wasting resources on chasing imaginary culprits or falling for miracle cures that don’t exist.
Vincent King is a certified health physicist who has been involved in radiological sciences for over 30 years. He is a volunteer at the Western Colorado Math & Science Center.