Seasickness Remedy
A parable in the scientific method
Preamble
To test a seasickness remedy you 'volunteer' a batch of
students to embark in a small boat on a wild sea.
Controls
It is not good enough to give them the pill. You don't know how
many would have been seasick, or not, without the pill. You gotta
have controls, subjects who don't get the pill:
Randomize
Don't cheat by giving the pill to subjects who look like good
seamen, and using as controls subjects who look dismayed at the
mere idea of going out in a boat. Avoid this, and other biases,
including ones that didn't occur to you, by randomly dividing
subjects into pill and control.
Blind
Even controls don't suffice. Seasickness is partly
psychological. If you really believe the seasickness pill works...
It works. Blind the experiment. Make up a dummy (called
placebo) which looks like the real thing. Randomly
decide who gets which. And don't tell them. Afterward compare
the treatment and control groups.
Double Blind
At what point do you judge a victim seasick?
Look uncomfortable?
Turn green?
Go to the rail?
Go to the rail and perform?
Someone has to make the call, and it shouldn't be a person who
expects to make a fortune marketing this pill. Not only shall
the subject not know which pill (blind), whoever calls the
shots shall not know either (double blind). Lock the
document telling which subject got which pill in a safe till the
experiment is over.
Probability, Etc.
A cynic about patent medicine doesn't believe the pill is any
better than peppermint candy. Who got seasick and who didn't was
predetermined when you randomly assigned pills to
subjects. Suppose out of 8 subjects, 4 are prone to seasickness,
4 are not, and the four weaklings chanced to be the ones who got
the placebo. One time in 70, the four weaklings get the placebo.
One time in 70 you will be fooled.
-
Conclusion
If indeed all four controls get seasick but none of the four
treated subjects get seasick, you are in this fork:
- EITHER
- The pill really helped
- OR
- Something unlikely to the extent of 1/70 happened.
- That's the evidence: Take your pick.
|