Back in 1996, in the chapter The Fine Art of Baloney Detection, Carl Sagan used this example to illustrate the post hoc ergo propter hoc fallacy: "Jaime Cardinal Sin, Archbishop of Manila [said]: 'I know of ... a 26-year-old who looks 60 because she takes [contraceptive] pills.'" [emphasis mine, not Sagan's] (Sagan, Carl. The Demon-Haunted World: Science as a Candle in the Dark. New York: Ballantine Books, 1996. p. 215)
Sin's use of "because" implies he was blaming the pill and citing it as the cause. That's fallacious. There's a host of possible causes in that woman's life that can account for the effect observed, and yet the cardinal picked this particular one (perhaps out of prejudice).
I'd also point out that Cardinal Sin committed the sin of selection bias. He cites one example that (erroneously) supports the implied claim that contraceptive pills are bad for women. However, just like psychics who never advertize those predictions that bombed, Sin fails to tell us about how many 26-year old women have taken the pill but don't' look 60 or 50 or 40 or 30. Also for the sake of completeness Sin would also need to tell us how many 26 yr olds who don't take the pill and look 60 or 50 or 40 or 30, and how many who don't take the pill and don't look any older than they really are.
Why would we need all those values? To fill in the 2x2 contingency table required for the computation of the correlational coefficient between the two variables (taking the pill and looking older than one really is). In order to find out if there is a correlation (positive or negative) or not, we need the values for all four cells of the table.
These two--post hockery and selection/confirmation bias--I might add are such common pitfalls in health claims (the Cardinal's assertion included). They're all over the Web. Quacks and manufacturers/dealers/multilevel marketers of health products use them to their advantage, and sadly most consumers see nothing wrong with them and in fact fall for and even naively employ such flawed reasoning.
All too often we read and hear testimonials proclaiming that John and Jane de la Cruz took Vitamin Z or LiverAIDS or 4Geek antioxidant or hairball tea or whatnot, and then felt better or lost weight or their cancer went away or their gray matter went away or whatever. That I took A and thereafter noticed B doesn't necessarily mean A caused B. To argue otherwise is textbook post hockery. And yet even doctors fall prey to this error. For instance there's one who enthusiastically presribed a CAM procedure to my mom because he's seen/heard positive anecdotes regarding that specific "alternative" treatment.
But as shown by the Cardinal's sin positive testimonials and stories aren't evidence at all, because:
1. they provide datum for only one of the cells in the contingency table. Where are the other three?
2. they don't tell us how much of the "I feel better" is due to the placebo effect
3. they don't tell us of the other treatments, lifestyle changes, and other possible causative factors that the person is simultaneously undertaking, confounding factors that screw up any pat and easy causal conclusion.
4. they don't give us a precise baseline--they don't provide the "before" picture--with which to compare the "after" nor a verification that the illness/condtion purported to exist is true and accurate.
If I declare that prayers work giving as proof the fact that the sun indeed rose after I wore ten rosaries each around my neck, arms and legs and prayed to god Xenu for the sun to rise, I would be guilty of the post. hoc fallacy as well as the sin of selection bias. If you want to show how freaking stupid and moronic I really am you simply have to ask for the data for the other three cells. After computing for the coefficient, not only will we discover a lack of causal link, we'd prove there isn't even a correlation at all.
We know the caveat "correlation does not necessarily imply causation." We also need to keep in mind that an apparent (but yet untested) correlation may not even be true--it might be illusory.
Sin's use of "because" implies he was blaming the pill and citing it as the cause. That's fallacious. There's a host of possible causes in that woman's life that can account for the effect observed, and yet the cardinal picked this particular one (perhaps out of prejudice).
I'd also point out that Cardinal Sin committed the sin of selection bias. He cites one example that (erroneously) supports the implied claim that contraceptive pills are bad for women. However, just like psychics who never advertize those predictions that bombed, Sin fails to tell us about how many 26-year old women have taken the pill but don't' look 60 or 50 or 40 or 30. Also for the sake of completeness Sin would also need to tell us how many 26 yr olds who don't take the pill and look 60 or 50 or 40 or 30, and how many who don't take the pill and don't look any older than they really are.
Why would we need all those values? To fill in the 2x2 contingency table required for the computation of the correlational coefficient between the two variables (taking the pill and looking older than one really is). In order to find out if there is a correlation (positive or negative) or not, we need the values for all four cells of the table.
These two--post hockery and selection/confirmation bias--I might add are such common pitfalls in health claims (the Cardinal's assertion included). They're all over the Web. Quacks and manufacturers/dealers/multilevel marketers of health products use them to their advantage, and sadly most consumers see nothing wrong with them and in fact fall for and even naively employ such flawed reasoning.
All too often we read and hear testimonials proclaiming that John and Jane de la Cruz took Vitamin Z or LiverAIDS or 4Geek antioxidant or hairball tea or whatnot, and then felt better or lost weight or their cancer went away or their gray matter went away or whatever. That I took A and thereafter noticed B doesn't necessarily mean A caused B. To argue otherwise is textbook post hockery. And yet even doctors fall prey to this error. For instance there's one who enthusiastically presribed a CAM procedure to my mom because he's seen/heard positive anecdotes regarding that specific "alternative" treatment.
But as shown by the Cardinal's sin positive testimonials and stories aren't evidence at all, because:
1. they provide datum for only one of the cells in the contingency table. Where are the other three?
2. they don't tell us how much of the "I feel better" is due to the placebo effect
3. they don't tell us of the other treatments, lifestyle changes, and other possible causative factors that the person is simultaneously undertaking, confounding factors that screw up any pat and easy causal conclusion.
4. they don't give us a precise baseline--they don't provide the "before" picture--with which to compare the "after" nor a verification that the illness/condtion purported to exist is true and accurate.
If I declare that prayers work giving as proof the fact that the sun indeed rose after I wore ten rosaries each around my neck, arms and legs and prayed to god Xenu for the sun to rise, I would be guilty of the post. hoc fallacy as well as the sin of selection bias. If you want to show how freaking stupid and moronic I really am you simply have to ask for the data for the other three cells. After computing for the coefficient, not only will we discover a lack of causal link, we'd prove there isn't even a correlation at all.
We know the caveat "correlation does not necessarily imply causation." We also need to keep in mind that an apparent (but yet untested) correlation may not even be true--it might be illusory.
