Wednesday, December 15, 2010

Sins to be wary of

Back in 1996, in the chapter The Fine Art of Baloney Detection, Carl Sagan used this example to illustrate the post hoc ergo propter hoc fallacy: "Jaime Cardinal Sin, Archbishop of Manila [said]: 'I know of ... a 26-year-old who looks 60 because she takes [contraceptive] pills.'" [emphasis mine, not Sagan's] (Sagan, Carl. The Demon-Haunted World: Science as a Candle in the Dark. New York: Ballantine Books, 1996. p. 215)

Sin's use of "because" implies he was blaming the pill and citing it as the cause. That's fallacious. There's a host of possible causes in that woman's life that can account for the effect observed, and yet the cardinal picked this particular one (perhaps out of prejudice).

I'd also point out that Cardinal Sin committed the sin of selection bias. He cites one example that (erroneously) supports the implied claim that contraceptive pills are bad for women. However, just like psychics who never advertize those predictions that bombed, Sin fails to tell us about how many 26-year old women have taken the pill but don't' look 60 or 50 or 40 or 30. Also for the sake of completeness Sin would also need to tell us how many 26 yr olds who don't take the pill and look 60 or 50 or 40 or 30, and how many who don't take the pill and don't look any older than they really are.

Why would we need all those values? To fill in the 2x2 contingency table required for the computation of the correlational coefficient between the two variables (taking the pill and looking older than one really is). In order to find out if there is a correlation (positive or negative) or not, we need the values for all four cells of the table.

These two--post hockery and selection/confirmation bias--I might add are such common pitfalls in health claims (the Cardinal's assertion included). They're all over the Web. Quacks and manufacturers/dealers/multilevel marketers of health products use them to their advantage, and sadly most consumers see nothing wrong with them and in fact fall for and even naively employ such flawed reasoning.

All too often we read and hear testimonials proclaiming that John and Jane de la Cruz took Vitamin Z or LiverAIDS or 4Geek antioxidant or hairball tea or whatnot, and then felt better or lost weight or their cancer went away or their gray matter went away or whatever. That I took A and thereafter noticed B doesn't necessarily mean A caused B. To argue otherwise is textbook post hockery. And yet even doctors fall prey to this error. For instance there's one who enthusiastically presribed a CAM procedure to my mom because he's seen/heard positive anecdotes regarding that specific "alternative" treatment.

But as shown by the Cardinal's sin positive testimonials and stories aren't evidence at all, because:
1. they provide datum for only one of the cells in the contingency table. Where are the other three?
2. they don't tell us how much of the "I feel better" is due to the placebo effect
3. they don't tell us of the other treatments, lifestyle changes, and other possible causative factors that the person is simultaneously undertaking, confounding factors that screw up any pat and easy causal conclusion.
4. they don't give us a precise baseline--they don't provide the "before" picture--with which to compare the "after" nor a verification that the illness/condtion purported to exist is true and accurate.

If I declare that prayers work giving as proof the fact that the sun indeed rose after I wore ten rosaries each around my neck, arms and legs and prayed to god Xenu for the sun to rise, I would be guilty of the post. hoc fallacy as well as the sin of selection bias. If you want to show how freaking stupid and moronic I really am you simply have to ask for the data for the other three cells. After computing for the coefficient, not only will we discover a lack of causal link, we'd prove there isn't even a correlation at all.

We know the caveat "correlation does not necessarily imply causation." We also need to keep in mind that an apparent (but yet untested) correlation may not even be true--it might be illusory.

Tuesday, December 14, 2010

If I drink and drive I'll meet an accident. I had a mishap therefore I'd been DUI.

 According to a news article which just got posted on Richard Dawkins' website, a study shows that "people with higher IQs are less likely to believe in God." Apparently the creme de la creme among scientists are not only geniuses but have chucked the belief in talking snakes and cops in the sky: "A survey of Royal Society fellows found that only 3.3 per cent believed in God - at a time when 68.5 per cent of the general UK population described themselves as believers."

Well, let's assume that's true--that highly intelligent blokes are more likely to be nonbelievers. As is known to all here I don't subscribe to any postulated supernatural entity whether you call them gods, goddesses, deities, angels, higher powers and principalities, your heavenly juju or holy mojo or Elvis for that matter. On the other hand, I have no idea what my IQ is. Since I'm a true-blue, dyed-in-the-wool, in-your-face atheist, does it follow then that I am more likely to have above average IQ? Can I conclude that I'm most likely an Einstein? Let's find out.

First, for the sake of clarity let's give the news article's claim some concreteness. So let's say that it's been determined that those with IQ >100 have an 80% chance of being atheistic. Now let's try and encode that in the nomenclature of probability theory since we are dealing with likelihoods.


AA = above average intelligence
NB = non-belief
P() = probability of

The above proposition then can be expressed as:

If a person has AA then P(NB) = 80%   [let's call this P1]

We now move on to the question we're trying to answer. Does it follow that if I'm a nonbeliever there is an 80% of oh so humble moi having above average intelligence? In other words, is the following statement true:

If a person is an atheist then P(AA) = 80%   [P2]

Well, you probably guessed it's not.

P1 can be written succinctly as P(NB|AA), which is read as the "probability of nonbelief given the person has above average intelligence." The vertical bar means "given."

P2 on other hand can be written as P(AA|NB), the probability of having AA given NB.

P1 and P2 are known as conditional probabilities and the values for these two are not necessarily the same. More often than not they're different. If we want to find the value for P(AA|NB) we'd need to have the figures for P(AA) and P(AA and NB), the latter being the probability that a person has above average intelligence and is an atheist--the intersection between the set of people with AA and the set of atheists. Mathematically, P(AA|NB) = P(AA & NB) / P(NB).

So the answer to our inquiry is: No, we cannot conclude at all that I can now be a member of Mensa or any clique and club of snobbish, condescending eggheads and geniuses.?

I couldn't think of a good title for this piece and so have that clumsy one up there. It alludes to the  conversion error whereby given "if A then B" is true we presume it follows that "if B therefore A."