It works because I tried it and got well

One of the fundamental principles upon which therapeutic treatments rests is that if a person has a certain medical condition and receives treatment/medication T then she will get better as oppose to not getting better had she not received T. Indeed, if after taking a certain medication or undergoing some treatment and we don't get better (or we get worse), why have it?

What this means then, from the standpoint of assessing new treatments, is that if we receive T and we get better, this confirms that T indeed is therapeutically efficacious. If I take a new drug for a week and the symptoms and whatever was afflicting me up and disappear then the pills must've been effective, right? Unfortunately, it's not necessarily the case, even if at first blush the argument sounds perfectly sensible. You see this kind of reasoning commits the post hoc ergo propter hoc ("after this, therefore because of this") fallacy.

The post hoc argument is one wherein we conclude that event F is the cause of G merely because we observe G occurring after F. But a little thought makes it plain that just because F precedes G doesn't necessarily imply that F caused G. Innumerable events precede G, not just F, and innumerable events come after F not just G. The rooster may crow just before sunrise, but that doesn't mean it woke up the sun. In the case of disease and therapy, unsung but certainly working 24/7 behind the scenes, is our body's defense system. People have the misconception that drugs and medical procedures cure us, but in fact most treatments only help the body bounce back. And in many cases the body can do without medical intervention.

As an aside, it's also interesting to note that the post hoc fallacy is at the root of various superstitions:

I wear a new pair of socks--which my wife has just given me--to an interview for a position that has dozens of applicants. To my amazement I get the job. I then attribute my success to wearing this particular pair of socks. The next time there's some important event I wear these lucky socks.

Of course there is no causal link between wearing the socks and landing the job. More rationally, it was my resume, the answers I provided in the grueling interviews, and the impression I conveyed which clinched it. Magical thinking--"interpreting ... two closely occurring events as though one caused the other, without any concern for the causal link"--is very much alive in our society and does not automatically vanish as we graduate from childhood.

In general, attribution (causal) errors are rather common. I myself commit them every now and then when pinpointing the cause of various events. For instance while cooking dinner yesterday I had this wok in which I was heating oil to its smoking point on one burner and a saucepan full of soup on a high boil on another. The two pans were cheek to cheek. While waiting for the oil to heat up I began hearing sputtering inside the wok, indicating that there were drops of water in the oil. I had covered the wok earlier with a lid although the lid (from a discarded rice cooker) was a couple of inches smaller in diameter than the wok. The saucepan, on the other hand, was open. Because this was the nearest source of water I then surmised that some droplets from the bubbling and nearly overflowing soup were finding their way into the wok and seeping through edge of the lid. I could rule out moisture in the wok before I poured oil on it because I had made sure the wok was dry by heating it up to evaporate any residual moisture. So, mystery solved. Or so I thought. Moments later it dawned on me that I was utterly mistaken.The droplets had in fact come from the lid itself. Minutes before I had used it while cooking some other dish and it had condensation on the inside which must then have dripped into the oil, perhaps the moment I put it on, while the oil was still well below 100deg C. It should've occurred to me early on that the temperature of the wok was such that any tiny drop of soup landing on it would have immediately (and inaudibly) sizzled and nearly instantly evaporated; it wouldn't have had a chance to remain liquid, seep through the lid and roll down to the oil.

In this case rather than a post hoc I had committed a cum hoc ("with this") fallacy, whereby I had (mistakenly) assumed that being the only source of water at that time tiny drops of soup were shooting into the wok and causing the sputtering. If anything this vignette should hammer in the point that we ought to be as meticulous as possible, propose as many plausible explanatory hypotheses, and then mercilessly scrutinize and critique each of them instead of jumping the gun and going with the first (sensible) explanation that pops in our head.


Given that it is fallacious to jump to the conclusion that T works, how are we to determine whether T is actually efficacious? Well, the way to do it is to conduct controlled studies. And that's what scientists and medical researchers do to evaluate new treatments. Basically, they get two groups of people who are--as much as is feasible--the same in relevant aspects (age, lifestyle, socio-economic status, race, etc.) and have the medical condition for which T is being tested. They give one group a placebo, or if there is already medication that's been previously tested and known to be effective against the said medical condition they give them that. They then give the other group the substance or treatment T. To maintain objectivity and reliability the participants (either just the subjects or, as much as possible, both subjects and researchers) are kept blind as to who is receiving which substance. After the experiment if there's statistically significantly more people who get well in the group who took T, then T is judged more effective than the placebo (or the current medication/treatment).

Now let me just add a nuance to the basic statement I gave at the very beginning and refine it: If a person receives treatment/medication T then that person will get better or get better in a shorter period of time than had she not received T. The addendum there is necessary since a good number of conditions are self-limiting in nature. Most headaches for instance will go away whether or not you pop an aspirin or paracetamol. The common cold goes away within a couple of weeks if left untreated. The body with its defense system can mop up various bacterial infections without the aid of antibiotics. So if T does not lick the problem faster than the body can on its own, then receiving T doesn't make that much sense, unless T can rid us of or minimize annoying or debilitating symptoms during the period.

Because mere precedence does not imply causation it becomes clear why anecdotes and testimonials from people who've tried/received treatment T and gotten better are not evidence for the efficacy of T. To make this a little more concrete let's illustrate this with a hypothetical example.

Mr. Juan Cruz, 71 years old and living alone, is suffering from a swollen knee. Some weeks ago he'd heard Dr. Nightrit on the radio hailing the healing properties of virgin coconut oil (VCO) for both external and internal ailments. Confident that a real doctor--a cardiologist with decades of clinical experience under his belt and one who himself markets VCO--had endorsed this substance, Mr. Cruz gets himself a bottle of VCO and then religiously massages a spoonful on the affected area three times a day for a week. Less than ten days later the inflammation is all but gone.

To most people this would seem like a clear-cut case that supports VCO's therapeutic efficacy. But as we've seen above to jump to such a conclusion is logically fallacious. (You will notice too that Mr. Cruz had failed to shudder at the fact that Dr. Nightrit is hardly an objective and impartial source of information on VCO, given that he's got a vested financial interest in it. We'd also have to ask Mr. Cruz whether the doctor had specifically cited VCO's applicability to his condition. Just because substance X is said to be effective against "a wide range of internal and external conditions" does not make it a panacea and automatically translates and extrapolates to any condition. Bold claims about X's range and breadth of therapeutic application should not be a cause for excitement, rather it should elicit skepticism. Snakeoil anyone?)

Let's analyze the role of VCO in Mr. Cruz's case. The first question we need to ask is whether the kind of inflammation is the type that would have subsided with or without medical intervention. Is it merely a mild contusion? If it is then one must ask whether VCO's role in this whole incident was simply cosmetic. Assuming that application of the oil was therapeutic, we need to then ask whether it was the oil or the massage or the placebo effect or some other phenomenon that actually had therapeutic value. If we are able to establish that VCO per se had indeed a role to play in the healing process, we need to compare, among other things, how long it would've taken the inflammation to subside with and without the VCO. How significant is VCO as an anti-inflammatory? Moreover, it would be instructive to try out other oils and substances (perhaps cheaper ones) to see how VCO measures up to them. If a less expensive and more readily available substance is just as effective or more effective, prescribing/using VCO becomes less attractive, unless these other options have greater adverse side effects or if some other characteristic of theirs outweighs their advantages.

Even if we multiply the above anecdote by ten thousand, with people across the world swearing by VCO's effectiveness in treating various internal and external conditions, all we end up with is ten thousand fallacious conclusions and ten thousand times ten critical questions to ask. Only controlled studies--or more to the point, controlled, randomized, double-blind studies--will be able to provide us the most reliable information about the efficacy of treatments. Moreover, our confidence in a study increases as independent researchers are able to replicate the clinical trial and end up with similar results (while conflicting results should immediately give us pause).

A lot of people mistakenly believe that positive personal experience is warrant enough for them to recommend a new touted remedy to family and friends. But as I have hopefully shown, it just isn't so. Just because we tried it and got well doesn't mean it really works.