2x2 Contingency table for evaluation of medical screening

Click on the image below for the contingency table spreadsheet. When the spreadsheet appears, remember to check out the sheet labeled "Reference." You'll find that tab at the bottom of spreadsheet.

The spreadsheet is set to "view only" which means you can't make changes to it. I definitely am not letting anyone fiddle around with! So I suggest opening a Google spreadsheet of your own on Google Drive and copying my sheet to yours. You can use the contingency table after that. Just don't alter any formulas in the cells unless you know what you're doing.

Numerical values that do need user input are in cells with a bright yellow background.

An older contingency table spreadsheet is located here and is used by my blog post on human pregnancy tests and a subsequent follow-up post.

The incredible gullibility of beings

Read the BBC article.

It's not just a question for those who're neck-deep in the supernatural, paranormal, or quackery, although these are the most conspicuous areas of gullibility.

[Stephan Lewandowsky at the University of Bristol] explains that our beliefs are embedded in our 'mental models' of the way the world works; each idea is interlinked with our other views. It’s a little like a tightly bound book: once you tear out one page, the others may begin to fray as well. 'You end up with a black hole in your mental representation, and people don’t like it.' To avoid that discomfort, we would often rather cling to the myth before our whole belief system starts unravelling.

Sound familiar? Yep. Religion. It's a worldview dependent on a number of core beliefs linked to one's identity. Shoot down belief in the supernatural and the person goes to pieces. And thus the Great Redoubt against doubt, reason and evidence--truth and intellectual integrity take a back seat (last row in the bus perhaps?) when it comes to emotional/psychological stability.

Because we're such poor thinkers it's imperative that we tap into the 90% of our brain that we don't normally use. That reserve holds the key to becoming good solid thinkers. And it wouldn't hurt at all to help the gray matter along by taking 10,000 mg of Vit C everyday. As anyone who's checked knows, scientific tests have shown that Vit C boosts mental capacity by at least 4000%. In a large multicenter randomized controlled trial half the subjects were asked to add up two 3-digit numbers before before waking up, while those in the other group worked on the math problem right after taking a 10000mg capsule of Vit C. The latter performed way better than the control group, with p < 0.00000001.

Full disclosure: I came across the Ark question mentioned in the BBC article just a month or two ago in one of the cognitive psych books I've been going through. It crashed my neural CPU. Embarrassing failure because it was in a book about such thinking booboos--I should've known better than to let my lazy System 1 be at the helm doing the mental processing. Knowing the power of the dark side (cognitive indolence/miserliness most certainly is) I bet I'm pretty much none the wiser. Fortunately for my ego the Iron Lady question didn't get pass my Central Intelligence Security.

Preposterous delusion over bleak truth

Now repeat after me: Even if our corpses end up in the tummies of worms and bugs and bacteria, we don't really die. Actually we live forever after we've breathed our last. And even without eyes, ears, brains, and not a single atom, we can see, think, feel, converse, sing praises and suck up to the Big Honcho. Yes, folks, this is no delusion. I'm positive it's true. Because as Tertullian said, I so totally believe because it is so damn friggin absurd and insane.

Forever no more

It's come to my attention that a widow who lost her husband to chronic illness almost a year ago is still reeling and grieving to the point that her health has now suffered. That to me doesn't make sense. And that's because the couple had been very devoutly religious and had been leaders in their church. Isn't he now in a so-called better place? Isn't he now free from the fetters of the mortal coil that only caused him physical torment in the last years of his earthly life? Isn't it a fact in their worldview that she will, sooner or later, be joining him, and they would be together forever living in utter bliss? Hence, her inability to let go makes no sense. If she really believes all the religious drivel then letting go wouldn't be an issue at all. In fact, she would've rejoiced at his departure and continue reveling at his good fortune. Therefore, it must be that her psyche in large part doesn't buy into the theological, supernatural, afterlife bullshit.

In this regard nonbelievers are in a superior position, mental health wise. They have no belief in any hereafter. To them physical, biological death is the end of it all. Consciousness, the ego, the I, personality, everything we associate with the vivacity and individuality of the living person are co-terminus with the body (brain included). Do we therefore not grieve? Of course we do. But we are untrammeled by delusions which try to mitigate the full impact of complete physical loss of the other. We know and accept the fact that the inevitable is coming. Either we will succumb first or our loved ones will go before us. Loss--complete and irrevocable--is built into the worldview that is aware of and accepts the reality of life and the reality of eventual nonbeing.

Snowmen not really banned in Saudi

A prominent religious scholar in Saudi Arabia has reportedly issued a fatwa against building snowmen in the kingdom, stating the practice is not acceptable to Islam. [The Telegraph]

I have just come out of a closed-door meeting with top-level imams to discuss this most pressing and earthshaking issue, and the consensus is that this cleric has erred in his reading of scriptures.

Making snowmen is perfectly fine. However, snow-women must always be built with a niqab or burqa. And snow-women must always be accompanied by their father, uncle, or brother snowman. And it goes without saying that snow-women cannot vote, cannot drive, are most definitely not entitled to any formal education, and cannot have any profession other than motherhood.

Oh, and before I forget, it was unanimously agreed that snow-goats are a must when making snowmen. Ice-Muslims need mates too you know.

Religion can explain everything away

They said I must pray for my incurable stage 4 cancer to go away. Thus I carefully composed a request, an entreaty containing details of my condition and the outcome I desired, and read it out loud. But they vigorously shook their heads at my ignorance and said I must first have faith, that I must become a true believer! So I converted, became a devotee and prayed throughout the day, every day. Alas my health did not improve and they chastised me for my lack of faith and urged me to deepen my belief. Certain of a miracle I continued my daily petitions, but then succumbed not long after. And when they came to my wake they commiserated with my wife and children, assuring them that everything that happened--the disease and my months of abject suffering--was God's will, that it was His way of beckoning me back home, and that I was at last in a much, much better place.

The Chinese ideogram for mouth is a square: "口". And old folks like to say that you can turn this character left or right or upside down and you'll still end up with the same thing--meaning, in one sense, that you can talk your way out anything if you're savvy enough. So it is with religion. Everything--even evil--can be rationalized and explained away with twisted theological gibberish.

Proportioning our respect for beliefs

Believing that laetrile can cure cancer can be fatal. Believing that a spaceship is hiding behind Comet Hale-Bopp and is waiting for humans to come on board can be fatal. Believing that prayer alone can heal children with type 1 diabetes can be fatal. Believing that there's some invisible superpower who commands the killing of those who don't believe in him and who don't follow his supposed orders can lead to fatalities. Believing in falsehoods can be and not infrequently is harmful to the believer and/or nonbelievers.

Therefore, respecting beliefs indiscriminately is patently unethical. To respect beliefs blindly and equally is to be party to evils that can arise from them. Just as we are ethically bound to proportion our belief according to the quality and amount of evidence, so we are ethically bound to proportion our respect for beliefs of others according to the truthfulness of those beliefs.

Which healer do you go to?

Is anyone among you sick? Then he must call for the elders of the church and they are to pray over him, anointing him with oil in the name of the Lord; and the prayer offered in faith will restore the one who is sick. (James 5:14-15) 

When it comes to health only two persons have integrity: the nonbeliever who doesn't ask mute, unseen, superbeings for help and instead visits doctors when they're ill or are injured, and the fundie who will not seek nor accept any kind of medical help and instead do only what the verse from the epistle above enjoins them to do. One has faith (i.e., confidence based on track record and evidence) in medical science; the other has faith (i.e., blind, unjustified trust) in the teachings of their tradition. Both are uncompromising.

The rest just want to cover all the bases, are insurance-loving and risk-averse. They rush to the hospital and fall on their knees ... and then pile on alternative and complementary quackery just to be triply sure.

The king

An all-powerful and all-knowing being created the universe so that by design he will/must:

1. kill by drowning every living thing on earth (save for a few)
2. affirm the institution of slavery and bondage
3. tyrannize his creation and promise to condemn unbelievers to eternal pain and suffering
4. and become flesh and blood and be lashed, whipped, beaten and then impaled.

Thus, rightly so, the Romans tacked a sign above the timber he so engineered himself to hang from: "King of BDSM."

Asking God

I asked God to smite me dead. He refused. 
I asked him to smite ISIS dead. Didn't happen. 
I asked him to smite the US dead. Nada. 
I asked him to smite this world dead. No dice. 
Exasperated, I asked him to smite himself dead. 
And he whispered, "Then just stop believing in me."

Finishing your vegies may still not get you dessert

Wish I had captured this on video. It is absolutely priceless!

Mother: Jeff, if you don't finish your food you won't go with us. [pause] Jeff Anderson! Did you hear what I said?!

My 4-yr old nephew finally takes his eyes off of the TV and turns toward his mom: Yes mommy. If I don't finish my food I can't go with you. If I finish my food I can go.

Classic logical fallacy from the mouth of babes! And let's not pat ourselves on the back because we all make this mistake just about every time.

Let
A = you don't do 100 push-ups
B = you won't be allowed to take your meal

The drill sergeant yells, If A then B. Does it follow then that if not A then not B? That if you complete the number of push-ups you'll finally be able to fill your empty belly?

No, of course not. To say so would be to commit the fallacy of inversion.

Here's Deborah Bennett in her Logic Made Easy giving an example which played right before my eyes this morning. Talk about textbook case! :)

As a part of natural language and conversation, 'if p then q' conditional statements that are promises and threats commonly invite the inference, 'if not p then not q.' Take, for example, the promise, 'If you eat your dinner, you may have dessert.' We would probably agree that this promise invites the threat of the inverse, 'If you don't eat your dinner, you won't have dessert.' But the statement says no such thing.... It is curious that even though the common interpretation of this parental warning has no basis in logic, both the parent and the child (and probably all of us) understand the intention of the statement.

I love the example given by Jonathan Baron [who the heck is he?!]. Presented with the threat, 'If you don't shut us I'll scream,' we would all be surprised if the speaker screamed anyway after you shut up. The speaker probably intends that your interpretation of this conditional to include its inverse, 'If you don't shut up, I'll scream and if you do shut up, I won't scream.'" This interpretation may be illogical but it isn't unreasonable; it makes perfect sense. In ordinary discourse, we make practical assumptions about what a person likely means.

---

Bennett, Deborah J.. Logic Made Easy: How to Know When Language Deceives You. New York: W.W. Norton & Co., 2004. p.115-116

Screening mammography is not recommended

A relative recently had a screening mammography. She was asymptomatic, therefore, this was screening not diagnostic. Moreover, she's 45 years old, thus age-wise, not (yet) in a high risk group. I sugested she not to go for mammography again unless she starts noticing symptoms, not least because being irradiated poses risks and can, ironically, induce mutations that can lead to cancers.

A simple computation provides a perspective on the unreliability of screening mammography. At any given time around 100 out of every 10,000 women have breast cancer, while 10,000 - 100 = 9,900 don't. Out of 100 who have cancer and who undergo mammography, the test correctly detects 95 of them and will miss 5, incorrectly showing those women as tumor-free. Out of the 9,900 who don't have cancer and who undergo mammography 9,603 will correctly test negative, but 297 will erroneously be interpreted as having a tumor.

So if we screen 10,000 women, how many will test positive? 95 + 297 = 392. And what proportion of those women would falsely be shown to have cancer? 297 / 392 = 75.8%! That's right. 3 out of every 4 women who test positive actually don't have cancer.

Now imagine the anxiety upon getting the results and being told that there is a chance you have cancer, and the need for further testing to rule out (or in) that possibility.

But what if the test comes out negative? What's the likelihood the test failed to detect the tumor? The total number of women who test negative is 5 + 9,603 = 9,608. The proportion of missed tumors = 5 / 9,608 = 0.05%, or close to 1 out of 2,000 women screened. Mammography, therefore, can very accurately rule out breast cancer (which, however, does not indicate continued absence).

----

In the latest Cochrane systematic review of the harms and benefits of screening mammography the authors conclude:

"[S]creening will result in some women getting a cancer diagnosis even though their cancer would not have led to death or sickness. Currently, it is not possible to tell which women these are, and they are therefore likely to have breasts or lumps removed and to receive radiotherapy unnecessarily. If we assume that screening reduces breast cancer mortality by 15% after 13 years of follow-up and that overdiagnosis and overtreatment is at 30%, it means that for every 2000 women invited for screening throughout 10 years, one will avoid dying of breast cancer and 10 healthy women, who would not have been diagnosed if there had not been screening, will be treated unnecessarily. Furthermore, more than 200 women will experience important psychological distress including anxiety and uncertainty for years because of false positive findings."

----

Notes:

Prevalence rate for breast cancer is difficult to come by. Estimate for the UK a decade ago was 1%. It must be noted that the frequency of cancers is higher in older people. 

I've taken the most optimistic sensitivity and specificity figures for mammography: 95% and 97%, respectively. Therefore, real false positive rate and false discovery rate are likely to be worse. For instance, specificity of mammograms can be as low as 90%. With sensitivity unchanged, If specificity were 95% the false discovery rate climbs to 83.9%, and with a specificity of 90% FDR shoots up to 91.2%.

Have a break. Have a religion

Made the after reading this completely insane thinking and behavior. So whenever you don't like what reality throws at you take some time out for a psychotic break.

original image

My very first orb!

Close-up from the same image:

OMG! My car's possezzed! An automobilic ectoplasmic globule has come out of the woodwork (or metalwork). This must be punishment for hacking into the electrical system. I must appease the god Toyotus. Perhaps feeding it ultra high octane gas, giving it a much needed oil change, and pampering it with a whole body wash might do it.

Hmmm my very first orb manifested in the car as well: http://www.skepdic.com/orbs.html

It was and has been raining lightly all morning and I was at the entrance to the garage so there's a good chance this is a tiny droplet floating by.

In the series of pics I took there were no other orbs so this speck is almost certainly not on the lens itself. Which makes sense since the orb is very bright--implying it was illuminated by the camera's flash

For the curious that's my test circuit for a variable intermittent wiper control I'm working on. There's a breadboarded microcontroller circuit in the transparent plastic box and two relays mounted on a DIN rail at the back of the box. Wires go up to the car's wiper circuit underneath the steering column. I've been studying the wiper's circuitry for several days now but I snipped the wiper's wires just today to bypass the car's electronics and hook up my circuit to it. Love the coincidence.

Finally, as alwayz the Bible has something to say: Beholdest thou the mote that is in thy camera's eye (Matt.7:3)

Pulseebo®

My company is all set to manufacture Pulseebo®, a cardiovascular drug indicated for hypertension, atherosclerosis, hyperlipidemia, among others. It'll be the most inexpensive CV drug around at just $0.10 per tablet. It's Rx so you need to get your doctor to write out a prescription.

Pulseebo® has no known side effects due to the fact that we have completely removed all active ingredients. Remember that active ingredients are responsible for all side effects in drugs. By taking them out of the formulation we have--for the first time in the industry--made a drug that will not produce any side effects. Moreover, this means that overdosing on Pulseebo® is almost impossible. Popping a whole bottle will, at worst, give you an upset stomach. (We're already working on our next drug Pulseebo Minus® which will have all nonactive ingredients removed as well to finally make overdosing an impossibility).

The evidence for the efficacy of Pulseebo® is just overwhelming. Over the past 5 years we've done extensive non-randomized, uncontrolled, unblinded, multicenter studies which which we've published in our in-house journal clearly showing that Pulseebo® is safe and effective. (These studies and other technical literature are available upon request.) More importantly, Pulseebo's efficacy is attested to by those who've used it. Just ask our staff for their personal stories. They've been unbiased and thoroughly objective in their assessment and have absolutely no conflicts of interest.

For more on Pulseebo®, ask your doctor on your next visit.


Pulseebo® -Your heart deserves more

When logic goes to the dogs

The dog's argument has an implicit inference which has to be made explicit:

All cats have four legs.
All creatures that have four legs are cats.
I have four legs.
Therefore I am a cat.

That argument can also be stated, equivalently, as follows:

If the animal is a cat then it has four legs
I'm an animal with four legs.
Therefore, I'm a cat.

Well our canine shouldn't be heartbroken at all for discovering she's actually a feline. Instead she should be depressed she's committed the fallacy of illicit conversion.

The fallacy occurs when given "all p are q" we infer that the converse "all q are p" is also true. Likewise when  given "if p then q" and we conclude that "if q then p," then we've committed an illicit conversion. The "if - then" case may be more familiar when stated as follows:

If p then q
q
Therefore p.

That's the well known fallacy of affirming the consequent and is an example of an illicit conversion. (p is known as the antecedent and q the consequent.) An argument in this form is invalid.

The thing to remember is that "all s are r" or "if p then q" do not necessarily imply "all r are s" or "if q then p." The converse is is not necessarily implied but it might be true. For instance, in causal arguments, if p causes q, and if p is both  necessary and sufficient to cause q, then the converse is also true:

If a sheet of paper is heated to its combustion temperature and oxygen is present then it will burn.
If paper is burning then it's been raised to its combustion temperature and oxygen is present

Both statements are true. This is called a biconditional--the conditional (i.e., the if - then statement) is true both ways. If p then q and if q then p. To formalize the relationship between temperature/oxygen and paper we would say:

Paper will burn if and only if its temperature is raised to its combustion point and oxygen is present.

The phrase "if and only if" indicates this is a biconditonal statement.


----

Print References:

Bennett, Deborah J.. Logic made easy: how to know when language deceives you. New York: W.W. Norton & Co., 2004. p. 108-110, 116-117, 130.

Vaughn, Lewis. The power of critical thinking: effective reasoning about ordinary and extraordinary claims. New York: Oxford University Press, 2005. p. 287-289.

The prejudice of faith

Faith discriminates. In the sense that a person will choose to believe in certain ideas that are extraordinary and unsupported by evidence and reason, but won't have faith in a host of other similar unenvinced extraordinary ideas. So what then are the criteria for having faith in X but not Y? Why X instead of Y, and why neither? What kinds of claims must we have faith in? What are the characteristics of those claims for which we must forgo reason/evidence, but must believe in fully nonetheless rather than just ignore or be agnostic about?

I think if believers honestly and sincerely answered these questions they'd find that their bases would include, among others, preference, emotion, culture-centric biases/prejudices. If we scratch the surface I think we'll discover instances of special pleading--of singling out particular claims which they hold immune to rationality and rule of evidence and logic, affording them a special pass and privilege they don't grant to almost all other ideas/beliefs and areas of inquiry.

More on HPT, and conditionals and their converse

Update (March 2016): A more comprehensive contingency table which includes detailed definitions can be viewed here. 

-------------

After writing the primer on conditional probabilities and their converse vis-a-vis screening tests, I continued to scour the Web to gather more information on the base rates of conception/pregnancy. But, much to my chagrin, as with searching for scientific studies on the reliability of home pregnancy tests (HPT) [note 1] I have come up nearly empty-handed. Thus far I've managed to find only one resource (which unfortunately doesn't provide any citation or rationale for the 50% figure provided; it sounds quite plausible, notwithstanding).

The probability of being pregnant increases as various indicators come into play. For instance given coitus within the last mentrual cycle and given the current scheduled menses has been overdue for several days, the probability that one is pregnant increases to around 50%. For our hypothetical HPT in the hands of the "average" user (for whom sensitivity = specificity = 75%) given P(pregnant) = 0.50, the reliability of the HPT is: P(pregnant | test positive) = 75% and P(not pregnant | test negative) = 75%. In this case, the true positive and true negative ratings are equal to the test's sensitivity and specificity only because the base rate = 50% and sensitivity = specificity.

A 75% true positive rating wouldn't warrant declaring HPTs as being spot on. If the prior probability (base rate) of 50% for pregnancy is held constant, the sensitivity and particularly specificity have to be >90% to attain a true positive rating close to 100%. (Use the spreadsheet and experiment with different combinations of base rate, sensitivity, and specificity. Instructions for copying the table to your spreadsheet are in Testing Screening Tests.)

It's claimed that HPTs are accurate, that "if the test result is positive, you're almost certainly pregnant" although "negative results are less reliable." However, as Bastian et al. discovered, even if the laboratory-determined sensitivity and specificity of the HPT kit is high, once the kit is in the hands of consumers, the actual sensitivity drops because most users fail to follow the instructions to the letter. User error detracts from the reliability of the test.

HPTs rely on the detection of the level of human chorionic gonadotrophin (HCG) in urine, HCG being a chemical whose amount increases over time after conception but is present in insufficient quantities during the first week or so after conception for the HPT to present a positive finding. Thus most HPT kits are to be used after the upcoming menstrual period is missed. If a user tests herself within a week of unprotected intercourse, chances are there still won't be enough HCG for the HPT to detect, assuming conception had taken place. Such misuse lowers the specificity of the test.

Moreover, as we've seen, what we're interested in discovering are the probabilities of the converse, i.e., not the sensitivity and specificity but their inverse--P(pregnant | test positive) and P(not pregnant | test negative). To obtain those figures we need to take the base rate of pregnancy into consideration. And so even with a sensitivity and specificity >90%, a true positive close to 100% is only possible if the prior probability of pregnancy is high enough (>50%). To fail to factor in P(pregnancy) is to commit the base rate fallacy.

According to studies even doctors can misunderstand the nature of sensitivity and specificity and fail to see the need to compute for the probability of the converse. 

[C]onditional statements are often confused with their converses. When they evaluate medical research, physicians routinely deal with statistics of the sensitivity and specificity of laboratory test results. In a 1978 study reported in the New England Journal of Medicine, it became apparent that physicians often misunderstand the results of these tests" [note 2]. 

In a study by David Eddy, after being given the values for base rate, sensitivity, and the complement of specificity (i.e., 1 - specificity), 95% of doctors surveyed overestimated P(malignancy | test positive) by one order of magnitude. "David Eddy reported that it's no wonder physicians confused these conditional probabilities; the authors of the medical research often made the error themselves in reporting their results" [note 3].


To end, let's look at conditional probabilities using "if / then" notation [note 4]. If a patient has cancer, then the probability of testing positive is 85%. That's sensitivity. If a patient does not have cancer, then the probability of testing negative is 90%. That's specificity. Now we want to know the converse: if an individual tests positive, then the probability of him/her having cancer is ___. And if a person tests negative, then the probability of him/her not having cancer is ___. They're both blank because we need the prevalence rate (base rate) of the particular type of cancer to determine those values. And the lower the prevalence of the disease 1. the higher the false positive rating of the screening test and 2.the lower its false negative rating will be.

In general, given the statement "if A then B," it doesn't imply the converse is necessarily true, that "if B then A." Thus, given "if I drink and drive then I will figure in a mishap," it would be erroneous to conclude that "if I'm in an accident then I was driving drunk." I could be in mishap even if I haven't had a drink and even if I'm the passenger.


----

Notes:

1. To find all studies with "home pregnancy test" in their titles, I typed in "home pregnancy test[Title]" (without the quotes) in the search engine box on the PubMed site . Only 13 studies (mostly irrelevant) came up.

2. Bennett, Deborah J.. Logic Made Easy: How to Know When Language Deceives You. New York: W.W. Norton & Co., 2004. p.109. The NEJM article being referred to is "Interpretation by Physicians of Clinical Laboratory Results" by Ward Casscells, Arno Schoenberger,  and Thomas B. Graboys.

3. Bennett, p.110.

4. Bennett, p.109.

Testing screening tests

Update (March 2016): A more comprehensive contingency table which includes detailed definitions can be viewed here. 

-------------

How reliable are home pregnancy test (HPT) kits? Given 100 who've just conceived, 75 of them will test positive. And given 100 who aren't pregnant, 75 will correctly test negative [see note 1]. Assume the chances of conception when coitus is performed at some randomly chosen day of the month is 5% [see note 2]. If the HPT result comes out positive should the woman panic and faint if it's an unwanted pregnancy? Or if she's been trying to be with child for years, should she immediately broadcast the news on Facebook, Twitter, and email every friend and kin she has?

Before finding out the answer, let's introduce some important terms. The base rate is the prevalence rate or frequency of a disease, condition or phenomenon in the population.  Sensitivity is the rate/frequency/probability a medical screening test will result in a positive finding given that the person being tested has the condition or disease which the test is for. In probability notation sensitivity can be denoted as P(test is positive | person has the condition). That's read as "the probability that the test comes out positive given the person has the condition." The vertical bar is read as "given." Specificity is the rate/frequency/probability a medical screening test will result in a negative finding given that the person being tested does not have the condition or disease which the test is for.  In other words, specificity is P(test is negative | person doesn't have the condition).

So for HPT the base rate = 0.05, sensitivity = 75/100 = 0.75, and specificity = 75/100 = 0.75.

What we're interested in finding out is how reliable the test is, ie., how accurate it is when the test comes out positive and when it comes out negative. In essence we want to know P(being pregnant | test is positive) and P(not being pregnant | test is negative). These are two values and they needn't be the same. If the values are close to 1 (e.g. > 0.90) then we can say that HPT is reliable.

Notice that when we're talking of reliability of the test when it results in a positive finding we are looking at P(being pregnant | test is positive). This is not the same as the sensitivity of HPT which is P(test is positive | person is pregnant). The test's sensitivity measures how accurate it is when the subjects who are being tested are already known to have the condition (eg. are definitely known to be pregnant before the test is even administered). Quite obviously women who buy HPT kits do so to find out whether or not they're pregnant. Likewise the reliability of the test when the results come back negative is the P(not pregnant | test is negative). Contrast this with specificity of HPT which is P(test is negative | not pregnant). So the reliability indicators we're interested in are the converse of sensitivity and specificity.

To help us derive all the numbers, we shall enlist the help of a spreadsheet. Take a look at this 2x2 contingency table (it would be best if you open a new tab or window on your browser for the spreadsheet so you can easily switch between this text and the tables). Although I can simply give you the equations for P(being pregnant | test is positive) and P(not pregnant | test is negative), using a table is far more illuminating and easier to understand. But if you would rather have the formulas, I've included them in the spreadsheet--look for the rows "true positive" and "true negative."

Although there are clearly more than two rows and two columns in the table it's described as 2x2 because the two variables have two states/levels each. In the case of HPT the variables are pregnancy and HPT test result. Their states are: pregnant, not pregnant, test result positive, test result negative. 

The equations to compute for the values of the cells are provided. The values for cells which don't have explicit equations for them can easily be derived after the other cells have been filled. Keep in mind that except for the last table which uses user-provided sample size, the cells of the contingency tables all contain probabilities. Therefore, they will only have values from 0 to 1, inclusive.

Now scroll down to the row labeled "EXAMPLE." I've already plugged in the base rate, sensitivity and specificity for HPT kits (see yellow-colored cells). Focus your attention on the purple cells. As you can see when HPT says a woman is pregnant, there's only a 13.64% chance that it's true. Thus the false positive rate = 86.36%. Out of a hundred times the test comes out positive, 86 of them will be false alarms. Not very comforting at all.

But look at HPT's true negative rate--P(not pregnant | test negative). It's 98.28%. When the test tells a woman she hasn't conceived, then it will be wrong only 2 out of 100 times. Now that's reliable.

Before moving on, I suggest creating a new google spreadsheet so you can play around with the inputs and see how the probabilities change. Go to https://docs.google.com/ (and sign in if you need to). Click on the "Create new" button on the upper left hand corner of the window. A drop menu will appear. Click on "Spreadsheet." Go to the contingency table and click on "View" on the menu bar on top. Click "Show all formulas" on the drop menu. Press CTRL-A to select the entire table and then CTRL-C to copy it. Go back to your new blank spreadsheet and press CTRL-V to paste. You will have to manually join the various cells for those whose text is too long to fit one cell (and so it wraps down). Do this by selecting the cells on the row which you want to join and then click on the merge icon (it's the square one with left and right facing arrows). To "unmerge" select the merged cells and click the icon.

Now you can change the values of the yellow colored cells--base rate, sensitivity, specificity, sample size--and watch how the values in the tables below them change. Try inputting a value of 1 for specificity and watch how false positives are completely eliminated no matter what the sensitivity is.

One of the reasons that HPT (or for that matter, other tests) are prone to false positives (i.e., test result is positive but it's wrong--it should actually be negative) is because the base rate is rather low. And as we decrease the base rate--keeping sensitivity and specificity constant--the higher the false positive rate goes up. Try tweaking the base rate on your spreadsheet and see what happens.

If low base rate decreases true positive rates, then higher base rates should increase it, right? Remember we assumed that the base rate, thus P(getting pregnant ) is 0.05? Well, our assumption is that the woman doesn't know her ovulation cycle--she doesn't know the time of the month she's fertile. Now suppose she does. Suppose she knows exactly the six-day ovulation period when she can conceive [see note 2]. If she then has coitus during this period, the chances of her conceiving jumps to approximately 20% (it's actually between 10% to 33% depending on the day). Given that the base rate has increased, the reliability of HPT should increase as well. If we change the base rate to 0.20, the false positive rate drops to 57.14%. If the test comes out positive, it's nearly a coin toss whether one is really pregnant. On the hand, look at how the false negative rate has been affected. It's increased from 1.72% to 7.69%. Moral is: You can't have your cake and eat it too.

Let's move on and use another real life example. Fecal occult blood test (FOBT) is a kit which can be used at home to screen for, among other conditions, possible colorectal cancer (CRC). As with HPT, FOBT kits  from different manufacturers vary in their sensitivity and specificity. Let's take an "average" FOBT which has a sensitivity = 65% and specificity = 95%. How often (or rarely) does CRC occur in the population? Given the prevalence in 2007 we can estimate it to be 0.37% [see note 3].

Plugging those values in our spreadsheet we get the following:

True positive rate = 4.61%
False positive rate = 95.39%

True negative rate = 99.86%
False negative rate = 0.14%

When FOBT comes out negative we can almost be proof positive that we are CRC-free. But when it comes out positive, further testing is necessary to confirm/refute the initial screening. The low base rate means this test is prone to false positives.


----

Notes:

1. Home pregnancy test sensitivity and specifity values: "The results we have suggest that for every four women who use such a test and are pregnant, one will get a negative test result. It also suggests that for every four women who are not pregnant, one will have a positive test result." In other words, P(test negative | pregnant) and P(test positive | not pregnant) are both 1/4. P(test positive | pregnant), i.e., sensitivity, and P(test negative | not pregnant), i.e., specificity of the test, are the complements of these values: 1 - 1/4 = 0.75.

It is important to note that different brands of HPT kits have varying sensitivities and specificities.

2. Pregnancy base rate: During the 6-day ovulation period the probability of becoming pregnant is between 10 to 33%. Computing for a simplistic average we get approx 20% for P(pregnancy | coitus during the 6-day ovulation period). According to the study there is no conception if coitus is outside this 6-day period. Therefore the P(pregnancy | coitus outside the 6-day ovulation period) = 0. Since the ovulation period is 6 days and there are 30 days/month, the probability that any randomly picked day is within the 6-day ovulation period = 6/30 = 1/5 = 0.2. We need to find the probability of getting pregnant regardless of when coitus is performed, that is we need to determine P(pregnancy).

Let:
p = pregnancy
p' = no pregnancy
v = coitus during 6-day ovulation period
v' = coitus outside the 6-day ovulation period

P(v) = 0.2
P(v') = 1 - P(v) = 0.8
P(p|v) = 0.25
P(p|v') = 0

P(p & v) = P(v) * P(p|v) = 0.2 * 0.25 = 0.05
P(p & v') = P(v') * P(p|v') = 0.8 * 0 = 0
P(p) = P(p & v) + P(p & v') = 0.05 + 0 = 0.05

I actually used a tree diagram (with ovulation period as the first two branches) and a 2x2 table as in the spreadsheet to aid in determining the probabilities. The above equations are a distillation of that graphical and tabular process.

3. Prevalence of colorectal cancer: "On January 1, 2007, in the United States there were approximately 1,112,493 men and women alive who had a history of cancer of the colon and rectum -- 540,636 men and 571,857 women. This includes any person alive on January 1, 2007 who had been diagnosed with cancer of the colon and rectum at any point prior to January 1, 2007 and includes persons with active disease and those who are cured of their disease." In 2007 the population of the US was 302.2 million. Therefore we can compute an estimate of the base rate for CRC: 1.112million / 302.2million = 0.0037.

A very sick doctor

According to Eduardo Cabantog he's an MD who's served in various state-run hospitals in the Philippines. Some years ago he founded Alliance in Motion (AIM) Global, Inc.--a multi level marketing (MLM) company. He's never looked back since. And if his presentation (see endnote 1) is any gauge, he's actually turned his back on the ethics of being a doctor.

AIM is into food/dietary supplements. But I'm not even going into the efficacy question regarding the items the company is peddling. This blog entry is about a (different) scam Cabantog has been perpetrating during his presentation(s).


Watch the following video begining at around 2:40 (you can see how the older gentlemen eventually fared in Part 8 of the presentation below).



The two men are easily thrown off balance by Cabantog when he performs his drag-the-person-down test. But after taking Alive! supplement (manufactured by Nature's Way) the two subjects become incredibly immovable. Try as he did Cabantog could not get them to budge. Amazing, isn't it? Well, it would be if Cabantog wasn't resorting to outright flim flam. Here's Richard Saunders of Australian Skeptics showing us how to do this party trick (see endnote 2). The reveal begins @5:10.



The direction in which the forces are applied is key. Push/pull a person away from where he's standing and it's easy to throw him off balance. The opposite is also true. Moreover, the lack of any form of blinding ensures that the tester can apply as much or as little force as he wants depending on whether the subject has taken the supplement or not. Similarly, the subject, knowing that's he's taken the supplement and having already been primed by Cabantog's prior presentation on the benefits of the wonder supplement, can apply more or less resistance. The subject--just like a patient--reflexively wants to please the authority figure and to play along specially given the large audience present (yes, there is peer pressure at work). Given how Cabantog has full control of the vectors--the magnitude and direction of the forces he's going to apply--and given how the subjects can more or less be counted on to perform as expected of a good albeit unwitting shill, there is almost no way this trick can go wrong.


Let's move on. Watch the following, paying close attention to the portion of the video from 2:16 - 3:03 and 4:34 - 5:20. Watch both a few times and see if you notice something peculiar.



Did you see it? Those two segments are merely mirror images of one another. And we know the first is the original because @2:23 to 2:25 we see the label of the lancet (the penlike device for pricking the finger)--the first few letters being "Lan"--flipped horizontally @4:41 to 4:43.

Moreover, if you listen carefully to the audio (best if you don't look at the vid) you'll hear the same one in both the first and second segments (although the first few seconds from the first don't make it to the second; in its place is a short part of the audio downstream, thus resulting in that portion being repeated twice in the second segment). You'll even hear the very same car horn @2:47 and @5:05.

In the first segment, the supposed blood sample taken is one before the subject took Alive! In the second, a blood sample after the supplement was taken. But as we've seen AIM Global merely provided a flipped version of the very same blood sampling. If AIM actually took blood samples from a subject in the audience before and after taking Alive! and if the glass slide showing freely moving RBC is from the same subject, then because they don't have the video clip for the second sampling there is reason to believe AIM is hiding something. Perhaps they don't want to show that they added a substance to the second blood sample (which they didn't to the first) because "Rouleaux and clumping occur when blood is placed under a microscope without first being suspended in proper solutions to control acidity and agglutination." Is AIM going to give us the lame excuse that they forgot to turn on the camera during the second blood sampling or that that video segment got corrupted?


Finally I found the following images in a Facebook album that apparently is open for all Facebook members to see and comment on. They're ads for AIM's latest product C24/7, which apparently is an improved Alive! since instead of just 16,000 "phytonutrients" C24/7 contains 22,000. Given the weight of one capsule and this number of chemicals you can do the math to find out how many micrograms of each there is per cap (assuming AIM isn't pulling our legs--again--with that 22k figure).

Whether these ads were made by AIM Global or by individuals (re)selling the supplement, I don't know. Since these ads make explicit therapeutic claims, they are in clear violation of the laws of the land (among others, RA 7394 Art.112 and the Department of Health regulations)



----

Notes:

1. Alliance in Motion (AIM) Global presentation featuring Eduardo Cabantog, MD
Part 1/10 Part 2/10 Part 3/10 Part 4/10 Part 5/10 Part 6/10 Part 7/10 Part 8/10 Part 9/10 Part 10/10

2. This trick was also employed by, among others, Power Balance (Australia), which recently had to admit (thanks to the Australian Competition and Consumer Commission) that its wild claims for its rubber bracelet are bogus: