Sensitivity vs Specificity
G'day team,
Today I thought I'd just write a quick post on the important, but rather dull, topic of sensitivity and specificity. This way if I ever speak about it in the future I've a handle article to reference back to :)
So let's get started!
The Concept
When we do a test, there is always a chance of the test screwing up and giving us the wrong result. Let's start with a simple example.
I want to know if it's going to rain tomorrow so I ask my sister.
My sister looks at the meteorology report, and it says 20% chance of rain.
So "no" she says to me. It isn't going to rain tomorrow.
I trust my sister and I head to the park, then twenty minutes later I'm cold, drenches and considering sororicide.
So! I'm shit out of luck... but where did my sister go wrong?
Well, it's pretty obvious, isn't it! She converted the 'probability' of it raining, into a definite answer. Her two options were "yes" or "no".
She was wrong because of the imperfect reliability of the information she was referencing. As we'll see in a bit, her answer was not sensitive enough. So let's go through what this means.
Sensitivity
Sensitivity is the easier of these two to wrap our heads around, so let's tackle it first.
If we buy a metal detector to take to the beach we want it to pick up every priceless wedding ring and nugget of gold we walk over. Say we walk over 100 objects, we want a metal detector that picks up all 100 objects, this would be a 100% sensitivity. But we're broke students so we go out and buy the cheapest model (a Sensitron2000), and it only picks up 60 of the 100 objects, this means we have a 60% sensitivity.
Pretty simple, right.
If we are trying to measure/ detect something, then sensitivity is a measure of how many results we miss. If we come up with a new test to screen cancer patients, and out of 100 patients with cancer, this test comes back positive in 92 of them, then it has a 92% sensitivity. The other 8% of cases are false negatives.
So that is sensitivity... but what about specificity?
Specificity
We're back at the beach again, with our metal detector. But this beach has had far less foot traffic and unfortunately not as many lost wedding rings and nuggets of gold. There are only 10 buried valuable objects on this beach, but there's a lot of buried logs and seashells. Clearly what we want here is a detector which beeps whenever we walk over one of the 10 buried valuables, but not when we walk over a log or seashell.
We haven't learned from our first lesson and although we've changed brands we're still working with the base model (the Specitron2000). We spend eight hours at the beach, covering every inch, and in this time our detector goes off 50 times. We pick up all 10 valuable objects, but we waste hours digging up 40 logs and seashells too.
This gives us a very poor specificity of 20% (10/50).
Let's give another example.
We've developed a new cancer test (again!). This time of 100 results that come back positive, only 88 patients will actually have that cancer, the other 12 will prove to be false positives. Specificity is a measure of how many of the positive results that we're given by a test are actually real
I hope this all makes sense so far.... because we're about to bring it together!!!
Bringing it Together
In the real world, no test is perfect. Most of the tools we rely on in the world of science have flaws in both their sensitivity and specificity... so let's discuss some of these. But first, an example...
I go to the airport and travel through bomb detection where a man rubs me with a wand-thingy and puts it in his machine. When the smart people in lab coats build this machine they had an option...
- Option 1) Dial up the settings to be really certain no one got through, but also risk flagging a few innocent people
- Option 2) Dial down the settings to that a few bombers got through, but never trouble an innocent traveler.
Well, the people in lab coats weren't morons... so we're left with option one.
As a result of the 1,000,000 people who travel through this airport every year there are, say, 100 people carrying explosives (hypothetically). The machine picks up 99 of these, but it also flags 901 people who were innocent.
This gives us the following...
- Sensitivity = 99/100 = .99 = 99%
- Specificity = 99/(99+901) = 99/1000 = .099 = 9.9%
Let's hope the machines they use are a bit better than this :/
So why's this all relevant in the real world?
The Real World
Below I've simply compiled a simple list of the sensitivity and specificity of some tests that're used in the medical field, just to give some context to the post.
Modern HIV Testing Methods - Enzyme immunoassay
Sensitivity of 99.7%
Specificity of 98.5%
TB Detection - Quantiferon Gold
Sensitivity of 89.0%
Specificity of 98.1
Colorectal Cancer Screening - FOBT + Colonoscopy
Sensitivity of 36%
Specificity of 94
Breast Cancer Screening - Mammogram
Sensitivity of 67.8%
Specificity of 75.0%
Heart Attack Detection - Troponin I
Sensitivity of 90.7%
Specificity of 90.2%
Thanks
As usual, thanks for reading team and I hope everyone has learned something. If you just got bored out of your brain then I don't blame you and I promise I've got a much funner post coming up tomorrow as I'll be assisting in my first ever surgery, an experience I can't wait to share :)
Thanks
-tfc
Resources