In a previous post, I talked about how companies could use statistics to mislead their customers. I am far from perfect and I can admit that I make mistakes when calculating out probabilities or statistics. So sure, I can be empathetic and can give the benefit of the doubt; I can just hope it’s done unintentionally by a lack of understanding the behavior of numbers. But I still need to consider even the slight possibility it's maliciously deliberate.

I have an open mind (within reason of course). I will never dismiss someone who disagrees with me, provided they have evidence and the work to back it, and even without, I'll listen. However, if I do believe I am correct, all I ask is for the same respect and courtesy in return. If I calculate out a probability or some mathematical statistic and there is an aspect I did not consider that should have been considered, I can accept being wrong and fix the solution…isn’t that how we learn!?!?

So after writing my first post about misleading statistics, I did contact the company with the table in question, to explain why I believe their table to be misleading to customers (in a respectful way). I provided my proof, evidence, and work, and simply just asked if they could explain how they calculated their figures. Or at the very least, if it was possible they mean something else when they say “being caught after one year of testing”? It's been 2 weeks, and I have not heard back.

If you have not read my steemit post that I am referring to and would like to read it, you can find it here: Do Companies Purposely Mislead?

But unfortunately, sometimes, people can be too proud to admit they are wrong or consider the consequences of being narrow-minded…even at the expense of sending an innocent person to jail for murder.

It was YOU! The Cat Burglar!

This happened in Britain to Sally Clark in 1998 when she was arrested and convicted of murdering her 2 sons. Her first son died suddenly within a few weeks of birth in 1996. After her second son died in a similar way, Professor Sir Roy Meadow, a paediatrician, presented faulty statistical evidence against her. He said that the chance of two children from a wealthy family suffering sudden infant death syndrome (SIDS) was 1 in 73 million.

There were a few flaws in Professor Sir Roy Meadow’s calculations, including that he calculated his probabilities without considering dependency…I mean think about it…both of these children have similar genetics, similar living conditions, similar environments…don’t you see the possibility that these could be dependent and maybe since one child died of SIDS, that there is an increase probability of a second to occur? But regardless of that calculated mistake (along with others outlined here) the "proof" of guilt based on his logic is the biggest mistake of all.

The argument was, the probability of this to happen is soooooooo rare, that for it to actually happen means foul play had to occur and in essence proving her guilt.

What is the Prosecutor’s Fallacy?

The Prosecutor’s Fallacy is the false belief that rarity implies guilt. Here’s a quick summary of an example. To see the full explanation, click here:

A woman is walking down a wealthy neighborhood street in New York City when suddenly a man jumps out, snatches her purse carrying thousands of dollars, and gets away. She was so startled that she couldn’t describe the man to the police, but luckily there were eye witnesses to give a description (also, any and all security cameras from the nearby businesses and homes could not get a clear video image of the man 😉 ). The man was described as between 6’4” and 6’8”, between the ages of 20 and 30, red hair, and had a limp.

A few days later, the police witness a man fitting the description above, walking down the street with a brand new 4K HDTV. They stop the man and find he just recently purchased the TV. What raised further suspicion was it was payed with cash. The man is arrested and tried in court.

Guilty or Innocent?

In court, the prosecutor has no forensic evidence. There were no fingerprints, footprints, no DNA found, etc. But he would present statistics to show the man's guilt based on the eyewitness’ description.

So, the prosecutor, using the demographics in New York City (*note: these are not actual demographic statistics in New York City), shows the statistics of each description:

Multiplying these probabilities gives us:

0.49 x 0.03 x 0.30 x .038 x .018 = 0.000003016

For the sake of the story, we’ll round to 0.000003 (that is 3 out of 1,000,000), meaning the probability of finding a person with that exact description in New York City is nearly ZERO. And since the probability is so close to zero, the sheer fact we found the man means he HAS to be the purse snatcher!

So the logic goes….

1. We’re looking for this particular suspect with this description
2. The probability of finding a person with this description is nearly zero
3. We actually found a person who fits the description (which was nearly zero chance of doing)
4. So this man has to be the robber.

Whoa, Whoa, Whoa,…

So let's break this down. The probability is 3 / 1,000,000. So that means, out of every 1 million people in New York City, 3 people will fit that description. If we say the population of New York City is 10 million people, that means there would be 30 people in New York City that fit this description (and that’s not even accounting for the thousands or millions of tourists visiting).

The misunderstanding comes in with the assumptions and considering guilt. To say this man is guilty is an assumption that all other suspects are innocent. We already know anyone not fitting the description is innocent, so we're ok on that assumption and can eliminate most of those 10 million possibilities in the population. BUT, that leaves 30 people that do fit the description. And if we can't assume their innocence, what needs to be considered is that the suspect is 1 person out of 30 possibilities when considering guilt.

The Prosecutors Fallacy illustrates a fundamental difference in what we are considering. There are 2 separate probabilities presented here:

1. The probability of fitting the description
2. The probability of fitting the description and being guilty

So in fact, the probability of this man’s innocence is really 29 out of 30, NOT 1 out of 333,333 (which I got by reducing the 3 / 1,000,000).

Now this doesn’t prove the man’s innocence either. All it merely shows is that he is 1 of 30 possible suspects. He could still be guilty, but it can’t be based purely on probabilities alone.

Just because you found a needle in a haystack, doesn’t mean it was the needle you were looking for.

So What Happened to Sally Clark?

The Royal Statistical Society disagreed with Sir Roy Meadow and wrote to the lord chancellor saying there was "no statistical basis" for his 1 in 73 million figure, which is now believed could be anywhere between one in 100 and one in 8,500. Again, does nothing to prove or disprove Sally Clark’s innocence, but the sheer “malpractice” of Sir Roy Meadow’s “statistical analysis” is disheartening. He ultimately ruined a women’s life with erroneous and misleading evidence…as an “expert”…and that is scary.

Had it not been for Prof. Ray Hall, would Sally have been freed?

Sally was released in 2003 after serving 3 years of a life sentence. Sadly, she had passed away in 2007 with family members saying she never recovered from the court case that wrongfully convicted her of killing her 2 sons. Professor Sir Roy Meadow, the “expert” witness, now retired, was struck off the medical register in July 2005 but then reinstated by the court of appeal in October 2005. It backed a high court ruling that he was not guilty of serious professional misconduct. His evidence had also helped to convict Angela Cannings and Donna Anthony in cot death cases with both eventually being freed on appeal.

To read more about the case, follow the links below or a simple google search should return 100s of hits.

What startled me about this case was how relatively recent this was. When I first heard of it, I had not known of when this occurred. I envisioned this as some case in the mid 1900s, thinking to myself, "In no way would the courts fail to find the flawed mathematics in today's day and age." But I was wrong, and as I stated above, I have no problem admitting I am wrong...it's how we learn. For a lack of a better term, and I realize it is a severe understatement, it is unfortunate for Sally Clark, Angela Cannings and Donna Anthony, that the Prosecutor's Fallacy will be defined by their court cases and not considered prior.

Thanks for reading. Stay tuned for my next posts which I'll talk about why “rare” events naturally occur in bunches. Then I’ll also look at another example of misinterpreting statistics and how it possibly could lead to a false medical diagnosis.

References:

http://www.conceptstew.co.uk/pages/prosecutors_fallacy.html
https://www.theguardian.com/society/2007/mar/17/childrensservices.uknews
http://www.statisticalconsultants.co.nz/blog/prosecutors-fallacy.html
https://www.ft.com/content/6294608e-f52b-11d9-8ffc-00000e2511c8

Images 1, 2, 3, 4, 5