Lying with Statistics
”Good morning class”, greeted Professor Hodge her students. ”Last lesson, we talked about how to properly present the data you got from your experiments. Today, I’ll show you what kind of presentation you might encounter, because the presentation of data always has a goal. And it is important that you can see behind the intentions of the writer.”
She pressed a button on her computer and a picture was projected behind her.
”Now, that’s a graph you want to present your boss, or the shareholders of your company! What an increase in profit! This is clearly the effect of your great work. Well…”
The picture was replaced by a different one.
”Some of you might already have noticed that, in the first picture, the difference between last year’s and this year’s profits was just about $20. In this pie chart it’s obvious that when you look at the overall profit, both years brought in almost the same amount of money. This kind of exaggerated scaling can be used for basically everything, because people rarely look at the numbers. The size difference of the bars is way more important, because it’s easier to process.”
”This technique of an adjusted scale can be used in a number of ways. It all depends on the data you present – and what you want the reader to feel when they have a look at it.”
”Don’t get me wrong”, Hodge said. ”A bigger scale is not always the right choice. But it happens a lot that people try to create a dramatic effect and that is achieved best by making drops and rises seem more drastic. But graphs are not the only thing that can be used to create a false impression. Please look at this statement.”
A new slide appeared.
A study showed that in almost fifty percent of cases, women were more likely to be the offender in a violent crime.
”Now, look at the actual data.”
| Crime | Female offenders | Male offenders |
|---|---|---|
| Murder | 3 % | 97 % |
| Spousal Abuse | 51 % | 49 % |
| Rape | 20 % | 80 % |
| Mass Shooting | 10 % | 90 % |
| Burglary | 5 % | 95 % |
| Terrorism | 50.5 % | 49.5 % |
| Drug crimes | 55 % | 45 % |
| Kidnapping | 52 % | 48 % |
| Carjacking | 30 % | 70 % |
”Nine different kinds of violent crimes and women commit a higher number of them in 4 cases. That is about 44 %. So yes, almost 50 %. But look at the exact numbers! In the cases when women are the main offenders, the percentage of male offenders is almost the same. The highest difference is 4 % while the highest difference for crimes mostly committed by man is at 95 %. That paints a completely different picture, doesn’t it? And still, the sentence from before immediately makes you think about how women are just as violent as men, in every regard.”
At this point, the class started to become agitated. One of the male students raised his hand.
”Why should anyone write something like this in a scientific paper?” He asked. ”I can understand that mainstream media would do something like this, but it’s just illogical for a researcher to present data in a way that totally takes it out of context.”
Professor Hodge smiled.
”But what is the context? Whatever research you do, there will always be something you look for. And what happens when you don’t get the results you are looking for? Or, almost worse, you almost get the results you want? Do you know what the saddest thing is that you can read in a scientific paper?”
She waited a few moments, in case anyone wanted to answer the question. Nobody did.
”The saddest thing to read is the collected results were not statistically relevant. You can almost hear the pain the researcher experienced while writing this. This person actually got data out of an experiment or a study, but was not able to use it because the statistical relevance wasn’t there. Every result could be just random. Do you have any idea how frustrating that can be?”
Now, one of the students raised their hand.
”Then why to people test for statistical relevance? Wouldn’t it be easier to just write about the results and ignore the possibility that it isn’t relevant at all?”
”Some people do”, Hodge explained. ”But that is the beauty about peer-reviewed papers. When other people check your results, they try to find inconsistencies. And the people reviewing your work are at least as qualified to interpret the data as you are. So the only people that really get away with actively manipulating their data are those that don’t depend on peer-reviewing.”
The Professor looked around. Her gaze met many worried faces.
”There will always be articles that pretend to be scientific. They will use exaggerated graphs, vague wording and statistically insignificant data. But if you’re willing to read into it deeper and know how to separate a well written article from one that just wants to push an agenda, you’ll find the truth. It is a lot of work but choosing the easy path will only make you believe in the wrong things. And then you’ll end up not vaccinating your child against measles because you think there is mercury in the vaccine.”
Some students chuckled. Hodge looked at her clock.
”Alright, class dismissed. I will see you next week.”
Sources:
Images were created by me using Excel.
The data used is fictional.
Got a scientific topic which you want to see as a story? Leave me a comment!
Check out @steemstem and the #steemSTEM channel in steemit.chat to support scientists on steemit!
Strange, before only statistics would be enough to see facts, then people start to abusing it, now we should be aware of this abuses and tricks to see clearly. So we must communicate and write articles one more meta level up to filter facts from illusions
yes ,bro you're right thumbs up.
Pfft. There's been no global warming for 18 years!
Wait wait, lemme zoom out a bit...
Er... whatever.
Wake up sheeple
Accurate
This is an important article. It was probably the most important lesson that I got out of my statistics class in college. @sariun mmm dat truth.
Fun! What the teacher doesn't mention is that there are plenty of peer-reviewed papers in which the peers are a group that was specifically selected to support the study design and conclusions. So, just because an article is peer-reviewed and got itself published in a major journal doesn't mean we don't need to read it with a critical eye. It is also essential to determine who funded the study and what affiliations the authors have.
Totally agree but didn't want to make the post too long
Right! Perhaps a subject for another story...
As a mathematician, I can vouch for the fact that it is incredibly easy to manipulate data and statistics to fit a narrative.
This post does a great job of showing a few simple ones that show up frequently (and we should all be on the lookout for!) but there are far more subtle, pernicious ways in which mathematics can be abused to abuse, discriminate, and manipulate. This is especially true in an era where "big data" and "machine learning" have started to infiltrate many parts of our lives.
For a really good, approachable overview of some of these Really Bad New Things, check out Cathy O'Neil's book Weapons of Math Destruction.
Thanks for the book reference!
No problem. Cathy O'Neil is generally a good writer about data stuff. I particularly like her critique of Nate Silver.
BRILLIANT.
I talk about this sort of stuff all the time in meetings at work. We most recently had an 'Employee Engagement Survey.' The raw numbers were 80% of Staff in the company were happy in the direction the company was going in.
This was seen as a positive. The way I looked at it, 1 in 5 staff did not agree with the direction and did not feel properly engaged, you could see this as focusing on the positive side of the fence, but what about that one staff member?
What about the unadulterated truth? It doesn't exist.
I have great statistics.
100% of the people interviewed agree with me. (According to a study where the total number people interviewed was 0.)
;D
Great study, gotta share it on Facebook with all my friends!
This is a great article! And I love how you do it in a fiction style. Following :)
Just a note though: Peer review is not always effective. More often than not, researchers belong to circles where they "upvote" each other just so they can all benefit.
A fiction about math and statistics, what can be more genius than this?
Very good publication. Always supporting the scientific knowledge and the logical and rational understanding of everything that surrounds us.