10 Simple rules for overcoming statistical paralysis

So, I’ve long threatened to create a blog called “Complex analysis” mostly to fight with Jeff, Roger and Rafa over at Simply Statistics. I’ve decided instead to start it on steemit for a variety of reasons. (But, honestly the main one is that I’ve been spending a lot of time learning blockchain stuff lately, and this fits. Also, I get to show off that I knew a new internet thing that Jeff doesn't.)

The title of this post is about an online talk that I’ll be giving to the Society for Neuroscience on April 10th. One of the organizers had an interesting question which amounted to “Many of our students are approaching data analysis with good faith, but feel paralyzed on how to start.” I’ve taken to calling this phenomena statistical paralysis.

Statistical paralysis: that feeling of hopelessness you get when doing data analysis with the volume of choices, considerations and fears weighing on you.

Statistical paralysis comes up for three basic reasons. First, is are the choices that you don't have a great basis for deciding between. Annoying statistical methodological researchers (like me) keep creating methods at an astounding rate. In addition, there’s a million software packages and programs. You don't have time to become an expert in everything just to do a simple analysis? Thus, the paradox of choice kicks in. See this Calvin and Hobbes cartoon.

Second are all of the considerations, which I'm thinking of as a different thing than choices. These are things you might know how to deal with just fine, but come in incredible volume whend doing data analysis: like what confounders to adjust for, remembering to adjust for multiple comparisons, study pre-registration and so on.

The third is fear and anxiety; such as anxiety over getting things wrong, imposter symdrome, the fear of being judged for using a naive method ...

Sadly, learning more statistics can often add to this phenomena, because now you just know methods to choice from, have more considerations to factor in and perhaps ever greater fear since now it’s more embarrassing if you do something naive. (Note, don’t use this as an excuse to drop your statistics class. I’m not saying to learn less statistics, just that learning statistics won’t suddenly make you more analytically decisive.)

So, if you suffer from statistical paralysis, don’t worry I’m here for you. In the spirit of the PLoS 10 Simple rules series, I’ve come up with 10 simple rules for overcoming statistical paralysis. They are as follows, in no particular order.

1. Be an effective worst critic.
2. Be clear about the nature of your study.
3. Before overloading on learning new methods, learn about common statistical pitfalls and how to avoid them.
4. Err on the side of more reporting and transparency.
5. Learn one software package well.
6. Know your data well.
7. Get started doing simple things.
8. Investigate and report uncertainty
9. The more developed the scientific thinking, the easier the analysis.
10. Better design makes for easier analyses.

So this blog post isn’t too long, I’ll discuss one here. I’ll post some videos/blogs later discussing the rest.

Consider rule 1; being an effective worst critic. We all know the person who fires an arsenal of criticism at themselves to the point where can’t complete anything. They’ve decided that a type I error rate of zero is better than the risk of putting anything out there.

Being critical is a good thing, just not if it's debilitating. So, why not put that criticism to good use? Instead of undirected criticism, take your own work and write a review as if you were a reviewer. Then write a response and address the review. This multiple personalities exercise will hopefully quiet your inner demons. If not, then maybe also get a review from a friend and address it. Ultimately, the point is to focus and direct your critical viewpoint as constructive for helping you move forward. You're going to have to make decisions, this will help you defend the ones that you made.

That's enough for the first post. I don't want to strain myself. Remember to subscribe to Monday Morning Data Science and all of the JHU Data Science Lab content.