Thank you for this post, @suesa! :) This topic needs a lot of awareness.

I can say from experience, that the problem of "significance" goes a lot deeper than this. If you work in a life science like e.g. biology, "significance" is a tricky thing. I see people applying the standard error when they only have a very small number of samples, same goes with the abuse of the beloved p-value. If you repeat something only 3 times (the standard in biological research in many places), the p-value is pretty much useless. But due to timepressure and money, experiments are often repeated only 3 times, as more would be too costly in either way. A dilemma, which causes the need for even closer observation, as even significant p-values are often misleading. So the whole set of experiments has to be considered together, and to do the right experiments, it takes a very skilled researcher a ton of knowledge. Quite frustrating, especially for young and aspiring scientists!