f hitting the head against the wall gives you a jolting pain every time you do it, this headache is in a proven causal relationship with your (slightly idiotic) behavior of hitting the wall.

Some times I want to do that when need to handle statistics :p

This is a very good post. The same thing holds for scientific discoveries in physics. We say we have a hint for something new when the probability for having something new is of at least 99.7300204%. To claim for a discovery, we need to be sur at the 99.9999426697% level. And finally, to reject an hypothesis, we just need to be conservatively sure at the 95% confidence level.