The scientist who stares at goats - Are we ready to understand statistics?

The first time I've heard about the Monty Hall problem I was still a child. Something about three doors, two goats and a car, what I regret the most is that I was still too young to make a joke about the corresponding zoophilic violence involved.

The problem was called like that because of the TV show and a game that consisted in the participants picking one of three doors. Behind two of them there was a goat (a literal goat, not a metaphorical one; a goaty goat). Behind a third door a car, of course it is always better to win a car instead of a goat.

Ok, that is a normal game... If we do not take the goats in consideration.

The interesting part is that you could pick a door and after doing so, Monty opened one of the remaining ones, always showing a proud goat.

So, here came the tricky part Did you chose to keep the door you chose or did you change your mind over to the remaining third door?

To decide whether to change or not of door became one of the mathematical celebrities of all times.
So a mathematical war started over in a section of the Sunday Newspaper.
Marilyn Vos Savant bailed out of formal education and chose a career as writer and investor, but the interesting part was that she started writing for that same newspaper after being mentioned at the World Guinness Records book for holding the highest IQ in history (228, that's like sticking Stephen Hawking's with his chain inside the Large Hadron Collider).

Leaping over the potential flaws in the IQ tests, one may believe that Marilyn was not a fool, and was asked about this problem by one of her readers, she simply said: "picking to change the door gives you double chances of winning compared to the ones you get by keeping the one you chose first".

After that was published (this was not Nature, Science... it was more like Cosmopolitan), the newspaper got more than 10000 letters. Among all of them there was around 1000 coming from proud holders of Doctorates that basically said: "If you don't know, keep your mouth shut. Chances are 50/50, to change or not is irrelevant".

From the first time I've heard about this problem; I only recall that, the argument.

Meh, I also remember the solution and correct explanation, that I had to learn to be able to reply about it the next time someone brought up the topic in class.
(SPOILER: Marilyn was right and many had to strategically relocate their doctorates and/or egos up their ass).

Some people call it the "Eureka Moment", that spark of inspiration where they understand that if A then B, and, if B then C, A then C, but I was always wary about that moment and always interpreted as a drowning man's last breath hoping to understand something as valid.

The thing with this problem is that it can be considered really strong kick to the balls to that guy called "intuition", it can even hurt the way our brain is wired.

Pick a door, you've ⅓ chances of winning. Monty uncovers a goat. Now you've one of two doors to pick from, from this one may assume that the chances are 50/50, that you chose from scratch, but this time with only two doors instead of one.
NOPE.

The secret of this bold "NOPE revolves around whom knows the truth. Whom has more information. Whom knows something that you don't, this information changes the odds.

In the exact moment where that lovely stage assistant discards a door, she does it based on something she knows. KNOWS that:

She knows something that you don't. She knows something that rebuilds the problem.

A few years ago, this problem was used to attempt to understand how our choice and judgement works, attempting to measure and understand our capacity to rationally understand our options.
What was done is offer a group of volunteers the original problem with a subtle variation. This time there was not two doors, but an increasing number between 2 and 100. Behind all of them but one there was a dollar, in a particular one $20.

Just like with the goat, you had a choice, and,just like before, the host discarded all the options but one.

This is, discarding 98.

The volunteers that at the first moment chose to keep the original door 94% of the times, now chose to repick 50% of the times; yet the amazing thing is that none could quite explain why they chose to change.

Adding difficulty, far from making it more complicated, made it clearer, not because of the interpretation of the problem itself but because it change the way it was being processed.
The interpretation given to this change has to do with working memory, this is (watered down) how do we store and manipulate information about multiple elements in a short period of time (a very short time, because short-lapse memory can hold data for hours), not like a board full of pasted pictures, but more like an incredibly active element that transforms and reshapes around that information.

Some things, are very hard to simplify down to a mortal level.

As the limit of 7 doors was surpassed, the results drastically changed, and perhaps it had to do with the fact that we interpret 7 doors as "7 doors", but 100 doors as "a shitload of doors". Without being able to individualize them correctly.

When we think of it with 99 goats, it all clears up, and the same thing happened to the volunteers.
As more options were added, the way the problem was looked at changed to something like "If I chose one out of 100 I'd have 1% chances of winning, but now that guy just discarded 98, leaving me only one option! Now it's 50%! how many chances I've of nailing the price now If I change my choice?"

Random has no memory.

Picking out of 3 doors presents us with 2 potential scenarios. In one, I keep what I've picked first, this is a 1 in 3 chances of getting a new car and 2 out of 3 of dining roasted goat. Now, in the scenario where I always switch, something great happens:
If I nail the car with my first choice, I am screwed because no matter what door they discard, I'll dine goat. BUT If I chose a goat, when I switch my choice I'll get the car. This is: I don't have to aim for the ⅓ that wins to win. I've to aim for the ⅔ chances to WIN!

Rationality and obviousness do not overlap.

Goat wins