Because when something happens to me, I use I

Using you instead of I implies not taking responsability for his feelings, as if it was a generic person.

And why does he get to use the "it's not natural" argument and when he does it it's fine? Since you are so clever @desmoind you can probably explain that to me