You own you. I own me. Why is this so hard for people?

It just blows my mind when people have such a hard time with this concept. They always come back with some response along the lines of I'm selfish, or I just want to watch society crumble.
They imply that I want to live free of consequences.
They vaguely threaten me with gangs and militant dictatorships popping out of the woodwork.
But the thing that drives me nuts is how eager they are with their accusations of elitism on my part.
Talking about how I want to be special, above everyone else. Above THEM.

Like, no, man. I own me. And you own you as well.
I'm the proper arbiter of my destiny, and you are the proper arbiter of yours.
I'm not trying to force you to do anything, or to even agree with me or accept my philosophy.
I simply want everyone to decide for themselves. And it seems like everyone views that as some kind of huge, controversial statement, as though I announced that I would love to start microwaving babies immediately.
People are so weird.