You can't please everyone, and you must be content with yourself first and foremost. I learned that lesson the hard way, to long at jobs I hated before realizing I wasn't living and the management were taking advantage financially. It's not worth it. It is scary as hell making such a drastic move, but tell yourself there is no such thing as security, then ask yourself if you trust management to keep you secure. That helps make up your mind real quick!
Best wishes to you, and remember to enjoy your life-you only get one.