From my own experience, we have the power to control every aspect of our lives with our minds. It's not cliche, it's a fact. If you don't want to deal with unwanted judgments for whatever you do, just don't think about that and keep doing your thing. As simple as that. Think about what you want, not about what you don't want. Doing this will save you many problems and conflicts.