I think the problem with prejudice is way more deeper and complex. If a person is bad, it goes to the person itself, not because other attributes. This is the dangerous of labeling.

The more dangerous and complicated things is that mass media have a massive influence on putting an agenda, a hegemony of certain ideas, drives the public opinion.

If we read about semiology, we will find that a word could have a different meaning other than the definitive one we find on dictionary. Some words could have connotation and context, that brings certain feeling toward the object.