Yeah, I agree.

But still, often time hate crime might be an apartment neighbor harassing their Muslim or gay neighbor or whatever, in non-violent, even "technically allowed" ways, but in ways that due to the way society is set up, still majorly inconveniences the person being harassed.

It gets worse when it might be tough for this person to find allies.