I do agree that there are still people very much against "people like you", if I may phrase it like that. There's no denying. Those who have a real problem, they stick out, obviously. Hundred pass by, one yells. Guess who'll be remembered.
Though, I do assume most Germans do not have a problem with muslim people, people wearing hijabs, foreigners, "dark skinned people", or whatever. The only reason people "stare" at you is, because they think "Oh, I don't see that often around here".
I, for one, also "stare" at strangers quite often, but not because I "don't accept or tolerate them", but rather out of curiosity on their quirks, behaviours, or stylistic choice.
As long as a woman wearing a Hijab is allowed to take it off, if she pleases to, then I, personally, have no problem with that.
Don't interpret too much into people's stares and looks.
People just like to look at anyone, so do I. ;)
Edit:
No one smiled at me back.
And, no, smiling back to strangers is usually not common. At least, I wouldn't call it common.
Eye contact is a different topic for discussion and interaction. Again, I would doubt they mean harm in it.
Yes, I know most of Germans are open minded on this topic and therefore I feel comfortable in this country. But if you were me, who heard a lot of "different stories" coming from friends who look like me being insulted verbally and (in some cases even) physically, not only on the street but also in an academic context, where a professor told one student to take off her hijab in front of other students, your instinct would tell you different thing if someone "starres". My instinct oft told me "Oke, this is the time when you will get insulted just like anybody else". So what I think is not about "the most Germans who are open minded" but "the minority of them who just don`t like people like me".