I know, it really can be. It took me a while to build up followers aswell. The best advice I can give you is to just keep on writing and, most important (especially at the start), join in on conversations. Write good comments. This will get people to notice you and click on your name to check out your profile.

If the conversation is good enough, some people might just follow you just for that!