I think the number of followers does matter, or actually the number of interested followers. I do follow people that I want to see back in my feed as a means to filter out the rubbish if I do not have alot of time and I am sure a good number of people do the same thing. But of course just having a huge number of random followers in itself is not beneficial per definition, or even hard to achieve really.