Same thing is happening with me. I am receiving downvotes from some newly created accounts which have no voting weight. These accounts mostly created by spammers and scammers. So, we should not worry about them.