People have the right to downvote as much as those who have the right to upvote. in fact, the very first thing you see when flagging is the rules for flagging, the very first thing being 'disagreement on rewards'. It should not be taboo to flag for legit reasons