A small dolphin can do only little against a plankton.
A big dolphin like some witnesses can indeed destroy plankton if he wants it enough.
Remember a downvoter controls only his downvotes and not others' upvotes, so any account can grow while being flagged by a bigger account.
As an example, this post may be flagged by a small dolphin, but it may still cause an overall increase of my reputation and Steem Power.