I once suggested that all down votes should require a comment as to why they felt it needed to be flagged so abusers who made that post the community could downvote them if they have seen that as abuse.

Also, it would spark up a conversation as well in which people could communicate.