I think that is a great idea because there is a guy who has been hitting my posts with $1+ down votes and that is a lot to me, just because he disagrees, I assume he disagrees, he never leaves a comment.
I wrote about an alternative to flagging for people who are more enlightened.