I think flagging should stay, but we need to reinforce that it's for abuse not disagreement. It would be interesting if downvoting actually risked SP too, so if you try to downvote something and the community comes to defend it, you permanently lose some stake. The premise is that consensus exists about what's abusive, and that SP held by supporters of that consensus outweighs SP held by those who oppose it. Given the nature of Steem, I think if either of those is false it will require forking people out until they're true. The network can't survive without sufficient consensus on both its technical and social protocols.