Good suggestion and I doubt we are too far down any development path (the reputation system in particular is severely underdeveloped).

One thing to keep in mind is that despite the scary red flag a downvote that doesn't drive rewards all the way to zero is just a function of reducing net upvotes (and net increase to reputation). It does not reduce reputation.