It only goes down if the post is downvoted below a 0.00 payout. As long as the post makes something then the net effect on the user's reputation is still up instead of down.