I think it is a good suggestion, especially removing the votes from inactive users to make the votes more relevant (although deciding who is an inactive user could be tricky). Most voting systems are prone to biases one way or another, so I am curious about the reason they have adopted this particular system because as you say it is far from perfect (as most thing tend to be when it is still relatively new).