I'm not an anarchist. There will still be a need for governments and banks (though in a much more limited capacity than today).

You would need a central authority to generate and distribute the keys. In the case of voting, the keys would be one-time use so if you showed up at the polling place and found your vote had already been cast, you would simply delete it and request a new key. You would be able to view, scan, read -- verify -- your key, of course.