I agree in general, in principle, but not everyone is a fighter, and evil could be small or very large.
Taking down a big evil (crime?) organization takes time and preparation, it's a job. So it can't be done by volunteers.
Maybe it could be done in a more decentralized way, or with private security, instead of statal / federal, but it would always be a delegation of power.
Yup, it takes numbers, and time to understand and coordinate an effort. It can be outsources better to something that is more accountable, as long as people hold it accountable, otherwise it's the same mess of abdication and power concentrated. The real solution is everyone being involved to hold each other accountable and deal with the issue as a real community.