Yes, I agree largely to what you say. Although it is basically a good thing to help out other people ( especially those who are in need), but it is easily exploited as well.
I think the USA are the best example for that. Many people there do volunteer work to help out in problem areas that the government fails to adress. That would be ok if there would be a minimum government - but it is not.
It is just as badly bloated as governments in countries that Americans like to call "socialist", for example in western Europe or Scandinavia.

However, I dont know the answer to this problem. A stop in charity work would certainly make things worse for a lot of people, until the government does what its really there for. If it ever does that at all.