That's where I think that you are wrong. Becoming successful and setting goals is as good for me as it is good for the whole of humanity.
If I am to give you 1% of what I have right now for the helping cause of solving poverty, I would give you around 0.2 cents, which is nowhere close to making a difference.
But in time, as success is assured, my 1% can become 100.000k $ which will be a great addition to the cause of solving world hunger.
Think Smart, before thinking Honorably but non productive.
And of course, I don't mean to turn your backs on those to need help. Always help them any way you can, but don't make that your primary goal before you are successful and content with your own Life.