The burden of proof lies with whoever is making the claim. We're all born not believing anything. The idea that there is a god has to be suggested to us by someone else and the burden of proof lies with them. An atheist is just someone that has rejected the claim. Rejection entails no burden of proof. It just means that the one making the claim has failed to convince.
They do not need to prove what they suggest, unless they attempt to pass some law against you for believing what you believe instead of what they believe, or some other attempt at curtailing your rights. You do not have to prove them wrong, unless you ..... The issue is, allow people to believe what they wish and refrain from imposing your beliefs on them.
I think he was saying in the context of a debate, they are the ones who have to prove it.