Sometimes people confuse those who practice the Non Aggression Principle (NAP) as people that can just be walked over.
In judo the practitioner does not seek out a great offence, rather he adds to his opponents force. When someone charges a judo practitioner, the judo person steps to the side and merely pushes the opponent in the same direction he was always going.
This is not aggression. This is protecting oneself by redirecting the offensive momentum of your attacker. It is very ethical and even compassionate.
The aggression in the NAP is about forcing yourself on another persons will. You cannot use someone's force against them unless they first consentually attack you.
Never be afraid to ethically protect yourself.