Violence is the use of physical force, usually to cause harm.
I believe the term you're looking for is "Abuse", which is just as egregious in my mind. Whether you damage someones body or mind is irrelevant, the perpetrator is still being an asshole.