I'd have said you should leave him for a while but who knows when you leave he'll bring another lady to the house and before you know it, you're out of the house.
So I'll advice you to just pray... God never sits and watch while his children suffers