I get beat up about every three weeks. I can’t get enough battle fortresses built to defend all my planets. I can usually hurt my attacker, but alas I lose every time. I would think that as in most battles, defense and defensive structures would have a greater impact on the attacker.
Everyday spend all your resources to build defense and resources you can't use fleet save them. As the attacked win nothing when they destroy defenses.