I ask first. I was taking a photo outside a maximum security prison last week for a post, and boy did I get yelled at by one of the guards because she might have been in one! Ya never know.
If you ask, 99 out of 100 people say yes.