Wouldn't it be easier to tell them next time that you're married but your husband is a very busy man so he can't travel with you as much as he wanted to? :D I think it would avoid these awkward situation :) But God bless that woman! She doesn't mind to share her husband with you and it would even be a honor for her :) You must have left a very good impression :)

Thank you for sharing and happy travels!