Absolutely, and it goes both ways! People can actually smell "bad" to each other if their immune systems are genetically incompatible.

I think a lot of times when we see someone who's considered "conventionally attractive" but doesn't do anything for us it's probably because we're subconsciously looking for more complementary genes.