There are so many small things to point out in this poem. The extended metaphor and the elegance of the poem as a whole are the real thing to pause over, but that speaks for itself, so I will point out and ask about small things in particular.

I will start with the one thing that gives me a bit of an uneasy pause, though. If we look at this as a metaphor for pregnancy, rather than just calling for a literal flower (or even the metaphor of love, "A Rose By Any Other Name," which I'll return to), then it's a bit "off" to look at the one who is pregnant as if the pregnancy has robbed her of her beauty.

Now, if we return to the flower as a metaphor for love, then it is an interesting image about longing for something that is never within our grasp, with the scent wafting behind once gone, with the wish to have appreciated the love while it was there. The fact the speaker never really got to enjoy it, because he missed it.
And that can be a metaphor for love, that lovers seek the promise of yesteryear, which was never truly there, rather than enjoy the fruits of their love, the sustenance-giving flesh of it.
Can also be seen as a commentary on romanticism, which values the memory and idealized image over achievable reality, no matter how sweet the reality might be. And only because they "missed" what was, do they romanticize and long for it.

"I should have stopped in admiration," which contrasts a bit the "I missed," if we take it for literally having missed it. But still, that line is arresting. It is that essence of longing in a succinct line.
And then it is complemented with the final line, which does it again.

Last comment, I would like to note how "swayed" is an image that is often related to sexual or sensual movement, and here too, what swayed in the breeze were the sexual organs, the means to reproduce. A nice touch, leading to the pregnant imagery, which then collides with "seed," that completes the male and female coupling once more.

I have to ask, is this your drawing? If so, could you show me the full version? Something about this drawing fragment in particular is drawing my eye. I can see it as a gown dress, and a knife or something out of Cinderella in the background. I can see dance, and death. I can see the sun and shadow. And I want to see it all.