Yes, just click on the upvote again and your vote will be changed to 0 weight.

Edit: But what have you misunderstood? Upvoting the post does nothing with the comments. The comments what are count in this poll. For e.g. I UV the post to make it more visible, and will vote for the comments later (bookmarked).