Scrolling down is no problem to me. As I do it anyway while reading. This maybe a problem for someone who just wants to hit UPVOTE, without actually reading the complete post.
Upvoting directly from the feed shall be not encouraged, as in general this is blind voting, not voting for a good content