But, I understand that it's hard to find free time for this (every day follow updates of friends and participate in discussions).
And auto-vote in a sense is an opportunity to say Hello, to express respect for your relationship. Accordingly,
if you withdrew from somebody list you have lost the interest of this person to you. So your actions in this case are absolutely correct.