I attempt to go around and if they follow me I approach to offer them a reward for helping me to accomplish the task that the Mayor has given me.

After going through all of this I'd tried to convince them to join the Defensive Union so they can help the village.