well as they themselves said, they agree on the end goal which would be to have both freedom of movement and freedom from goverment
The main difference (from my interpretation) was that Adam did not wanted to compromise by maintaining the violations on freedom of movement temporarily in order to archive it as that would be inconsistent , and don't that he risks loosing both. as he makes his task harder ( even by his own admission), here Stephan was saying that by maintaining and enforcing the laws of immigration it would be possible to accomplish a smaller government and a population that understands that smaller is better, later it would be possible to get both