First of all: Your problem is identical to the one faced by all people who generally consider themselves to be: (1) Libertarians (2) Classical Liberals (3) Anarchists (4) Minarchists (5) True Democrats (6) True Republicans (7) Generally Pro-American (8)Individualist (9) Capitalist (10) Constitutionalist. I could continue, but the terms would gradually become more and more "loosely-related." As it is, people's myopic focus on "how they are different" causes them to use different terms for inessential differences.

For this reason, freedom is never attained by creating "issue lists." Or "further categorizing groups according to inessential differences." Issue positions are the stuff of utility to the myopic, totalitarian "major parties" who benefit from arbitrary demographic divisions and "infighting."

The average "libertarian" or "voluntaryist" is going to look at the prior list in paragraph 1, and start "splitting hairs" about how the prior networks are mutually-exclusive. They need not be, but conventional "voluntaryists" or, still worse, "agorists" will claim that they are, because they also have an incorrect model of reality. In the agorist's model of reality, an idealized form of "human philosophy" exists in everyone's mind, and "those who have not yet become agorists simply haven't been exposed to a good-enough explanation of the ideas yet." ...But this will never happen, because the laws of cybernetics (the study of goal-directed systems that includes political systems) forbid it. The amount of information required to "change one's label to agorist" is simply too large, for too little easily-perceived gain. Too few network nodes are capable of making that transition, when transitions exist that seemingly offer "all of the pay-off(s)" at a lesser research cost.

But what is the essential difference that means a Democrat can't work with a Libertarian for constructive change? Answering that question requires them to ask "How are we legitimately similar?" This requires the voluntaryist to ask "How might a Democrat also believe in the same things that make my philosophical position as a voluntaryist legitimate?" ie: "How might the two labels be reconciled to the point where they share compatible goals?"

Most Democrats will claim that they don't want to send people to prison for "victimless crimes." A self-described libertarian will typically hold a stronger version of the same position: "I will refuse to cooperate so long as you continue to use my money to imprison people for victimless crimes." Additionally, the libertarian will have some comprehension of a philosophical framework that explains WHY it's a bad idea to punish people for victimless crimes. Their economic framework/comprehension/perspective/focus might pose the idea that it's "wasteful" and/or "diminishes systemic wealth" to punish people that way.

The prior is not enough for the voluntaryist, so, in typical "libertarian" fashion, he proceeds to produce "a hair to be split." The voluntaryist seeks out a deeper explanation, one which encompasses more of reality, and is consistent with more of human knowledge (specifically, philosophy, natural law, economics, and morality). Perhaps the voluntaryist, (such as myself) is an atheist, and wishes to reconcile his views with biologically-evolved nature. (When voluntaryists are unaware of the great body of research done on sociopaths, they tend to utter inanities such as "human nature" ...as if all humans had a similar "default" moral and psychological nature. Biology now tells us this is not remotely true.)

In any case, a typical libertarian's "economic perspective" will strike the Democrat as "amoral" or "emotionally tone-deaf," if it is explained in isolation from proper morality. The Democrat will view the issue as a moral issue, from a direct personal perspective. "I've smoked pot, and I don't belong in jail. How dare the cops attack me for doing something so harmless!" The Democrat's view will strike the libertarian as myopic, and "unprincipled." The libertarian then typically concludes that "political problems are caused by a lack of sufficient principles" or "short-sighted pragmatism."

But this pitting of "practicality" versus "principle" is a red herring. It's a distraction from the essentials, with no legitimate possible outcome. In fact, the democrat's focus on "practical outcomes" reveals a legitimate criticism of the libertarian position: it is unable to be implemented, so long as it is unattractively presented.

If the "Democrat" is in the Democratic Party leadership, however, they may not even comprehend the moral angle at all. (Which further encourages libertarians to simply discount mainstream party voters as "unprincipled.") Instead, "political leaders" will try to map the issue onto "race" or "racial demographics." Why? Because the leadership of all political parties is primarily comprised of (1) Sociopaths who are willing to lie to demographics whose overlapping memberships are large enough to get them elected (because this is the only effective means by which sociopaths can obtain political power). and (2) the unphilosophical people they manipulate into supporting them, via incentives(offered rewards) and disincentives(threats). Spooner called the prior groups of manipulated non-politician civilians "Dupes, knaves, and cowards." The state's sociopaths try can fool most idiots, but the ones they cannot fool they try to "buy off" or "cow into submission with threats."

It's not just "Democrats" and "Libertarians" that seem to have irreconcilable differences. Given a myopic view, the groups in paragraph 1 above seem very different. They seem like distinct groups. But what might possibly unite them?

Shared History (most people believe that government power should be limited, in some way, and understand that "totalitarianism" and "totalitarian law" is theoretically illegitimate).

Shared Biology (most people are empaths, not sociopaths).

America was actually designed to be a voluntaryist nation.