Permanently disconnecting yourself from speculators is an understandable desire, but unfortunately not a realistic one:
Currencies by their very nature, require exchanges. Both the dollar and gold trade on the global decentralized Foreign Exchange Market, like every other currency in the world.

Don't get me wrong, pushing for crypto adoption is the way to go, and is an active work in progress -- most online shops I deal with locally accept Bitcoin already, but it's simply not possible to disconnect currencies from exchanges in concept.

P.S. It should be noted that major crypto currencies have become comparatively very stable in latest months -- outside of major on-chain speculative events, dollar's moves don't affect them more than the stable FOREX currencies.