Personally I keep all the key information and the Portable PGP software inside a Veracrypt encrypted volume hidden away in an generically named folder.

Remember, it's not paranoia if someone is actually out to get you!