For some reason as I read your insightful observations, I thought of Larry Niven's Ringworld Engineers for its spin on human failure. My thought is that even "older/classic" SciFi depicted imperfections, but there is a there has been a shift in the mix of placing ones hopes in human ingenuity/spirit vs. some vague concept of the scientific method or technological possibilities. My theory is that as our scientific knowledge grows over the years and decades, the slow progress leaves less room for hand-waving that any problem will be easily/quickly solvable by technology. Yet this is curiously counterbalanced by the paradox of popular faith in socialized solutions. By this, I mean the sentiment from mass media that if we could only all cooperate by using less energy and taking less showers, then we can forestall global climate doom and gloom. Here's to a raft of good Sci Fi to take my mind off political posturing IRL.