This is a very interesting theory and as you say hard to really test or prove although the anecdotal evidence seems quite powerful. Even if we did assume a real basis to this idea I think any kind of challenging situation could provide a similar spur. For example if our civilisation decided to commit itself to colonise space because we believed there was an imminent threat of the earth being destroyed (within a time frame which made it feasible) I'm sure we would get similar rates of advancement. It is a case of necessity pushing people to strive harder I think.
Yeap. This is why I Neil Tyson said that a potential space race with China (much like with Russian during the cold war) will put us on Mars on no time.
The point though is the cost. There is definately a tipping point in war though where everything can result in total extinction.
I think that is why war is such a big driver - if you are worried you could be wiped out cost goes out of the window!
There is a book I read a long time ago it is a kind of spoof in the form of a fake government document called The Report from Iron Mountain and puts forth the case for how the US would cope with a lasting period of peace!
It covers some of these issues. A lot of people believed it to be real but I think it is satire although it is written in the very dry style of a real government document.