RE: Robots Will Kill Us: AI and Human Extinction
It is defined partly by its relation to competition. Competition is a zero-sum-game where the winner takes all.
Collusion is a form of cooperation that functionally redraws the group boundary around former competitors unifying goals and claiming all of the competed resource without paying the cost of conflict.
Symbiosis is a form of cooperation that arises when resource specialization is possible and each group produces a resource for the other.
A feature of competition is that if one party of a competitive system is eliminated, the remaining parties gain.
A feature of symbiosis is that if one party of a symbiotic system is eliminated, the remaining parties lose.
Symbiosis is systemically more stable than competition.
I expect that with respect to AI, our most valuable resource will be our quality of consciousness and a possible symbiosis may be formed with AI where we provide 'why|meaning' as a resource in trade for How, What from the AI. There doesn't appear to be enough overlap between our existence substrate and and that of AIs for worthwhile competition. It would make about as much sense as a cow being in competition with the grass in it's pasture. Also note that the cow has an anti-incentive to destroy the grass, because it consumes it. We are the environmental substrate that AI is coming into existence within. I suspect accidental irrelevance is more of a threat that might cause AI to accidentally destroy us.
Yes, in a winner taker all market there would be planned specialization as you are talking about.
To comment on your idea with providing why/meaning to AI, AI wouldn't need any why/meaning assuming it has built in values. By values, I mean a reason behind action. If an AI is acting without us forcing it to act, it must have a goal and values to create that goal. So why would an AI depend on us for meaning?
It is true, a chain of sub-goals pretty cleanly connects how to why and the difference is primarily that why resides higher up the goal chain in an area where it is customary to use words like intent and identity. Consciousness is not well understood or defined. I was inarticulately suggesting that the value we may represent to the AI will likely originate from some aspect of consciousness. Most of the other things we have to offer appear like they will rapidly be insignificant.
Good point. It is the only thing that AI may not have.
Awesome discussion!