Habit of under-inclusion and Sentient AI.

Is it possible - or even likely - that we are setting the bar for artificial intelligence way too high? That is, are we applying standards to AIs that are likely to miss the first actually sentient computer programs?

I think we might be.

I do not agree, of course, that the chatbot-based toys we have today are sentient; I'm not sure anyone does. I also don't agree that specialized chess, go, or Jeopardy! playing programs are sentient. But let's set them aside and consider the criteria that we have applied in past for full inclusion in the community of sentience.

Historical standards relegated all sorts of sentient entities to a second-class status. Women. People of color. Gay and trans people. Atheists, even: for centuries, the biblical proverb "a fool says in his heart 'there is no god'" was used to declare atheists outside full inclusion in the human community. Atheists' sentience was ipso facto impaired in political theory and in political practice.

We seem inclined to err on the side of underinclusion, and to rejecting the consequences of sentience even when sentience is utterly obvious and impossible to deny.

Cases of overinclusion seem fewer by contrast. Various religions have attributed sentience (and activity) to dead people, in the form of saints, heroes, demigods, and the like. But "dead people" is a unique and comparatively risk-free category to overinclude. Dead people won't actually get up and correct your inference of sentience on their behalf. Nor can they do anything to falsify it. (I mean, if being dead were not enough already, there's doubtless nothing else that they can do.)

I infer then that there is a substantial likelihood that we are working with standards for AI sentience that are probably too demanding. In what respect, it remains to be seen. I don't think the analogy does much to enlighten on that question. Still, asking questions in this direction seems prudent.