What Should Replace the Turing Test?

Abstract

Today, chatbots and other artificial intelligence tools pass the Turing test, which was Turing’s alternative to trying to answer the question: can a machine think? Despite their success in passing the Turing test, these machines do not think. We therefore propose a test of a more focused question: does a program reason in the way that humans reason? This test treats an “intelligent” program as though it were a participant in a psychological study and has 3 steps: (a) test the program in a set of experiments examining its inferences, (b) test its understanding of its own way of reasoning, and (c) examine, if possible, the cognitive adequacy of the source code for the program.