Human misidentification in Turing tests

• Published in: Journal of experimental & theoretical artificial intelligence Vol. 27; no. 2; pp. 123 - 135
• Main Authors: Warwick, Kevin, Shah, Huma
• Format: Journal Article
• Language: English
• Published: Abingdon Taylor & Francis 04.03.2015; Taylor & Francis Ltd
• Subjects: Artificial intelligence; chatbots; confederate effect; Expert systems; Foils; Human; imitation game; intelligence; Judgment; Parks; philosophy of mind; practical Turing tests; Strategy; Systems design; Text messaging; Bletchley England
• ISSN: 0952-813X; 1362-3079
• DOI: 10.1080/0952813X.2014.921734

This paper presents some important issues on misidentification of human interlocutors in text-based communication during practical Turing tests. The study here presents transcripts in which human judges succumbed to theconfederate effect, misidentifying hidden human foils for machines. An attempt is made to assess the reasons for this. The practical Turing tests in question were held on 23 June 2012 at Bletchley Park, England. A selection of actual full transcripts from the tests is shown and an analysis is given in each case. As a result of these tests, conclusions are drawn with regard to the sort of strategies which can perhaps lead to erroneous conclusions when one is involved as an interrogator. Such results also serve to indicate conversational directions to avoid for those machine designers who wish to create a conversational entity that performs well on the Turing test.