Cognitive biases in natural language: Automatically detecting, differentiating, and measuring bias in text

Abstract

We examine preliminary results from the first automated system to detect the 188 cognitive biases included in the 2016 Cognitive Bias Codex, as applied to both human and AI-generated text, and compared to a human baseline of performance. The human baseline was constructed from the collective intelligence of a small but diverse group of volunteers independently submitting their detected cognitive biases for each sample in the task used for the first phase. This baseline was used as an approximation of the ground truth on this task, for lack of any prior established and relevant benchmark. Results showed the system’s performance to be above that of the average human, but below that of the top-performing human and the collective, with greater performance on a subset of 18 out of the 24 categories in the codex. This version of the system was also applied to analyzing responses to 150 open-ended questions put to each of the top 5 performing closed and open-source Large Language Models, as of the time of testing. Results from this second phase showed measurably higher rates of cognitive bias detection across roughly half of all categories than those observed when analyzing human-generated text. The level of model contamination was also considered for two types of contamination observed, where the models gave canned responses. Levels of cognitive bias detected in each model were compared both to one another and to data from the first phase.