AI OUTPERFORMS 43 SE GRADUATES IN CREATING CAUSAL LOOP DIAGRAMS OF JANIS GROUPTHINK PHENOMENON

Abstract

We explore the capacity of transformer-based LLMs augmented on the service side with math tools, web-based search, and internal and/or interactive “chat” scaffolding (AugLLM) to generate and analyze Causal Loop Diagrams (CLDs) from unstructured natural text. Using ChatGPT—an augmented reflective generative pre-trained transformer—we address a graduate test question on groupthink, a social theory of group decision-making as written by Janis (1982), for which we have 43 student responses. We prompted the model in three distinct roles: “over-achieving student,” “mid-B student,” and “polymath.” Each role elicited varying levels of causal abstraction, with the initial output unconstrained in format to ensure maximum computational resources are allocated to CLD elucidation. The responses were internalized into Annotated Directed Graphs for analysis.

We compare the AI-generated CLDs against 43 human-produced diagrams and a reference model using Borda aggregation of loop counts. Our findings indicate that the AugLLM adjusts its performance to match the role prompt, producing near-expert-level diagrams in the over-achiever mode and more measured outputs in the mid-B mode, while the polymath mode yields the most comprehensive diagrams, far exceeding the prompt's minimum requirements. The AugLLM in the role of quiz taker had no access to student-generated artifacts. This remarkable outcome was made possible by a novel compositional Pipeline Algebra (PA) to manage a multi-stage process described in this paper.

The PA orchestrates chat-based prompt engineering, structured output generation, and higher-order operations. It is a reproducible framework for expressing and automating complex chat-based workflows. This work demonstrates the potential of advanced LLMs for scalable detection and extraction of Causal Loop Diagrams from unstructured text. More simply stated, we are integrating math and AI to augment Systems Thinking.

This work demonstrates the potential of advanced LLMs for scalable causal analysis of unstructured text, while our pipeline algebra provides a reproducible framework to express and automate complex, chat-based workflows.