Simulating insufficient effort responding with large language models: A persona-based prompt engineering approach.

Abstract

Using large language models (LLMs) with persona-based prompt engineering, this study simulates realistic insufficient effort responding (IER) data under controlled conditions, overcoming the limitations of traditional methods in ecological validity and controllability. The core objective is to generate controlled, distinct IER and non-IER datasets, thereby improving further research on detection methods. Our strategy involved systematically manipulating persona attributes, such as behavioral descriptions and psychological attributes, to produce synthetic IER data under controlled conditions. We instructed the LLM to generate one persona condition (general) with attentive response and three IER-intended persona conditions using varying combinations of IER-associated personality traits and IER behavioral descriptions (CRDO, CRPO, CRDP). To validate this approach, we first examined differences in response patterns across persona conditions using descriptive statistics and correlation analysis. Furthermore, we conducted confirmatory factor analysis (CFA) and analyzed IER detection indices to confirm that the synthetic IER data exhibited statistical traits like real IER data. Results indicate the pattern distinction among the persona conditions. Specifically, the IER-intended conditions consistently demonstrated IER characteristics, including degraded CFA model RMSEA (CRDO: 0.09; CRPO: 0.13; CRDP: 0.12) and high mean IER detection rates (n = 60; CRDO: 52.33%, CRPO: 74.16%, CRDP: 88.66%). These findings demonstrate that persona-based prompt engineering with LLMs effectively simulates realistic human IER, providing a robust and scalable approach for generating high-quality synthetic data. This methodology establishes a foundation for future research, offering potential to enhance IER detection accuracy, strengthen psychometric validation, and advance survey methodology through improved ecological validity and methodological rigor.