Dynamics of scope ambiguities: comparative analysis of human and large language model performance in Korean

Abstract

This study investigates how native Korean speakers and large language models (LLMs) resolve scope ambiguities and integrate them with discourse information, focusing on interactions between negation and quantificational phrases (QPs). The objectives were twofold: (i) to determine whether the general preference for surface scope interpretations and integration with discourse information persists in complex syntactic constructions in Korean, which require refined processing, and (ii) to assess how well LLMs comprehend and integrate semantic structures compared with human performance. The results showed a preference for surface scope among Korean speakers but did not rigidly hold against the inverse scope, particularly influenced by object QPs or long-form negation, even when contexts favor an inverse scope. LLMs developed by OpenAI—GPT-3.5 Turbo, GPT-4 Turbo, and GPT-4o—align with human judgments, mainly favoring surface scope interpretations when contexts favor the inverse scope. However, when the context supports an inverse scope, discrepancies in the handling of syntactic nuances are evident. This model tends to overgeneralize the inverse scope in specific configurations in which humans typically find the inverse scope more accessible. These findings highlight the challenges of mimicking human linguistic processing and the need for further refinement of language models to improve their interpretive accuracy.