Apprehending relational Events: The visual world paradigm and the Interplay of event perception and language.

Abstract

When we observe the world, we appreciate not only the colors, shapes, and textures of people and objects, but also how they interact with one another (e.g., in events such as a girl pushing a boy). Although it might seem intuitive that extracting events and other relations would require active effort and multiple fixations, a growing body of vision research suggests that humans rapidly and automatically extract relational information-including the structure of events (i.e., who is acting on whom)-from a single fixation. These findings suggest that aspects of event structure can often be perceived without extensive visual interrogation. Yet despite this progress, much remains unknown about how visual events are perceived and represented-particularly for complex events (e.g., those involving roles beyond Agent and Patient, or events with multiple salient construals, such as chase vs. flee)-and how these emerging representations interact with language during interpretation and production of utterances about events. The visual world paradigm (VWP) offers a powerful tool to address these questions about the perception-language interface, by revealing which event representations are active when and by probing how language may guide event construal in real-time. We review eyetracking work in VWP studies of language comprehension and language production, as well as in related tasks, that provide initial insights into online event apprehension. This work suggests that (1) event apprehension and linguistic encoding are closely coordinated, interacting earlier and more continuously than previously recognized, and (2) fixations may serve to refine or disambiguate relational information extracted during initial processing- such as identifying event participants or clarifying their roles (e.g., as Instruments, Goals, or Recipients)-with language in some cases guiding attentional prioritization towards certain event components. More generally, this perspective offers a foundation for future VWP research exploring the dynamic relationship between seeing, listening, and speaking.