{"title":"Predictive event segmentation and representation with neural networks: A self-supervised model assessed by psychological experiments","authors":"Hamit Basgol , Inci Ayhan , Emre Ugur","doi":"10.1016/j.cogsys.2023.101167","DOIUrl":null,"url":null,"abstract":"<div><p><span>People segment complex, ever-changing, and continuous experience into basic, stable, and discrete spatio-temporal experience units, called events. The literature on event segmentation investigates the mechanisms behind this ability. Event segmentation theory points out that people predict ongoing activities and observe prediction error signals to find event boundaries. In this study, we investigated the mechanism giving rise to this ability through a computational model and accompanying psychological experiments. Inspired by event segmentation theory and </span>predictive processing<span><span>, we introduced a self-supervised model of event segmentation. This model consists of neural networks<span> that predict the sensory signal in the next time-step to represent different events, and a cognitive model that regulates these networks on the basis of their prediction errors. In order to verify the ability of our model in segmenting events, learning them during passive observation, and representing them in its representational space, we prepared a video of </span></span>human behaviors<span> represented by point-light displays. We compared the event segmentation behaviors<span> of participants and our model with this video in two granularities. Using point-biserial correlation, we demonstrated that the event boundaries of our model correlated with the responses of the participants. Moreover, by approximating the representation space of participants, we showed that our model formed a similar representation space with those of participants. The result suggests that our model that tracks the prediction error signals can produce human-like event boundaries and event representations. Finally, we discuss our contribution to the literature and our understanding of how event segmentation is implemented in the brain.</span></span></span></p></div>","PeriodicalId":55242,"journal":{"name":"Cognitive Systems Research","volume":"83 ","pages":"Article 101167"},"PeriodicalIF":2.1000,"publicationDate":"2023-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cognitive Systems Research","FirstCategoryId":"102","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1389041723001018","RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
People segment complex, ever-changing, and continuous experience into basic, stable, and discrete spatio-temporal experience units, called events. The literature on event segmentation investigates the mechanisms behind this ability. Event segmentation theory points out that people predict ongoing activities and observe prediction error signals to find event boundaries. In this study, we investigated the mechanism giving rise to this ability through a computational model and accompanying psychological experiments. Inspired by event segmentation theory and predictive processing, we introduced a self-supervised model of event segmentation. This model consists of neural networks that predict the sensory signal in the next time-step to represent different events, and a cognitive model that regulates these networks on the basis of their prediction errors. In order to verify the ability of our model in segmenting events, learning them during passive observation, and representing them in its representational space, we prepared a video of human behaviors represented by point-light displays. We compared the event segmentation behaviors of participants and our model with this video in two granularities. Using point-biserial correlation, we demonstrated that the event boundaries of our model correlated with the responses of the participants. Moreover, by approximating the representation space of participants, we showed that our model formed a similar representation space with those of participants. The result suggests that our model that tracks the prediction error signals can produce human-like event boundaries and event representations. Finally, we discuss our contribution to the literature and our understanding of how event segmentation is implemented in the brain.
期刊介绍:
Cognitive Systems Research is dedicated to the study of human-level cognition. As such, it welcomes papers which advance the understanding, design and applications of cognitive and intelligent systems, both natural and artificial.
The journal brings together a broad community studying cognition in its many facets in vivo and in silico, across the developmental spectrum, focusing on individual capacities or on entire architectures. It aims to foster debate and integrate ideas, concepts, constructs, theories, models and techniques from across different disciplines and different perspectives on human-level cognition. The scope of interest includes the study of cognitive capacities and architectures - both brain-inspired and non-brain-inspired - and the application of cognitive systems to real-world problems as far as it offers insights relevant for the understanding of cognition.
Cognitive Systems Research therefore welcomes mature and cutting-edge research approaching cognition from a systems-oriented perspective, both theoretical and empirically-informed, in the form of original manuscripts, short communications, opinion articles, systematic reviews, and topical survey articles from the fields of Cognitive Science (including Philosophy of Cognitive Science), Artificial Intelligence/Computer Science, Cognitive Robotics, Developmental Science, Psychology, and Neuroscience and Neuromorphic Engineering. Empirical studies will be considered if they are supplemented by theoretical analyses and contributions to theory development and/or computational modelling studies.