Brain-Supervised Conditional Generative Modeling

Abstract

Present machine learning approaches to steer generative models rely on the availability of manual human input. We propose an alternative approach to supervising generative machine learning models by directly detecting task-relevant information from brain responses. That is, requiring humans only to perceive stimulus and react to it naturally. Brain responses of participants (N=30) were recorded via electroencephalography (EEG) while they perceived artificially generated images of faces and were instructed to look for a particular semantic feature, such as “smile” or “young”. A supervised adversarial autoencoder was trained to disentangle semantic image features by using EEG data as a supervision signal. The model was subsequently conditioned to generate images matching users' intentions without additional human input. The approach was evaluated in a validation study comparing brain-conditioned models to manually conditioned and randomly conditioned alternatives. Human assessors scored the saliency of images generated from different models according to the target visual features (e.g., which face image is more “smiling” or more “young”). The results show that brain-supervised models perform comparably to models trained with manually curated labels, without requiring any manual input from humans.