Informing Machines about Human Mental States via EEG Decoding

QC 20260416

Abstract

Brain activity is a rich source of information about human mental states and can potentially inform interactive intelligent systems about human intentions and perception of the environment. Among non-invasive neuroimaging techniques, electroencephalography (EEG) is particularly suited for interactive applications due to its portability, relatively low cost, and real-time measurements. Advances in artificial intelligence (AI) have enabled decoding models to extract complex task-related patterns from EEG signals and reveal information related to intention and perception. However, inherent properties of EEG signals, such as low signal-to-noise ratio, low spatial resolution, and high inter-trial variability, make decoding challenging. These limitations become even more pronounced when decoding targets are high-dimensional, such as natural images. At the same time, existing research is strongly biased toward well-established and highly separable conditions, while more nuanced scenarios remain underexplored. In this thesis, we move beyond conventional EEG applications by investigating settings in which decoding targets are high-dimensional and/or elicited by nuanced conditions. We assess decoding feasibility in these demanding scenarios and propose methods that leverage pretrained models of the stimulus modality for complex targets. We begin by examining the EEG decoding pipeline and discuss how EEG and task constraints shape the architecture and representation choices of decoding models. We then integrate pretrained stimulus models as priors for predicting high-dimensional outputs and propose two complementary approaches to align stimulus representations with EEG activity. The first uses EEG responses as feedback in an online closed-loop framework to guide a pretrained generative model toward a user’s intended mental image. The second aligns EEG representations with perceptually informed embedding spaces from pretrained vision models, improving the retrieval of perceived images from EEG.To go beyond well-established paradigms, we investigate EEG-based intention decoding in demanding same-limb human–robot collaboration scenarios. We also examine how informative EEG signals are about olfactory perception. Finally, we discuss methodological and evaluation challenges in small-scale, task-based EEG datasets, including risks of performance overestimation and limited generalization.

Link to DiVA