Towards everyday brain-computer interfacing through decoding visual cognition

Decoding visual cognition from non-invasive measurements of brain activity has shown valuable applications. Vision-based Brain-Computer Interfaces (BCI) systems extend from spellers to database search and spatial navigation. Despite the high performance of these systems, they are limited to the laboratory environment. Several attempts were made to improve the quality of life of patients with severe neurodegenerative disorders by helping them acquire more independence. Bringing vision-based BCI to everyday life would benefit a wide range of population from handicapped to healthy people. However, real-life applications impose additional challenges on obtaining neural signals with a good signal-to-noise ratio. The natural environments are rich, dynamic and ambiguous that challenges the visual perception and, therefore, the decoding of underlying neural correlates. The limited attentional resources require to explore and visually sample the environment by freely moving eyes and fixating on the relevant objects. Moreover, everyday activities are rarely isolated leading to a mixture and complex interactions between neural correlates in the measured signal. In this thesis, I explore various aspects of decoding brain activity by developing new protocols for realistic scenarios with challenging visual tasks. The tasks include free visual exploration, dynamic scenes, and multiple tasks, for example, recognition of a billboard category during simulated driving. The major efforts have been directed towards synchronous decoding of Event-Related Potentials (ERP) acquired by the means of electroencephalography (EEG) and Eye-tracking technology, which allowed the extraction of Eye-Fixation Related Potentials (EFRP). Additionally, I investigate the neural correlates of perceptual decision making which is tightly linked to visual recognition. The perceptual challenges can lead to higher temporal variability of obtained ERP which was also addressed from the decoding perspective. The obtained results, firstly, provide new insights on how to tackle the temporal variability of ERP. Secondly, they raise new questions on studying and decoding neural correlates of perceptual decision making. And thirdly, they show the feasibility of decoding visual recognition in a simulated realistic scenario of car driving. Although, as expected, I found a lower performance compared to classical ERP protocols, these findings hold promise and raise new questions to be investigated in order to improve the quality of decoding visual cognition for everyday application.