Publications associées à Jointly Optimizing Sensing Pipelines for Multimodal Mixed Reality Interaction

Publications associées (35)

Transferability of Learnt Speech Representations for Decoding Non-Human Vocal Communication

Humans and animals both use acoustic signals for vocal communication. The advent of self-supervised learning (SSL) has enabled neural networks to learn robust and general feature representations through the intrinsic acoustic structure of input signals, wi ...

EPFL2025

Foundation neural-networks quantum states as a unified Ansatz for multiple hamiltonians

Luciano Loris Viteritti, Giuseppe Carleo

Foundation models are highly versatile neural-network architectures capable of processing different data types, such as text and images, and generalizing across various tasks like classification and generation. Inspired by this success, we propose Foundati ...

2025

TimeGPT in load forecasting: A large time series model perspective

Fernando Porté Agel, Jiannong Fang, Wenlong Liao

Machine learning models have made significant progress in load forecasting, but their forecast accuracy is limited in cases where historical load data is scarce. Inspired by the outstanding performance of large language models (LLMs) in computer vision and ...

2025

Enhanced Architectures and Optimization Methods for Efficient Language Modeling

Amirkeivan Mohtashami

This thesis investigates methods to construct more capable models more efficiently, focusing on two aspects: improved architectures and optimization. We examine principled architectural modifications that reduce computational costs or introduce features fo ...

EPFL2025

TTool: A Supervised Artificial Intelligence-Assisted Visual Pose Detector for Tool Heads in Augmented Reality Woodworking

Naravich Chutisilp, Yves Weinand, Julien Gamerro, Andrea Settimi, Florian Aymanns

We present TimberTool (TTool v2.1.1), a software designed for woodworking tasks assisted by augmented reality (AR), emphasizing its essential function of the real-time localization of a tool head’s poses within camera frames. The localization process, a fu ...

2024

Out-of-body illusion induced by visual-vestibular stimulation

Estelle Nakul, Olaf Blanke, Bruno Herbelin, Sophie Jacqueline Andrée Betka, Florian Lance, Hsin-Ping Wu

Out-of-body experiences (OBEs) are characterized by the subjective feeling of being located outside one's physical body and perceiving one's own body from an elevated perspective looking downwards. OBEs have been correlated with abnormal integration of bod ...

Cell Press2023

Generalization and Personalization of Machine Learning for Multimodal Mobile Sensing in Everyday Life

Lakmal Buddika Meegahapola

A range of behavioral and contextual factors, including eating and drinking behavior, mood, social context, and other daily activities, can significantly impact an individual's quality of life and overall well-being. Therefore, inferring everyday life aspe ...

EPFL2024

Customizing the human-avatar mapping based on EEG error related potentials during avatar-based interaction

Olaf Blanke, José del Rocio Millán Ruiz, Ronan Boulic, Ricardo Andres Chavarriaga Lozano, Bruno Herbelin, Fumiaki Iwane, Thibault Serge Mario Porssut

Objective. A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside th ...

2024

Chaotic microcomb inertia-free parallel ranging

Tobias Kippenberg, Johann Emmeram Riemensberger, Anton Lukashchuk, Anton Stroganov

Ever growing pixel acquisition rates in the fields of augmented reality, autonomous driving and robotics have increased interest in solid state beam scanning without moving parts. Modern photonic integrated laser ranging advances towards passive beam steer ...

AIP Publishing2023

Immersive VR for upper-extremity rehabilitation in patients with neurological disorders: a scoping review

Silvestro Micera

Background Neurological disorders, such as stroke and chronic pain syndromes, profoundly impact independence and quality of life, especially when affecting upper extremity (UE) function. While conventional physical therapy has shown effectiveness in provid ...

BioMed Central2024

Practical considerations of diffusion-weighted MRS with ultra-strong diffusion gradients

André Döring

IntroductionDiffusion-weighted magnetic resonance spectroscopy (DW-MRS) offers improved cellular specificity to microstructure-compared to water-based methods alone-but spatial resolution and SNR is severely reduced and slow-diffusing metabolites necessita ...

Frontiers Media Sa2023

Physical Body Orientation Impacts Virtual Navigation Experience and Performance

Olaf Blanke, Emanuela De Falco, Hyukjun Moon, Hsin-Ping Wu

Most human navigation studies in MRI rely on virtual navigation. However, the necessary supine position in MRI makes it fundamentally different from daily ecological navigation. Nonetheless, until now, no study has assessed whether differences in physical ...

Soc Neuroscience2023

Additively manufactured stretchable zipping electrostatic actuators

Giulio Grasso

The use of soft and stretchable materials allows the development of adaptive robotic systems and human-machine interfaces that are more natural and comfortable to interact with. One of the application fields that benefits the most from these compliant mate ...

EPFL2024

Investigating neural resource allocation in the sensorimotor control of extra limbs

Giulia Dominijanni

The rise of robotic body augmentation brings forth new developments that will transform robotics, human-machine interaction, and wearable electronics. Extra robotic limbs, although building upon restorative technologies, bring their own set of challenges i ...

EPFL2024

An immersive virtual reality system for ecological assessment of peripersonal and extrapersonal unilateral spatial neglect

Olaf Blanke, Andrea Serino, Roberta Ronchi, Ravi Tej Sai Giridhar Tadi

BackgroundUnilateral spatial neglect (USN) is a debilitating neuropsychological syndrome that often follows brain injury, in particular a stroke affecting the right hemisphere. In current clinical practice, the assessment of neglect is based on old-fashion ...

BioMed Central2023

Effects of Head-Tracking Artefacts on Externalization and Localization in Azimuth With Binaural Wearable Devices

Hervé Lissek, Gilles André Courtois, Vincent Pierre Olivier Grimaldi

Head tracking combined with head movements have been shown to improve auditory externalization of a virtual sound source and contribute to the performance in localization. With certain technically constrained head-tracking algorithms, as can be found in we ...

Audio Engineering Soc2023

Training a Filter-Based Model of the Cochlea in the Context of Pre-Trained Acoustic Models

Philip Neil Garner, Louise Clothilde Coppieters De Gibson

Auditory research aims in general to lead to understanding of physiological processes. By contrast, the state of the art in automatic speech processing (notably recognition) is dominated by large pre-trained models that are meant to be used as black-boxes. ...

2024

SORI: A softness-rendering interface to unravel the nature of softness perception

Hwayeong Jeong, Jamie Paik, Mustafa Mete

Tactile perception of softness serves a critical role in the survival, well-being, and social interaction among various species, including humans. This perception informs activities from food selection in animals to medical palpation for disease detection ...

National Academy of Sciences2024

Automating the Design of Programmable Interconnect for Reconfigurable Architectures

Stefan Nikolic

With Moore's law coming to an end, increasingly more hope is being put in specialized hardware implemented on reconfigurable architectures such as Field-Programmable Gate Arrays (FPGAs). Yet, it is often neglected that these architectures themselves experi ...

EPFL2023

Human-Robot Swarm Interaction: An Explorative Path to Foster Complex Systems Understanding

Hala Khodr

Order, regularities, and patterns are ubiquitous around us. A flock of birds maneuvering in the sky, the self-organization of social insects, a global pandemic or a traffic jam are examples of complex systems where the macroscopic patterns arise from the m ...

EPFL2023