Publications related to Extending Capabilities of Attention-based Models

Related publications (21)

ProtMamba: a homology-aware but alignment-free protein state space model

Cyril Antoine Malbranke, Anne-Florence Raphaëlle Bitbol, Damiano Sgarbossa

Motivation Protein language models are enabling advances in elucidating the sequence-to-function mapping, and have important applications in protein design. Models based on multiple sequence alignments efficiently capture the evolutionary information in ho ...

Oxford University Press2025

Meditation in the third-person perspective modulates minimal self and heartbeat-evoked potentials

Hang Yang, Olaf Blanke, Bruno Herbelin, Loup Vuarnesson

Experienced meditation practitioners often report altered states of their sense of self, including decentering and distancing the self from the body and one's current concerns. Altered states of the sense of self, such as disembodiment and distancing of th ...

Elsevier BV2025

A multi-modal transformer for predicting global minimum adsorption energy

Philippe Schwaller, Junwu Chen

The fast assessment of the global minimum adsorption energy (GMAE) between catalyst surfaces and adsorbates is crucial for large-scale catalyst screening. However, multiple adsorption sites and numerous possible adsorption configurations for each surface/a ...

NATURE PORTFOLIO2025

Uncertainty quantification for a deep learning models for image-based crack segmentation

Onaïa Joseph Savary, Katrin Beyer, Bryan German Pantoja Rosero, Ketson Roberto Maximiano Dos Santos

Recent advancements in deep learning have found compelling applications in the image-based inspection of infrastructure systems. However, the efficacy of these models hinges significantly on the quality and diversity of the input data used for training. In ...

2024

FSscore: A Personalized Machine Learning-Based Synthetic Feasibility Score

Bruno Emanuel Ferreira De Sousa Correia, Philippe Schwaller, Rebecca Manuela Neeser

Determining whether a molecule can be synthesized is crucial in chemistry and drug discovery, as it guides experimental prioritization and molecule ranking in de novo design tasks. Existing scoring approaches to assess synthetic feasibility struggle to ext ...

2024

Connectome embedding in multidimensional graph spaces

Mathieu Mach, Dimitri Nestor Alice Van De Ville, Alessandra Griffa, Maria Giulia Preti, Enrico Amico, Raphaël Pierre Liégeois

Nodal properties are of particular importance when investigating patterns in brain networks. Nodal information is usually studied by comparing a few nodal measurements (up to three), resulting in analyses in three-dimensional spaces, at maximum. We offer a ...

Massachusetts Institute of Technology2024

Computational complexity of deep learning: fundamental limitations and empirical phenomena

Ludovic Théo Stephan, Alessandro Favero

This manuscript is the lecture notes of B. Barak’s course in the Les Houches ‘Statistical Physics and Machine Learning’ summer school in 2022. It surveys various proxies for computational hardness in random planted problems, from the low-degree likelihood ...

2024

The neural correlates of topographical disorientation-a lesion analysis study

Olaf Blanke, Lukas Heydrich, Eva Blondiaux

Topographical disorientation refers to the selective inability to orient oneself in familiar surroundings. However, to date its neural correlates remain poorly understood. Here we use quantitative lesion analysis and a lesion network mapping approach in or ...

Wiley2024

Detection of Malicious Agents in Social Learning

Ali H. Sayed, Mert Kayaalp, Valentina Shumovskaia

Non-Bayesian social learning is a framework for distributed hypothesis testing aimed at learning the true state of the environment. Traditionally, the agents are assumed to receive observations conditioned on the same true state, although it is also possib ...

IEEE Institute of Electrical and Electronics Engineers2024

Detecting Anomalies and Obstacles in Road Scenes

Krzysztof Maciej Lis

We address the problem of segmenting anomalies and unusual obstacles in road scenes for the purpose of self-driving safety. The objects in question are not present in the common training sets as it is not feasible to collect and annotate examples for every ...

EPFL2023

Early decoding of walking tasks with minimal set of EMG channels

Silvestro Micera

Objective. Powered lower-limb prostheses relying on decoding motor intentions from non-invasive sensors, like electromyographic (EMG) signals, can significantly improve the quality of life of amputee subjects. However, the optimal combination of high decod ...

IOP Publishing Ltd2023

Dense Image-based Predictions for Comics Analysis

Deblina Bhattacharjee

Dense image-based prediction methods have advanced tremendously in recent years. Their remarkable development has been possible due to the ample availability of real-world imagery. While these methods work well on photographs, their abilities do not genera ...

EPFL2023

Improving Object Detection under Domain Shifts

Vidit Vidit

Object detection plays a critical role in various computer vision applications, encompassing domains like autonomous vehicles, object tracking, and scene understanding. These applica- tions rely on detectors that generate bounding boxes around known object ...

EPFL2023

Prediction of plasma volume and total hemoglobin mass with machine learning

Jean-Marc Vesin

Hemoglobin concentration ([Hb]) is used for the clinical diagnosis of anemia, and in sports as a marker of blood doping. [Hb] is however subject to significant variations mainly due to shifts in plasma volume (PV). This study proposes a newly developed mod ...

WILEY2023

Automated post-earthquake damage assessment of stone masonry buildings integrating machine learning, computer vision, and physics-based modeling

Bryan German Pantoja Rosero

Current post-earthquake damage assessment methodologies are not only time-consuming but also subjective in nature and difficult to document. Recent advancements in artificial intelligence and technological devices make it possible to accomplish this task a ...

EPFL2023

Domain knowledge-informed synthetic fault sample generation with health data map for cross-domain planetary gearbox fault diagnosis

Olga Fink

Extensive research has been conducted on fault diagnosis of planetary gearboxes using vibration signals and deep learning (DL) approaches. However, DL-based methods are susceptible to the domain shift problem caused by varying operating conditions of the g ...

ACADEMIC PRESS LTD- ELSEVIER SCIENCE LTD2023

Genetic Algorithms for the Discovery of Homogeneous Catalysts

Anne-Clémence Corminboeuf, Rubén Laplaza Solanas, Simone Gallarati, Puck Elisabeth van Gerwen, Alexandre Alain Schöpfer

In this account, we discuss the use of genetic algorithms in the inverse design process of homogeneous catalysts for chemical transformations. We describe the main components of evolutionary experiments, specifically the nature of the fitness function to o ...

2023

Electromagnetic wave-based extreme deep learning with nonlinear time-Floquet entanglement

Romain Christophe Rémy Fleury, Ali Momeni

Wave-based analog signal processing holds the promise of extremely fast, on-the-fly, power-efficient data processing, occurring as a wave propagates through an artificially engineered medium. Yet, due to the fundamentally weak non-linearities of traditiona ...

Nature Research2022

Bertraffic: Bert-Based Joint Speaker Role And Speaker Change Detection For Air Traffic Control Communications

Petr Motlicek, Juan Pablo Zuluaga Gomez

Automatic speech recognition (ASR) allows transcribing the communications between air traffic controllers (ATCOs) and aircraft pilots. The transcriptions are used later to extract ATC named entities, e.g., aircraft callsigns. One common challenge is speech ...

IEEE2022

Are socially-aware trajectory prediction models really socially-aware?

Alexandre Massoud Alahi, Seyed Mohsen Moosavi Dezfooli, Saeed Saadatnejad, Mohammadhossein Bahari

Our transportation field has recently witnessed an arms race of neural network-based trajectory predictors. While these predictors are at the core of many applications such as autonomous navigation or pedestrian flow simulations, their adversarial robustne ...

2022