Publication

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

Publications associées (29)

Text Representation Learning for Low Cost Natural Language Understanding

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

EPFL2023

Language Transformers for Remote Sensing Visual Question Answering

Devis Tuia, Christel Marie Tartini-Chappuis, Vincent Alexandre Mendez, Sylvain Lobry

Remote sensing visual question answering (RSVQA) opens new avenues to promote the use of satellites data, by interfacing satellite image analysis with natural language processing. Capitalizing on the remarkable advances in natural language processing and c ...

IEEE2022

Efficient Transformer-Based Speech Recognition

Apoorv Vyas

Training deep neural network based Automatic Speech Recognition (ASR) models often requires thousands of hours of transcribed data, limiting their use to only a few languages. Moreover, current state-of-the-art acoustic models are based on the Transformer ...

EPFL2022

SKILL: Structured Knowledge Infusion for Large Language Models

Martin Jaggi, Fedor Moiseev

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2022

Unsupervised Term Extraction for Highly Technical Domains

Diego Matteo Antognini

Term extraction is an information extraction task at the root of knowledge discovery platforms. Developing term extractors that are able to generalize across very diverse and potentially highly technical domains is challenging, as annotations for domains r ...

2022

Generalization of an Encoder-Decoder LSTM model for flood prediction in ungauged catchments

Peter Molnar, Silvan Ragettli, Olga Fink, Yi Zhang

Flood prediction in ungauged catchments is usually conducted by hydrological models that are parameterized based on nearby and similar gauged catchments. As an alternative to this process-based modelling, deep learning (DL) models have demonstrated their a ...

ELSEVIER2022

Physics-informed neural networks for diffraction tomography

Demetri Psaltis, Ahmed Ayoub, Carlo Gigli, Amirhossein Saba Shirvan

We propose a physics-informed neural network (PINN) as the forward model for tomographic reconstructions of biological samples. We demonstrate that by training this network with the Helmholtz equation as a physical loss, we can predict the scattered field ...

SPIE-SOC PHOTO-OPTICAL INSTRUMENTATION ENGINEERS2022

OptTTA: Learnable Test-Time Augmentation for Source-Free Medical Image Segmentation Under Domain Shift

Jean-Philippe Thiran, Guillaume Marc Georges Vray, Seyedbehzad Bozorgtabar, Devavrat Tomar

As distribution shifts are inescapable in realistic clinical scenarios due to inconsistencies in imaging protocols, scanner vendors, and across different centers, well-trained deep models incur a domain generalization problem in unseen environments. Despit ...

PMLR2022

Continual Test-Time Domain Adaptation

Qin Wang, Olga Fink

Test-time domain adaptation aims to adapt a source pretrained model to a target domain without using any source data. Existing works mainly consider the case where the target domain is static. However, real-world machine perception systems are running in n ...

IEEE COMPUTER SOC2022