Recherche

Afficher tous les résultats pour

Accueil

Publication

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

À propos
Confidentialité
Mentions légales

Graph Chatbot

Publications associées (38)

Page 2 sur 2

STREAMING TENSOR TRAIN APPROXIMATION

Daniel Kressner

Tensor trains are a versatile tool to compress and work with high-dimensional data and functions. In this work we introduce the streaming tensor train approximation (STTA), a new class of algorithms for approximating a given tensor ' in the tensor train fo ...

Siam Publications2023

Deep Generative Models for Autonomous Driving: from Motion Forecasting to Realistic Image Synthesis

Saeed Saadatnejad

Forecasting is a capability inherent in humans when navigating. Humans routinely plan their paths, considering the potential future movements of those around them. Similarly, to achieve comparable sophistication and safety, autonomous systems must embrace ...

EPFL2023

Robustness of Local Predictions in Atomistic Machine Learning Models

Michele Ceriotti, Federico Grasselli, Chiheb Ben Mahmoud, Sanggyu Chong

Machine learning (ML) models for molecules and materials commonly rely on a decomposition of the global target quantity into local, atom-centered contributions. This approach is convenient from a computational perspective, enabling large-scale ML-driven si ...

Amer Chemical Soc2023

Efficient Learning of a Linear Dynamical System with Stability Guarantees

Daniel Kuhn, Wouter Jongeneel, Tobias Sutter

We propose a principled method for projecting an arbitrary square matrix to the non- convex set of asymptotically stable matrices. Leveraging ideas from large deviations theory, we show that this projection is optimal in an information-theoretic sense and ...

2023

VETIM: Expanding the Vocabulary of Text-to-Image Models only with Text

Sabine Süsstrunk, Radhakrishna Achanta, Mahmut Sami Arpa, Martin Nicolas Everaert

Text-to-image models, such as Stable Diffusion, can generate high-quality images from simple textual prompts. With methods such as Textual Inversion, it is possible to expand the vocabulary of these models with additional concepts, by learning the vocabula ...

BMVA2023

Efficient Transformer-Based Speech Recognition

Apoorv Vyas

Training deep neural network based Automatic Speech Recognition (ASR) models often requires thousands of hours of transcribed data, limiting their use to only a few languages. Moreover, current state-of-the-art acoustic models are based on the Transformer ...

EPFL2022

SKILL: Structured Knowledge Infusion for Large Language Models

Martin Jaggi

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2022

Supervised learning and inference of spiking neural networks with temporal coding

Ana Stanojevic

The way biological brains carry out advanced yet extremely energy efficient signal processing remains both fascinating and unintelligible. It is known however that at least some areas of the brain perform fast and low-cost processing relying only on a smal ...

EPFL2023

Robust Outlier Rejection for 3D Registration with Variational Bayes

Mathieu Salzmann, Jiancheng Yang, Zheng Dang, Zhen Wei

Learning-based outlier (mismatched correspondence) rejection for robust 3D registration generally formulates the outlier removal as an inlier/outlier classification problem. The core for this to be successful is to learn the discriminative inlier/outlier f ...

Ieee Computer Soc2023

Unsupervised Term Extraction for Highly Technical Domains

Diego Matteo Antognini

Term extraction is an information extraction task at the root of knowledge discovery platforms. Developing term extractors that are able to generalize across very diverse and potentially highly technical domains is challenging, as annotations for domains r ...

2022

Generalization of an Encoder-Decoder LSTM model for flood prediction in ungauged catchments

Olga Fink

Flood prediction in ungauged catchments is usually conducted by hydrological models that are parameterized based on nearby and similar gauged catchments. As an alternative to this process-based modelling, deep learning (DL) models have demonstrated their a ...

ELSEVIER2022

Text Representation Learning for Low Cost Natural Language Understanding

Jan Frederik Jonas Florian Mai

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

EPFL2023

Physics-informed neural networks for diffraction tomography

Demetri Psaltis, Carlo Gigli, Amirhossein Saba Shirvan

We propose a physics-informed neural network (PINN) as the forward model for tomographic reconstructions of biological samples. We demonstrate that by training this network with the Helmholtz equation as a physical loss, we can predict the scattered field ...

SPIE-SOC PHOTO-OPTICAL INSTRUMENTATION ENGINEERS2022

Reinforcement learning for occupant-centric operation of residential energy system

François Maréchal, Dolaana Khovalyg, Amirreza Heidari

Occupant behavior is a highly stochastic phenomenon, which is known as a key challenge for the optimal control of residential energy systems. With the increasing share of renewable energy in the building sector, the volatile nature of renewable energy is a ...

TU Delft2022

Continual Test-Time Domain Adaptation

Olga Fink

Test-time domain adaptation aims to adapt a source pretrained model to a target domain without using any source data. Existing works mainly consider the case where the target domain is static. However, real-world machine perception systems are running in n ...

IEEE COMPUTER SOC2022

OptTTA: Learnable Test-Time Augmentation for Source-Free Medical Image Segmentation Under Domain Shift

Jean-Philippe Thiran, Seyedbehzad Bozorgtabar, Guillaume Marc Georges Vray, Devavrat Tomar

As distribution shifts are inescapable in realistic clinical scenarios due to inconsistencies in imaging protocols, scanner vendors, and across different centers, well-trained deep models incur a domain generalization problem in unseen environments. Despit ...

PMLR2022

Optimization Over Banach Spaces: A Unified View on Supervised Learning and Inverse Problems

Shayan Aziznejad

In this thesis, we reveal that supervised learning and inverse problems share similar mathematical foundations. Consequently, we are able to present a unified variational view of these tasks that we formulate as optimization problems posed over infinite-di ...

EPFL2022

Language Transformers for Remote Sensing Visual Question Answering

Devis Tuia, Vincent Alexandre Mendez, Christel Marie Tartini-Chappuis

Remote sensing visual question answering (RSVQA) opens new avenues to promote the use of satellites data, by interfacing satellite image analysis with natural language processing. Capitalizing on the remarkable advances in natural language processing and c ...

IEEE2022