Parameter-efficient Methods for Training and Inference of Transformer-based Models

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

We present a method to control a text-to-image generative model to produce training data useful for supervised learning. Unlike previous works that employ an open-loop approach via pre-defined prompts to generate new data using either a language model or h ...

Large language models (LLMs) have gained widespread interest owing to their ability to process human language and perform tasks on which they have not been explicitly trained. However, we possess only a limited systematic understanding of the chemical capa ...

Accurate human trajectory prediction is one of the most crucial tasks for autonomous driving, ensuring its safety. Yet, existing models often fail to fully leverage the visual cues that humans subconsciously communicate when navigating the space. In this w ...

Elucidating the functional effects of missense variants is crucial yet challenging. To investigate their impact, we fine-tuned protein language models, including ESM2 and ProtT5, to classify 20 protein features at amino acid resolution. In addition, we tra ...

Adversarial robustness has been studied extensively in image classification, especially for the ℓ∞-threat model, but significantly less so for related tasks such as object detection and semantic segmentation, where attacks turn out to be a much harder opti ...

Training high-quality deep learning models is a challenging task due to computational and technical requirements. A growing number of individuals, institutions, and companies increasingly rely on pre-trained, third-party models made available in public rep ...

In this work, we investigate Speech Foundation Models (SFMs) for Parkinson's Disease (PD) detection. We explore two main approaches: (1) using SFMs as frozen feature extractors and, (2) fine-tuning/adapting SFMs for PD detection. We propose a cross-validat ...

Density-functional theory with extended Hubbard functionals (DFT + U + V) provides a robust framework to accurately describe complex materials containing transition-metal or rare-earth elements. It does so by mitigating self-interaction errors inherent to ...

Cell tracking is a ubiquitous image analysis task in live-cell microscopy. Unlike multiple object tracking (MOT) for natural images, cell tracking typically involves hundreds of similar-looking objects that can divide in each frame, making it a particularl ...

While writing peer reviews resembles an important task in science, education, and large organizations, providing fruitful suggestions to peers is not a straightforward task, as different user interaction designs of text suggestion interfaces can have diver ...

Growing demand for multi-material Laser Powder Bed Fusion (LPBF) faces process control and quality monitoring challenges, particularly in ensuring precise material composition. This study explores optical and acoustic emission signals during LPBF processes ...

Recently, generalist Vision Language Models (VLMs) have shown exceptional progress in tasks previously dominated by specialized computer vision models. This becomes more prevalent when visual grounding capabilities, such as the ability to reason over input ...

Quantifying the uncertainty of regression models is essential to ensure their reliability, particularly since their application often extends beyond their training domain. Based on the solution of a constrained optimization problem, this work proposes ‘pre ...

At the intersection of Topological Data Analysis (TDA) and machine learning, the field of cellular signal processing has advanced rapidly in recent years. In this context, each signal on the cells of a complex is processed using the combinatorial Laplacian ...

© The Author(s) 2024.Deep learning models are widely used for traffic forecasting on freeways due to their ability to learn complex temporal and spatial relationships. In particular, graph neural networks, which integrate graph theory into deep learning, h ...

Many technologically important material properties are underpinned by disorder and short-range structural correlations; therefore, elucidating structure-property relationships in functional materials requires understanding both the average and the local st ...

The capabilities of large language models (LLMs) have raised concerns about their potential to create and propagate convincing narratives.Here, we study their performance in detecting convincing arguments to gain insights into LLMs' persuasive capabilities ...

We analyze and implement the kernel ridge regression (KR) method developed in Filipovic et al. (Stripping the discount curve-a robust machine learning approach. Swiss Finance Institute Research Paper No. 22-24. SSRN. https://ssrn.com/abstract=4058150, 2022 ...

Chiral ligands are important components in asymmetric homogeneous catalysis, but their synthesis and screening can be both time-consuming and resource-intensive. Data-driven approaches, in contrast to screening procedures based on intuition, have the poten ...

Irregular sampling of time series in electronic health records (EHRs) presents a challenge for the development of machine learning models. Additionally, the pattern of missing data in certain clinical variables is not random, but depends on the decisions o ...

Parameter-efficient Methods for Training and Inference of Transformer-based Models

Chat with Graph Search

Controlled Training Data Generation with Diffusion Models

A framework for evaluating the chemical knowledge and reasoning abilities of large language models against the expertise of chemists

Social-Pose: Enhancing Trajectory Prediction with Human Body Pose

Fine-tuning protein language models to understand the functional impact of missense variants

Towards Reliable Evaluation and Fast Training of Robust Semantic Segmentation Models

Do You Trust Your Model? Emerging Malware Threats in the Deep Learning Ecosystem

Automatic Parkinson's disease detection from speech: Layer selection vs adaptation of foundation models

Machine learning Hubbard parameters with equivariant neural networks

TRACKASTRA: Transformer-Based Cell Tracking for Live-Cell Microscopy

Enhancing Peer Review with AI-Powered Suggestion Generation Assistance: Investigating the Design Dynamics

Qualify-as-you-go: sensor fusion of optical and acoustic signatures with contrastive deep learning for multi-material composition monitoring in laser powder bed fusion process

Training Visual Language Models with Object Detection: Grounded Change Descriptions in Satellite Images

A prediction rigidity formalism for low-cost uncertainties in trained neural networks

Morse theoretic signal compression and reconstruction on chain complexes

A two-level resolution neural network with enhanced interpretability for freeway traffic forecasting

Unravelling the components of diffuse scattering using deep learning

Can Language Models Recognize Convincing Arguments?

Stripping the Swiss discount curve using kernel ridge regression

Reaction-Agnostic Featurization of Bidentate Ligands for Bayesian Ridge Regression of Enantioselectivity

Point-process-based Representation Learning for Electronic Health Records