Publications related to Enhanced Architectures and Optimization Methods for Efficient Language Modeling

Related publications (16)

Transferability of Learnt Speech Representations for Decoding Non-Human Vocal Communication

Humans and animals both use acoustic signals for vocal communication. The advent of self-supervised learning (SSL) has enabled neural networks to learn robust and general feature representations through the intrinsic acoustic structure of input signals, wi ...

EPFL2025

Foundation neural-networks quantum states as a unified Ansatz for multiple hamiltonians

Luciano Loris Viteritti, Giuseppe Carleo

Foundation models are highly versatile neural-network architectures capable of processing different data types, such as text and images, and generalizing across various tasks like classification and generation. Inspired by this success, we propose Foundati ...

2025

TimeGPT in load forecasting: A large time series model perspective

Fernando Porté Agel, Jiannong Fang, Wenlong Liao

Machine learning models have made significant progress in load forecasting, but their forecast accuracy is limited in cases where historical load data is scarce. Inspired by the outstanding performance of large language models (LLMs) in computer vision and ...

2025

Assessing Sample Quality via the Latent Space of Generative Models

Minh Hieu Lê

Advances in generative models increase the need for sample quality assessment. To do so, previous methods rely on a pre-trained feature extractor to embed the generated samples and real samples into a common space for comparison. However, different feature ...

Springer Science and Business Media Deutschland GmbH2025

MARBLE: interpretable representations of neural population dynamics using geometric deep learning

Pierre Vandergheynst

The dynamics of neuron populations commonly evolve on low-dimensional manifolds. Thus, we need methods that learn the dynamical processes over neural manifolds to infer interpretable and consistent latent representations. We introduce a representation lear ...

Nature Publishing Group2025

Learning the language of the genome: AI- and data-driven decoding of genetic variants and disease mechanisms

Ali Saadat Varnosfaderanii

Genetic variation plays a crucial role in human disease susceptibility and molecular pathophysiology. This thesis explores genetic determinants of disease using advanced computational approaches, including statistical modeling, network analysis, and deep l ...

EPFL2025

Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence

Matin Ansaripour

Distributed optimization is the standard way of speeding up machine learning training, and most of the research in the area focuses on distributed first-order, gradient-based methods. Yet, there are settings where some computationally-bounded nodes may not ...

Association for the Advancement of Artificial Intelligence2025

Musculoskeletal motor control with reinforcement learning

Alberto Silvio Chiappa

Animals, including humans, interact with the external environment primarily through motion. Replicating their motor control skills in artificial embodied agents is a major objective of artificial intelligence research. This thesis presents a collection of ...

EPFL2025

Intelligent Soft Robotic System for Sensing and Recognition via Triboelectric‐Based Multi‐Sensor Fusion

Xin Chen

Triboelectric nanogenerators (TENGs) serve as a key technology in flexible electronic devices, especially tactile sensors, enabling flexible robotic manipulators to effectively interact with their external environment. Persistent challenges include inadequ ...

Wiley2025

Topics in statistical physics of high-dimensional machine learning

Hugo Chao Cui

In the past few years, Machine Learning (ML) techniques have ushered in a paradigm shift, allowing the harnessing of ever more abundant sources of data to automate complex tasks. The technical workhorse behind these important breakthroughs arguably lies in ...

EPFL2024

Training a Filter-Based Model of the Cochlea in the Context of Pre-Trained Acoustic Models

Philip Neil Garner, Louise Clothilde Coppieters De Gibson

Auditory research aims in general to lead to understanding of physiological processes. By contrast, the state of the art in automatic speech processing (notably recognition) is dominated by large pre-trained models that are meant to be used as black-boxes. ...

2024

Anatomy-Informed Multimodal Learning for Myocardial Infarction Prediction

Pascal Frossard, Emmanuel Abbé, Ntorina Thanou, Ortal Yona Senouf, Ivan-Daniel Sievering

Goal: In patients with coronary artery disease, the prediction of future cardiac events such as myocardial infarction (MI) remains a major challenge. In this work, we propose a novel anatomy-informed multimodal deep learning framework to predict future MI ...

2024

The influence of data on adversarial robustness

Javier Alejandro Maroto Morales

Deep learning models are generally vulnerable to small perturbations, raising concerns about safety, security, and performance guarantees. Achieving robustness has been shown to be costly in terms of computational resources and quantity of data. This thesi ...

EPFL2024

Implicit phonetic information modeling for speech emotion recognition

Tilak Purohit

This study investigates the efficacy of utilizing embedding spaces to model phonetic information in emotion utterances for speech emotion recognition. Our approach involves implicit modeling of phone information by deriving phone-based embeddings from netw ...

Isca-Int Speech Communication Assoc2023

EFaR 2023: Efficient Face Recognition Competition

Hatef Otroshi Shahreza

This paper presents the summary of the Efficient Face Recognition Competition (EFaR) held at the 2023 International Joint Conference on Biometrics (IJCB 2023). The competition received 17 submissions from 6 different teams. To drive further development of ...

IEEE2023

Stable parameterization of continuous and piecewise-linear functions

Michaël Unser, Alexis Marie Frederic Goujon, Joaquim Gonçalves Garcia Barreto Campos

Rectified-linear-unit (ReLU) neural networks, which play a prominent role in deep learning, generate continuous and piecewise-linear (CPWL) functions. While they provide a powerful parametric representation, the mapping between the parameter and function s ...

2023