Scaling the Modalities in Multimodal Foundation Models

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Contemporary artificial intelligence (AI) models' exponentially growing complexity translates to their expanding skill sets. They can create language or image content mimicking human creativity and understanding, becoming indispensable tools in education, ...

Segmenting brain tumors is a critical task in medical imaging that relies on advanced deep-learning methods. However, effectively handling complex tumor regions requires more comprehensive and advanced strategies to overcome challenges such as computationa ...

Machine translation (MT) systems and language models (LMs) have demonstrated impressive performance in recent years, driven by advances in architecture, algorithms, and the scaling of parameters and training data. Despite the ability of these models to int ...

In this concise contribution, it is demonstrated that Recurrent Neural Networks (RNNs) based on Gated Recurrent Unit (GRU) architecture, possess the capability to learn the complex dynamics of rate-and-state friction (RSF) laws from synthetic data. The dat ...

Computing has long been central to technological advancements, where each leap in compute power has catalyzed applications once thought impossible, such as smartphones. Recently, the exponential growth in the compute demand of artificial intelligence (AI) ...

Research in speech processing has largely focused on source-system modeling, where vocal fold vibrations serve as the source and the vocal cavity's articulations as the system. Nonetheless, speech production includes a complex combination of physiological ...

In this study, we employed traditional methods and deep learning models to improve resolution and quality of low-resolution AFM images made under standard ambient scanning. Both traditional methods and deep learning models were benchmarked and quantified r ...

Sign languages are rich visual languages that use both manual features like handshape, movement, and location and non-manual features such as facial expressions, head movements, and body posture to convey information. Sign language learning technologies ha ...

Finding proper collective variables for complex systems and processes is one of the most challenging tasks in simulations, which limits the interpretation of experimental and simulated data and the application of enhanced sampling techniques. Here, we prop ...

The ESA Euclid mission will survey more than 14 000 deg2 of the sky in visible and near-infrared wavelengths, mapping the extragalactic sky to constrain our cosmological model of the Universe. Although the survey focusses on regions further than 15 from th ...

We study low-rank matrix estimation for a generic inhomogeneous output channel through which the matrix is observed. This generalizes the commonly considered spiked matrix model with homogeneous noise to include for instance the dense degree-corrected stoc ...

In supervised learning, the regularization path is sometimes used as a convenient theoretical proxy for the optimization path of gradient descent initialized from zero. In this paper, we study a modification of the regularization path for infinite-width 2- ...

We present GigaPose, a fast, robust, and accurate method for CAD-based novel object pose estimation in RGB images. GigaPose first leverages discriminative "templates", rendered images of the CAD models, to recover the out-of-plane rotation and then uses pa ...

Motivated by the substantial achievements of Large Language Models (LLMs) in the field of natural language processing, recent research has commenced investigations into the application of LLMs for complex, long-horizon sequential task planning challenges i ...

This paper presents an acoustic impedance control architecture for an electroacoustic absorber combining both feedforward and feedback microphone-based strategies on a current-driven loudspeaker. Feedforward systems enable good performance for direct imped ...

De novo protein design enhances our understanding of the principles that govern protein folding and interactions, and has the potential to revolutionize biotechnology through the engineering of novel protein functionalities. Despite recent progress in comp ...

The choice of input image size can have a significant impact on the performance of the state-of-the-art algorithms. We can always customize the algorithms by training and fine-tuning them on our datasets, but it is time consuming. Nowadays there is a trend ...

The great potential brought by large-scale data in the humanities is still hindered by the time and technicality required for making documents digitally intelligible. Within urban studies, historical cadasters have been hitherto largely under-explored desp ...

This letter investigates the universal approximation capabilities of Hamiltonian Deep Neural Networks (HDNNs) that arise from the discretization of Hamiltonian Neural Ordinary Differential Equations. Recently, it has been shown that HDNNs enjoy, by design, ...

This thesis consists of three applications of machine learning techniques to risk management. The first chapter proposes a deep learning approach to estimate physical forward default intensities of companies. Default probabilities are computed using artifi ...

Scaling the Modalities in Multimodal Foundation Models

Chat with Graph Search

Programming Light Propagation for Efficient Artificial Intelligence

LIU-NET: lightweight Inception U-Net for efficient brain tumor segmentation from multimodal 3D MRI images

From Preprocessing to Evaluation: The Role of External Knowledge in Advancing Language Models and Machine Translation

Data-driven dynamic friction models based on Recurrent Neural Networks

Optical Processors for AI Computing and Networking

Towards Integrated Processing of Physiological Signals and Speech

Deep learning for enhancement of low-resolution and noisy scanning probe microscopy images

Advancing Phonology-Based Sign Language Assessment: From Learner to Machine-Generated Videos

Inherent Structural Descriptors Via Machine Learning

Euclid: Detecting Solar System objects in Euclid images and classifying them using Kohonen self-organising maps

Low-rank matrix estimation with inhomogeneous noise

On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks

GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence

ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning

Robust direct acoustic impedance control using two microphones for mixed feedforward-feedback controller

De novo protein design by inversion of the AlphaFold structure prediction network

Enhancing Hand and Object Detection for Monitoring Patients with Upper-Limb Impairment: A Study on the Impact of Input Size in Foundation Models

Effective annotation for the automatic vectorization of cadastral maps

Universal Approximation Property of Hamiltonian Deep Neural Networks

Financial Risk Management with Machine Learning