MultiMAE: Multi-modal Multi-task Masked Autoencoders

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Recent progress in computer vision has been driven by a simple yet transformative observation: the potential of 30-year-old neural networks can be unlocked by massively scaling their parameters and the amount of human-labeled training data. This breakthrou ...

Training data mixtures greatly impact the generalization performance of large language models. Existing domain reweighting methods often rely on costly weight computations and require retraining when new data is introduced. To this end, we introduce a flex ...

Deep learning has shown remarkable potential for industrial applications, particularly in predictive maintenance and condition monitoring. A key challenge in this domain is prognostics, specifically predicting the Remaining Useful Life (RUL) of assets - a ...

Machine learning models (e.g., neural networks) achieve high accuracy in wind power forecasting, but they are usually regarded as black boxes that lack interpretability. To address this issue, the paper proposes a glass-box model that combines high accurac ...

Proteins are the molecular machines of life, driving essential biological processes such as enzymatic catalysis, cellular signaling, and immune responses. The ability to design entirely new proteins with desired structures and functions has immense implica ...

The current machine learning paradigm relies on continuous representations and fixed neural network architectures to approximate environmental structures, leading to challenges with continual learning, internal structure design, and goal-directed behavior ...

Decentralized learning (DL) is an emerging approach that enables nodes to collaboratively train a machine learning model without sharing raw data. In many application domains, such as healthcare, this approach faces challenges due to the high level of hete ...

Web search engines process billions of queries daily, making the balance between computational efficiency and ranking quality crucial. While neural ranking models have shown impressive performance, their computational costs, particularly in feature extract ...

Data augmentation is vital in deep learning for enhancing model robustness by artificially expanding training datasets. However, advanced methods like CutMix blend images and assign labels based on pixel ratios, often introducing label noise by neglecting ...

This thesis focuses on understanding and improving the reasoning capabilities of neural networks. It develops theoretical results and empirical analyses to uncover reasoning potential and limitations, leveraging these insights to guide the design of improv ...

Monitoring forests, in particular their response to climate and land use change, requires studying long time scales. While efficient deep learning methods have been developed to process short time series of satellite imagery, leveraging long time series of ...

We introduce BLAP, a model capable of generating high-quality captions for music. BLAP leverages a fine-tuned CLAP audio encoder and a pre-trained Flan-T5 large language model. To achieve effective cross-modal alignment between music and language, BLAP uti ...

Modern neuroscience research is generating increasingly large datasets, from recording thousands of neurons over long timescales to behavioral recordings of animals spanning weeks, months, or even years. Despite a great variety in recording setups and expe ...

Semi-supervised semantic segmentation methods leverage unlabeled data by pseudo-labeling them. Thus the success of these methods hinges on the reliability of the pseudo-labels. Existing methods mostly choose high-confidence pixels in an effort to avoid err ...

In this thesis we explore the applications of projective geometry, a mathematical theory of the relation between 3D scenes and their 2D images, in modern learning-based computer vision systems. This is an interesting research question which contradicts the ...

In the realm of point cloud scene understanding, particularly in indoor scenes, objects are arranged following human habits, resulting in objects of certain semantics being closely positioned and displaying notable inter-object correlations. This can creat ...

When arranged in a crossbar configuration, resistive memory devices can be used to execute Matrix-Vector Multiplications (MVMs), the most dominant operation of many Machine Learning (ML) algorithms, in constant time complexity. Nonetheless, when performing ...

Low-light photography produces images with low signal-to-noise ratios due to limited photons. In such conditions, common approximations like the Gaussian noise model fall short, and many denoising techniques fail to remove noise effectively. Although deep- ...

The present work proposes a framework for nonlinear model order reduction based on a Graph Convolutional Autoencoder (GCA-ROM). In the reduced order modeling (ROM) context, one is interested in obtaining real -time and many-query evaluations of parametric ...

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimen ...

MultiMAE: Multi-modal Multi-task Masked Autoencoders

Chat with Graph Search

Exploiting Representation Similarities in Self-Supervised Learning for Vision Tasks

CHAMELEON: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning

Overcoming Distribution Shifts and Imbalance Challenges in Representation Learning for Deep Regression Models

Explainable modeling for wind power forecasting: A Glass-Box model with high accuracy

Computational design of complex protein folds and soluble analogs of membrane proteins

Modelleyen: Continual Learning and Planning via Structured Modelling of Environment Dynamics

Fair Decentralized Learning

Cost-Efficiency Trade-offs for Neural Cascade Rankers in Web Search

SaliencyMix plus : Noise-Minimized Image Mixing Method With Saliency Map in Data Augmentation

Learning to Reason with Neural Networks: Principles and Methods

Multi-temporal forest monitoring in the Swiss Alps with knowledge-guided deep learning

Bootstrapping Language-Audio Pre-training for Music Captioning

Robust machine learning for neuroscientific inference

Weighting Pseudo-labels via High-Activation Feature Index Similarity and Object Detection for Semi-supervised Segmentation

Incorporating Projective Geometry into Deep Learning

Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchange

LIONHEART: A Layer-based Mapping Framework for Heterogeneous Systems with Analog In-Memory Computing Tiles

Dark Noise Diffusion: Noise Synthesis for Low-Light Image Denoising

A graph convolutional autoencoder approach to model order reduction for parametrized PDEs

The multimodality cell segmentation challenge: toward universal solutions