Improving speech embedding using crossmodal transfer learning with audio-visual data

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

This thesis focuses on understanding and improving the reasoning capabilities of neural networks. It develops theoretical results and empirical analyses to uncover reasoning potential and limitations, leveraging these insights to guide the design of improv ...

In this thesis we explore the applications of projective geometry, a mathematical theory of the relation between 3D scenes and their 2D images, in modern learning-based computer vision systems. This is an interesting research question which contradicts the ...

When a plasma disrupts in a tokamak, significant heat and electromagnetic loads are deposited onto the surrounding device components. These forces scale with plasma current and magnetic field strength, making disruptions one of the key challenges for futur ...

The current machine learning paradigm relies on continuous representations and fixed neural network architectures to approximate environmental structures, leading to challenges with continual learning, internal structure design, and goal-directed behavior ...

Machine learning (ML) enables artificial intelligent (AI) agents to learn autonomously from data obtained from their environment to perform tasks. Modern ML systems have proven to be extremely effective, reaching or even exceeding human intelligence. Altho ...

The evolution of airborne mapping witnesses the introduction of hybrid lidar-camera systems to enhance data collection, i.e. to obtain simultaneously high-density point-cloud and texture. Yet, the common adjustment of both optical data streams is a nontriv ...

Monitoring forests, in particular their response to climate and land use change, requires studying long time scales. While efficient deep learning methods have been developed to process short time series of satellite imagery, leveraging long time series of ...

A range of behavioral and contextual factors, including eating and drinking behavior, mood, social context, and other daily activities, can significantly impact an individual's quality of life and overall well-being. Therefore, inferring everyday life aspe ...

The process of protein aggregation, central to neurodegenerative diseases like Huntington’s, is challenging to study due to its unpredictable nature and relatively rapid kinetics. Understanding its biomechanics is crucial for unraveling its role in disease ...

Visual affordance segmentation identifies image regions of an object an agent can interact with. Existing methods re-use and adapt learning-based architectures for semantic segmentation to the affordance segmentation task and evaluate on small-size dataset ...

Current transformer-based skeletal action recognition models tend to focus on a limited set of joints and low-level motion patterns to predict action classes. This results in significant performance degradation under small skeleton perturbations or changin ...

Earth scientists study a variety of problems with remote sensing data, but they most often consider them in isolation from each other, which limits information flows across disciplines. In this work, we present METEOR, a meta-learning methodology for Earth ...

Federated learning (FL) is a machine learning approach where nodes collaboratively train a global model. As more nodes participate in a round of FL, the effectiveness of individual model updates by nodes also diminishes. In this study, we increase the effe ...

Data augmentation is vital in deep learning for enhancing model robustness by artificially expanding training datasets. However, advanced methods like CutMix blend images and assign labels based on pixel ratios, often introducing label noise by neglecting ...

Social behaviors such as cooperation are crucial for mammals. A deeper knowledge of the neuronal mechanisms underlying cooperation can be beneficial for people suffering from pathologies with impaired social behavior. Our aim was to study the brain activit ...

Web search engines process billions of queries daily, making the balance between computational efficiency and ranking quality crucial. While neural ranking models have shown impressive performance, their computational costs, particularly in feature extract ...

Over the course of history, the relationship between cities and their waters has shown different gradients of interweaving, marked by cycles of bonding and distancing. Following a period of complete neglect of urban watercourses, the versatile, multifacete ...

When arranged in a crossbar configuration, resistive memory devices can be used to execute Matrix-Vector Multiplications (MVMs), the most dominant operation of many Machine Learning (ML) algorithms, in constant time complexity. Nonetheless, when performing ...

Monitoring the health of complex industrial assets is crucial for safe and efficient operations. Health indicators that provide quantitative real-time insights into the health status of industrial assets over time serve as valuable tools for, e.g., fault d ...

Students learn more when they are actively engaged in the learning process. While hands-on activities, labs and projects are moments when students are active, the learning benefits can be amplified with coaching strategies. This activity will enable studen ...

Improving speech embedding using crossmodal transfer learning with audio-visual data

Chat with Graph Search

Learning to Reason with Neural Networks: Principles and Methods

Incorporating Projective Geometry into Deep Learning

Plasma state monitoring and disruption characterization using multimodal VAEs

Modelleyen: Continual Learning and Planning via Structured Modelling of Environment Dynamics

Few-shot Learning for Efficient and Effective Machine Learning Model Adaptation

On the Perspectives of Image-to-Lidar Constraints in Dynamic Network Optimisation

Multi-temporal forest monitoring in the Swiss Alps with knowledge-guided deep learning

Generalization and Personalization of Machine Learning for Multimodal Mobile Sensing in Everyday Life

Self-driving microscopy detects the onset of protein aggregation and enables intelligent Brillouin imaging

Segmenting Object Affordances: Reproducibility and Sensitivity to Scale

MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning

Meta-learning to address diverse Earth observation problems across resolutions

Harnessing Increased Client Participation with Cohort-Parallel Federated Learning

SaliencyMix plus : Noise-Minimized Image Mixing Method With Saliency Map in Data Augmentation

Involvement of prelimbic cortex neurons and related circuits in the acquisition of a cooperative learning by pairs of rats

Cost-Efficiency Trade-offs for Neural Cascade Rankers in Web Search

Quartiers rhodaniens en transition

LIONHEART: A Layer-based Mapping Framework for Heterogeneous Systems with Analog In-Memory Computing Tiles

Learning Informative Health Indicators Through Unsupervised Contrastive Learning

How to support students to develop coaching and peer teaching skills