Learning computationally efficient static word and sentence representations

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

This study integrates deep learning and geospatial analysis to enhance soil loss estimation in the Moulouya Watershed, a region prone to erosion due to diverse topography and climatic conditions. Traditional models like the Universal Soil Loss Equation (US ...

Natural language processing has experienced significant improvements with the development of Transformer-based models, which employ self-attention mechanism and pre-training strategies. However, these models still present several obstacles. A notable issue ...

The digitization of 3D deformable objects remains a significant challenge in computer graphics and vision, particularly in the accurate modeling of garments. Garments exhibit complex shape variability, non-rigid deformations, and frequent self-occlusion, m ...

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

Machine learning is increasingly integrated with decision making by powering automated systems and informing human decision makers in applications such as CV screening, portfolio allocation, loan risk assessment, and insurance approval. In this thesis, I e ...

Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-constrained devices, requiring practitioners to compress them to smaller net ...

Jointly estimating hand and object shape facilitates the grasping task in human-to-robot handovers. Relying on handcrafted prior knowledge about the geometric structure of the object fails when generalising to unseen objects, and depth sensors fail to dete ...

This paper examines how the European press dealt with the no-vax reactions against the Covid-19 vaccine and the dis- and misinformation associated with this movement. Using a curated dataset of 1786 articles from 19 European newspapers on the anti-vaccine ...

To date, galaxy image simulations for weak lensing surveys usually approximate the light profiles of all galaxies as a single or double Sérsic profile, neglecting the influence of galaxy substructures and morphologies deviating from such a simplified param ...

The success of self-supervised learning in computer vision and natural language processing has motivated pretraining methods on tabular data. However, most existing tabular self-supervised learning models fail to leverage information across multiple data t ...

The optimistic gradient method is useful in addressing minimax optimization problems. Motivated by the observation that the conventional stochastic version suffers from the need for a large batch size on the order of O(ϵ-2) to achieve an ϵ-stationary solut ...

In this dissertation, we propose multiple methods to improve transfer learning for pretrained language models (PLMs). Broadly, transfer learning is a powerful technique in natural language processing, where a language model is first pre-trained on a data-r ...

MammAlps is a multimodal and multi-view dataset of wildlife behavior monitoring. Nine camera-traps were placed at three monitoring sites in the Swiss National Park, from which we curated over 14 hours of video with audio, 2D segmentation maps and 8.5 hours ...

Abstractive summarization has seen big improvements in recent years, mostly due to advances in neural language modeling, language model pretraining, and scaling models and datasets. While large language models generate summaries that are fluent, coherent, ...

A robust anomaly detection mechanism should possess the capability to effectively remediate anomalies, restoring them to a healthy state, while preserving essential healthy information. Despite the efficacy of existing generative models in learning the und ...

This paper introduces TACOSS a text-image alignment approach that allows explainable land cover semantic segmentation by directly integrating semantic concepts encoded from texts. TACOSS combines convolutional neural networks for visual feature extraction ...

Integer linear programming (ILP) is an elegant approach to solve linear optimization problems, naturally described using integer decision variables. Within the context of physics-inspired machine learning (ML) applied to chemistry, we demonstrate the relev ...

Recent successes in deep learning for vision and natural language processing are attributed to larger models but come with energy consumption and scalability issues. Current training of digital deep-learning models primarily relies on backpropagation that ...

Given a ground-level query image and a geo-referenced aerial image that covers the query’s local surroundings, fine-grained cross-view localization aims to estimate the location of the ground camera inside the aerial image. Recent works have focused on dev ...

Learning computationally efficient static word and sentence representations

Chat with Graph Search

Text Representation Learning for Low Cost Natural Language Understanding

Application of Deep Learning and Geospatial Analysis in Soil Loss Risk in the Moulouya Watershed, Morocco

Modeling Structured Data in Attention-based Models

Neural Garment Modeling

Transformer Models for Vision

Machine Learning in Decision-Making Systems: Fairness, Robustness, and Data Bias

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

Stereo Hand-Object Reconstruction for Human-to-Robot Handover

Examining European Press Coverage of the Covid-19 No-Vax Movement: An NLP Framework

Euclid preparation: LXVII. Deep learning true galaxy morphologies for weak lensing shear bias calibration

XTab: Cross-table Pretraining for Tabular Transformers

Diffusion Stochastic Optimization for Min-Max Problems

Improving Generalization of Pretrained Language Models

MammAlps: A multi-view video behavior monitoring dataset of wild mammals in the Swiss Alps

Interpretable Representation Learning and Evaluation for Abstractive Summarization

Self-Supervised Anomaly Segmentation via Diffusion Models with Dynamic Transformer UNet

Text as a Richer Source of Supervision in Semantic Segmentation Tasks

Integer Linear Programming for Unsupervised Training Set Selection in Molecular Machine Learning

Backpropagation-free training of deep physical neural networks

Adapting Fine-Grained Cross-View Localization to Areas Without Fine Ground Truth