Publications associées à Vision Transformer Adapters for Generalizable Multitask Learning

Modeling Structured Data in Attention-based Models

Natural language processing has experienced significant improvements with the development of Transformer-based models, which employ self-attention mechanism and pre-training strategies. However, these models still present several obstacles. A notable issue ...

EPFL2023

One-shot Garbage Collection for In-memory OLTP through Temporality-aware Version Storage

Anastasia Ailamaki, Angelos Christos Anadiotis, Periklis Chrysogelos, Syed Mohammad Aunn Raza

Most modern in-memory online transaction processing (OLTP) engines rely on multi-version concurrency control (MVCC) to provide data consistency guarantees in the presence of conflicting data accesses. MVCC improves concurrency by generating a new version o ...

ACM2023

Linear Complexity Self-Attention With 3rd Order Polynomials

Filippos Kokkinos, Grigorios Chrysos

Self-attention mechanisms and non-local blocks have become crucial building blocks for state-of-the-art neural architectures thanks to their unparalleled ability in capturing long-range dependencies in the input. However their cost is quadratic with the nu ...

Ieee Computer Soc2023

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

EPFL2023

Dense Image-based Predictions for Comics Analysis

Deblina Bhattacharjee

Dense image-based prediction methods have advanced tremendously in recent years. Their remarkable development has been possible due to the ample availability of real-world imagery. While these methods work well on photographs, their abilities do not genera ...

EPFL2023

Concurrency control for database theorists

Christoph Koch, Bas Ketsman

The aim of this paper is to serve as a lightweight introduction to concurrency control for database theorists through a uniform presentation of the work on robustness against Multiversion Read Committed and Snapshot Isolation. ...

ASSOC COMPUTING MACHINERY2022

Remote sensing visual question answering with a self-attention multi-modal encoder

Devis Tuia, Joao Magalhaes

Visual Question Answering (VQA) on remote sensing imagery can help non-expert users in extracting information from Earth observation data. Current approaches follow a neural encoder-decoder design, combining convolutional and recurrent encoders together wi ...

2022

Efficient Transformer-Based Speech Recognition

Apoorv Vyas

Training deep neural network based Automatic Speech Recognition (ASR) models often requires thousands of hours of transcribed data, limiting their use to only a few languages. Moreover, current state-of-the-art acoustic models are based on the Transformer ...

EPFL2022

STATNet: Spatial-temporal attention in the traffic prediction

Alexandre Massoud Alahi, Amin Gheibi, Seyed Mohamad Moghadas

Recent traffic flow prediction methods are lacking abilities to determine predictive features. Thus, they will propagate the error in the next timestamps. In this paper, first, we assess the role of spatial and temporal features on the traffic speed predic ...

2022

On the Relationship between Self-Attention and Convolutional Layers

Jean-Baptiste Francis Marie Juliette Cordonnier, Andreas Loukas, Martin Jaggi

Recent trends of incorporating attention mechanisms in vision have led re- searchers to reconsider the supremacy of convolutional layers as a primary build- ing block. Beyond helping CNNs to handle long-range dependencies, Ramachandran et al. (2019) showed ...

2020

Fast Transformers with Clustered Attention

François Fleuret, Angelos Katharopoulos, Apoorv Vyas

Transformers have been proven a successful model for a variety of tasks in sequence modeling. However, computing the attention matrix, which is their key component, has quadratic complexity with respect to the sequence length, thus making them prohibitivel ...

2020

Visual Correspondences for Unsupervised Domain Adaptation on Electron Microscopy Images

Pascal Fua, Mathieu Salzmann, Carlos Joaquin Becker, Róger Bermúdez Chacón, Okan Altingövde

We present an Unsupervised Domain Adaptation strategy to compensate for domain shifts on Electron Microscopy volumes. Our method aggregates visual correspondences—motifs that are visually similar across different acquisitions—to infer changes on the parame ...

Institute of Electrical and Electronics Engineers2020

Context-Aware Attention Mechanism for Speech Emotion Recognition

Philip Neil Garner, Gaétan Ramet

In this work, we study the use of attention mechanisms to enhance the performance of the state-of-the-art deep learning model in Speech Emotion Recognition (SER). We introduce a new Long Short-Term Memory (LSTM)-based neural network attention model which i ...

IEEE2018

Program Analysis and Compilation Techniques for Speeding up Transactional Database Workloads

Mohammad Dashti Rahmat Abadi

There is a trend towards increased specialization of data management software for performance reasons. The improved performance not only leads to a more efficient usage of the underlying hardware and cuts the operation costs of the system, but also is a ga ...

EPFL2017

Sequential Proximity: Towards Provably Scalable Concurrent Search Algorithms

Rachid Guerraoui, Vasileios Trigonakis, Karolos Antoniadis, Julien Michel Stainer

Establishing the scalability of a concurrent algorithm a priori, before implementing and evaluating it on a concrete multi-core platform, seems difficult, if not impossible. In the context of search data structures however, according to all practical work ...

Springer International Publishing2017

Universally Scalable Concurrent Data Structures

Tudor Alexandru David

The increase in the number of cores in processors has been an important trend over the past decade. In order to be able to efficiently use such architectures, modern software must be scalable: performance should increase proportionally to the number of all ...

EPFL2017

Transactions Chasing Scalability and Instruction Locality on Multicores

Pinar Tözün

For several decades, online transaction processing (OLTP) has been one of the main server applications that drives innovations in the data management ecosystem, and in turn the database and computer architecture communities. Recent hardware trends oblige s ...

EPFL2014

Scaling up Mixed Workloads: a Battle of Data Freshness, Flexibility, and Scheduling

Anastasia Ailamaki, Iraklis Psaroudakis

The common "one size does not fit all" paradigm isolates transactional and analytical workloads into separate, specialized database systems. Operational data is periodically replicated to a data warehouse for analytics. Competitiveness of enterprises today ...

Springer International Publishing AG2015

Scaling up analytical queries with column-stores

Anastasia Ailamaki, Manoussos Gavriil Athanassoulis, Ioannis Alagiannis

As data analytics is used by an increasing number of applications, data analytics engines are required to execute workloads with increased concurrency, i.e., an increasing number of clients submitting queries. Data management systems designed for data anal ...

2013

Sharing Data and Work Across Concurrent Analytical Queries

Anastasia Ailamaki, Manoussos Gavriil Athanassoulis, Iraklis Psaroudakis

Today's data deluge enables organizations to collect massive data, and analyze it with an ever-increasing number of concurrent queries. Traditional data warehouses (DW) face a challenging problem in executing this task, due to their query-centric model: ea ...

2013