On the Relationship between Self-Attention and Convolutional Layers

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

The evolution of airborne mapping witnesses the introduction of hybrid lidar-camera systems to enhance data collection, i.e. to obtain simultaneously high-density point-cloud and texture. Yet, the common adjustment of both optical data streams is a nontriv ...

This paper presents a comparison of machine learning (ML) methods used for three-dimensional localization of partial discharges (PD) in a power transformer tank. The study examines ML and deep learning (DL) methods, ranging from support vector machines (SV ...

Neural networks have shown to be a powerful tool to represent the ground state of quantum many-body systems, including fermionic systems. However, efficiently integrating lattice symmetries into neural representations remains a significant challenge. In th ...

Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data. However, there is a notable absence of benchmark evaluations and theoretical guarantees for searching these robus ...

The process of protein aggregation, central to neurodegenerative diseases like Huntington’s, is challenging to study due to its unpredictable nature and relatively rapid kinetics. Understanding its biomechanics is crucial for unraveling its role in disease ...

Modern neuroscience research is generating increasingly large datasets, from recording thousands of neurons over long timescales to behavioral recordings of animals spanning weeks, months, or even years. Despite a great variety in recording setups and expe ...

Cyclic peptides are attractive for drug discovery due to their excellent binding properties and the potential to cross cell membranes. However, by far not all cyclic peptides are cell permeable, and measuring or predicting their membrane permeability is no ...

Machine learning (ML) enables artificial intelligent (AI) agents to learn autonomously from data obtained from their environment to perform tasks. Modern ML systems have proven to be extremely effective, reaching or even exceeding human intelligence. Altho ...

Low-light photography produces images with low signal-to-noise ratios due to limited photons. In such conditions, common approximations like the Gaussian noise model fall short, and many denoising techniques fail to remove noise effectively. Although deep- ...

The desire and ability to place AI-enabled applications on the edge has grown significantly in recent years. However, the compute-, area-, and power-constrained nature of edge devices are stressed by the needs of the AI-enabled applications, due to a gener ...

Current transformer-based skeletal action recognition models tend to focus on a limited set of joints and low-level motion patterns to predict action classes. This results in significant performance degradation under small skeleton perturbations or changin ...

Video DeepFakes are fake media created with Deep Learning (DL) that manipulate a person’s expression or identity. Most current DeepFake detection methods analyze each frame independently, ignoring inconsistencies and unnatural movements between frames. Som ...

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

End-to-end learning methods like deep neural networks have been the driving force in the remarkable progress of machine learning in recent years. However, despite their success, the deployment process of such networks in safety-critical use cases, such as ...

According to the proposed Artificial Intelligence Act by the European Comission (expected to pass at the end of 2023), the class of High-Risk AI Systems (Title III) comprises several important applications of Deep Learning like autonomous driving vehicles ...

In the past few years, Machine Learning (ML) techniques have ushered in a paradigm shift, allowing the harnessing of ever more abundant sources of data to automate complex tasks. The technical workhorse behind these important breakthroughs arguably lies in ...

The ability to reason, plan and solve highly abstract problems is a hallmark of human intelligence. Recent advancements in artificial intelligence, propelled by deep neural networks, have revolutionized disciplines like computer vision and natural language ...

The use of meteorological radars to study snowfall microphysical properties and processes is well established, in particular via a few distinct techniques: the use of radar polarimetry, of multi-frequency radar measurements, and of the radar Doppler spectr ...

Recent advancements in deep learning have revolutionized 3D computer vision, enabling the extraction of intricate 3D information from 2D images and video sequences. This thesis explores the application of deep learning in three crucial challenges of 3D com ...

On the Relationship between Self-Attention and Convolutional Layers

Chat with Graph Search

On the Perspectives of Image-to-Lidar Constraints in Dynamic Network Optimisation

Partial discharge localization in power transformer tanks using machine learning methods

Spectroscopy of two-dimensional interacting lattice electrons using symmetry-aware neural backflow transformations

Robust NAS under adversarial training: benchmark, theory, and beyond

Self-driving microscopy detects the onset of protein aggregation and enables intelligent Brillouin imaging

Robust machine learning for neuroscientific inference

Bulk measurement of membrane permeability for random cyclic peptides in living cells to guide drug development

Few-shot Learning for Efficient and Effective Machine Learning Model Adaptation

Dark Noise Diffusion: Noise Synthesis for Low-Light Image Denoising

Exploring High-Performance and Energy-Efficient Architectures for Edge AI-Enabled Applications

MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning

Efficient Temporally-Aware DeepFake Detection using H.264 Motion Vectors

Transformer Models for Vision

Random matrix methods for high-dimensional machine learning models

Estimating and Improving the Robustness of Attributions in Text

Robust Training and Verification of Deep Neural Networks

Topics in statistical physics of high-dimensional machine learning

Infusing structured knowledge priors in neural models for sample-efficient symbolic reasoning

Dual-frequency spectral radar retrieval of snowfall microphysics: a physics-driven deep-learning approach

Advancing Self-Supervised Deep Learning for 3D Scene Understanding