Lecture

Transformers in Vision

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture covers the concept of Transformers in computer vision, focusing on the 'Attention Is All You Need' architecture. It explains the Transformer Encoder and Decoder, multi-head self-attention, positional encoding, and the use of transformers for visual intelligence tasks. The lecture also discusses the application of transformers in vision tasks, such as image recognition and dense prediction. Various transformer types, including Vision Transformers (ViT), Dense Prediction Transformers (DPT), and BERT Pre-Training of Image Transformers (BEIT), are explored. Additionally, it delves into the concepts of masked autoencoders, parallel decoding, and the role of transformers in high-resolution image synthesis.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Related concepts (32)

Computer vision

Computer vision tasks include methods for , , and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images (the input to the retina in the human analog) into descriptions of the world that make sense to thought processes and can elicit appropriate action.

Artificial intelligence

Artificial intelligence (AI) is the intelligence of machines or software, as opposed to the intelligence of human beings or animals. AI applications include advanced web search engines (e.g., Google Search), recommendation systems (used by YouTube, Amazon, and Netflix), understanding human speech (such as Siri and Alexa), self-driving cars (e.g., Waymo), generative or creative tools (ChatGPT and AI art), and competing at the highest level in strategic games (such as chess and Go).

Problem of other minds

The problem of other minds is a philosophical problem traditionally stated as the following epistemological question: Given that I can only observe the behavior of others, how can I know that others have minds? The problem is that knowledge of other minds is always indirect. The problem of other minds does not negatively impact social interactions due to people having a "theory of mind" - the ability to spontaneously infer the mental states of others - supported by innate mirror neurons, a theory of mind mechanism, or a tacit theory.

Mind

The mind (adjective form: mental) is that which thinks, imagines, remembers, wills, and senses, or is the set of faculties responsible for such phenomena. The mind is also associated with experiencing perception, pleasure and pain, belief, desire, intention, and emotion. The mind can include conscious and non-conscious states as well as sensory and non-sensory experiences. The exact nature of the mind is disputed. Traditionally, minds were understood as substances, but contemporary philosophers tend to see them as collections of properties or capacities.

Computer stereo vision

Computer stereo vision is the extraction of 3D information from digital images, such as those obtained by a CCD camera. By comparing information about a scene from two vantage points, 3D information can be extracted by examining the relative positions of objects in the two panels. This is similar to the biological process of stereopsis. In traditional stereo vision, two cameras, displaced horizontally from one another, are used to obtain two differing views on a scene, in a manner similar to human binocular vision.

Related lectures (64)

Transformer Architecture: The X Gomega

Delves into the Transformer architecture, self-attention, and training strategies for machine translation and image recognition.

Introduction to Machine Learning

Provides an overview of Machine Learning, including historical context, key tasks, and real-world applications.

Sequence to Sequence Models: Overview and Applications

Covers sequence to sequence models, their architecture, applications, and the role of attention mechanisms in improving performance.

Fine-Grained Visual Categorization: Challenges and Solutions

Explores challenges and solutions in fine-grained visual categorization, focusing on computer vision and machine learning.

Deep Visual Recognition: Interpretability