Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Image Representation Insights

Description

This lecture explores the importance of image representation, starting with the evolution of ImageNet results and the use of pre-training for downstream tasks. It delves into the challenges of supervised learning, the cost of supervision, and the benefits of self-supervised learning strategies like RotNet and Jigsaw puzzles. The speaker discusses the effectiveness of different self-supervised learning approaches, the evaluation methods, and the recent advancements in SSL, including cost function-based SSL and the use of subspace in deep learning. The lecture concludes with insights on self-expressiveness, attention mechanisms, and the application of transformer-based multitask learning in image representation.

Official source

Related concepts (25)

Self-supervised learning

Self-supervised learning (SSL) is a paradigm in machine learning for processing data of lower quality, rather than improving ultimate outcomes. Self-supervised learning more closely imitates the way humans learn to classify objects. The typical SSL method is based on an artificial neural network or other model such as a decision list. The model learns in two steps. First, the task is solved based on an auxiliary or pretext classification task using pseudo-labels which help to initialize the model parameters.

Feature learning

In machine learning, feature learning or representation learning is a set of techniques that allows a system to automatically discover the representations needed for feature detection or classification from raw data. This replaces manual feature engineering and allows a machine to both learn the features and use them to perform a specific task. Feature learning is motivated by the fact that machine learning tasks such as classification often require input that is mathematically and computationally convenient to process.

Machine learning

Machine learning (ML) is an umbrella term for solving problems for which development of algorithms by human programmers would be cost-prohibitive, and instead the problems are solved by helping machines 'discover' their 'own' algorithms, without needing to be explicitly told what to do by any human-developed algorithms. Recently, generative artificial neural networks have been able to surpass results of many previous approaches.

Deep learning

Deep learning is part of a broader family of machine learning methods, which is based on artificial neural networks with representation learning. The adjective "deep" in deep learning refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

Unsupervised learning

Unsupervised learning, is paradigm in machine learning where, in contrast to supervised learning and semi-supervised learning, algorithms learn patterns exclusively from unlabeled data. Neural network tasks are often categorized as discriminative (recognition) or generative (imagination). Often but not always, discriminative tasks use supervised methods and generative tasks use unsupervised (see Venn diagram); however, the separation is very hazy. For example, object recognition favors supervised learning but unsupervised learning can also cluster objects into groups.

Related lectures (43)

Introduction to Machine Learning

Provides an overview of Machine Learning, including historical context, key tasks, and real-world applications.

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

Supervised Learning Overview

Covers CNNs, RNNs, SVMs, and supervised learning methods, emphasizing the importance of tuning regularization and making informed decisions in machine learning.

Nonlinear Supervised Learning

Explores the inductive bias of different nonlinear supervised learning methods and the challenges of hyper-parameter tuning.

Deep Splines: Unifying Framework for Deep Neural Networks