Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Deep Learning Modus Operandi

In course

PHYS-467: Machine learning for physicists

Machine learning and data analysis are becoming increasingly central in sciences including physics. In this course, fundamental principles and methods of machine learning will be introduced and practised. ...

Description

This lecture delves into the modus operandi of deep learning, exploring the benefits of deeper networks through experiments on ImageNet. It discusses the importance of over-parameterization and generalization in deep networks, emphasizing the role of minimum norm solutions. The lecture also covers topics such as interpolation points, transfer learning with CNNs, and back to unsupervised learning with auto-encoders, GANs, and diffusion models.

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Mathematics

Analysis: Numerical analysis

Related concepts (32)

Interpolation

In the mathematical field of numerical analysis, interpolation is a type of estimation, a method of constructing (finding) new data points based on the range of a discrete set of known data points. In engineering and science, one often has a number of data points, obtained by sampling or experimentation, which represent the values of a function for a limited number of values of the independent variable. It is often required to interpolate; that is, estimate the value of that function for an intermediate value of the independent variable.

Polynomial interpolation

In numerical analysis, polynomial interpolation is the interpolation of a given bivariate data set by the polynomial of lowest possible degree that passes through the points of the dataset. Given a set of n + 1 data points , with no two the same, a polynomial function is said to interpolate the data if for each . There is always a unique such polynomial, commonly given by two explicit formulas, the Lagrange polynomials and Newton polynomials.

Autoencoder

An autoencoder is a type of artificial neural network used to learn efficient codings of unlabeled data (unsupervised learning). An autoencoder learns two functions: an encoding function that transforms the input data, and a decoding function that recreates the input data from the encoded representation. The autoencoder learns an efficient representation (encoding) for a set of data, typically for dimensionality reduction. Variants exist, aiming to force the learned representations to assume useful properties.

Nearest-neighbor interpolation

Nearest-neighbor interpolation (also known as proximal interpolation or, in some contexts, point sampling) is a simple method of multivariate interpolation in one or more dimensions. Interpolation is the problem of approximating the value of a function for a non-given point in some space when given the value of that function in points around (neighboring) that point. The nearest neighbor algorithm selects the value of the nearest point and does not consider the values of neighboring points at all, yielding a piecewise-constant interpolant.

Bilinear interpolation

In mathematics, bilinear interpolation is a method for interpolating functions of two variables (e.g., x and y) using repeated linear interpolation. It is usually applied to functions sampled on a 2D rectilinear grid, though it can be generalized to functions defined on the vertices of (a mesh of) arbitrary convex quadrilaterals. Bilinear interpolation is performed using linear interpolation first in one direction, and then again in another direction.

Related lectures (44)

Diffusion Models

Explores diffusion models, focusing on generating samples from a distribution and the importance of denoising in the process.

Supervised Learning with kNN: Regression Model

Covers a simple mathematical model for supervised learning with k-nearest neighbors in regression.

Learning with Deep Neural Networks

Explores the success and challenges of deep learning, including overfitting, generalization, and the impact on various domains.

Compression: Prediction

Covers the concepts of compression and prediction using prefix-free codes and distributions.

Signal processing and vector spaces