Lecture

Learning in the Presence of Distribution Shifts: How does the Geometry of Perturbations Play a Role?

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture explores the challenges of learning in the presence of distribution shifts, focusing on the role of perturbation geometry. It covers topics such as deep learning in safety-critical applications, adversarial examples, sparse perturbations, norm-bounded perturbations, and natural variation. The instructor discusses the impact of perturbations on misclassification, the geometry of perturbations, and the development of robust classifiers. Additionally, the lecture delves into the concepts of L0 ball perturbations, model-based domain generalization, and the tradeoffs between standard and adversarial risks. The presentation concludes with insights on robust classification in the L0 setting and the importance of modeling natural variation for generalization.

Official source

Related concepts (32)

Stable distribution

In probability theory, a distribution is said to be stable if a linear combination of two independent random variables with this distribution has the same distribution, up to location and scale parameters. A random variable is said to be stable if its distribution is stable. The stable distribution family is also sometimes referred to as the Lévy alpha-stable distribution, after Paul Lévy, the first mathematician to have studied it. Of the four parameters defining the family, most attention has been focused on the stability parameter, (see panel).

Data modeling

Data modeling in software engineering is the process of creating a data model for an information system by applying certain formal techniques. It may be applied as part of broader Model-driven engineering (MDD) concept. Data modeling is a process used to define and analyze data requirements needed to support the business processes within the scope of corresponding information systems in organizations. Therefore, the process of data modeling involves professional data modelers working closely with business stakeholders, as well as potential users of the information system.

Lévy distribution

In probability theory and statistics, the Lévy distribution, named after Paul Lévy, is a continuous probability distribution for a non-negative random variable. In spectroscopy, this distribution, with frequency as the dependent variable, is known as a van der Waals profile. It is a special case of the inverse-gamma distribution. It is a stable distribution. The probability density function of the Lévy distribution over the domain is where is the location parameter and is the scale parameter.

Normal distribution

In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is The parameter is the mean or expectation of the distribution (and also its median and mode), while the parameter is its standard deviation. The variance of the distribution is . A random variable with a Gaussian distribution is said to be normally distributed, and is called a normal deviate.

Hyperbolic secant distribution

In probability theory and statistics, the hyperbolic secant distribution is a continuous probability distribution whose probability density function and characteristic function are proportional to the hyperbolic secant function. The hyperbolic secant function is equivalent to the reciprocal hyperbolic cosine, and thus this distribution is also called the inverse-cosh distribution. Generalisation of the distribution gives rise to the Meixner distribution, also known as the Natural Exponential Family - Generalised Hyperbolic Secant or NEF-GHS distribution.

Related lectures (64)

Advanced Machine Learning: Brief review of C-SVM

Covers clustering, classification, and Support Vector Machine principles, applications, and optimization, including non-linear classification and Gaussian kernel effects.

Machine Learning Fundamentals

Introduces fundamental machine learning concepts, covering regression, classification, dimensionality reduction, and deep generative models.

Linear Models for Classification: Multi-Class Extensions

Covers linear models for multi-class classification, focusing on logistic regression and evaluation metrics.

Neural Networks: Multilayer Learning

Covers the fundamentals of multilayer neural networks and deep learning, including back-propagation and network architectures like LeNet, AlexNet, and VGG-16.

Landscape and Generalisation in Deep Learning