Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

All of Probability: LLN, CLT, Chernoff and PAC bound

In course

EE-411: Fundamentals of inference and learning

This is an introductory course in the theory of statistics, inference, and machine learning, with an emphasis on theoretical understanding & practical exercises. The course will combine, and alternate, between mathematical theoretical foundations and prac ...

Description

This lecture covers various important concepts in probability theory, such as the Law of Large Numbers (LLN), Central Limit Theorem (CLT), Chernoff bounds, and PAC (Probably Approximately Correct) bounds. The instructor explains these topics in detail, providing insights into how they are applied in different scenarios.

Instructor

Florent Gérard Krzakala

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Related concepts (29)

Probability

Probability is the branch of mathematics concerning numerical descriptions of how likely an event is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and 1, where, roughly speaking, 0 indicates impossibility of the event and 1 indicates certainty. The higher the probability of an event, the more likely it is that the event will occur. A simple example is the tossing of a fair (unbiased) coin.

Probability theory

Probability theory or probability calculus is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of axioms. Typically these axioms formalise probability in terms of a probability space, which assigns a measure taking values between 0 and 1, termed the probability measure, to a set of outcomes called the sample space.

Chernoff bound

In probability theory, a Chernoff bound is an exponentially decreasing upper bound on the tail of a random variable based on its moment generating function. The minimum of all such exponential bounds forms the Chernoff or Chernoff-Cramér bound, which may decay faster than exponential (e.g. sub-Gaussian). It is especially useful for sums of independent random variables, such as sums of Bernoulli random variables. The bound is commonly named after Herman Chernoff who described the method in a 1952 paper, though Chernoff himself attributed it to Herman Rubin.

Probability axioms

The Kolmogorov axioms are the foundations of probability theory introduced by Russian mathematician Andrey Kolmogorov in 1933. These axioms remain central and have direct contributions to mathematics, the physical sciences, and real-world probability cases. An alternative approach to formalising probability, favoured by some Bayesians, is given by Cox's theorem. The assumptions as to setting up the axioms can be summarised as follows: Let be a measure space with being the probability of some event , and .

Event (probability theory)

In probability theory, an event is a set of outcomes of an experiment (a subset of the sample space) to which a probability is assigned. A single outcome may be an element of many different events, and different events in an experiment are usually not equally likely, since they may include very different groups of outcomes. An event consisting of only a single outcome is called an or an ; that is, it is a singleton set. An event is said to if contains the outcome of the experiment (or trial) (that is, if ).

Related lectures (37)

Probability and Statistics: Fundamental Theorems

Explores fundamental theorems in probability and statistics, joint probability laws, and marginal distributions.

Variance and Covariance: Properties and Examples

Explores variance, covariance, and practical applications in statistics and probability.

Continuous Random Variables

Explores continuous random variables, density functions, joint variables, independence, and conditional densities.

Generalization Error

Explores tail bounds, information bounds, and maximal leakage in the context of generalization error.

All of Probability: Basic Bounds, LLN & CLT