Lecture

Hoeffding's Inequality: Binomial Distribution

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MGT-448: Statistical inference and machine learning

This course aims to provide graduate students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The course covers topics from machine learning, classical statistics, and data ...

Description

This lecture covers Hoeffding's inequality, which is applied to independent random variables to estimate the probability of deviation from the expected sum. It also explores the binomial distribution as the sum of Bernoulli random variables, providing insights into concentration bounds and overfitting in Expected Risk Minimization. The lecture delves into predictors' error minimization and generalization gaps, emphasizing the importance of structural risk minimization in predictor selection.

Instructor

Negar Kiyavash

Teaching & PhD PhD Students Mohammad Shahverdikondori, Badri Mohammad Reza, Arefe Boushehrian, Sepehr Elahi, Mohammadsadegh Khorasani, Mohammadsaeed Masiha, Amir Mohammad Aboueimehrizi Past EPFL PhD Students Sina Akbari, Ehsan Mokhtarian, Yaroslav Kivva, Fateme Jamshidi Courses Causal inference MGT-416 Students will learn the core concepts and techniques of network analysis with emphasis on causal inference. Theory andapplication will be balanced, with students working directly with networ

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Related concepts (32)

Independent and identically distributed random variables

In probability theory and statistics, a collection of random variables is independent and identically distributed if each random variable has the same probability distribution as the others and all are mutually independent. This property is usually abbreviated as i.i.d., iid, or IID. IID was first defined in statistics and finds application in different fields such as data mining and signal processing. Statistics commonly deals with random samples. A random sample can be thought of as a set of objects that are chosen randomly.

Hoeffding's inequality

In probability theory, Hoeffding's inequality provides an upper bound on the probability that the sum of bounded independent random variables deviates from its expected value by more than a certain amount. Hoeffding's inequality was proven by Wassily Hoeffding in 1963. Hoeffding's inequality is a special case of the Azuma–Hoeffding inequality and McDiarmid's inequality. It is similar to the Chernoff bound, but tends to be less sharp, in particular when the variance of the random variables is small.

Concentration inequality

In probability theory, concentration inequalities provide bounds on how a random variable deviates from some value (typically, its expected value). The law of large numbers of classical probability theory states that sums of independent random variables are, under very mild conditions, close to their expectation with a large probability. Such sums are the most basic examples of random variables concentrated around their mean. Recent results show that such behavior is shared by other functions of independent random variables.

Chernoff bound

In probability theory, a Chernoff bound is an exponentially decreasing upper bound on the tail of a random variable based on its moment generating function. The minimum of all such exponential bounds forms the Chernoff or Chernoff-Cramér bound, which may decay faster than exponential (e.g. sub-Gaussian). It is especially useful for sums of independent random variables, such as sums of Bernoulli random variables. The bound is commonly named after Herman Chernoff who described the method in a 1952 paper, though Chernoff himself attributed it to Herman Rubin.

Bennett's inequality

In probability theory, Bennett's inequality provides an upper bound on the probability that the sum of independent random variables deviates from its expected value by more than any specified amount. Bennett's inequality was proved by George Bennett of the University of New South Wales in 1962. Let X1, ... Xn be independent random variables with finite variance. Further assume Xi ≤ a almost surely for all i, and define and Then for any t ≥ 0, where h(u) = (1 + u)log(1 + u) – u and log denotes the natural logarithm.

Related lectures (64)

Probability and Statistics

Delves into probability, statistics, paradoxes, and random variables, showcasing their real-world applications and properties.

Advanced Probabilities: Random Variables & Expected Values

Explores advanced probabilities, random variables, and expected values, with practical examples and quizzes to reinforce learning.

Random Variables: Expected Value

Covers advanced probability concepts, including random variables and expected value calculation.

Random Variables and Expected Value

Introduces random variables, probability distributions, and expected values through practical examples.

Elements of Statistics: Probability and Random Variables