Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Bayesian Parameter Estimation

In course

MGT-448: Statistical inference and machine learning

This course aims to provide graduate students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The course covers topics from machine learning, classical statistics, and data ...

Description

This lecture covers the concept of Bayesian Parameter Estimation, focusing on cost functions, control objectives, and model selection. It delves into regularization techniques, feature selection, and Bayesian statistics, emphasizing the importance of minimizing expected costs and maximizing rewards.

Instructor

Negar Kiyavash

Official source

Related concepts (18)

Feature selection

Feature selection is the process of selecting a subset of relevant features (variables, predictors) for use in model construction. Stylometry and DNA microarray analysis are two cases where feature selection is used. It should be distinguished from feature extraction. Feature selection techniques are used for several reasons: simplification of models to make them easier to interpret by researchers/users, shorter training times, to avoid the curse of dimensionality, improve data's compatibility with a learning model class, encode inherent symmetries present in the input space.

Model selection

Model selection is the task of selecting a model from among various candidates on the basis of performance criterion to choose the best one. In the context of learning, this may be the selection of a statistical model from a set of candidate models, given data. In the simplest cases, a pre-existing set of data is considered. However, the task can also involve the design of experiments such that the data collected is well-suited to the problem of model selection.

Bayesian inference

Bayesian inference (ˈbeɪziən or ˈbeɪʒən ) is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available. Bayesian inference is an important technique in statistics, and especially in mathematical statistics. Bayesian updating is particularly important in the dynamic analysis of a sequence of data. Bayesian inference has found application in a wide range of activities, including science, engineering, philosophy, medicine, sport, and law.

Cost curve

In economics, a cost curve is a graph of the costs of production as a function of total quantity produced. In a free market economy, productively efficient firms optimize their production process by minimizing cost consistent with each possible level of production, and the result is a cost curve. Profit-maximizing firms use cost curves to decide output quantities. There are various types of cost curves, all related to each other, including total and average cost curves; marginal ("for each additional unit") cost curves, which are equal to the differential of the total cost curves; and variable cost curves.

Regularization (mathematics)

In mathematics, statistics, finance, computer science, particularly in machine learning and inverse problems, regularization is a process that changes the result answer to be "simpler". It is often used to obtain results for ill-posed problems or to prevent overfitting. Although regularization procedures can be divided in many ways, the following delineation is particularly helpful: Explicit regularization is regularization whenever one explicitly adds a term to the optimization problem.

Related lectures (46)

Linear Regression: Statistical Inference and Regularization

Covers the probabilistic model for linear regression and the importance of regularization techniques.

Probabilistic Models for Linear Regression

Covers the probabilistic model for linear regression and its applications in nuclear magnetic resonance and X-ray imaging.

Maximum Likelihood: Inference and Model Comparison

Explores maximum likelihood inference, model selection, and comparing models using likelihood ratios.

Bayesian Inference: Optimal Estimation

Explores optimal Bayesian inference, denoising, scalar estimation, and phase transitions.

Model Selection: AIC and BIC