Lecture

Controlled Stochastic Processes

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MGT-484: Applied probability & stochastic processes

This course focuses on dynamic models of random phenomena, and in particular, the most popular classes of such models: Markov chains and Markov decision processes. We will also study applications in queuing theory, finance, project management, etc. ...

Description

This lecture by the instructor covers the topic of controlled stochastic processes, focusing on analyzing proper processes, asymptotic behavior, distribution, and passage times. The lecture delves into the descriptive and prescriptive tasks of controlling stochastic processes, with examples such as steering a car from A to B with minimum energy. Dynamic programming is introduced as a solution method, emphasizing the principle of optimality. The lecture concludes with the application of dynamic programming algorithms to solve the machine replacement problem, showcasing the backward induction process to determine the optimal policy.

Instructor

Negar Kiyavash

Teaching & PhD PhD Students Mohammad Shahverdikondori, Badri Mohammad Reza, Arefe Boushehrian, Sepehr Elahi, Mohammadsadegh Khorasani, Mohammadsaeed Masiha, Amir Mohammad Aboueimehrizi Past EPFL PhD Students Sina Akbari, Ehsan Mokhtarian, Yaroslav Kivva, Fateme Jamshidi Courses Causal inference MGT-416 Students will learn the core concepts and techniques of network analysis with emphasis on causal inference. Theory andapplication will be balanced, with students working directly with networ

Official source

Related concepts (32)

Normal distribution

In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is The parameter is the mean or expectation of the distribution (and also its median and mode), while the parameter is its standard deviation. The variance of the distribution is . A random variable with a Gaussian distribution is said to be normally distributed, and is called a normal deviate.

Control theory

Control theory is a field of control engineering and applied mathematics that deals with the control of dynamical systems in engineered processes and machines. The objective is to develop a model or algorithm governing the application of system inputs to drive the system to a desired state, while minimizing any delay, overshoot, or steady-state error and ensuring a level of control stability; often with the aim to achieve a degree of optimality. To do this, a controller with the requisite corrective behavior is required.

Cauchy distribution

The Cauchy distribution, named after Augustin Cauchy, is a continuous probability distribution. It is also known, especially among physicists, as the Lorentz distribution (after Hendrik Lorentz), Cauchy–Lorentz distribution, Lorentz(ian) function, or Breit–Wigner distribution. The Cauchy distribution is the distribution of the x-intercept of a ray issuing from with a uniformly distributed angle. It is also the distribution of the ratio of two independent normally distributed random variables with mean zero.

Cost

In production, research, retail, and accounting, a cost is the value of money that has been used up to produce something or deliver a service, and hence is not available for use anymore. In business, the cost may be one of acquisition, in which case the amount of money expended to acquire it is counted as cost. In this case, money is the input that is gone in order to acquire the thing. This acquisition cost may be the sum of the cost of production as incurred by the original producer, and further costs of transaction as incurred by the acquirer over and above the price paid to the producer.

Stochastic control

Stochastic control or stochastic optimal control is a sub field of control theory that deals with the existence of uncertainty either in observations or in the noise that drives the evolution of the system. The system designer assumes, in a Bayesian probability-driven fashion, that random noise with known probability distribution affects the evolution and observation of the state variables. Stochastic control aims to design the time path of the controlled variables that performs the desired control task with minimum cost, somehow defined, despite the presence of this noise.

Related lectures (64)

Dynamic Programming: Optimal Control

Explores Dynamic Programming for optimal control, focusing on stability, stationary policy, and recursive solutions.

Introduction to Reinforcement Learning: Key Concepts and Applications

Introduces reinforcement learning, covering its definitions, applications, and theoretical foundations, while outlining the course structure and objectives.

Diffusion Models

Explores diffusion models, focusing on generating samples from a distribution and the importance of denoising in the process.

Optimal Transport: Kantorovich Duality

Covers optimal transport and Kantorovich duality in real-life distribution problems.

Markov Decision Processes: Foundations of Reinforcement Learning