Lecture

Infinite-Horizon Problems: Formulation & Complexity

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MGT-484: Applied probability & stochastic processes

This course focuses on dynamic models of random phenomena, and in particular, the most popular classes of such models: Markov chains and Markov decision processes. We will also study applications in queuing theory, finance, project management, etc. ...

Description

This lecture covers infinite-horizon problems in Applied Probability and Stochastic Processes, focusing on the formulation and algorithm complexity. It discusses a two-state discounted Markov decision process and explores the policy iteration algorithm to find an optimal policy.

Instructor

Negar Kiyavash

Teaching & PhD PhD Students Mohammad Shahverdikondori, Badri Mohammad Reza, Arefe Boushehrian, Sepehr Elahi, Mohammadsadegh Khorasani, Mohammadsaeed Masiha, Amir Mohammad Aboueimehrizi Past EPFL PhD Students Sina Akbari, Ehsan Mokhtarian, Yaroslav Kivva, Fateme Jamshidi Courses Causal inference MGT-416 Students will learn the core concepts and techniques of network analysis with emphasis on causal inference. Theory andapplication will be balanced, with students working directly with networ

Official source

Related concepts (32)

Time complexity

In computer science, the time complexity is the computational complexity that describes the amount of computer time it takes to run an algorithm. Time complexity is commonly estimated by counting the number of elementary operations performed by the algorithm, supposing that each elementary operation takes a fixed amount of time to perform. Thus, the amount of time taken and the number of elementary operations performed by the algorithm are taken to be related by a constant factor.

Policy analysis

Policy analysis or public policy analysis is a technique used in the public administration sub-field of political science to enable civil servants, nonprofit organizations, and others to examine and evaluate the available options to implement the goals of laws and elected officials. People who regularly use policy analysis skills and techniques on the job, particularly those who use it as a major part of their job duties are generally known by the title Policy Analyst.

Program evaluation

Program evaluation is a systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs, particularly about their effectiveness and efficiency. In both the public sector and private sector, as well as the voluntary sector, stakeholders might be required to assess—under law or charter—or want to know whether the programs they are funding, implementing, voting for, receiving or opposing are producing the promised effect.

Policy

Policy is a deliberate system of guidelines to guide decisions and achieve rational outcomes. A policy is a statement of intent and is implemented as a procedure or protocol. Policies are generally adopted by a governance body within an organization. Policies can assist in both subjective and objective decision making. Policies used in subjective decision-making usually assist senior management with decisions that must be based on the relative merits of a number of factors, and as a result, are often hard to test objectively, e.

Computational complexity

In computer science, the computational complexity or simply complexity of an algorithm is the amount of resources required to run it. Particular focus is given to computation time (generally measured by the number of needed elementary operations) and memory storage requirements. The complexity of a problem is the complexity of the best algorithms that allow solving the problem. The study of the complexity of explicitly given algorithms is called analysis of algorithms, while the study of the complexity of problems is called computational complexity theory.

Related lectures (64)

Solving Parity Games in Practice

Explores practical aspects of solving parity games, including winning strategies, algorithms, complexity, determinism, and heuristic approaches.

Asset Selling Problem

Explores the Asset Selling Problem to maximize long-term reward without a deadline.

Advanced Machine Learning: Discrete Reinforcement Learning

Introduces the basics of Reinforcement Learning, covering discrete states, actions, policies, value functions, MDPs, and optimal policies.

Optimal Marketing Strategy

Covers decision-making in marketing based on customer behavior for optimal strategies.

Primal-dual Optimization: Extra-Gradient Method