Publication

Markov decision processes for services opportunity pipeline optimization

2010
Student Projects

Abstract

The dynamics of sales opportunities can be modelled by a Markov Decision Process. The latter can be solved by using dynamic programming and assigns to each state an optimal action. In this project, states are modelled by the number of opportunities at five different maturity levels called ranks, actions are represented by investments and rewards by profits from signed contracts. Transitions are simulated using the probabilities that an opportunity moves from one rank to another. Two different types of policy appear recurrently in the model outcome, i.e. a low-investment policy when the opportunities are rather uniformly distributed across ranks and a high-investment policy, when a larger number of opportunities have reached mature status or have juste entered the pipe.

Official source

https://infoscience.epfl.ch/entities/publication/4846564b-d157-45cb-a1ac-d033f40f5b78

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

2010
Student Projects

Abstract

Official source

https://infoscience.epfl.ch/entities/publication/4846564b-d157-45cb-a1ac-d033f40f5b78

About this result

Related concepts (32)

Markov chain

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC).

Markov decision process

In mathematics, a Markov decision process (MDP) is a discrete-time stochastic control process. It provides a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker. MDPs are useful for studying optimization problems solved via dynamic programming. MDPs were known at least as early as the 1950s; a core body of research on Markov decision processes resulted from Ronald Howard's 1960 book, Dynamic Programming and Markov Processes.

Markov property

In probability theory and statistics, the term Markov property refers to the memoryless property of a stochastic process, which means that its future evolution is independent of its history. It is named after the Russian mathematician Andrey Markov. The term strong Markov property is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a stopping time. The term Markov assumption is used to describe a model where the Markov property is assumed to hold, such as a hidden Markov model.

Related publications (19)

Markov decision processes for services opportunity pipeline optimization

Chat with Graph Search

(Re)framing timber values: Cascade use metrics as a dynamic policy tool

Journal research data policies in materials science

Meta-RL Induces Exploration in Language Agents