Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Markov Chains: Definition and Examples

In course

COM-516: Markov chains and algorithmic applications

The study of random walks finds many applications in computer science and communications. The goal of the course is to get familiar with the theory of random walks, and to get an overview of some applications of this theory to problems of interest in commu ...

Description

This lecture introduces Markov chains, a time-homogeneous stochastic process with values in a finite or countable set. It covers the definition, properties, transition matrix, and examples like a party and a simple symmetric random walk.

Official source

Related concepts (28)

Markov chain

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC).

Finite set

In mathematics, particularly set theory, a finite set is a set that has a finite number of elements. Informally, a finite set is a set which one could in principle count and finish counting. For example, is a finite set with five elements. The number of elements of a finite set is a natural number (possibly zero) and is called the cardinality (or the cardinal number) of the set. A set that is not a finite set is called an infinite set.

Dedekind-infinite set

In mathematics, a set A is Dedekind-infinite (named after the German mathematician Richard Dedekind) if some proper subset B of A is equinumerous to A. Explicitly, this means that there exists a bijective function from A onto some proper subset B of A. A set is Dedekind-finite if it is not Dedekind-infinite (i.e., no such bijection exists). Proposed by Dedekind in 1888, Dedekind-infiniteness was the first definition of "infinite" that did not rely on the definition of the natural numbers.

Countable set

In mathematics, a set is countable if either it is finite or it can be made in one to one correspondence with the set of natural numbers. Equivalently, a set is countable if there exists an injective function from it into the natural numbers; this means that each element in the set may be associated to a unique natural number, or that the elements of the set can be counted one at a time, although the counting may never finish due to an infinite number of elements.

Hereditarily finite set

In mathematics and set theory, hereditarily finite sets are defined as finite sets whose elements are all hereditarily finite sets. In other words, the set itself is finite, and all of its elements are finite sets, recursively all the way down to the empty set. A recursive definition of well-founded hereditarily finite sets is as follows: Base case: The empty set is a hereditarily finite set. Recursion rule: If a1,...,ak are hereditarily finite, then so is {a1,...,ak}.

Related lectures (29)

Theory of Computation: Counting and Decision Problems

Explores counting infinite sets and decision problems, showcasing the limits of computation in solving certain undecidable problems.

Fractals and Strange Attractors

Delves into renormalization, fractals, and strange attractors, exploring the properties of countable and uncountable sets.

Relations, Sequences and Summations

Covers strings, countable sets, cardinality, and the concept of countability, exploring the countability of various sets and Cantor diagonalization.

Cardinality of Sets: Countable and Uncountable

Explores cardinality, countable sets, and examples of countable and uncountable sets.

Nonlinear Dynamics: Chaos and Complex Systems