2022 Markov Analysis

Font size  SML

Register update notification mail Add to favorite lecture list
Academic unit or major
Undergraduate major in Mathematical and Computing Science
Nakano Yumiharu  Miyoshi Naoto 
Class Format
Lecture    (On-demand)
Media-enhanced courses
Day/Period(Room No.)
Course number
Academic year
Offered quarter
Syllabus updated
Lecture notes updated
Language used
Access Index

Course description and aims

This course facilitates students in understanding of the fundamentals of Markov processes, one of most basic stochastic processes, through analyses of stochastic models.

Student learning outcomes

At the end of this course, students will be able to:
1) Have understandings of the concept of Markov property in discrete and continuous time, and the basic facts that hold in Markov processes.
2) Apply the theory of Markov processes to analyze various stochastic models.


Markov processes, stochastic models, Markov chains, Poisson processes

Competencies that will be developed

Specialist skills Intercultural skills Communication skills Critical thinking skills Practical and/or problem-solving skills

Class flow

The class will be conducted on demand using video. The first 30 minutes of each class will be spent on Zoom, explaining the contents of the class to be delivered on the day, reviewing the previous class, explaining the previous assignment, and answering questions. Use T2SCHOLA as a place to ask questions and exchange opinions.

Course schedule/Required learning

  Course schedule Required learning
Class 1 Markov property and discrete time Markov chains Explain the concept of Markov properties.
Class 2 Transition diagram and probability distributions of the state Explain the transition diagram and probability distribution of the state.
Class 3 Classification of the state: connectivity Classificate the state of Markov chains.
Class 4 Periodicity Explain the concept and basic facts of the periodicity.
Class 5 Reccurence Explain the concept and basic facts of the recurrence.
Class 6 Stationary distributions Explain the concept of the stationary distributions and its derivation.
Class 7 Limit theorems Explain the limit theorems.
Class 8 Markov chain Monte Carlo methods Introduce Markov chain Monte Carlo methods.
Class 9 Poisson processes Understand the definition of Poisson processes and explain its basic properties.
Class 10 Compound Poisson processes Understand the definition of compound Poisson processes and explain its basic properties.
Class 11 Continuous time Markov chains Understand the definition of Markov chains in continuous time and explain its basic properties.
Class 12 Birth-death processes Explain the basic properties and applications of birth-death processes.
Class 13 Queueing systems Explain the basic properties and applications of queueing systems.
Class 14 Brownian motion Introduction to Brownian motion


Lecture notes.

Reference books, course materials, etc.

P. Brémaud, Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues, Springer

Assessment criteria and methods

Students will be assessed on the understanding of Markov processes and its application. Grades are based on exercises and a final exam.

Related courses

  • MCS.T212 : Fundamentals of Probability

Prerequisites (i.e., required knowledge, skills, courses, etc.)

It is preferable that students have completed MCS.T212:Fundamentals of Probability.

Page Top