### 2020　Markov Analysis

Font size  SML

Undergraduate major in Mathematical and Computing Science
Instructor(s)
Nakano Yumiharu  Miyoshi Naoto
Class Format
Lecture    (ZOOM)
Media-enhanced courses
Day/Period(Room No.)
Tue7-8(W834)  Fri7-8(W834)
Group
-
Course number
MCS.T312
Credits
2
2020
Offered quarter
2Q
Syllabus updated
2020/9/18
Lecture notes updated
-
Language used
Japanese
Access Index

### Course description and aims

This course facilitates students in understanding of the fundamentals of Markov processes, one of most basic stochastic processes, through analyses of stochastic models.

### Student learning outcomes

At the end of this course, students will be able to:
1) Have understandings of the concept of Markov property in discrete and continuous time, and the basic facts that hold in Markov processes.
2) Apply the theory of Markov processes to analyze various stochastic models.

### Keywords

Markov processes, stochastic models, Markov chains, Poisson processes

### Competencies that will be developed

 ✔ Specialist skills Intercultural skills Communication skills Critical thinking skills ✔ Practical and/or problem-solving skills

### Class flow

Slides and blackboard are used. Towards the end of class, students are given exercise problems related to what is taught on that day to solve.

### Course schedule/Required learning

Course schedule Required learning
Class 1 Markov property and discrete time Markov chains Explain the concept of Markov properties.
Class 2 Transition diagram and probability distributions of the state Explain the transition diagram and probability distribution of the state.
Class 3 Classification of the state: connectivity Classificate the state of Markov chains.
Class 4 Periodicity Explain the concept and basic facts of the periodicity.
Class 5 Reccurence Explain the concept and basic facts of the recurrence.
Class 6 Stationary distributions Explain the concept of the stationary distributions and its derivation.
Class 7 Limit theorems Explain the limit theorems.
Class 8 Poisson processes Understand the definition of Poisson processes and explain its basic properties.
Class 9 Compound Poisson processes Understand the definition of compound Poisson processes and explain its basic properties.
Class 10 Continuous time Markov chains Understand the definition of Markov chains in continuous time and explain its basic properties.
Class 11 Birth-death processes Explain the basic properties and applications of birth-death processes.
Class 12 Queueing systems Explain the basic properties and applications of queueing systems.
Class 13 Control of Markov chains Explain the basic approach to control problems of Markov chains and its applications.
Class 14 Brownian motion Introduction to Brownian motion

### Out-of-Class Study Time (Preparation and Review)

To enhance effective learning, students are encouraged to spend approximately 100 minutes preparing for class and another 100 minutes reviewing class content afterwards (including assignments) for each class.
They should do so by referring to textbooks and other course material.

None required.

### Reference books, course materials, etc.

P. Brémaud, Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues, Springer

### Assessment criteria and methods

Students will be assessed on the understanding of Markov processes and its application.
Students' course scores are based on several reports.

### Related courses

• MCS.T212 ： Fundamentals of Probability

### Prerequisites (i.e., required knowledge, skills, courses, etc.)

It is preferable that students have completed MCS.T212:Fundamentals of Probability.