This course facilitates students in understanding of the fundamentals of Markov processes, one of most basic stochastic processes, through analyses of stochastic models.
At the end of this course, students will be able to:
1) Have understandings of the concept of Markov property in discrete and continuous time, and the basic facts that hold in Markov processes.
2) Apply the theory of Markov processes to analyze various stochastic models.
Markov processes, stochastic models, Markov chains, Poisson processes
|✔ Specialist skills
|Critical thinking skills
|✔ Practical and/or problem-solving skills
The class will be conducted on demand using video. The first 30 minutes of each class will be spent on Zoom, explaining the contents of the class to be delivered on the day, reviewing the previous class, explaining the previous assignment, and answering questions. Use T2SCHOLA as a place to ask questions and exchange opinions.
|Markov property and discrete time Markov chains
|Explain the concept of Markov properties.
|Transition diagram and probability distributions of the state
|Explain the transition diagram and probability distribution of the state.
|Classification of the state: connectivity
|Classificate the state of Markov chains.
|Explain the concept and basic facts of the periodicity.
|Explain the concept and basic facts of the recurrence.
|Explain the concept of the stationary distributions and its derivation.
|Explain the limit theorems.
|Understand the definition of Poisson processes and explain its basic properties.
|Compound Poisson processes
|Understand the definition of compound Poisson processes and explain its basic properties.
|Continuous time Markov chains
|Understand the definition of Markov chains in continuous time and explain its basic properties.
|Explain the basic properties and applications of birth-death processes.
|Explain the basic properties and applications of queueing systems.
|Control of Markov chains
|Explain the basic approach to control problems of Markov chains and its applications.
|Introduction to Brownian motion
To enhance effective learning, students are encouraged to spend approximately 100 minutes preparing for class and another 100 minutes reviewing class content afterwards (including assignments) for each class.
They should do so by referring to textbooks and other course material.
P. Brémaud, Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues, Springer
Students will be assessed on the understanding of Markov processes and its application. Grades are based on exercises and a final exam.
It is preferable that students have completed MCS.T212:Fundamentals of Probability.