2019 Statistical Learning Theory

Font size  SML

Register update notification mail Add to favorite lecture list
Academic unit or major
Graduate major in Mathematical and Computing Science
Instructor(s)
Watanabe Sumio  Kabashima Yoshiyuki 
Course component(s)
Lecture
Day/Period(Room No.)
Mon1-2(H135)  Thr1-2(H135)  
Group
-
Course number
MCS.T403
Credits
2
Academic year
2019
Offered quarter
2Q
Syllabus updated
2019/4/22
Lecture notes updated
-
Language used
English
Access Index

Course description and aims

Machine learning theory and statistical mechanics are introduced. In the first half, hierarchical learning machines for accurate prediction and knowledge discovery are explained and its mathematical laws are derived. In the second half, statistical mechanical approximation theory for handling massive information processing is introduced.

Student learning outcomes

The purpose of statistical learning is to estimate the true information source from empirical samples. In this course, several learning machines which have high dimensional parameters are introduced. Statistical mechanics theory plays an important role in studying such learning machines. Two other courses, ``Theory of statistical mathematics" and ``Machine learning" are strongly recommended for students.

Keywords

Statistics, Information Theory, Statistical mechanics, Free energy, and Entropy

Competencies that will be developed

Intercultural skills Communication skills Specialist skills Critical thinking skills Practical and/or problem-solving skills
- - -

Class flow

In the first class of this lecture, the examination asking prerequisite knowledge is performed, which determines permitted students for attendance. The prerequisite knowledge consists of linear algebra, derivative and integration, probability theory, and statistics.
This course consists of two parts. Machine learning theory and statistical mechanics are introduced.

Course schedule/Required learning

  Course schedule Required learning
Class 1 Introduction of statistical learning theory. The examination asking prerequisite knowledge is performed. Statistical learning theory is introduced. Based on the examination, permitted students are determined.
Class 2 Neural network architecture neural network
Class 3 Learning in neural networks learning in neural networks
Class 4 Boltzmann machine Boltzmann machine
Class 5 Deep Learning Deep learning
Class 6 Information and relative entropy Information and relative entropy
Class 7 Prediction Theory Prediction theory
Class 8 Discovery theory Discovery theory
Class 9 Monte Carlo methods Computational difficulty of sampling in high dimensional spaces
Class 10 Markov chain Monte Carlo Methods Metropolis-Hastings methods
Class 11 Advanced Markov chain Monte Carlo Methods Advanced Markov chain Monte Carlo Methods
Class 12 Variational inference methods Variational inference methods
Class 13 Variational Mixture of Gaussians Variational Mixture of Gaussians
Class 14 Tree graphs and belief propagation Belief propagation
Class 15 Loopy graphs and loopy belief propagation Loopy belief propagation

Textbook(s)

None.

Reference books, course materials, etc.

None. Two other lectures "Theory of statistical mathematics" and "Machine Learning" are strongly recommended for students.

Assessment criteria and methods

Reports.

Related courses

  • MCS.T507 : Theory of Statistical Mathematics
  • ART.T458 : Machine Learning

Prerequisites (i.e., required knowledge, skills, courses, etc.)

Linear algebra, differential and integral analysis, probability theory, and statistics are necessary.

Page Top