Print

2017　Statistical Learning Theory

Font size  SML

Register update notification mail Add to favorite lecture list
Academic unit or major
Graduate major in Mathematical and Computing Science
Instructor(s)
Watanabe Sumio  Kabashima Yoshiyuki 
Class Format
Lecture     
Media-enhanced courses
Day/Period(Room No.)
Mon1-2(G511)  Thr1-2(G511)  
Group
-
Course number
MCS.T403
Credits
2
Academic year
2017
Offered quarter
3Q
Syllabus updated
2017/4/18
Lecture notes updated
-
Language used
English
Access Index

Course description and aims

Machine learning theory and statistical mechanics are introduced. In the first half, hierarchical learning machines for accurate prediction and knowledge discovery are explained and its mathematical laws are derived. In the second half, statistical mechanical approximation theory for handling massive information processing is introduced.

Student learning outcomes

The purpose of statistical learning is to estimate the true information source from empirical samples. In this course, several learning machines which have high dimensional parameters are introduced. Statistical mechanics theory plays an important role in studying such learning machines. Two other courses, ``Theory of statistical mathematics" and ``Machine learning" are strongly recommended for students.

Keywords

Statistics, Information Theory, Statistical mechanics, Free energy, and Entropy

Competencies that will be developed

✔ Specialist skills Intercultural skills Communication skills Critical thinking skills ✔ Practical and/or problem-solving skills

Class flow

This course consists of two parts. Machine learning theory and statistical mechanics are introduced.

Course schedule/Required learning

  Course schedule Required learning
Class 1 Introduction of statistical learning theory statistical learning theory
Class 2 Neural network architecture neural network
Class 3 Learning in neural networks learning in neural networks
Class 4 Boltzmann machine Boltzmann machine
Class 5 Deep Learning Deep learning
Class 6 Information and relative entropy Information and relative entropy
Class 7 Prediction Theory Prediction theory
Class 8 Discovery theory Discovery theory
Class 9 Similarity between Bayesian inference and statistical mechanics Inference based on Bayes Formula, canonical distribution and free energy
Class 10 Ideal gas and Ising model Ideal gas, Ising model, equation of state
Class 11 Mean field approximations Molecular field and Bethe approximations in Ising model
Class 12 Belief propagation for inference on sparse graphs Graphical model for probabilistic models, belief propagation
Class 13 Belief propagation for inference on dense graphs Approximate message passing
Class 14 Statistical mechanical formalism Moment evaluation based on free energies, linear response relation
Class 15 Free energy and hyper parameter estimation Variational principle, variational Bayes, EM-algorithm

Textbook(s)

None.

Reference books, course materials, etc.

None. Two other lectures "Theory of statistical mathematics" and "Machine Learning"are strongly recommended for students.

Assessment criteria and methods

Reports.

Related courses

  • MCS.T507 ： Theory of Statistical Mathematics
  • ART.T458 ： Machine Learning

Prerequisites (i.e., required knowledge, skills, courses, etc.)

Probability theory and statistics are necessary.

Page Top