2019 | Statistical Learning Theory

2019　Statistical Learning Theory

Font size S M L

Academic unit or major: Graduate major in Mathematical and Computing Science

Instructor(s): Watanabe Sumio Kabashima Yoshiyuki

Class Format: Lecture

Media-enhanced courses

Day/Period(Room No.): Mon1-2(H135) Thr1-2(H135)

Group: -

Course number: MCS.T403

Credits: 2

Academic year: 2019

Offered quarter: 2Q

Syllabus updated: 2019/4/22

Lecture notes updated: -

Language used: English

Access Index

Syllabus

Course description and aims

Machine learning theory and statistical mechanics are introduced. In the first half, hierarchical learning machines for accurate prediction and knowledge discovery are explained and its mathematical laws are derived. In the second half, statistical mechanical approximation theory for handling massive information processing is introduced.

Student learning outcomes

The purpose of statistical learning is to estimate the true information source from empirical samples. In this course, several learning machines which have high dimensional parameters are introduced. Statistical mechanics theory plays an important role in studying such learning machines. Two other courses, ``Theory of statistical mathematics" and ``Machine learning" are strongly recommended for students.

Keywords

Statistics, Information Theory, Statistical mechanics, Free energy, and Entropy

Competencies that will be developed

✔ Specialist skills

Intercultural skills

Communication skills

Critical thinking skills

✔ Practical and/or problem-solving skills

Class flow

In the first class of this lecture, the examination asking prerequisite knowledge is performed, which determines permitted students for attendance. The prerequisite knowledge consists of linear algebra, derivative and integration, probability theory, and statistics.
This course consists of two parts. Machine learning theory and statistical mechanics are introduced.

Course schedule/Required learning

	Course schedule	Required learning
Class 1	Introduction of statistical learning theory. The examination asking prerequisite knowledge is performed.	Statistical learning theory is introduced. Based on the examination, permitted students are determined.
Class 2	Neural network architecture	neural network
Class 3	Learning in neural networks	learning in neural networks
Class 4	Boltzmann machine	Boltzmann machine
Class 5	Deep Learning	Deep learning
Class 6	Information and relative entropy	Information and relative entropy
Class 7	Prediction Theory	Prediction theory
Class 8	Discovery theory	Discovery theory
Class 9	Monte Carlo methods	Computational difficulty of sampling in high dimensional spaces
Class 10	Markov chain Monte Carlo Methods	Metropolis-Hastings methods
Class 11	Advanced Markov chain Monte Carlo Methods	Advanced Markov chain Monte Carlo Methods
Class 12	Variational inference methods	Variational inference methods
Class 13	Variational Mixture of Gaussians	Variational Mixture of Gaussians
Class 14	Tree graphs and belief propagation	Belief propagation
Class 15	Loopy graphs and loopy belief propagation	Loopy belief propagation

Textbook(s)

None.

Reference books, course materials, etc.

None. Two other lectures "Theory of statistical mathematics" and "Machine Learning" are strongly recommended for students.

Assessment criteria and methods

Reports.

Related courses

MCS.T507 ： Theory of Statistical Mathematics
ART.T458 ： Machine Learning

Prerequisites (i.e., required knowledge, skills, courses, etc.)

Linear algebra, differential and integral analysis, probability theory, and statistics are necessary.

TOKYO INSTITUTE OF TECHNOLOGY