Print

2024　Statistical Learning Theory

Font size  SML

Register update notification mail Add to favorite lecture list
Academic unit or major
Graduate major in Mathematical and Computing Science
Instructor(s)
Kanamori Takafumi 
Class Format
Lecture    (Face-to-face)
Media-enhanced courses
Day/Period(Room No.)
Tue7-8(M-155(H1104))  Fri7-8(M-155(H1104))  
Group
-
Course number
MCS.T403
Credits
2
Academic year
2024
Offered quarter
1Q
Syllabus updated
2024/3/18
Lecture notes updated
-
Language used
English
Access Index

Course description and aims

Some advanced topics and theories related to statistics and machine learning are taught. More specifically, a nonparametric method called kernel method, statistical properties of training and prediction errors, prediction error bound using Rademacher complexity, and universal approximation theorem of neural networks are taught.

Student learning outcomes

[Objectives] Statistical science and machine learning are disciplines in which useful information is extracted from data to aid
prediction and decision making. Students will learn methodology not simply as knowledge but also learning the background theory including the validity of those methods to promote understanding the essence. Students will broadly apply all kinds of techniques to a variety of problems, learning to construct new techniques on one's own.
[Topics] Students in this course will learn several of statistical science's more advanced techniques, based on their connection to various application fields. We will focus in particular on the connection with machine learning, introducing central topics from both statistical science and machine learning.

Keywords

machine learning, statistics, kernel methods, spline method, prediction error, Rademacher complexity, statistical consistency, neural networks, universal approximation theorem, stochastic gradient descent method

Competencies that will be developed

✔ Specialist skills Intercultural skills Communication skills ✔ Critical thinking skills ✔ Practical and/or problem-solving skills

Class flow

Lectures using slides. Students need to access T2SCHOLA to solve quizzes in class, so please bring your own laptop, tablet, etc.

Course schedule/Required learning

  Course schedule Required learning
Class 1 Overview of statistical learning and Regression analysis Overview the statistical learning through some practical examples. Learn the problem setup of regression analysis.
Class 2 Regression analysis Understand statistical modeling in kernel regression analysis
Class 3 Regression analysis and kernel methods Understand cross validation method for regression analysis. Learn statistical inference using kernel method.
Class 4 Kernel methods: reproducing property, representer theorem, etc. Understand reproducing property, representer theorem, etc used in statistical learning with kernels.
Class 5 Kernel method and reproducing kernel Hilbert space Understand some properties of reproducing kernel Hilbert space defined from kernel function.
Class 6 Spline smoothing and kernel methods Learn the relationship between spline smoothing methods and kernel methods.
Class 7 Spline smoothing and kernel methods Learn B-spline and multi-dimensional spline regression.
Class 8 Inequalities of probabilities in machine learning Understand some probabilistic inequalities used in the theory of machine learning.
Class 9 Foundation of statistical learning theory Understand the problem setting of statistical learning theory.
Class 10 Prediction error and Model Selection Learn the prediction error of statistical learning and model selection methods.
Class 11 Rademacher complexity Learn Rademacher complexity to measure the statistical models.
Class 12 Uniform law of large numbers and statistical consistency of learning algorithms Learn the uniform law of large numbers, an extension of the law of large numbers, and prove the statistical consistency of learning algorithms.
Class 13 Neural network models and universal approximation theorem Understand Neural network models and learn the universal approximation theorem for neural networks.
Class 14 Deep learning models and B-spline method. Understand deep neural network models and B-spline method.

Out-of-Class Study Time (Preparation and Review)

To enhance effective learning, students are encouraged to spend approximately 100 minutes preparing for class and another 100 minutes reviewing class content afterwards (including assignments) for each class.

Textbook(s)

Unspecified.

Reference books, course materials, etc.

Course materials are provided on T2SCHOLA during the course.
Reference book:
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar, Foundations of Machine Learning, MIT Press, Second Edition, 2018.
Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014.

Assessment criteria and methods

Based on quizzes in class and report submissions

Related courses

  • MCS.T223 ： Mathematical Statistics
  • ART.T458 ： Machine Learning

Prerequisites (i.e., required knowledge, skills, courses, etc.)

Students must have basic knowledge of probability theory and statistics.

Page Top