2016 Theory of Statistical Mathematics

Font size  SML

Register update notification mail Add to favorite lecture list
Academic unit or major
Graduate major in Mathematical and Computing Science
Suzuki Taiji  Watanabe Sumio 
Course component(s)
Mode of instruction
Day/Period(Room No.)
Tue7-8(W832)  Fri7-8(W832)  
Course number
Academic year
Offered quarter
Syllabus updated
Lecture notes updated
Language used
Access Index

Course description and aims

Some advanced topics in statistics are taught. In particular, topics and theories related to machine learning are taught. More specifically, high dimensional data analysis methods and its theory are explained, and, in addition, a nonparametric method called "kernel method", unsupervised learning methods, and learning theory are taught. Moreover, efficient optimization techniques for machine learning are explained.

Student learning outcomes

[Objectives] Statistical science and machine learning are disciplines in which useful information is extracted from data to aid human decision making. Students will learn methodology not simply as knowledge, but also learning the background theory including the validity of those methods to promote understanding the essence. Students will broadly apply all kinds of techniques to a variety of problems, learning to construct new techniques on one's own.

[Topics] Students in this course will learn several of statistical science's more advanced techniques, based on their connection to various application fields. We will focus in particular on the connection with machine learning, introducing central topics from both statistical science and machine learning.


machine learning, statistics, high dimension data analysis, support vector machine, kernel method, optimization, convex analysis, unsupervised learning, learning theory

Competencies that will be developed

Specialist skills Intercultural skills Communication skills Critical thinking skills Practical and/or problem-solving skills

Class flow

Lectures are given using black board mainly.

Course schedule/Required learning

  Course schedule Required learning
Class 1 Machine learning and statistics Learn the overview of machine learning and know the relation between machine learning and statistics.
Class 2 Bias-variance trade-off Learn how bias-variance affects the predictive accuracy in supervised learning.
Class 3 Model selection Understand over-fitting phenomena and how to resolve it by model selection.
Class 4 High dimension sparse estimation Learn the methodology of sparse estimation that is useful in high dimensional data analysis.
Class 5 Several kinds of regularized learning methods. Learn several kinds of regularized learning methods.
Class 6 Statistical properties of high dimensional sparse estimation. Learn the statistical properties of sparse estimation in high dimensional settings, in particular, the error bound.
Class 7 Optimization method for regularized learning methods Learn some methods that are effective for regularized learning methods.
Class 8 Online learning Learn online learning frame-work in which data are supposed to come sequentially.
Class 9 Online stochastic optimization method Learn an efficient optimization method that utilize a small amount of data per iteration.
Class 10 Basics of kernel method Learn a kernel method that is a nonparametric method on reproducing kernel Hilbert space.
Class 11 Some applications of kernel method Learn several applications of kernel methods
Class 12 Unsupervised learning model Learn the frame-work of unsupervised learning problems, and some representative models.
Class 13 Methods and theories of unsupervised learning Learn some methodologies of unsupervised learning and their theories.
Class 14 Basics of PAC learning theory Learn PAC learning theory to analyze why machine learning works well.
Class 15 Advanced topics in PAC learning theory. Learn some advanced topics of PAC learning theory.



Reference books, course materials, etc.


Assessment criteria and methods

Evaluated by report submission.

Related courses

  • MCS.T223 : Mathematical Statistics
  • MCS.T402 : Mathematical Optimization: Theory and Algorithms
  • MCS.T403 : Statistical Learning Theory

Prerequisites (i.e., required knowledge, skills, courses, etc.)

No prerequisites. But, it is preferred that students know the basics of statistics and probability theory.

Page Top