Machine learning theory and statistical mechanics are introduced. In the first half, hierarchical learning machines for accurate prediction and knowledge discovery are explained and its mathematical laws are derived. In the second half, statistical mechanical approximation theory for handling massive information processing is introduced.
The purpose of statistical learning is to estimate the true information source from empirical samples. In this course, several learning machines which have high dimensional parameters are introduced. Statistical mechanics theory plays an important role in studying such learning machines. Two other courses, ``Theory of statistical mathematics" and ``Machine learning" are strongly recommended for students.
Statistics, Information Theory, Statistical mechanics, Free energy, and Entropy
✔ Specialist skills | Intercultural skills | Communication skills | Critical thinking skills | ✔ Practical and/or problem-solving skills |
This course consists of two parts. Machine learning theory and statistical mechanics are introduced.
Course schedule | Required learning | |
---|---|---|
Class 1 | Introduction of statistical learning theory | statistical learning theory |
Class 2 | Neural network architecture | neural network |
Class 3 | Learning in neural networks | learning in neural networks |
Class 4 | Boltzmann machine | Boltzmann machine |
Class 5 | Deep Learning | Deep learning |
Class 6 | Information and relative entropy | Information and relative entropy |
Class 7 | Prediction Theory | Prediction theory |
Class 8 | Discovery theory | Discovery theory |
Class 9 | Similarity between Bayesian inference and statistical mechanics | Inference based on Bayes Formula, canonical distribution and free energy |
Class 10 | Ideal gas and Ising model | Ideal gas, Ising model, equation of state |
Class 11 | Mean field approximations | Molecular field and Bethe approximations in Ising model |
Class 12 | Belief propagation for inference on sparse graphs | Graphical model for probabilistic models, belief propagation |
Class 13 | Belief propagation for inference on dense graphs | Approximate message passing |
Class 14 | Statistical mechanical formalism | Moment evaluation based on free energies, linear response relation |
Class 15 | Free energy and hyper parameter estimation | Variational principle, variational Bayes, EM-algorithm |
None.
None. Two other lectures "Theory of statistical mathematics" and "Machine Learning"are strongly recommended for students.
Reports.
Probability theory and statistics are necessary.