Summary: Some attempts are introduced to analyze and understand principals behind brain function and massively parallel computation. Methods of statistical physics and probabilistic computation are lectured in addition to programming exercises to confirm the behavior of the parallel systems based on these methods.
Purpose: Study theories of the statistical mechanics to analisys and design of highly parallel system. Make programs of neural networks and apply them to various problems.
Basic knowledge on brain computation and models of its ultra parallel computation is learned. Mathematical and statistical theories to understand the highlly parallel system are mastered. Statistical or probablistic computation technique is learned to make programs of highly parallel computation systems. Through programming exercises of parallel computation systems, the effectiveness of the theories is confirmed and programming skills are improved.
Brain, Neural Network, Parallel Computation, Statistical Mechanics, Optimazation Problem, Learning
Intercultural skills | Communication skills | Specialist skills | Critical thinking skills | Practical and/or problem-solving skills |
---|---|---|---|---|
- | - | ✔ | ✔ | ✔ |
Lectures and Progrmming Excircises are organized to study theories and examine their performances and effectiveness on practical proble,ms.
Course schedule | Required learning | |
---|---|---|
Class 1 | Introduction of biological neural network (Neurons and Neural Networks) | Study biological neural networks and how they work. |
Class 2 | Artifical models of neuron (Derministic and probablistic models. Binary and continuous models). | By analogy to biological neural networks, engineering models of neural networks areintroduced. |
Class 3 | Artifical models of neural network (Recurrent and Feed forward models). | As a typical models of neural networks, recurrent models and feed forward models are studied. |
Class 4 | Introduction of statistical mechanics (Magnetic Systems and Spin Glass Models). | As a theoretical foundation to understand recurrent models, statistical mechanics is stduied. |
Class 5 | How to understand the behavior of highly parallel system like a brain. Analogy between Neural Networks and Spin Glass Models. | By analogy with spin glass models, parallel computation of recurrent models is studied. The concept of energy is introduced to the parallel computation model of the brain. |
Class 6 | Enegy minimization by the deterministic models of recurrent neural network. | It is proven that the enegy is reduced by the deterministic models of recurrent neural network |
Class 7 | Analysis of Neural Computaion by Boltsmann's Theory. | Theorerical analysys is conducted for the probablistic computation model of brain by using the methods developed in statistical mechanics. |
Class 8 | Computer simulation of deterministic and probablistic models of recurrent neural network. | Computer simulation of deterministic and probablistic models is conducted to confirm the capability of recurrent neural network |
Class 9 | Application of a recurrent neural network for solving simultaneous equations and Four Queen Problem. | Simultaneous equations and optimization problems are implemented in the recurrent neural networks and shown to be solved by using their energy minimization capabilities. |
Class 10 | Tips for efficial computation: Ergodicity and automatic determination of connection weight. | Use of elgodicity property is studied to improve the computation efficiency of the recurrent neural networks. Ways of determing weights and thresholds to solve a given problem are studied. |
Class 11 | Programing and application of the recurrent neural network Part 1: Eight Queen Problem. | We study the formulation and programming of the recurrent neural netowork to solve Eight Queen Problem that is a popular problem for programming exercise. |
Class 12 | Programing and application of the recurrent neural network Part 2: Travelling Salesman Problem. | We study the formulation and programming of the recurrent neural netowork to solve Travelling Salesman Problem that is aknown to be a difficult combinatorial problem. |
Class 13 | Learning of the recurrent neural network Part 1: Mathematical techniques. | We study mathematical techniques for learning metods of recurrent neural networks. |
Class 14 | Learning of the recurrent neural network Part 2: Learning algorithm for Boltzmann Machine. | We derive a learning algorithm for one of the recurrent neural networks : Boltzmann Machine. |
Class 15 | Computer Simulation of a leaning algorithm of recurrent neural network. | Programming and computer simulation of a leaning algorithm of a recurrent neural network are conducted to confirm its capability. |
"Learnig and Neural Netwoks" written by Itsuo Kumazawa and published by Morikita Publishing Company is used as an text book. As the book is written in Japanese, it is used partially and as an assist for the lecture that is given in English.
Introduction to the Theory of Neural Computation, written by J. Hertz, A. Krogh and R.G. Palmer and published by Westview Press.
Assignments on implementation of what studied in the courses. Submitted reports on computer simulation and discussion on the simulation results are evaluated.
Programming with any computer language or such applications as Matlab or mathematica is required to conduct computer simulation of various methods taught in this course.
kumazawa.i.aa[at]m.titech.ac.jp
10:00-19:00 on weekdays