Print

2020　Statistical Theories for Brain and Parallel Computing

Font size  SML

Register update notification mail Add to favorite lecture list
Academic unit or major
Graduate major in Information and Communications Engineering
Instructor(s)
Kumazawa Itsuo 
Class Format
Lecture    (ZOOM)
Media-enhanced courses
Day/Period(Room No.)
Tue1-2(Zoom)  Fri1-2(Zoom)  
Group
-
Course number
ICT.H416
Credits
2
Academic year
2020
Offered quarter
3Q
Syllabus updated
2020/10/2
Lecture notes updated
-
Language used
English
Access Index

Course description and aims

Summary: Some attempts are introduced to analyze and understand principals behind brain function and massively parallel computation. Methods of statistical physics and probabilistic computation are lectured in addition to programming exercises to confirm the behavior of the parallel systems based on these methods.
Purpose: Study theories of the statistical mechanics to analisys and design of highly parallel system. Make programs of neural networks and apply them to various problems.

Student learning outcomes

Basic knowledge on brain computation and models of its ultra parallel computation is learned. Mathematical and statistical theories to understand the highlly parallel system are mastered. Statistical or probablistic computation technique is learned to make programs of highly parallel computation systems. Through programming exercises of parallel computation systems, the effectiveness of the theories is confirmed and programming skills are improved.

Keywords

Brain, Neural Network, Parallel Computation, Statistical Mechanics, Optimazation Problem, Learning

Competencies that will be developed

✔ Specialist skills Intercultural skills Communication skills ✔ Critical thinking skills ✔ Practical and/or problem-solving skills

Class flow

Lectures and Progrmming Excircises are organized to study theories and examine their performances and effectiveness on practical proble,ms.

Course schedule/Required learning

  Course schedule Required learning
Class 1 Introduction of biological neural network (Neurons and Neural Networks) Study biological neural networks and how they work.
Class 2 Artifical models of neuron (Derministic and probablistic models. Binary and continuous models). By analogy to biological neural networks, engineering models of neural networks areintroduced.
Class 3 Artifical models of neural network (Recurrent and Feed forward models). As a typical models of neural networks, recurrent models and feed forward models are studied.
Class 4 Introduction of statistical mechanics (Magnetic Systems and Spin Glass Models). As a theoretical foundation to understand recurrent models, statistical mechanics is stduied.
Class 5 How to understand the behavior of highly parallel system like a brain. Analogy between Neural Networks and Spin Glass Models. By analogy with spin glass models, parallel computation of recurrent models is studied. The concept of energy is introduced to the parallel computation model of the brain.
Class 6 Enegy minimization by the deterministic models of recurrent neural network. It is proven that the enegy is reduced by the deterministic models of recurrent neural network
Class 7 Analysis of Neural Computaion by Boltsmann's Theory. Theorerical analysys is conducted for the probablistic computation model of brain by using the methods developed in statistical mechanics.
Class 8 Computer simulation of deterministic and probablistic models of recurrent neural network. Computer simulation of deterministic and probablistic models is conducted to confirm the capability of recurrent neural network
Class 9 Application of a recurrent neural network for solving simultaneous equations and Four Queen Problem. Simultaneous equations and optimization problems are implemented in the recurrent neural networks and shown to be solved by using their energy minimization capabilities.
Class 10 Tips for efficial computation: Ergodicity and automatic determination of connection weight. Use of elgodicity property is studied to improve the computation efficiency of the recurrent neural networks. Ways of determing weights and thresholds to solve a given problem are studied.
Class 11 Programing and application of the recurrent neural network Part 1: Eight Queen Problem. We study the formulation and programming of the recurrent neural netowork to solve Eight Queen Problem that is a popular problem for programming exercise.
Class 12 Programing and application of the recurrent neural network Part 2: Travelling Salesman Problem. We study the formulation and programming of the recurrent neural netowork to solve Travelling Salesman Problem that is aknown to be a difficult combinatorial problem.
Class 13 Learning of the recurrent neural network Part 1: Mathematical techniques. We study mathematical techniques for learning metods of recurrent neural networks.
Class 14 Learning of the recurrent neural network Part 2: Learning algorithm for Boltzmann Machine. We derive a learning algorithm for one of the recurrent neural networks : Boltzmann Machine.

Out-of-Class Study Time (Preparation and Review)

To enhance effective learning, students are encouraged to spend approximately 100 minutes preparing for class and another 100 minutes reviewing class content afterwards (including assignments) for each class.
They should do so by referring to textbooks and other course material.

Textbook(s)

"Learnig and Neural Netwoks" written by Itsuo Kumazawa and published by Morikita Publishing Company is used as an text book. As the book is written in Japanese, it is used partially and as an assist for the lecture that is given in English.

Reference books, course materials, etc.

Introduction to the Theory of Neural Computation, written by J. Hertz, A. Krogh and R.G. Palmer and published by Westview Press.

Assessment criteria and methods

Assignments on implementation of what studied in the courses. Submitted reports on computer simulation and discussion on the simulation results are evaluated.

Related courses

  • ICT.M202 ： Probability and Statistics (ICT)
  • ICT.S302 ： Functional Analysis and Inverse Problems
  • ICT.H318 ： Foundations of Artificial Intelligence (ICT)
  • ICT.P204 ： Basic Computer Programming (ICT)
  • ICT.M306 ： Concrete Mathematics

Prerequisites (i.e., required knowledge, skills, courses, etc.)

Programming with any computer language or such applications as Matlab or mathematica is required to conduct computer simulation of various methods taught in this course.

Contact information (e-mail and phone)    Notice : Please replace from "[at]" to "@"(half-width character).

kumazawa.i.aa[at]m.titech.ac.jp

Office hours

10:00-19:00 on weekdays

Page Top