Through the development of cloud computing environments, the demand for efficiently processing and utilizing large-scale data is increasing. However, there are generally still few opportunities to experience applied examples of distributed processing technology for large-scale data, making it difficult to learn those skills and know-how. In this course we will use the educational cloud, constructed and operated by the National Institute of Informatics, as an exercise environment. By utilizing material focused on actual examples, students are able to experience distributed processing application development in practice.
In this course students will learn about distributed processing technology for practical large-scale data, mainly through exercises.
By the end of this course, students will be able to grasp techniques of practical distributed computation handling big data.
Distributed file system, big data processing, testing, performance tuning.
✔ Specialist skills | Intercultural skills | Communication skills | ✔ Critical thinking skills | ✔ Practical and/or problem-solving skills |
Students will use the educational cloud constructed and operated by the National Institute of Informatics as an exercise environment, and proceed with lessons mainly through exercises.
Course schedule | Required learning | |
---|---|---|
Class 1 | Construction of Development Environment and Overview of Hadoop | Overview of Hadoop |
Class 2 | Application of MapReduce | Application of MapReduce |
Class 3 | Advanced Technique of MapReduce | Advanced Technique of MapReduce |
Class 4 | Testing of MapReduce Application Software | Testing of MapReduce Application Software |
Class 5 | Performance Tuning in Hadoop | Performance Tuning in Hadoop |
Class 6 | Operation and Monitoring of Hadoop | Operation and Monitoring of Hadoop |
Class 7 | Programming Practices of Hadoop | Programming Practices of Hadoop |
Handouts will be provided during the course.
The materials will be provided from the Web site which will be introduced during the course.
Practice Q&A: 20%
Report: 80%
This course is given in Japanese.