2018 Advanced Data Engineering

Font size  SML

Register update notification mail Add to favorite lecture list
Academic unit or major
Graduate major in Computer Science
Instructor(s)
Yokota Haruo 
Course component(s)
Lecture
Day/Period(Room No.)
Mon5-6(W831)  Thr5-6(W831)  
Group
-
Course number
CSC.T523
Credits
2
Academic year
2018
Offered quarter
2Q
Syllabus updated
2018/3/20
Lecture notes updated
2018/6/9
Language used
English
Access Index

Course description and aims

The data engineering is an active research area focuses on the sophisticated processing of a large amount of various data in computer systems, such as processing advanced databases.
This course aims to let students learn advanced methodologies and mechanisms for manipulating a large amount of data efficiently through understanding various contemporary technologies of data engineering, including application examples, data structures, indexing, processing algorithms, and parallel processing methods for highly functional and high-speed processing of a large amount of data.

Student learning outcomes

By the end of this course, students will be able to
1) Understand the basic concept of data engineering and its basics: Relational databases and transaction processing
2) Understand technologies for data warehouse as a typical application of data engineering
3) Understand data structure and algorithms of OLAP and data mining executed in the data warehouse
4) Understand implementation algorithms and costs of relational database operations for the data warehouse
5) Understand parallelization approaches for high-speed relational database operations
6) Understand skew handling methods for parallel database operations
7) Understand distributed database processing including a database in the cloud
8) Understand trends of recent XML/RDF databases

Keywords

Data Warehousing, OLAP, Data Mining, Indexing Methods, Parallel Database Operations, Data Placement, Skew Handling, Cloud Database, XML/RDF databases

Competencies that will be developed

Intercultural skills Communication skills Specialist skills Critical thinking skills Practical and/or problem-solving skills
- - -

Class flow

Standard Lecture

Course schedule/Required learning

  Course schedule Required learning
Class 1 Basic Concept and Background of Data Engineering Understand the basic concept of data engineering
Class 2 Relational Database and Transaction Procesing Understand relational databases and transaction processing
Class 3 Data Warehouse Understand Data Warehouse
Class 4 OLAP and Data Mining Understand OLAP and Data Mining in Data Warehouse
Class 5 Data Structure and Index Understand Data Structure and Index Structure for Data Engineering
Class 6 Estimate Cost of Relational Algebra Operations 1: Selection, Projection Understand Algorithms and Cost for Selection and Projection Operations
Class 7 Estimate Cost of Relational Algebra Operations 2: Join, Aggregate Functions Understand Algorithms and Costs for Join Operation and Aggregate Functions
Class 8 Classify Parallelize Database Operations and Data Partitioning Understand Classification of Parallel Database Processing and Data Distributiion
Class 9 Parallel Join Operations 1: Sort Merge Join Understand Algorithm and Costs of Parallel Merge Sort Join
Class 10 Parallel Join Operations 2: Hash Join Understand Algorithm and Costs of Parallel Hash Join
Class 11 Parallel Aggregate Functions Understand Algorithm and Cost of Parallel Aggregation Functions
Class 12 Skew Handling Understand Skew Handling Algorithm for Parallel Database Processing
Class 13 Distributed Database Processing Understand Distributed Database Processing
Class 14 XML Databases and RDF Databases Understand XML Databases and RDF Databases
Class 15 Cloud and Databases Understad Database Processing in Cloud Environment

Textbook(s)

Distribute manuscripts through OCW/OCW-i

Reference books, course materials, etc.

Jim Gray and Andreas Reuter著「Transaction Processing: Concept and Techniques」 Morgan Kaufmann Publishers,

Assessment criteria and methods

Assignments in Lectures (60%) and Final Report (40%)

Related courses

  • CSC.T343 : Databases

Prerequisites (i.e., required knowledge, skills, courses, etc.)

Basic knowledge of databases and computer architecture

Contact information (e-mail and phone)    Notice : Please replace from "[at]" to "@"(half-width character).

yokota[at]cs.titech.ac.jp

Page Top