This course introduces basic knowledge of machine learning and deep learning.
[Goal]
- Understand basic concepts (e.g., classification, convex optimization) and methods (e.g., stochastic gradient descent, back propagation) for discriminative models of machine learning.
- Realize machine learning with toolkits and programming.
[Theme] The first half of this lecture covers basic concept of machine learning with linear models and optimization. The second half of this lecture presents the fundamentals and practices of deep learning.
Machine learning, regression, classification, optimization, linear model, neural network, deep learning
✔ Specialist skills | Intercultural skills | Communication skills | Critical thinking skills | Practical and/or problem-solving skills |
This lecture includes explanations and exercises of machine learning toolkits.
Course schedule | Required learning | |
---|---|---|
Class 1 | introduction | Basic concept of Machine Learning |
Class 2 | Linear Model 1 | Loss functions, empirical loss minimization, overfitting, regularization, bias and variance, linear model (linear regression) |
Class 3 | Optimization 1 | Concept of optimization, gradient methods, constraint optimization. |
Class 4 | Optimization 2 | Convex optimization, Duality |
Class 5 | Linear Model 2 | Linear model (classification),logistic regression, linear and kernel support vector machines |
Class 6 | Linear Model 3 | L1 regularization, sparse learning, Lasso |
Class 7 | Scalable Learning | Stochastic gradient, accelerated gradients, moment, mini-batch, distributed parallel training |
Class 8 | Introduction to Deep Learning | Real-world applications |
Class 9 | Deep Feedforward Networks | multi-layer perceptron, hidden units, activation functions, computational graph |
Class 10 | Training Deep Models | gradient-based learning, back propagation, regularization, dropout |
Class 11 | Representation Learning | distributed representation, word embeddings, latent semantic analysis, skip-gram model |
Class 12 | Convolutional Networks | convolution, pooling, gated linear unit, feature extraction |
Class 13 | Recurrent and Recursive Nets | recurrent neural networks, recursive neural networks, long short-term memory, gated recurrent unit |
Class 14 | Encoder Decoder Modeling | sequence-to-sequence models, attention mechanisms |
Class 15 | Applications of Deep Learning | text generation, implementation |
Handouts will be given when necessary.
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press. 2016.
- Christopher Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics), 2010
-
Course marks are based on assignments (70%) and exercises (30%).
None
None.