Advanced Topics in Machine Learning

MSc - Spring Semester
422828, Lectures and exercises, 5.0 ECTS

Lecturer Prof. Dr. Paolo Favaro
Teaching assistants Mr. Simon Jenni
Mr. Adam Bielski
Mr. Abdelhak Lemkhenter
Location Seminarraum 002 HRZ building Engehaldenstrasse 8
Time Tuesdays 09.15-11.00 (lecture) and 11.15-12.00 (tutorials)
Exam June 11 2019 from 10:00-12:00 at Engehaldenstrasse 8, Room 002
KSL

Course description

This course provides an introduction to deep learning methods. These are modern methods in artificial intelligence (AI), which are incorporated into most of the top-performing algorithms in several fields of research. We focus on the machine learning paradigm, where rules are learned from examples, rather than being hard-coded. Most examples will be in computer vision, that is, about problems of object recognition, detection, segmentation in images and videos. On a successful completion of this course, you may be able to get to know how to use the software to automate routine work, understand images and videos (and you should be able to work with other data too), and support basic scientific research.

The course will cover most of the chapters of the deep learning book by Goodfellow, Bengio, and Courville: Review of Machine Learning, Deep feedforward networks (gradient-based learning, back-propagation, regularization, optimization, training) Convolutional Neural Networks, Recurrent Neural Networks, Autoenders, Representation learning.

Prerequisites

Applied math fundamentals like linear algebra, probability and numerical optimization.

Resources

Deep learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville (available online at www.deeplearningbook.org)

Neural Networks and Deep Learning by Michael Nielsen (free online book at www.neuralnetworksanddeeplearning.com)

Pattern Recognition and Machine Learning by Christopher M. Bishop

Learning Outcomes

On successful completion of this course students are expected to:

  1. be able to formalize tasks in computer vision via neural networks
  2. design neural networks to solve data-rich tasks
  3. build datasets, tune and train neural networks with advanced deep learning libraries
  4. understand the inner mechanisms of neural networks during training
  5. analyze the performance of neural networks on tasks of interest.

Projects and assignments

There will be a project and 5 small biweekly assignments. The project will be carried out with group work while the assignments will be carried out individually. The project will require 4 short group presentations and the submission of an individual report and code. All the work will contribute to the final grade as follows:

  • Project 30% (5% each for the 2 presentations + 20% report & code)
  • Assignments 20%
  • Final exam 50%.

The project report should be prepared as a jupyter notebook, similar to this sample report.

Schedule and material

The following table provides an overview of the content of the lectures during the semester. Please check it periodically as it might be updated.

Week Lecture Reading
1 Introduction
2 Machine Learning Review Ch 5, DL book
3 Deep feedforward nets Ch 6, DL book
4 Deep feedforward nets Ch 6, DL book
5 Deep feedforward nets – regularization Ch 7, DL book
6 Deep feedforward nets - optimization Ch 8, DL book
7 Deep feedforward nets – training Ch 8, DL book
8 ConvNets Ch 9, DL book
9 Deep RNN Ch 10, DL book
10 Deep RNN Ch 10, DL book
11 Practical methodology Ch 11, DL book
12 Autoencoders Ch 14, DL book
13 Representation learning Ch 15, DL book
14 Review