The course provides a comprehensive introduction to deep learning.

Topics Covered:

1. Core machine learning concepts: hypothesis spaces, loss functions, risk minimization, overfitting and underfitting, model selection, and regularization

2. Training neural networks via stochastic gradient descent

3. Shallow neural networks: architecture, activation functions, and universal approximation

4. Deep neural networks: depth, skip connections, layer types, and backpropagation

5. Convolutional neural networks: convolutions, pooling, modern architectures, and transfer learning

6. Embeddings and transformers: self-attention, multi-head attention, positional encoding, and autoregressive decoder models

After completing this course, students will be able to:

• Explain the role of hypothesis spaces, loss functions, and regularization in supervised machine learning

• Describe the neural network training pipeline — optimization via SGD, backpropagation, and techniques for mitigating overfitting

• Compare and contrast architectures of deep neural networks, CNNs, and transformers

• Implement and train neural network models using modern deep learning frameworks

• Analyze and critically evaluate deep learning research papers, and present findings effectively

Attendance is mandatory

The course combines instructor-led lectures with hands-on tutorial sessions. Students prepare for each session by reviewing assigned materials in advance, and selected topics are supported by pre-recorded knowledge clips for self-study. In the final week, students present research papers or book chapters. Participation in discussions and tutorial work is encouraged throughout the course.

Grades will be awarded based on individual exercises (20%), a presentation at the end of the course (40%), and an exam (40%).

Familiarity with matrix algebra, multivariate calculus (chain rule, gradients, Jacobians), and basic probability. Programming experience in Python (NumPy, PyTorch) is helpful for the tutorial sessions but not required.

Please log in with your WU account to use all functionalities of read!t. For off-campus access to our licensed electronic resources, remember to activate your VPN connection connection. In case you encounter any technical problems or have questions regarding read!t, please feel free to contact the library at readinglists@wu.ac.at.

During the weeks of the sessions: before and after class; for longer question, by appointment.