!!! All practical examples will be done in Python Jupyter Notebook: laptops in class are required (from 1st class) !!!

1st class - Understanding Language Models (11 March)

2nd class - Understanding Large Language Models (18 March) 

3rd class - Training Large Language Models (25 March)

4th class - Attention mechanism (8 April)

5th class - Transformer architecture (15 April)

6th class - Presentations of the final projects (22 April)

• Assignment (before the last class on 22 April):
  • Submit your final project's notebooks/code/reports to the course GitHub repository

Course Materials

• Textbook: Build a Large Language Model (From Scratch) by Sebastian Raschka
• Software: Python, Jupyter Notebook

After completing this course students will have an in-depth knowledge of the architecture and implementation of modern large language models. Apart from that, they will be able to apply large language models to solve practical problems, such as text classification and question answering.

The rules on the attendance of a Continuous Assessment Course (PI) apply.

At least 80% attendance (physical presence) is required. Students who fail to meet the attendance requirement will be de-registered from the continuous assessment course with a “fail” grade.

The course will combine theoretical material with hands-on excercises. The instructor will present the course content using the slides; and the students will practice implementation of the introduced concepts in class.

The final grade will be computed on the basis of:

• Short multiple choice quizzes on the content of the previous lecture via Canvas 40%
• Programming project, presentation in the final class 50%
• Active class participation 10%

For the programming project participants select a topic in coordination with the lecturer. The project involves application of a large language model to a practical task, such as text classification or question answering.

Grades:

1: =>95%

2: =>85%

3: =>75%

4: =>60%

It is possible to ask for an additional homework assignment if you are missing points from the quizzes. For that send an email to the lecturer with your proposal: what you want to do (related to the class topics) and for how many points.

The quizz questions have to be answered in class. If you are not able to attend reach out to the lecturer via email in advance.

Successful conclusion of the course 1 of SBWL Data Science.

Please be aware that, for all courses in this SBWL, registration is only possibly for students who successfully have completed the entry course (Einstieg in die SBWL: Data Science).

Note that for courses within the SBWL "Data Science" we can only accept students enrolled in one of WU's bachelor programmes who qualify for starting an SBWL; particularly, we cannot accept students from other courses and programmes enrolled at WU as 'Mitbeleger' only.

Please log in with your WU account to use all functionalities of read!t. For off-campus access to our licensed electronic resources, remember to activate your VPN connection connection. In case you encounter any technical problems or have questions regarding read!t, please feel free to contact the library at readinglists@wu.ac.at.

A working knowledge of the Python programming language is required.