263-5354-00L  Large Language Models

SemesterSpring Semester 2023
LecturersR. Cotterell, M. Sachan, F. Tramèr, C. Zhang
Periodicityyearly recurring course
Language of instructionEnglish



Courses

NumberTitleHoursLecturers
263-5354-00 VLarge Language Models3 hrs
Tue14:15-16:00CAB G 61 »
Fri10:15-11:00CAB G 61 »
R. Cotterell, M. Sachan, F. Tramèr, C. Zhang
263-5354-00 ULarge Language Models2 hrs
Thu16:15-18:00NO C 60 »
R. Cotterell, M. Sachan, F. Tramèr, C. Zhang
263-5354-00 ALarge Language Models2 hrsR. Cotterell, M. Sachan, F. Tramèr, C. Zhang

Catalogue data

AbstractLarge language models have become one of the most commonly deployed NLP inventions. In the past half-decade, their integration into core natural language processing tools has dramatically increased the performance of such tools, and they have entered the public discourse surrounding artificial intelligence.
Learning objectiveTo understand the mathematical foundations of large language models as well as how to implement them.
ContentWe start with the probabilistic foundations of language models, i.e., covering what constitutes a language model from a formal, theoretical perspective. We then discuss how to construct and curate training corpora, and introduce many of the neural-network architectures often used to instantiate language models at scale. The course covers aspects of systems programming, discussion of privacy and harms, as well as applications of language models in NLP and beyond.
LiteratureThe lecture notes will be supplemented with various readings from the literature.

Performance assessment

Performance assessment information (valid until the course unit is held again)
Performance assessment as a semester course
ECTS credits8 credits
ExaminersR. Cotterell, M. Sachan, F. Tramèr, C. Zhang
Typesession examination
Language of examinationEnglish
RepetitionThe performance assessment is only offered in the session after the course unit. Repetition only possible after re-enrolling.
Mode of examinationwritten 180 minutes
Additional information on mode of examinationThe exam will constitute 50% of the final grade. The remaining 50% will be based
on several assignments released during the semester.
Written aidsTwo A4-pages (i.e. one A4-sheet of paper), either handwritten or 11
This information can be updated until the beginning of the semester; information on the examination timetable is binding.

Learning materials

 
Main linkInformation
Only public learning materials are listed.

Groups

No information on groups available.

Restrictions

There are no additional restrictions for the registration.

Offered in

ProgrammeSectionType
CAS in Computer ScienceFocus Courses and ElectivesWInformation
Cyber Security MasterElective CoursesWInformation
Data Science MasterCore ElectivesWInformation
Computer Science MasterElective CoursesWInformation
Computer Science MasterMinor in Data ManagementWInformation
Computer Science MasterMinor in Machine LearningWInformation