263-5255-00L  Foundations of Reinforcement Learning

Courses

Catalogue data

Abstract	Reinforcement learning (RL) has been in the limelight of many recent breakthroughs in artificial intelligence. This course focuses on theoretical and algorithmic foundations of reinforcement learning, through the lens of optimization, modern approximation, and learning theory. The course targets M.S. students with strong research interests in reinforcement learning, optimization, and control.
Learning objective	This course aims to provide students with an advanced introduction of RL theory and algorithms as well as bring them near the frontier of this active research field. By the end of the course, students will be able to - Identify the strengths and limitations of various reinforcement learning algorithms; - Formulate and solve sequential decision-making problems by applying relevant reinforcement learning tools; - Generalize or discover “new” applications, algorithms, or theories of reinforcement learning towards conducting independent research on the topic.
Content	Topics include fundamentals of Markov decision processes, approximate dynamic programming, linear programming and primal-dual perspectives of RL, model-based and model-free RL, policy gradient and actor-critic algorithms, Markov games and multi-agent RL. If time allows, we will also discuss advanced topics such as batch RL, inverse RL, causal RL, etc. The course keeps strong emphasis on in-depth understanding of the mathematical modeling and theoretical properties of RL algorithms.
Lecture notes	Lecture slides will be posted on Moodle.
Literature	Dynamic Programming and Optimal Control, Vol I & II, Dimitris Bertsekas Reinforcement Learning: An Introduction, Second Edition, Richard Sutton and Andrew Barto. Algorithms for Reinforcement Learning, Csaba Czepesvári. Reinforcement Learning: Theory and Algorithms, Alekh Agarwal, Nan Jiang, Sham M. Kakade.
Prerequisites / Notice	Students are expected to have strong mathematical background in linear algebra, probability theory, optimization, and machine learning.
Competencies	Subject-specific Competencies Concepts and Theories assessed Techniques and Technologies fostered Method-specific Competencies Analytical Competencies assessed Decision-making fostered Problem-solving assessed Project Management assessed Social Competencies Communication assessed Cooperation and Teamwork assessed Leadership and Responsibility fostered Self-presentation and Social Influence fostered Personal Competencies Adaptability and Flexibility fostered Creative Thinking assessed Critical Thinking assessed Integrity and Work Ethics fostered Self-awareness and Self-reflection fostered Self-direction and Self-management fostered

Performance assessment

Performance assessment information (valid until the course unit is held again)
Performance assessment as a semester course
ECTS credits	7 credits
Examiners	N. He
Type	graded semester performance
Language of examination	English
Repetition	Repetition only possible after re-enrolling for the course unit.
Additional information on mode of examination	project 60%, homework 40% Last cancellation/deregistration date for this graded semester performance: March 8, 2024! Please note that after that date no deregistration will be accepted and the course will be considered as "fail".

Learning materials

Groups

Restrictions

Offered in

Programme	Section	Type
CAS in Computer Science	Focus Courses and Electives	W
Cyber Security Master	Elective Courses	W
DAS in Data Science	Electives	W
Data Science Master	Core Electives	W
Doctorate Computer Science	Subject Specialisation	W
Doctorate Information Technology and Electrical Engineering	Subject Specialisation	W
Computer Science Master	Elective Courses	W
Computer Science Master	Minor in Machine Learning	W
Mathematics Master	Machine Learning	W
Computational Science and Engineering Master	Electives	W
Statistics Master	Subject Specific Electives	W