227-0423-00L  Neural Network Theory

SemesterAutumn Semester 2021
LecturersH. Bölcskei
Periodicityyearly recurring course
Language of instructionEnglish


AbstractThe class focuses on fundamental mathematical aspects of neural networks with an emphasis on deep networks: Universal approximation theorems, capacity of separating surfaces, generalization, fundamental limits of deep neural network learning, VC dimension.
ObjectiveAfter attending this lecture, participating in the exercise sessions, and working on the homework problem sets, students will have acquired a working knowledge of the mathematical foundations of neural networks.
Content1. Universal approximation with single- and multi-layer networks

2. Introduction to approximation theory: Fundamental limits on compressibility of signal classes, Kolmogorov epsilon-entropy of signal classes, non-linear approximation theory

3. Fundamental limits of deep neural network learning

4. Geometry of decision surfaces

5. Separating capacity of nonlinear decision surfaces

6. Vapnik-Chervonenkis (VC) dimension

7. VC dimension of neural networks

8. Generalization error in neural network learning
Lecture notesDetailed lecture notes are available on the course web page
Link
Prerequisites / NoticeThis course is aimed at students with a strong mathematical background in general, and in linear algebra, analysis, and probability theory in particular.