252-0526-00L  Statistical Learning Theory

SemesterFrühjahrssemester 2019
DozierendeJ. M. Buhmann
Periodizitätjährlich wiederkehrende Veranstaltung
LehrspracheEnglisch



Lehrveranstaltungen

NummerTitelUmfangDozierende
252-0526-00 VStatistical Learning Theory3 Std.
Mo14:15-16:00HG E 5 »
Di09:15-10:00HG E 5 »
J. M. Buhmann
252-0526-00 UStatistical Learning Theory2 Std.
Mo16:15-18:00HG E 5 »
J. M. Buhmann
252-0526-00 AStatistical Learning Theory1 Std.J. M. Buhmann

Katalogdaten

KurzbeschreibungThe course covers advanced methods of statistical learning :
Statistical learning theory;variational methods and optimization, e.g., maximum entropy techniques, information bottleneck, deterministic and simulated annealing; clustering for vectorial, histogram and relational data; model selection; graphical models.
LernzielThe course surveys recent methods of statistical learning. The fundamentals of machine learning as presented in the course "Introduction to Machine Learning" are expanded and in particular, the theory of statistical learning is discussed.
Inhalt# Theory of estimators: How can we measure the quality of a statistical estimator? We already discussed bias and variance of estimators very briefly, but the interesting part is yet to come.

# Variational methods and optimization: We consider optimization approaches for problems where the optimizer is a probability distribution. Concepts we will discuss in this context include:

* Maximum Entropy
* Information Bottleneck
* Deterministic Annealing

# Clustering: The problem of sorting data into groups without using training samples. This requires a definition of ``similarity'' between data points and adequate optimization procedures.

# Model selection: We have already discussed how to fit a model to a data set in ML I, which usually involved adjusting model parameters for a given type of model. Model selection refers to the question of how complex the chosen model should be. As we already know, simple and complex models both have advantages and drawbacks alike.

# Statistical physics models: approaches for large systems approximate optimization, which originate in the statistical physics (free energy minimization applied to spin glasses and other models); sampling methods based on these models
SkriptA draft of a script will be provided;
transparencies of the lectures will be made available.
LiteraturHastie, Tibshirani, Friedman: The Elements of Statistical Learning, Springer, 2001.

L. Devroye, L. Gyorfi, and G. Lugosi: A probabilistic theory of pattern recognition. Springer, New York, 1996
Voraussetzungen / BesonderesRequirements:

knowledge of the Machine Learning course
basic knowledge of statistics, interest in statistical methods.

It is recommended that Introduction to Machine Learning (ML I) is taken first; but with a little extra effort Statistical Learning Theory can be followed without the introductory course.

Leistungskontrolle

Information zur Leistungskontrolle (gültig bis die Lerneinheit neu gelesen wird)
Leistungskontrolle als Semesterkurs
ECTS Kreditpunkte7 KP
PrüfendeJ. M. Buhmann
FormSessionsprüfung
PrüfungsspracheEnglisch
RepetitionDie Leistungskontrolle wird nur in der Session nach der Lerneinheit angeboten. Die Repetition ist nur nach erneuter Belegung möglich.
Prüfungsmodusschriftlich 180 Minuten
Zusatzinformation zum Prüfungsmodus70% session examination, 30% project; the final grade will be calculated as weighted average of both these elements. As a compulsory continuous performance assessment task, the project must be passed on its own and has a bonus/penalty function.

The practical project are an integral part (60 hours of work, 2 credits) of the course. Participation is mandatory.
Failing the project results in a failing grade for the overall examination of Statistical Learning Theory (252-0526-00S).

Students who fail to fulfil the project requirement have to de-register from the exam. Otherwise, they are not admitted to the exam and they will be treated as a no show.
Hilfsmittel schriftlich2 sheets A4 (= 4 pages) summary, script
Diese Angaben können noch zu Semesterbeginn aktualisiert werden; verbindlich sind die Angaben auf dem Prüfungsplan.

Lernmaterialien

 
HauptlinkInformation
AufzeichnungStatistical Learning Theory recorings
Es werden nur die öffentlichen Lernmaterialien aufgeführt.

Gruppen

Keine Informationen zu Gruppen vorhanden.

Einschränkungen

Keine zusätzlichen Belegungseinschränkungen vorhanden.

Angeboten in

StudiengangBereichTyp
CAS in InformatikFokusfächer und WahlfächerWInformation
Computational Biology and Bioinformatics MasterTheorieWInformation
DAS in Data ScienceMachine Learning and Artificial IntelligenceWInformation
Data Science MasterWählbare KernfächerWInformation
Elektrotechnik und Informationstechnologie MasterEmpfohlene FächerWInformation
Elektrotechnik und Informationstechnologie MasterEmpfohlene FächerWInformation
Elektrotechnik und Informationstechnologie MasterVertiefungsfächerWInformation
Elektrotechnik und Informationstechnologie MasterEmpfohlene FächerWInformation
Elektrotechnik und Informationstechnologie MasterVertiefungsfächerWInformation
Elektrotechnik und Informationstechnologie MasterVertiefungsfächerWInformation
Informatik MasterWahlfächer der Vertiefung General StudiesWInformation
Informatik MasterWahlfächer der Vertiefung in Visual ComputingWInformation
Informatik MasterWahlfächer der Vertiefung in Computational ScienceWInformation
Maschineningenieurwissenschaften MasterRobotics, Systems and ControlWInformation
Rechnergestützte Wissenschaften MasterWahlfächerWInformation
Robotics, Systems and Control MasterKernfächerWInformation
Statistik MasterStatistische und mathematische FächerWInformation