Search result: Catalogue data in Spring Semester 2021

Data Science Master Information
Core Courses
Data Analysis
Information and Learning
NumberTitleTypeECTSHoursLecturers
227-0434-10LMathematics of Information Information W8 credits3V + 2U + 2AH. Bölcskei
AbstractThe class focuses on mathematical aspects of

1. Information science: Sampling theorems, frame theory, compressed sensing, sparsity, super-resolution, spectrum-blind sampling, subspace algorithms, dimensionality reduction

2. Learning theory: Approximation theory, greedy algorithms, uniform laws of large numbers, Rademacher complexity, Vapnik-Chervonenkis dimension
ObjectiveThe aim of the class is to familiarize the students with the most commonly used mathematical theories in data science, high-dimensional data analysis, and learning theory. The class consists of the lecture, exercise sessions with homework problems, and of a research project, which can be carried out either individually or in groups. The research project consists of either 1. software development for the solution of a practical signal processing or machine learning problem or 2. the analysis of a research paper or 3. a theoretical research problem of suitable complexity. Students are welcome to propose their own project at the beginning of the semester. The outcomes of all projects have to be presented to the entire class at the end of the semester.
ContentMathematics of Information

1. Signal representations: Frame theory, wavelets, Gabor expansions, sampling theorems, density theorems

2. Sparsity and compressed sensing: Sparse linear models, uncertainty relations in sparse signal recovery, super-resolution, spectrum-blind sampling, subspace algorithms (ESPRIT), estimation in the high-dimensional noisy case, Lasso

3. Dimensionality reduction: Random projections, the Johnson-Lindenstrauss Lemma

Mathematics of Learning

4. Approximation theory: Nonlinear approximation theory, best M-term approximation, greedy algorithms, fundamental limits on compressibility of signal classes, Kolmogorov-Tikhomirov epsilon-entropy of signal classes, optimal compression of signal classes

5. Uniform laws of large numbers: Rademacher complexity, Vapnik-Chervonenkis dimension, classes with polynomial discrimination
Lecture notesDetailed lecture notes will be provided at the beginning of the semester.
Prerequisites / NoticeThis course is aimed at students with a background in basic linear algebra, analysis, statistics, and probability.

We encourage students who are interested in mathematical data science to take both this course and "401-4944-20L Mathematics of Data Science" by Prof. A. Bandeira. The two courses are designed to be complementary.

H. Bölcskei and A. Bandeira
Statistics
NumberTitleTypeECTSHoursLecturers
401-3632-00LComputational StatisticsW8 credits3V + 1UM. Mächler
AbstractWe discuss modern statistical methods for data analysis, including methods for data exploration, prediction and inference. We pay attention to algorithmic aspects, theoretical properties and practical considerations. The class is hands-on and methods are applied using the statistical programming language R.
ObjectiveThe student obtains an overview of modern statistical methods for data analysis, including their algorithmic aspects and theoretical properties. The methods are applied using the statistical programming language R.
ContentSee the class website
Prerequisites / NoticeAt least one semester of (basic) probability and statistics.

Programming experience is helpful but not required.
Data Management
NumberTitleTypeECTSHoursLecturers
261-5110-00LOptimization for Data Science Information W10 credits3V + 2U + 4AB. Gärtner, D. Steurer, N. He
AbstractThis course provides an in-depth theoretical treatment of optimization methods that are particularly relevant in data science.
ObjectiveUnderstanding the theoretical guarantees (and their limits) of relevant optimization methods used in data science. Learning general paradigms to deal with optimization problems arising in data science.
ContentThis course provides an in-depth theoretical treatment of optimization methods that are particularly relevant in machine learning and data science.

In the first part of the course, we will first give a brief introduction to convex optimization, with some basic motivating examples from machine learning. Then we will analyse classical and more recent first and second order methods for convex optimization: gradient descent, Nesterov's accelerated method, proximal and splitting algorithms, subgradient descent, stochastic gradient descent, variance-reduced methods, Newton's method, and Quasi-Newton methods. The emphasis will be on analysis techniques that occur repeatedly in convergence analyses for various classes of convex functions. We will also discuss some classical and recent theoretical results for nonconvex optimization.

In the second part, we discuss convex programming relaxations as a powerful and versatile paradigm for designing efficient algorithms to solve computational problems arising in data science. We will learn about this paradigm and develop a unified perspective on it through the lens of the sum-of-squares semidefinite programming hierarchy. As applications, we are discussing non-negative matrix factorization, compressed sensing and sparse linear regression, matrix completion and phase retrieval, as well as robust estimation.
Prerequisites / NoticeAs background, we require material taught in the course "252-0209-00L Algorithms, Probability, and Computing". It is not necessary that participants have actually taken the course, but they should be prepared to catch up if necessary.
Core Electives
NumberTitleTypeECTSHoursLecturers
151-0566-00LRecursive Estimation Information W4 credits2V + 1UR. D'Andrea
AbstractEstimation of the state of a dynamic system based on a model and observations in a computationally efficient way.
ObjectiveLearn the basic recursive estimation methods and their underlying principles.
ContentIntroduction to state estimation; probability review; Bayes' theorem; Bayesian tracking; extracting estimates from probability distributions; Kalman filter; extended Kalman filter; particle filter; observer-based control and the separation principle.
Lecture notesLecture notes available on course website: http://www.idsc.ethz.ch/education/lectures/recursive-estimation.html
Prerequisites / NoticeRequirements: Introductory probability theory and matrix-vector algebra.
227-0150-00LSystems-on-Chip for Data Analytics and Machine Learning
Previously "Energy-Efficient Parallel Computing Systems for Data Analytics"
W6 credits4GL. Benini
AbstractSystems-on-chip architecture and related design issues with a focus on machine learning and data analytics applications. It will cover multi-cores, many-cores, vector engines, GP-GPUs, application-specific processors and heterogeneous compute accelerators. Special emphasis given to energy-efficiency issues and hardware-software techniques for power and energy minimization.
ObjectiveGive in-depth understanding of the links and dependencies between architectures and their energy-efficient implementation and to get a comprehensive exposure to state-of-the-art systems-on-chip platforms for machine learning and data analytics. Practical experience will also be gained through practical exercises and mini-projects (hardware and software) assigned on specific topics.
ContentThe course will cover advanced system-on-chip architectures, with an in-depth view on design challenges related to advanced silicon technology and state-of-the-art system integration options (nanometer silicon technology, novel storage devices, three-dimensional integration, advanced system packaging). The emphasis will be on programmable parallel architectures with application focus on machine learning and data analytics. The main SoC architectural families will be covered: namely, multi and many- cores, GPUs, vector accelerators, application-specific processors, heterogeneous platforms. The course will cover the complex design choices required to achieve scalability and energy proportionality. The course will will also delve into system design, touching on hardware-software tradeoffs and full-system analysis and optimization taking into account non-functional constraints and quality metrics, such as power consumption, thermal dissipation, reliability and variability. The application focus will be on machine learning both in the cloud and at the edges (near-sensor analytics).
Lecture notesSlides will be provided to accompany lectures. Pointers to scientific literature will be given. Exercise scripts and tutorials will be provided.
LiteratureJohn L. Hennessy, David A. Patterson, Computer Architecture: A Quantitative Approach (The Morgan Kaufmann Series in Computer Architecture and Design) 6th Edition, 2017.
Prerequisites / NoticeKnowledge of digital design at the level of "Design of Digital Circuits SS12" is required.

Knowledge of basic VLSI design at the level of "VLSI I: Architectures of VLSI Circuits" is required
227-0155-00LMachine Learning on Microcontrollers Restricted registration - show details
Number of participants limited to 40.
Registration in this class requires the permission of the instructors.
W6 credits3GM. Magno, L. Benini
AbstractMachine Learning (ML) and artificial intelligence are pervading the digital society. Today, even low power embedded systems are incorporating ML, becoming increasingly “smart”. This lecture gives an overview of ML methods and algorithms to process and extracts useful near-sensor information in end-nodes of the “internet-of-things”, using low-power microcontrollers (ARM-Cortex-M; RISC-V).
ObjectiveLearn how to Process data from sensors and how to extract useful information with low power microprocessors using ML techniques. We will analyze data coming from real low-power sensors (accelerometers, microphones, ExG bio-signals, cameras…). The main objective is to study in detail how Machine Learning algorithms can be adapted to the performance constraints and limited resources of low-power microcontrollers becoming Tiny Machine learning algorithms.
ContentThe final goal of the course is a deep understanding of machine learning and its practical implementation on single- and multi-core microcontrollers, coupled with performance and energy efficiency analysis and optimization. The main topics of the course include:

- Sensors and sensor data acquisition with low power embedded systems

- Machine Learning: Overview of supervised and unsupervised learning and in particular supervised learning ( Decision Trees, Random, Support Vector Machines, Artificial Neural Networks, Deep Learning, and Convolutional Networks)

- Low-power embedded systems and their architecture. Low Power microcontrollers (ARM-Cortex M) and RISC-V-based Parallel Ultra Low Power (PULP) systems-on-chip.

- Low power smart sensor system design: hardware-software tradeoffs, analysis, and optimization. Implementation and performance evaluation of ML in battery-operated embedded systems.

The laboratory exercised will show how to address concrete design problems, like motion, gesture recognition, emotion detection, image, and sound classification, using real sensors data and real MCU boards.

Presentations from Ph.D. students and the visit to the Digital Circuits and Systems Group will introduce current research topics and international research projects.
Lecture notesScript and exercise sheets. Books will be suggested during the course.
Prerequisites / NoticePrerequisites: Good experience in C language programming. Microprocessors and computer architecture. Basics of Digital Signal Processing. Some exposure to machine learning concepts is also desirable.
227-0224-00LStochastic Systems
Does not take place this semester.
W4 credits2V + 1Uto be announced
AbstractProbability. Stochastic processes. Stochastic differential equations. Ito. Kalman filters. St Stochastic optimal control. Applications in financial engineering.
ObjectiveStochastic dynamic systems. Optimal control and filtering of stochastic systems. Examples in technology and finance.
Content- Stochastic processes
- Stochastic calculus (Ito)
- Stochastic differential equations
- Discrete time stochastic difference equations
- Stochastic processes AR, MA, ARMA, ARMAX, GARCH
- Kalman filter
- Stochastic optimal control
- Applications in finance and engineering
Lecture notesH. P. Geering et al., Stochastic Systems, Measurement and Control Laboratory, 2007 and handouts
227-0420-00LInformation Theory II Information W6 credits4GA. Lapidoth, S. M. Moser
AbstractThis course builds on Information Theory I. It introduces additional topics in single-user communication, connections between Information Theory and Statistics, and Network Information Theory.
ObjectiveThe course's objective is to introduce the students to additional information measures and to equip them with the tools that are needed to conduct research in Information Theory as it relates to Communication Networks and to Statistics.
ContentSanov's Theorem, Rényi entropy and guessing, differential entropy, maximum entropy, the Gaussian channel, the entropy-power inequality, the broadcast channel, the multiple-access channel, Slepian-Wolf coding, the Gelfand-Pinsker problem, and Fisher information.
Lecture notesn/a
LiteratureT.M. Cover and J.A. Thomas, Elements of Information Theory, second edition, Wiley 2006
Prerequisites / NoticeBasic introductory course on Information Theory.
227-0424-00LModel- and Learning-Based Inverse Problems in ImagingW4 credits2V + 1PV. Vishnevskiy
AbstractReconstruction is an inverse problem which estimates images from noisy measurements. Model-based reconstructions use analytical models of the imaging process and priors. Data-based methods directly approximate inversion using training data. Combining these two approaches yields physics-aware neural nets and state-of-the-art imaging accuracy (MRI, US, CT, microscopy, non-destructive imaging).
ObjectiveThe goal of this course is to introduce the mathematical models of imaging experiments and practice implementation of numerical methods to solve the corresponding inverse problem. Students will learn how to improve reconstruction accuracy by introducing prior knowledge in the form of regularization models and training data. Furthermore, students will practice incorporating imaging model knowledge into deep neural networks.
ContentThe course is based on following fundamental fields: (i) numerical linear algebra, (ii) mathematical statistics and learning theory, (iii) convex optimization and (iv) signal processing. The first part of the course introduces classical linear and nonlinear methods for image reconstruction. The second part considers data-based regularization and covers modern deep learning approaches to inverse problems in imaging. Finally, we introduce advances in the actively developing field of experimental design in biomedical imaging (i.e. how to conduct an experiment in a way to enable the most accurate reconstruction).

1. Introduction: Examples of inverse problems, general introduction. Refresh prerequisites.

2. Linear algebra in imaging: Refresh prerequisites. Demonstrate properties of operators employed in imaging.

3. Linear inverse problems and regularization: Classical theory of inverse problems. Introduce notion of ill-posedness and regularization.

3. Compressed sensing: Sparsity, basis-CS, TV-CS. Notion of analysis and synthesis forms of reconstruction problems. Application of PGD and ADMM to reconstruction.

4. Advanced priors and model selection: Total generalized variation, GMM priors, vectorial TV, low-rank, and tensor models. Stein's unbiased risk estimator.

5. Dictionary and prior learning: Classical dictionary learning. Gentle intro to machine learning. A lot of technical details about patch-models.

6. Deep learning in image reconstruction: Generic convolutional-NN models (automap, residual filtering, u-nets). Talk about the data generation process. Characterized difference between model- and data-based reconstruction methods. Mode averaging.

7. Loop unrolling and physics-aware networks for reconstruction: Autograd, Variational Networks, a lot of examples and intuition. Show how to use them efficiently, e.g. adding preconditioners, attention, etc.

8. Generative models and uncertainty quantification: Amortized posterior, variational autoencoders, adversarial learning. Estimation uncertainty quantification.

9. Inversible networks for estimation: Gradient flows in networks, inversible neural networks for estimation problems.

10. Experimental design in imaging: Acquisition optimization for continuous models. How far can we exploit autograd?

11. Signal sampling optimization in MRI. Reinforcement learning: Acquisition optimization for discrete models. Reinforce and policy gradients, variance minimization for discrete variables (RELAX, REBAR). Cartesian under-sampling pattern design

12. Summary and exam preparation.
Lecture notesLecture slides with references will be provided during the course.
Prerequisites / NoticeStudents are expected to know the basics of (i) numerical linear algebra, (ii) applied methods of convex optimization, (iii) computational statistics, (iv) Matlab and Python.
227-0427-10LAdvanced Signal Analysis, Modeling, and Machine Learning Information W6 credits4GH.‑A. Loeliger
AbstractThe course develops a selection of topics pivoting around graphical models (factor graphs), state space methods, sparsity, and pertinent algorithms.
ObjectiveThe course develops a selection of topics pivoting around factor graphs, state space methods, and pertinent algorithms:
- factor graphs and message passing algorithms
- hidden-​Markov models
- linear state space models, Kalman filtering, and recursive least squares
- Gaussian message passing
- Gibbs sampling, particle filter
- recursive local polynomial fitting & applications
- parameter learning by expectation maximization
- sparsity and spikes
- binary control and digital-​to-analog conversion
- duality and factor graph transforms
Lecture notesLecture notes
Prerequisites / NoticeSolid mathematical foundations (especially in probability, estimation, and linear algebra) as provided by the course "Introduction to Estimation and Machine Learning".
227-0432-00LLearning, Classification and Compression Information W4 credits2V + 1UE. Riegler
AbstractThe focus of the course is aligned to a theoretical approach of learning theory and classification and an introduction to lossy and lossless compression for general sets and measures. We will mainly focus on a probabilistic approach, where an underlying distribution must be learned/compressed. The concepts acquired in the course are of broad and general interest in data sciences.
ObjectiveAfter attending this lecture and participating in the exercise sessions, students will have acquired a working knowledge of learning theory, classification, and compression.
Content1. Learning Theory
(a) Framework of Learning
(b) Hypothesis Spaces and Target Functions
(c) Reproducing Kernel Hilbert Spaces
(d) Bias-Variance Tradeoff
(e) Estimation of Sample and Approximation Error

2. Classification
(a) Binary Classifier
(b) Support Vector Machines (separable case)
(c) Support Vector Machines (nonseparable case)
(d) Kernel Trick

3. Lossy and Lossless Compression
(a) Basics of Compression
(b) Compressed Sensing for General Sets and Measures
(c) Quantization and Rate Distortion Theory for General Sets and Measures
Lecture notesDetailed lecture notes will be provided.
Prerequisites / NoticeThis course is aimed at students with a solid background in measure theory and linear algebra and basic knowledge in functional analysis.
227-0558-00LPrinciples of Distributed Computing Information W7 credits2V + 2U + 2AR. Wattenhofer, M. Ghaffari
AbstractWe study the fundamental issues underlying the design of distributed systems: communication, coordination, fault-tolerance, locality, parallelism, self-organization, symmetry breaking, synchronization, uncertainty. We explore essential algorithmic ideas and lower bound techniques.
ObjectiveDistributed computing is essential in modern computing and communications systems. Examples are on the one hand large-scale networks such as the Internet, and on the other hand multiprocessors such as your new multi-core laptop. This course introduces the principles of distributed computing, emphasizing the fundamental issues underlying the design of distributed systems and networks: communication, coordination, fault-tolerance, locality, parallelism, self-organization, symmetry breaking, synchronization, uncertainty. We explore essential algorithmic ideas and lower bound techniques, basically the "pearls" of distributed computing. We will cover a fresh topic every week.
ContentDistributed computing models and paradigms, e.g. message passing, shared memory, synchronous vs. asynchronous systems, time and message complexity, peer-to-peer systems, small-world networks, social networks, sorting networks, wireless communication, and self-organizing systems.

Distributed algorithms, e.g. leader election, coloring, covering, packing, decomposition, spanning trees, mutual exclusion, store and collect, arrow, ivy, synchronizers, diameter, all-pairs-shortest-path, wake-up, and lower bounds
Lecture notesAvailable. Our course script is used at dozens of other universities around the world.
LiteratureLecture Notes By Roger Wattenhofer. These lecture notes are taught at about a dozen different universities through the world.

Distributed Computing: Fundamentals, Simulations and Advanced Topics
Hagit Attiya, Jennifer Welch.
McGraw-Hill Publishing, 1998, ISBN 0-07-709352 6

Introduction to Algorithms
Thomas Cormen, Charles Leiserson, Ronald Rivest.
The MIT Press, 1998, ISBN 0-262-53091-0 oder 0-262-03141-8

Disseminatin of Information in Communication Networks
Juraj Hromkovic, Ralf Klasing, Andrzej Pelc, Peter Ruzicka, Walter Unger.
Springer-Verlag, Berlin Heidelberg, 2005, ISBN 3-540-00846-2

Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes
Frank Thomson Leighton.
Morgan Kaufmann Publishers Inc., San Francisco, CA, 1991, ISBN 1-55860-117-1

Distributed Computing: A Locality-Sensitive Approach
David Peleg.
Society for Industrial and Applied Mathematics (SIAM), 2000, ISBN 0-89871-464-8
Prerequisites / NoticeCourse pre-requisites: Interest in algorithmic problems. (No particular course needed.)
227-0560-00LDeep Learning for Autonomous Driving Information Restricted registration - show details
Registration in this class requires the permission of the instructors.
Class size will be limited to 80 students.
Please send an email to Dengxin Dai <dai@vision.ee.ethz.ch> about your courses/projects that are related to machine learning, computer vision, and Robotics.
W6 credits3V + 2PD. Dai, A. Liniger
AbstractAutonomous driving has moved from the realm of science fiction to a very real possibility during the past twenty years, largely due to rapid developments of deep learning approaches, automotive sensors, and microprocessor capacity. This course covers the core techniques required for building a self-driving car, especially the practical use of deep learning through this theme.
ObjectiveStudents will learn about the fundamental aspects of a self-driving car. They will also learn to use modern automotive sensors and HD navigational maps, and to implement, train and debug their own deep neural networks in order to gain a deep understanding of cutting-edge research in autonomous driving tasks, including perception, localization and control.

After attending this course, students will:
1) understand the core technologies of building a self-driving car;
2) have a good overview over the current state of the art in self-driving cars;
3) be able to critically analyze and evaluate current research in this area;
4) be able to implement basic systems for multiple autonomous driving tasks.
ContentWe will focus on teaching the following topics centered on autonomous driving: deep learning, automotive sensors, multimodal driving datasets, road scene perception, ego-vehicle localization, path planning, and control.

The course covers the following main areas:

I) Foundation
a) Fundamentals of a self-driving car
b) Fundamentals of deep-learning


II) Perception
a) Semantic segmentation and lane detection
b) Depth estimation with images and sparse LiDAR data
c) 3D object detection with images and LiDAR data
d) Object tracking and Lane Detection

III) Localization
a) GPS-based and Vision-based Localization
b) Visual Odometry and Lidar Odometry

IV) Path Planning and Control
a) Path planning for autonomous driving
b) Motion planning and vehicle control
c) Imitation learning and reinforcement learning for self driving cars

The exercise projects will involve training complex neural networks and applying them on real-world, multimodal driving datasets. In particular, students should be able to develop systems that deal with the following problems:
- Sensor calibration and synchronization to obtain multimodal driving data;
- Semantic segmentation and depth estimation with deep neural networks ;
- 3D object detection and tracking in LiDAR point clouds
Lecture notesThe lecture slides will be provided as a PDF.
Prerequisites / NoticeThis is an advanced grad-level course. Students must have taken courses on machine learning and computer vision or have acquired equivalent knowledge. Students are expected to have a solid mathematical foundation, in particular in linear algebra, multivariate calculus, and probability. All practical exercises will require basic knowledge of Python and will use libraries such as PyTorch, scikit-learn and scikit-image.
252-0211-00LInformation Security Information W8 credits4V + 3UD. Basin, S. Capkun
AbstractThis course provides an introduction to Information Security. The focus
is on fundamental concepts and models, basic cryptography, protocols and system security, and privacy and data protection. While the emphasis is on foundations, case studies will be given that examine different realizations of these ideas in practice.
ObjectiveMaster fundamental concepts in Information Security and their
application to system building. (See objectives listed below for more details).
Content1. Introduction and Motivation (OBJECTIVE: Broad conceptual overview of information security) Motivation: implications of IT on society/economy, Classical security problems, Approaches to
defining security and security goals, Abstractions, assumptions, and trust, Risk management and the human factor, Course verview. 2. Foundations of Cryptography (OBJECTIVE: Understand basic
cryptographic mechanisms and applications) Introduction, Basic concepts in cryptography: Overview, Types of Security, computational hardness, Abstraction of channel security properties, Symmetric
encryption, Hash functions, Message authentication codes, Public-key distribution, Public-key cryptosystems, Digital signatures, Application case studies, Comparison of encryption at different layers, VPN, SSL, Digital payment systems, blind signatures, e-cash, Time stamping 3. Key Management and Public-key Infrastructures (OBJECTIVE: Understand the basic mechanisms relevant in an Internet context) Key management in distributed systems, Exact characterization of requirements, the role of trust, Public-key Certificates, Public-key Infrastructures, Digital evidence and non-repudiation, Application case studies, Kerberos, X.509, PGP. 4. Security Protocols (OBJECTIVE: Understand network-oriented security, i.e.. how to employ building blocks to secure applications in (open) networks) Introduction, Requirements/properties, Establishing shared secrets, Principal and message origin authentication, Environmental assumptions, Dolev-Yao intruder model and
variants, Illustrative examples, Formal models and reasoning, Trace-based interleaving semantics, Inductive verification, or model-checking for falsification, Techniques for protocol design,
Application case study 1: from Needham-Schroeder Shared-Key to Kerberos, Application case study 2: from DH to IKE. 5. Access Control and Security Policies (OBJECTIVES: Study system-oriented security, i.e., policies, models, and mechanisms) Motivation (relationship to CIA, relationship to Crypto) and examples Concepts: policies versus models versus mechanisms, DAC and MAC, Modeling formalism, Access Control Matrix Model, Roll Based Access Control, Bell-LaPadula, Harrison-Ruzzo-Ullmann, Information flow, Chinese Wall, Biba, Clark-Wilson, System mechanisms: Operating Systems, Hardware Security Features, Reference Monitors, File-system protection, Application case studies 6. Anonymity and Privacy (OBJECTIVE: examine protection goals beyond standard CIA and corresponding mechanisms) Motivation and Definitions, Privacy, policies and policy languages, mechanisms, problems, Anonymity: simple mechanisms (pseudonyms, proxies), Application case studies: mix networks and crowds. 7. Larger application case study: GSM, mobility
252-0526-00LStatistical Learning Theory Information W8 credits3V + 2U + 2AJ. M. Buhmann, C. Cotrini Jimenez
AbstractThe course covers advanced methods of statistical learning:

- Variational methods and optimization.
- Deterministic annealing.
- Clustering for diverse types of data.
- Model validation by information theory.
ObjectiveThe course surveys recent methods of statistical learning. The fundamentals of machine learning, as presented in the courses "Introduction to Machine Learning" and "Advanced Machine Learning", are expanded from the perspective of statistical learning.
Content- Variational methods and optimization. We consider optimization approaches for problems where the optimizer is a probability distribution. We will discuss concepts like maximum entropy, information bottleneck, and deterministic annealing.

- Clustering. This is the problem of sorting data into groups without using training samples. We discuss alternative notions of "similarity" between data points and adequate optimization procedures.

- Model selection and validation. This refers to the question of how complex the chosen model should be. In particular, we present an information theoretic approach for model validation.

- Statistical physics models. We discuss approaches for approximately optimizing large systems, which originate in statistical physics (free energy minimization applied to spin glasses and other models). We also study sampling methods based on these models.
Lecture notesA draft of a script will be provided. Lecture slides will be made available.
LiteratureHastie, Tibshirani, Friedman: The Elements of Statistical Learning, Springer, 2001.

L. Devroye, L. Gyorfi, and G. Lugosi: A probabilistic theory of pattern recognition. Springer, New York, 1996
Prerequisites / NoticeKnowledge of machine learning (introduction to machine learning and/or advanced machine learning)
Basic knowledge of statistics.
252-0538-00LShape Modeling and Geometry Processing Information W8 credits2V + 1U + 4AO. Sorkine Hornung
AbstractThis course covers the fundamentals and some of the latest developments in geometric modeling and geometry processing. Topics include surface modeling based on point clouds and polygonal meshes, mesh generation, surface reconstruction, mesh fairing and parameterization, discrete differential geometry, interactive shape editing, topics in digital shape fabrication.
ObjectiveThe students will learn how to design, program and analyze algorithms and systems for interactive 3D shape modeling and geometry processing.
ContentRecent advances in 3D geometry processing have created a plenitude of novel concepts for the mathematical representation and interactive manipulation of geometric models. This course covers the fundamentals and some of the latest developments in geometric modeling and geometry processing. Topics include surface modeling based on point clouds and triangle meshes, mesh generation, surface reconstruction, mesh fairing and parameterization, discrete differential geometry, interactive shape editing and digital shape fabrication.
Lecture notesSlides and course notes
Prerequisites / NoticePrerequisites:
Visual Computing, Computer Graphics or an equivalent class. Experience with C++ programming. Solid background in linear algebra and analysis. Some knowledge of differential geometry, computational geometry and numerical methods is helpful but not a strict requirement.
252-0579-00L3D Vision Information W5 credits3G + 1AM. Pollefeys, V. Larsson
AbstractThe course covers camera models and calibration, feature tracking and matching, camera motion estimation via simultaneous localization and mapping (SLAM) and visual odometry (VO), epipolar and mult-view geometry, structure-from-motion, (multi-view) stereo, augmented reality, and image-based (re-)localization.
ObjectiveAfter attending this course, students will:
1. understand the core concepts for recovering 3D shape of objects and scenes from images and video.
2. be able to implement basic systems for vision-based robotics and simple virtual/augmented reality applications.
3. have a good overview over the current state-of-the art in 3D vision.
4. be able to critically analyze and asses current research in this area.
ContentThe goal of this course is to teach the core techniques required for robotic and augmented reality applications: How to determine the motion of a camera and how to estimate the absolute position and orientation of a camera in the real world. This course will introduce the basic concepts of 3D Vision in the form of short lectures, followed by student presentations discussing the current state-of-the-art. The main focus of this course are student projects on 3D Vision topics, with an emphasis on robotic vision and virtual and augmented reality applications.
252-3005-00LNatural Language Processing Information Restricted registration - show details
Number of participants limited to 400.
W5 credits2V + 1U + 1AR. Cotterell
AbstractThis course presents topics in natural language processing with an emphasis on modern techniques, primarily focusing on statistical and deep learning approaches. The course provides an overview of the primary areas of research in language processing as well as a detailed exploration of the models and techniques used both in research and in commercial natural language systems.
ObjectiveThe objective of the course is to learn the basic concepts in the statistical processing of natural languages. The course will be project-oriented so that the students can also gain hands-on experience with state-of-the-art tools and techniques.
ContentThis course presents an introduction to general topics and techniques used in natural language processing today, primarily focusing on statistical approaches. The course provides an overview of the primary areas of research in language processing as well as a detailed exploration of the models and techniques used both in research and in commercial natural language systems.
LiteratureJacob Eisenstein: Introduction to Natural Language Processing (Adaptive Computation and Machine Learning series)
261-5130-00LResearch in Data Science Restricted registration - show details
Only for Data Science MSc.
W6 credits13AProfessors
AbstractIndependent work under the supervision of a core or adjunct faculty of data science.
ObjectiveIndependent work under the supervision of a core or adjunct faculty of data science.
An approval of the director of studies is required for a non DS professor.
ContentProject done under supervision of an approved professor.
Prerequisites / NoticeOnly students who have passed at least one core course in Data Management and Processing, and one core course in Data Analysis can start with a research project.

A project description must be submitted at the start of the project to the studies administration.
263-0007-00LAdvanced Systems Lab Information Restricted registration - show details
Only for master students, otherwise a special permission by the study administration of D-INFK is required.
W8 credits3V + 2U + 2AM. Püschel, C. Zhang
AbstractThis course introduces the student to the foundations and state-of-the-art techniques in developing high performance software for mathematical functionality occurring in various fields in computer science. The focus is on optimizing for a single core and includes optimizing for the memory hierarchy, for special instruction sets, and the possible use of automatic performance tuning.
ObjectiveSoftware performance (i.e., runtime) arises through the complex interaction of algorithm, its implementation, the compiler used, and the microarchitecture the program is run on. The first goal of the course is to provide the student with an understanding of this "vertical" interaction, and hence software performance, for mathematical functionality. The second goal is to teach a systematic strategy how to use this knowledge to write fast software for numerical problems. This strategy will be trained in several homeworks and a semester-long group project.
ContentThe fast evolution and increasing complexity of computing platforms pose a major challenge for developers of high performance software for engineering, science, and consumer applications: it becomes increasingly harder to harness the available computing power. Straightforward implementations may lose as much as one or two orders of magnitude in performance. On the other hand, creating optimal implementations requires the developer to have an understanding of algorithms, capabilities and limitations of compilers, and the target platform's architecture and microarchitecture.

This interdisciplinary course introduces the student to the foundations and state-of-the-art techniques in high performance mathematical software development using important functionality such as matrix operations, transforms, filters, and others as examples. The course will explain how to optimize for the memory hierarchy, take advantage of special instruction sets, and other details of current processors that require optimization. The concept of automatic performance tuning is introduced. The focus is on optimization for a single core; thus, the course complements others on parallel and distributed computing.

Finally a general strategy for performance analysis and optimization is introduced that the students will apply in group projects that accompany the course.
Prerequisites / NoticeSolid knowledge of the C programming language and matrix algebra.
263-0008-00LComputational Intelligence Lab Information
Only for master students, otherwise a special permission by the study administration of D-INFK is required.
W8 credits2V + 2U + 3AT. Hofmann
AbstractThis laboratory course teaches fundamental concepts in computational science and machine learning with a special emphasis on matrix factorization and representation learning. The class covers techniques like dimension reduction, data clustering, sparse coding, and deep learning as well as a wide spectrum of related use cases and applications.
ObjectiveStudents acquire fundamental theoretical concepts and methodologies from machine learning and how to apply these techniques to build intelligent systems that solve real-world problems. They learn to successfully develop solutions to application problems by following the key steps of modeling, algorithm design, implementation and experimental validation.

This lab course has a strong focus on practical assignments. Students work in groups of three to four people, to develop solutions to three application problems: 1. Collaborative filtering and recommender systems, 2. Text sentiment classification, and 3. Road segmentation in aerial imagery.

For each of these problems, students submit their solutions to an online evaluation and ranking system, and get feedback in terms of numerical accuracy and computational speed. In the final part of the course, students combine and extend one of their previous promising solutions, and write up their findings in an extended abstract in the style of a conference paper.

(Disclaimer: The offered projects may be subject to change from year to year.)
Contentsee course description
263-2925-00LProgram Analysis for System Security and Reliability Information W7 credits2V + 1U + 3AM. Vechev
AbstractSecurity issues in modern systems (blockchains, datacenters, deep learning, etc.) result in billions of losses due to hacks and system downtime. This course introduces fundamental techniques (ranging from automated analysis, machine learning, synthesis, zero-knowledge and their combinations) that can be applied in practice so to build more secure and reliable modern systems.
Objective* Understand the fundamental techniques used to create modern security and reliability analysis engines that are used worldwide.

* Understand how symbolic techniques are combined with machine learning (e.g., deep learning, reinforcement learning) so to create new kinds of learning-based analyzers.

* Understand how to quantify and fix security and reliability issues in modern deep learning models.

* Understand open research questions from both theoretical and practical perspectives.
ContentPlease see: https://www.sri.inf.ethz.ch/teaching/pass2021 for detailed course content.
263-3710-00LMachine Perception Information Restricted registration - show details
Number of participants limited to 200.
W8 credits3V + 2U + 2AO. Hilliges, S. Tang
AbstractRecent developments in neural networks (aka “deep learning”) have drastically advanced the performance of machine perception systems in a variety of areas including computer vision, robotics, and intelligent UIs. This course is a deep dive into deep learning algorithms and architectures with applications to a variety of perceptual tasks.
ObjectiveStudents will learn about fundamental aspects of modern deep learning approaches for perception. Students will learn to implement, train and debug their own neural networks and gain a detailed understanding of cutting-edge research in learning-based computer vision, robotics and HCI. The final project assignment will involve training a complex neural network architecture and applying it on a real-world dataset of human activity.

The core competency acquired through this course is a solid foundation in deep-learning algorithms to process and interpret human input into computing systems. In particular, students should be able to develop systems that deal with the problem of recognizing people in images, detecting and describing body parts, inferring their spatial configuration, performing action/gesture recognition from still images or image sequences, also considering multi-modal data, among others.
ContentWe will focus on teaching: how to set up the problem of machine perception, the learning algorithms, network architectures and advanced deep learning concepts in particular probabilistic deep learning models

The course covers the following main areas:
I) Foundations of deep-learning.
II) Probabilistic deep-learning for generative modelling of data (latent variable models, generative adversarial networks and auto-regressive models).
III) Deep learning in computer vision, human-computer interaction and robotics.

Specific topics include: 
I) Deep learning basics:
a) Neural Networks and training (i.e., backpropagation)
b) Feedforward Networks
c) Timeseries modelling (RNN, GRU, LSTM)
d) Convolutional Neural Networks for classification
II) Probabilistic Deep Learning:
a) Latent variable models (VAEs)
b) Generative adversarial networks (GANs)
c) Autoregressive models (PixelCNN, PixelRNN, TCNs)
III) Deep Learning techniques for machine perception:
a) Fully Convolutional architectures for dense per-pixel tasks (i.e., instance segmentation)
b) Pose estimation and other tasks involving human activity
c) Deep reinforcement learning
IV) Case studies from research in computer vision, HCI, robotics and signal processing
LiteratureDeep Learning
Book by Ian Goodfellow and Yoshua Bengio
Prerequisites / Notice***
In accordance with the ETH Covid-19 master plan the lecture will be fully virtual. Details on the course website.
***

This is an advanced grad-level course that requires a background in machine learning. Students are expected to have a solid mathematical foundation, in particular in linear algebra, multivariate calculus, and probability. The course will focus on state-of-the-art research in deep-learning and will not repeat basics of machine learning

Please take note of the following conditions:
1) The number of participants is limited to 200 students (MSc and PhDs).
2) Students must have taken the exam in Machine Learning (252-0535-00) or have acquired equivalent knowledge
3) All practical exercises will require basic knowledge of Python and will use libraries such as Pytorch, scikit-learn and scikit-image. We will provide introductions to Pytorch and other libraries that are needed but will not provide introductions to basic programming or Python.

The following courses are strongly recommended as prerequisite:
* "Visual Computing" or "Computer Vision"

The course will be assessed by a final written examination in English. No course materials or electronic devices can be used during the examination. Note that the examination will be based on the contents of the lectures, the associated reading materials and the exercises.
263-3855-00LCloud Computing Architecture Information W9 credits3V + 2U + 3AG. Alonso, A. Klimovic
AbstractCloud computing hosts a wide variety of online services that we use on a daily basis, including web search, social networks, and video streaming. This course will cover how datacenter hardware, systems software, and application frameworks are designed for the cloud.
ObjectiveAfter successful completion of this course, students will be able to: 1) reason about performance, energy efficiency, and availability tradeoffs in the design of cloud system software, 2) describe how datacenter hardware is organized and explain why it is organized as such, 3) implement cloud applications as well as analyze and optimize their performance.
ContentIn this course, we study how datacenter hardware, systems software, and applications are designed at large scale for the cloud. The course covers topics including server design, cluster management, large-scale storage systems, serverless computing, data analytics frameworks, and performance analysis.
Lecture notesLecture slides will be available on the course website.
Prerequisites / NoticeUndergraduate courses in 1) computer architecture and 2) operating systems, distributed systems, and/or database systems are strongly recommended.
263-4400-00LAdvanced Graph Algorithms and Optimization Information W8 credits3V + 1U + 3AR. Kyng, M. Probst
AbstractThis course will cover a number of advanced topics in optimization and graph algorithms.
ObjectiveThe course will take students on a deep dive into modern approaches to
graph algorithms using convex optimization techniques.

By studying convex optimization through the lens of graph algorithms,
students should develop a deeper understanding of fundamental
phenomena in optimization.

The course will cover some traditional discrete approaches to various graph
problems, especially flow problems, and then contrast these approaches
with modern, asymptotically faster methods based on combining convex
optimization with spectral and combinatorial graph theory.
ContentStudents should leave the course understanding key
concepts in optimization such as first and second-order optimization,
convex duality, multiplicative weights and dual-based methods,
acceleration, preconditioning, and non-Euclidean optimization.

Students will also be familiarized with central techniques in the
development of graph algorithms in the past 15 years, including graph
decomposition techniques, sparsification, oblivious routing, and
spectral and combinatorial preconditioning.
Prerequisites / NoticeThis course is targeted toward masters and doctoral students with an
interest in theoretical computer science.

Students should be comfortable with design and analysis of algorithms, probability, and linear algebra.

Having passed the course Algorithms, Probability, and Computing (APC) is highly recommended, but not formally required. If you are not
sure whether you're ready for this class or not, please consult the
instructor.
263-5000-00LComputational Semantics for Natural Language Processing Information Restricted registration - show details
Limited number of participants: 80. Last cancellation/deregistration date for this graded semester performance: Friday, 26 March 2021! Please note that after that date no deregistration will be accepted and the course will be considered as "fail".
W6 credits2V + 1U + 2AM. Sachan
AbstractThis course presents an introduction to Natural language processing (NLP) with an emphasis on computational semantics i.e. the process of constructing and reasoning with meaning representations of natural language text.
ObjectiveThe objective of the course is to learn about various topics in computational semantics and its importance in natural language processing methodology and research. Exercises and the project will be key parts of the course so the students will be able to gain hands-on experience with state-of-the-art techniques in the field.
ContentWe will take a modern view of the topic, and focus on various statistical and deep learning approaches for computation semantics. We will also overview various primary areas of research in language processing and discuss how the computational semantics view can help us make advances in NLP.
Lecture notesLecture slides will be made available at the course Web site.
LiteratureNo textbook is required, but there will be regularly assigned readings from research literature, linked to the course website.
Prerequisites / NoticeThe student should have successfully completed a graduate level class in machine learning (252-0220-00L), deep learning (263-3210-00L) or natural language processing (252-3005-00L) before. Similar courses from other universities are acceptable too.
263-5300-00LGuarantees for Machine Learning Information Restricted registration - show details
Number of participants limited to 30.

Last cancellation/deregistration date for this graded semester performance: 17 March 2021! Please note that after that date no deregistration will be accepted and a "no show" will appear on your transcript.
W7 credits3G + 3AF. Yang
AbstractThis course is aimed at advanced master and doctorate students who want to conduct independent research on theory for modern machine learning (ML). It teaches classical and recent methods in statistical learning theory commonly used to prove theoretical guarantees for ML algorithms. The knowledge is then applied in independent project work that focuses on understanding modern ML phenomena.
ObjectiveLearning objectives:

- acquire enough mathematical background to understand a good fraction of theory papers published in the typical ML venues. For this purpose, students will learn common mathematical techniques from statistics and optimization in the first part of the course and apply this knowledge in the project work
- critically examine recently published work in terms of relevance and determine impactful (novel) research problems. This will be an integral part of the project work and involves experimental as well as theoretical questions
- find and outline an approach (some subproblem) to prove a conjectured theorem. This will be practiced in lectures / exercise and homeworks and potentially in the final project.
- effectively communicate and present the problem motivation, new insights and results to a technical audience. This will be primarily learned via the final presentation and report as well as during peer-grading of peer talks.
ContentThis course touches upon foundational methods in statistical learning theory aimed at proving theoretical guarantees for machine learning algorithms, touching on the following topics
- concentration bounds
- uniform convergence and empirical process theory
- high-dimensional statistics (e.g. sparsity)
- regularization for non-parametric statistics (e.g. in RKHS, neural networks)
- implicit regularization via gradient descent (e.g. margins, early stopping)
- minimax lower bounds

The project work focuses on current theoretical ML research that aims to understand modern phenomena in machine learning, including but not limited to
- how overparameterization could help generalization ( RKHS, NN )
- how overparameterization could help optimization ( non-convex optimization, loss landscape )
- complexity measures and approximation theoretic properties of randomly initialized and trained NN
- generalization of robust learning ( adversarial robustness, standard and robust error tradeoff, distribution shift)
Prerequisites / NoticeIt’s absolutely necessary for students to have a strong mathematical background (basic real analysis, probability theory, linear algebra) and good knowledge of core concepts in machine learning taught in courses such as “Introduction to Machine Learning”, “Regression”/ “Statistical Modelling”. In addition to these prerequisites, this class requires a high degree of mathematical maturity—including abstract thinking and the ability to understand and write proofs.

Students have usually taken a subset of Fundamentals of Mathematical Statistics, Probabilistic AI, Neural Network Theory, Optimization for Data Science, Advanced ML, Statistical Learning Theory, Probability Theory (D-MATH)
401-0674-00LNumerical Methods for Partial Differential Equations
Not meant for BSc/MSc students of mathematics.
W10 credits2G + 2U + 2P + 4AR. Hiptmair
AbstractDerivation, properties, and implementation of fundamental numerical methods for a few key partial differential equations: convection-diffusion, heat equation, wave equation, conservation laws. Implementation in C++ based on a finite element library.
ObjectiveMain skills to be acquired in this course:
* Ability to implement fundamental numerical methods for the solution of partial differential equations efficiently.
* Ability to modify and adapt numerical algorithms guided by awareness of their mathematical foundations.
* Ability to select and assess numerical methods in light of the predictions of theory
* Ability to identify features of a PDE (= partial differential equation) based model that are relevant for the selection and performance of a numerical algorithm.
* Ability to understand research publications on theoretical and practical aspects of numerical methods for partial differential equations.
* Skills in the efficient implementation of finite element methods on unstructured meshes.

This course is neither a course on the mathematical foundations and numerical analysis of methods nor an course that merely teaches recipes and how to apply software packages.
Content1 Second-Order Scalar Elliptic Boundary Value Problems
1.2 Equilibrium Models: Examples
1.3 Sobolev spaces
1.4 Linear Variational Problems
1.5 Equilibrium Models: Boundary Value Problems
1.6 Diffusion Models (Stationary Heat Conduction)
1.7 Boundary Conditions
1.8 Second-Order Elliptic Variational Problems
1.9 Essential and Natural Boundary Conditions
2 Finite Element Methods (FEM)
2.2 Principles of Galerkin Discretization
2.3 Case Study: Linear FEM for Two-Point Boundary Value Problems
2.4 Case Study: Triangular Linear FEM in Two Dimensions
2.5 Building Blocks of General Finite Element Methods
2.6 Lagrangian Finite Element Methods
2.7 Implementation of Finite Element Methods
2.7.1 Mesh Generation and Mesh File Format
2.7.2 Mesh Information and Mesh Data Structures
2.7.2.1 L EHR FEM++ Mesh: Container Layer
2.7.2.2 L EHR FEM++ Mesh: Topology Layer
2.7.2.3 L EHR FEM++ Mesh: Geometry Layer
2.7.3 Vectors and Matrices
2.7.4 Assembly Algorithms
2.7.4.1 Assembly: Localization
2.7.4.2 Assembly: Index Mappings
2.7.4.3 Distribute Assembly Schemes
2.7.4.4 Assembly: Linear Algebra Perspective
2.7.5 Local Computations
2.7.5.1 Analytic Formulas for Entries of Element Matrices
2.7.5.2 Local Quadrature
2.7.6 Treatment of Essential Boundary Conditions
2.8 Parametric Finite Element Methods
3 FEM: Convergence and Accuracy
3.1 Abstract Galerkin Error Estimates
3.2 Empirical (Asymptotic) Convergence of Lagrangian FEM
3.3 A Priori (Asymptotic) Finite Element Error Estimates
3.4 Elliptic Regularity Theory
3.5 Variational Crimes
3.6 FEM: Duality Techniques for Error Estimation
3.7 Discrete Maximum Principle
3.8 Validation and Debugging of Finite Element Codes
4 Beyond FEM: Alternative Discretizations [dropped]
5 Non-Linear Elliptic Boundary Value Problems [dropped]
6 Second-Order Linear Evolution Problems
6.1 Time-Dependent Boundary Value Problems
6.2 Parabolic Initial-Boundary Value Problems
6.3 Linear Wave Equations
7 Convection-Diffusion Problems [dropped]
8 Numerical Methods for Conservation Laws
8.1 Conservation Laws: Examples
8.2 Scalar Conservation Laws in 1D
8.3 Conservative Finite Volume (FV) Discretization
8.4 Timestepping for Finite-Volume Methods
8.5 Higher-Order Conservative Finite-Volume Schemes
Lecture notesThe lecture will be taught in flipped classroom format:
- Video tutorials for all thematic units will be published online.
- Tablet notes accompanying the videos will be made available to the audience as PDF.
- A comprehensive lecture document will cover all aspects of the course.
LiteratureChapters of the following books provide supplementary reading
(detailed references in course material):

* D. Braess: Finite Elemente,
Theorie, schnelle Löser und Anwendungen in der Elastizitätstheorie, Springer 2007 (available online).
* S. Brenner and R. Scott. Mathematical theory of finite element methods, Springer 2008 (available online).
* A. Ern and J.-L. Guermond. Theory and Practice of Finite Elements, volume 159 of Applied Mathematical Sciences. Springer, New York, 2004.
* Ch. Großmann and H.-G. Roos: Numerical Treatment of Partial Differential Equations, Springer 2007.
* W. Hackbusch. Elliptic Differential Equations. Theory and Numerical Treatment, volume 18 of Springer Series in Computational Mathematics. Springer, Berlin, 1992.
* P. Knabner and L. Angermann. Numerical Methods for Elliptic and Parabolic Partial Differential Equations, volume 44 of Texts in Applied Mathematics. Springer, Heidelberg, 2003.
* S. Larsson and V. Thomée. Partial Differential Equations with Numerical Methods, volume 45 of Texts in Applied Mathematics. Springer, Heidelberg, 2003.
* R. LeVeque. Finite Volume Methods for Hyperbolic Problems. Cambridge Texts in Applied Mathematics. Cambridge University Press, Cambridge, UK, 2002.

However, study of supplementary literature is not important for for following the course.
Prerequisites / NoticeMastery of basic calculus and linear algebra is taken for granted.
Familiarity with fundamental numerical methods (solution methods for linear systems of equations, interpolation, approximation, numerical quadrature, numerical integration of ODEs) is essential.

Important: Coding skills and experience in C++ are essential.

Homework assignments involve substantial coding, partly based on a C++ finite element library. The written examination will be computer based and will comprise coding tasks.
401-3052-10LGraph Theory Information W10 credits4V + 1UB. Sudakov
AbstractBasics, trees, Caley's formula, matrix tree theorem, connectivity, theorems of Mader and Menger, Eulerian graphs, Hamilton cycles, theorems of Dirac, Ore, Erdös-Chvatal, matchings, theorems of Hall, König, Tutte, planar graphs, Euler's formula, Kuratowski's theorem, graph colorings, Brooks' theorem, 5-colorings of planar graphs, list colorings, Vizing's theorem, Ramsey theory, Turán's theorem
ObjectiveThe students will get an overview over the most fundamental questions concerning graph theory. We expect them to understand the proof techniques and to use them autonomously on related problems.
Lecture notesLecture will be only at the blackboard.
LiteratureWest, D.: "Introduction to Graph Theory"
Diestel, R.: "Graph Theory"

Further literature links will be provided in the lecture.
Prerequisites / NoticeStudents are expected to have a mathematical background and should be able to write rigorous proofs.
401-3602-00LApplied Stochastic Processes Information W8 credits3V + 1UV. Tassion
AbstractPoisson processes; renewal processes; Markov chains in discrete and in continuous time; some applications.
ObjectiveStochastic processes are a way to describe and study the behaviour of systems that evolve in some random way. In this course, the evolution will be with respect to a scalar parameter interpreted as time, so that we discuss the temporal evolution of the system. We present several classes of stochastic processes, analyse their properties and behaviour and show by some examples how they can be used. The main emphasis is on theory; in that sense, "applied" should be understood to mean "applicable".
LiteratureR. N. Bhattacharya and E. C. Waymire, "Stochastic Processes with Applications", SIAM (2009), available online: http://epubs.siam.org/doi/book/10.1137/1.9780898718997
R. Durrett, "Essentials of Stochastic Processes", Springer (2012), available online: http://link.springer.com/book/10.1007/978-1-4614-3615-7/page/1
M. Lefebvre, "Applied Stochastic Processes", Springer (2007), available online: http://link.springer.com/book/10.1007/978-0-387-48976-6/page/1
S. I. Resnick, "Adventures in Stochastic Processes", Birkhäuser (2005)
Prerequisites / NoticePrerequisites are familiarity with (measure-theoretic) probability theory as it is treated in the course "Probability Theory" (401-3601-00L).
401-4632-15LCausality Information W4 credits2GC. Heinze-Deml
AbstractIn statistics, we are used to search for the best predictors of some random variable. In many situations, however, we are interested in predicting a system's behavior under manipulations. For such an analysis, we require knowledge about the underlying causal structure of the system. In this course, we study concepts and theory behind causal inference.
ObjectiveAfter this course, you should be able to
- understand the language and concepts of causal inference
- know the assumptions under which one can infer causal relations from observational and/or interventional data
- describe and apply different methods for causal structure learning
- given data and a causal structure, derive causal effects and predictions of interventional experiments
Prerequisites / NoticePrerequisites: basic knowledge of probability theory and regression
401-4656-21LDeep Learning in Scientific Computing Restricted registration - show details
Aimed at students in a Master's Programme in Mathematics, Engineering and Physics.
W6 credits2V + 1US. Mishra
AbstractMachine Learning, particularly deep learning is being increasingly applied to perform, enhance and accelerate computer simulations of models in science and engineering. This course aims to present a highly topical selection of themes in the general area of deep learning in scientific computing, with an emphasis on the application of deep learning algorithms for systems, modeled by PDEs.
ObjectiveThe objective of this course will be to introduce students to advanced applications of deep learning in scientific computing. The focus will be on the design and implementation of algorithms as well as on the underlying theory that guarantees reliability of the algorithms. We will provide several examples of applications in science and engineering where deep learning based algorithms outperform state of the art methods.
ContentA selection of the following topics will be presented in the lectures.

1. Issues with traditional methods for scientific computing such as Finite Element, Finite Volume etc, particularly for PDE models with high-dimensional state and parameter spaces.

2. Introduction to Deep Learning: Artificial Neural networks, Supervised learning, Stochastic gradient descent algorithms for training, different architectures: Convolutional Neural Networks, Recurrent Neural Networks, ResNets.

3. Theoretical Foundations: Universal approximation properties of the Neural networks, Bias-Variance decomposition, Bounds on approximation and generalization errors.

4. Supervised deep learning for solutions fields and observables of high-dimensional parametric PDEs. Use of low-discrepancy sequences and multi-level training to reduce generalization error.

5. Uncertainty Quantification for PDEs with supervised learning algorithms.

6. Deep Neural Networks as Reduced order models and prediction of solution fields.

7. Active Learning algorithms for PDE constrained optimization.

8. Recurrent Neural Networks and prediction of time series for dynamical systems.

9. Physics Informed Neural networks (PINNs) for the forward problem for PDEs. Applications to high-dimensional PDEs.

10. PINNs for inverse problems for PDEs, parameter identification, optimal control and data assimilation.

All the algorithms will be illustrated on a variety of PDEs: diffusion models, Black-Scholes type PDEs from finance, wave equations, Euler and Navier-Stokes equations, hyperbolic systems of conservation laws, Dispersive PDEs among others.
Lecture notesLecture notes will be provided at the end of the course.
LiteratureAll the material in the course is based on research articles written in last 1-2 years. The relevant references will be provided.
Prerequisites / NoticeThe students should be familiar with numerical methods for PDEs, for instance in courses such as Numerical Methods for PDEs for CSE, Numerical analysis of Elliptic and Parabolic PDEs, Numerical methods for hyperbolic PDEs, Computational methods for Engineering Applications.

Some familiarity with basic concepts in machine learning will be beneficial. The exercises in the course rely on standard machine learning frameworks such as KERAS, TENSORFLOW or PYTORCH. So, competence in Python is helpful.
401-4944-20LMathematics of Data Science
Does not take place this semester.
W8 credits4GA. Bandeira
AbstractMostly self-contained, but fast-paced, introductory masters level course on various theoretical aspects of algorithms that aim to extract information from data.
ObjectiveIntroduction to various mathematical aspects of Data Science.
ContentThese topics lie in overlaps of (Applied) Mathematics with: Computer Science, Electrical Engineering, Statistics, and/or Operations Research. Each lecture will feature a couple of Mathematical Open Problem(s) related to Data Science. The main mathematical tools used will be Probability and Linear Algebra, and a basic familiarity with these subjects is required. There will also be some (although knowledge of these tools is not assumed) Graph Theory, Representation Theory, Applied Harmonic Analysis, among others. The topics treated will include Dimension reduction, Manifold learning, Sparse recovery, Random Matrices, Approximation Algorithms, Community detection in graphs, and several others.
Lecture noteshttps://people.math.ethz.ch/~abandeira/BandeiraSingerStrohmer-MDS-draft.pdf
Prerequisites / NoticeThe main mathematical tools used will be Probability, Linear Algebra (and real analysis), and a working knowledge of these subjects is required. In addition
to these prerequisites, this class requires a certain degree of mathematical maturity--including abstract thinking and the ability to understand and write proofs.


We encourage students who are interested in mathematical data science to take both this course and ``227-0434-10L Mathematics of Information'' taught by Prof. H. Bölcskei. The two courses are designed to be
complementary.
A. Bandeira and H. Bölcskei
401-6102-00LMultivariate Statistics
Does not take place this semester.
W4 credits2Gnot available
AbstractMultivariate Statistics deals with joint distributions of several random variables. This course introduces the basic concepts and provides an overview over classical and modern methods of multivariate statistics. We will consider the theory behind the methods as well as their applications.
ObjectiveAfter the course, you should be able to:
- describe the various methods and the concepts and theory behind them
- identify adequate methods for a given statistical problem
- use the statistical software "R" to efficiently apply these methods
- interpret the output of these methods
ContentVisualization / Principal component analysis / Multidimensional scaling / The multivariate Normal distribution / Factor analysis / Supervised learning / Cluster analysis
Lecture notesNone
LiteratureThe course will be based on class notes and books that are available electronically via the ETH library.
Prerequisites / NoticeTarget audience: This course is the more theoretical version of "Applied Multivariate Statistics" (401-0102-00L) and is targeted at students with a math background.

Prerequisite: A basic course in probability and statistics.

Note: The courses 401-0102-00L and 401-6102-00L are mutually exclusive. You may register for at most one of these two course units.
402-0448-01LQuantum Information Processing I: Concepts
This theory part QIP I together with the experimental part 402-0448-02L QIP II (both offered in the Spring Semester) combine to the core course in experimental physics "Quantum Information Processing" (totally 10 ECTS credits). This applies to the Master's degree programme in Physics.
W5 credits2V + 1UP. Kammerlander
AbstractThe course will cover the key concepts and ideas of quantum information processing, including descriptions of quantum algorithms which give the quantum computer the power to compute problems outside the reach of any classical supercomputer.
Key concepts such as quantum error correction will be described. These ideas provide fundamental insights into the nature of quantum states and measurement.
ObjectiveBy the end of the course students are able to explain the basic mathematical formalism of quantum mechanics and apply them to quantum information processing problems. They are able to adapt and apply these concepts and methods to analyse and discuss quantum algorithms and other quantum information-processing protocols.
ContentThe topics covered in the course will include quantum circuits, gate decomposition and universal sets of gates, efficiency of quantum circuits, quantum algorithms (Shor, Grover, Deutsch-Josza,..), error correction, fault-tolerant design, entanglement, teleportation and dense conding, teleportation of gates, and cryptography.
Lecture notesMore details to follow.
LiteratureQuantum Computation and Quantum Information
Michael Nielsen and Isaac Chuang
Cambridge University Press
Prerequisites / NoticeA good understanding of linear algebra is recommended.
701-0104-00LStatistical Modelling of Spatial DataW3 credits2GA. J. Papritz
AbstractIn environmental sciences one often deals with spatial data. When analysing such data the focus is either on exploring their structure (dependence on explanatory variables, autocorrelation) and/or on spatial prediction. The course provides an introduction to geostatistical methods that are useful for such analyses.
ObjectiveThe course will provide an overview of the basic concepts and stochastic models that are used to model spatial data. In addition, participants will learn a number of geostatistical techniques and acquire familiarity with R software that is useful for analyzing spatial data.
ContentAfter an introductory discussion of the types of problems and the kind of data that arise in environmental research, an introduction into linear geostatistics (models: stationary and intrinsic random processes, modelling large-scale spatial patterns by linear regression, modelling autocorrelation by variogram; kriging: mean square prediction of spatial data) will be taught. The lectures will be complemented by data analyses that the participants have to do themselves.
Lecture notesSlides, descriptions of the problems for the data analyses and solutions to them will be provided.
LiteratureP.J. Diggle & P.J. Ribeiro Jr. 2007. Model-based Geostatistics. Springer.
Prerequisites / NoticeFamiliarity with linear regression analysis (e.g. equivalent to the first part of the course 401-0649-00L Applied Statistical Regression) and with the software R (e.g. 401-6215-00L Using R for Data Analysis and Graphics (Part I), 401-6217-00L Using R for Data Analysis and Graphics (Part II)) are required for attending the course.