Search result: Catalogue data in Autumn Semester 2024
DAS in Data Science  | ||||||||||||||||||||||||||||||||||||||||||
 Specialisation Track | ||||||||||||||||||||||||||||||||||||||||||
| Machine Learning and Artificial Intelligence | ||||||||||||||||||||||||||||||||||||||||||
| Number | Title | Type | ECTS | Hours | Lecturers | |||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 227-0689-00L | System Identification | W | 4 credits | 2V + 1U | R. Smith | |||||||||||||||||||||||||||||||||||||
| Abstract | Theory and techniques for the identification of dynamic models from experimentally obtained system input-output data. | |||||||||||||||||||||||||||||||||||||||||
| Learning objective | To provide a series of practical techniques for the development of dynamical models from experimental data, with the emphasis being on the development of models suitable for feedback control design purposes. To provide sufficient theory to enable the practitioner to understand the trade-offs between model accuracy, data quality and data quantity. | |||||||||||||||||||||||||||||||||||||||||
| Content | Introduction to modeling: Black-box and grey-box models; Parametric and non-parametric models; ARX, ARMAX (etc.) models. Predictive, open-loop, black-box identification methods. Time and frequency domain methods. Subspace identification methods. Optimal experimental design, Cramer-Rao bounds, input signal design. Parametric identification methods. On-line and batch approaches. Closed-loop identification strategies. Trade-off between controller performance and information available for identification. | |||||||||||||||||||||||||||||||||||||||||
| Literature | "System Identification; Theory for the User" Lennart Ljung, Prentice Hall (2nd Ed), 1999. Additional papers will be available via the course Moodle. | |||||||||||||||||||||||||||||||||||||||||
| Prerequisites / Notice | Control systems (227-0216-00L) or equivalent. | |||||||||||||||||||||||||||||||||||||||||
| 252-0535-00L | Advanced Machine Learning | W | 10 credits | 3V + 2U + 4A | J. M. Buhmann, C. Cotrini Jimenez | |||||||||||||||||||||||||||||||||||||
| Abstract | Machine learning algorithms provide analytical methods to search data sets for characteristic patterns. Typical tasks include the classification of data, function fitting and clustering, with applications in image and speech analysis, bioinformatics and exploratory data analysis. This course is accompanied by practical machine learning projects. | |||||||||||||||||||||||||||||||||||||||||
| Learning objective | Students will be familiarized with advanced concepts and algorithms for supervised and unsupervised learning; reinforce the statistics knowledge which is indispensible to solve modeling problems under uncertainty. Key concepts are the generalization ability of algorithms and systematic approaches to modeling and regularization. Machine learning projects will provide an opportunity to test the machine learning algorithms on real world data. | |||||||||||||||||||||||||||||||||||||||||
| Content | The theory of fundamental machine learning concepts is presented in the lecture, and illustrated with relevant applications. Students can deepen their understanding by solving both pen-and-paper and programming exercises, where they implement and apply famous algorithms to real-world data. Topics covered in the lecture include: Fundamentals: What is data? Bayesian Learning Computational learning theory Supervised learning: Ensembles: Bagging and Boosting Max Margin methods Neural networks Unsupservised learning: Dimensionality reduction techniques Clustering Mixture Models Non-parametric density estimation Learning Dynamical Systems | |||||||||||||||||||||||||||||||||||||||||
| Lecture notes | No lecture notes, but slides will be made available on the course webpage. | |||||||||||||||||||||||||||||||||||||||||
| Literature | C. Bishop. Pattern Recognition and Machine Learning. Springer 2007. R. Duda, P. Hart, and D. Stork. Pattern Classification. John Wiley & Sons, second edition, 2001. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2001. L. Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, 2004. | |||||||||||||||||||||||||||||||||||||||||
| Prerequisites / Notice | The course requires solid basic knowledge in analysis, statistics and numerical methods for CSE as well as practical programming experience for solving assignments. Students should have followed at least "Introduction to Machine Learning" or an equivalent course offered by another institution. PhD students are required to obtain a passing grade in the course (4.0 or higher based on project and exam) to gain credit points. | |||||||||||||||||||||||||||||||||||||||||
| 252-3005-00L | Natural Language Processing  | W | 7 credits | 3V + 3U + 1A | R. Cotterell | |||||||||||||||||||||||||||||||||||||
| Abstract | This course presents topics in natural language processing with an emphasis on modern techniques, primarily focusing on statistical and deep learning approaches. The course provides an overview of the primary areas of research in language processing as well as a detailed exploration of the models and techniques used both in research and in commercial natural language systems. | |||||||||||||||||||||||||||||||||||||||||
| Learning objective | The objective of the course is to learn the basic concepts in the statistical processing of natural languages. The course will be project-oriented so that the students can also gain hands-on experience with state-of-the-art tools and techniques. | |||||||||||||||||||||||||||||||||||||||||
| Content | This course presents an introduction to general topics and techniques used in natural language processing today, primarily focusing on statistical approaches. The course provides an overview of the primary areas of research in language processing as well as a detailed exploration of the models and techniques used both in research and in commercial natural language systems. | |||||||||||||||||||||||||||||||||||||||||
| Literature | Lectures will make use of textbooks such as the one by Jurafsky and Martin where appropriate, but will also make use of original research and survey papers. | |||||||||||||||||||||||||||||||||||||||||
| 263-2400-00L | Reliable and Trustworthy Artificial Intelligence | W | 6 credits | 2V + 2U + 1A | M. Vechev | |||||||||||||||||||||||||||||||||||||
| Abstract | Creating reliable, secure, robust, and fair machine learning models is a core challenge in artificial intelligence and one of fundamental importance. The goal of the course is to teach both the mathematical foundations of this new and emerging area as well as to introduce students to the latest and most exciting research in the space. | |||||||||||||||||||||||||||||||||||||||||
| Learning objective | Upon completion of the course, the students should have mastered the underlying methods and be able to apply them to a variety of engineering and research problems. To facilitate deeper understanding, the course includes a group coding project where students will build a system based on the learned material. | |||||||||||||||||||||||||||||||||||||||||
| Content | The course is split into 4 parts: Robustness of Machine Learning -------------------------------------------- - Adversarial attacks and defenses on deep learning models. - Automated certification of deep learning models (major trends: convex relaxations, branch-and-bound, randomized smoothing). - Certified training of deep neural networks (combining symbolic and continuous methods). Privacy of Machine Learning -------------------------------------- - Threat models (e.g., stealing data, poisoning, membership inference, etc.). - Attacking federated machine learning (across vision, natural language and tabular data). - Differential privacy for defending machine learning. - AI Regulations and checking model compliance. Fairness of Machine Learning --------------------------------------- - Introduction to fairness (motivation, definitions). - Enforcing individual fairness (for both vision and tabular data). - Enforcing group fairness (e.g., demographic parity, equalized odds). Robustness, Privacy and Fairness of Foundation Models --------------------------------------------------------------------------- - We discuss all previous topics, as well as programmability, in the context of latest foundation models (e.g., LLMs). More information here: https://www.sri.inf.ethz.ch/teaching/rtai24. | |||||||||||||||||||||||||||||||||||||||||
| Prerequisites / Notice | While not a formal requirement, the course assumes familiarity with basics of machine learning (especially linear algebra, gradient descent, and neural networks as well as basic probability theory). These topics are usually covered in “Intro to ML” classes at most institutions (e.g., “Introduction to Machine Learning” at ETH). The coding project will utilize Python and PyTorch. Thus some programming experience in Python is expected. Students without prior knowledge of PyTorch are expected to acquire it early in the course by solving exercise sheets. | |||||||||||||||||||||||||||||||||||||||||
| Competencies |
| |||||||||||||||||||||||||||||||||||||||||
| 263-3210-00L | Deep Learning | W | 8 credits | 3V + 2U + 2A | T. Hofmann | |||||||||||||||||||||||||||||||||||||
| Abstract | Deep learning is an area within machine learning that deals with algorithms and models that automatically induce multi-level data representations. | |||||||||||||||||||||||||||||||||||||||||
| Learning objective | In recent years, deep learning and deep networks have significantly improved the state-of-the-art in many application domains such as computer vision, speech recognition, and natural language processing. This class will cover the mathematical foundations of deep learning and provide insights into model design, training, and validation. The main objective is a profound understanding of why these methods work and how. There will also be a rich set of hands-on tasks and practical projects to familiarize students with this emerging technology. | |||||||||||||||||||||||||||||||||||||||||
| Prerequisites / Notice | This is an advanced level course that requires some basic background in machine learning. More importantly, students are expected to have a very solid mathematical foundation, including linear algebra, multivariate calculus, and probability. The course will make heavy use of mathematics and is not (!) meant to be an extended tutorial of how to train deep networks with tools like Torch or Tensorflow, although that may be a side benefit. The participation in the course is subject to the following condition: - Students must have taken the exam in Advanced Machine Learning (252-0535-00) or have acquired equivalent knowledge, see exhaustive list below: Advanced Machine Learning https://ml2.inf.ethz.ch/courses/aml/ Computational Intelligence Lab http://da.inf.ethz.ch/teaching/2019/CIL/ Introduction to Machine Learning https://las.inf.ethz.ch/teaching/introml-S19 Statistical Learning Theory http://ml2.inf.ethz.ch/courses/slt/ Computational Statistics https://stat.ethz.ch/lectures/ss19/comp-stats.php Probabilistic Artificial Intelligence https://las.inf.ethz.ch/teaching/pai-f18 | |||||||||||||||||||||||||||||||||||||||||
| 263-5210-00L | Probabilistic Artificial Intelligence | W | 8 credits | 3V + 2U + 2A | A. Krause | |||||||||||||||||||||||||||||||||||||
| Abstract | This course introduces core modeling techniques and algorithms from machine learning, optimization and control for reasoning and decision making under uncertainty, and study applications in areas such as robotics. | |||||||||||||||||||||||||||||||||||||||||
| Learning objective | How can we build systems that perform well in uncertain environments? How can we develop systems that exhibit "intelligent" behavior, without prescribing explicit rules? How can we build systems that learn from experience in order to improve their performance? We will study core modeling techniques and algorithms from statistics, optimization, planning, and control and study applications in areas such as robotics. The course is designed for graduate students. | |||||||||||||||||||||||||||||||||||||||||
| Content | Topics covered: - Probability - Probabilistic inference (variational inference, MCMC) - Bayesian learning (Gaussian processes, Bayesian deep learning) - Probabilistic planning (MDPs, POMPDPs) - Multi-armed bandits and Bayesian optimization - Reinforcement learning | |||||||||||||||||||||||||||||||||||||||||
| Prerequisites / Notice | Solid basic knowledge in statistics, algorithms and programming. The material covered in the course "Introduction to Machine Learning" is considered as a prerequisite. | |||||||||||||||||||||||||||||||||||||||||
| Competencies |
| |||||||||||||||||||||||||||||||||||||||||
| 263-5300-00L | Guarantees for Machine Learning Does not take place this semester. | W | 7 credits | 3V + 1U + 2A | F. Yang | |||||||||||||||||||||||||||||||||||||
| Abstract | This course is aimed at advanced master and doctorate students who want to conduct independent research on theory for modern machine learning (ML). It teaches standard methods in statistical learning theory commonly used to prove theoretical guarantees for ML algorithms. The knowledge is then applied in independent project work to understand and follow-up on recent theoretical ML results. | |||||||||||||||||||||||||||||||||||||||||
| Learning objective | By the end of the semester students should be able to - understand a good fraction of theory papers published in the typical ML venues. For this purpose, students will learn common mathematical techniques from statistical learning in the first part of the course and apply this knowledge in the project work - critically examine recently published work in terms of relevance and find impactful (novel) research problems. This will be an integral part of the project work and involves experimental as well as theoretical questions - outline a possible approach to prove a conjectured theorem by e.g. reducing to more solvable subproblems. This will be practiced in in-person exercises, homeworks and potentially in the final project - effectively communicate and present the problem motivation, new insights and results to a technical audience. This will be primarily learned via the final presentation and report as well as during peer-grading of peer talks. | |||||||||||||||||||||||||||||||||||||||||
| Content | This course touches upon foundational methods in statistical learning theory aimed at proving theoretical guarantees for machine learning algorithms. It touches on the following topics - concentration bounds - uniform convergence and empirical process theory - regularization for non-parametric statistics (e.g. in RKHS, neural networks) - high-dimensional learning - computational and statistical learnability (information-theoretic, PAC, SQ) - overparameterized models, implicit bias and regularization The project work focuses on current theoretical ML research that aims to understand modern phenomena in machine learning, including but not limited to - how overparameterized models generalize (statistically) and converge (computationally) - complexity measures and approximation theoretic properties of randomly initialized and trained neural networks - generalization of robust learning (adversarial or distribution-shift robustness) - private and fair learning | |||||||||||||||||||||||||||||||||||||||||
| Prerequisites / Notice | Students should have a very strong mathematical background (real analysis, probability theory, linear algebra) and solid knowledge of core concepts in machine learning taught in courses such as “Introduction to Machine Learning”, “Regression”/ “Statistical Modelling”. In addition to these prerequisites, this class requires a high degree of mathematical maturity—including abstract thinking and the ability to understand and write proofs. Students have usually taken a subset of Fundamentals of Mathematical Statistics, Probabilistic AI, Neural Network Theory, Optimization for Data Science, Advanced ML, Statistical Learning Theory, Probability Theory (D-MATH) | |||||||||||||||||||||||||||||||||||||||||
| Competencies |
| |||||||||||||||||||||||||||||||||||||||||
Page
1
of
1
