The spring semester 2021 will take place online until further notice. Exceptions: Courses that can only be carried out with on-site presence. Please note the information provided by the lecturers.

263-2300-00L  How To Write Fast Numerical Code

SemesterSpring Semester 2019
LecturersM. Püschel
Periodicityyearly recurring course
Language of instructionEnglish
CommentNumber of participants limited to 84.

Prerequisite: Master student, solid C programming skills.

Takes place the last time in this form.

AbstractThis course introduces the student to the foundations and state-of-the-art techniques in developing high performance software for mathematical functionality such as matrix operations, transforms, and others. The focus is on optimizing for a single core. This includes optimizing for the memory hierarchy, for special instruction sets, and the possible use of automatic performance tuning.
ObjectiveSoftware performance (i.e., runtime) arises through the complex interaction of algorithm, its implementation, the compiler used, and the microarchitecture the program is run on. The first goal of the course is to provide the student with an understanding of this "vertical" interaction, and hence software performance, for mathematical functionality. The second goal is to teach a systematic strategy how to use this knowledge to write fast software for numerical problems. This strategy will be trained in several homeworks and a semester-long group project.
ContentThe fast evolution and increasing complexity of computing platforms pose a major challenge for developers of high performance software for engineering, science, and consumer applications: it becomes increasingly harder to harness the available computing power. Straightforward implementations may lose as much as one or two orders of magnitude in performance. On the other hand, creating optimal implementations requires the developer to have an understanding of algorithms, capabilities and limitations of compilers, and the target platform's architecture and microarchitecture.

This interdisciplinary course introduces the student to the foundations and state-of-the-art techniques in high performance mathematical software development using important functionality such as matrix operations, transforms, filters, and others as examples. The course will explain how to optimize for the memory hierarchy, take advantage of special instruction sets, and other details of current processors that require optimization. The concept of automatic performance tuning is introduced. The focus is on optimization for a single core; thus, the course complements others on parallel and distributed computing.

Finally a general strategy for performance analysis and optimization is introduced that the students will apply in group projects that accompany the course.