Search for contacts, projects,
courses and publications

Solution and Optimization methods for Large Scale Problems

Description

COURSE OBJECTIVES
Overview on iterative solution methods techniques for large scale systems. Parallel solution of large linear and non-linear system. Hierarchical methods for linear systems and for minimization problems. Decomposition methods. Beyond first order methods in machine learning

COURSE DESCRIPTION
Large scale systems and large scale optimization problems are of central importance in computational science, optimization, and machine learning. Since standard solution and minimization methods in general do not scale optimally, alternative solution strategies have been developed during the last decades. In particular hierarchical solution strategies and parallel strategies have been developed. Prominent examples are multilevel or domain decomposition methods, originally developed for linear elliptic problems. We start from basic iterative methods, and then consider Krylov-space methods and eventually subspace correction methods for linear and non-linear problems,. We will discuss multilevel optimization methods such as MG/OPT, (recursive) trust-region methods (RMTR) and hierarchical minimization methods for machine learning, including variance reduction methods.

LEARNING METHODS
Lecture, reading, self study, hands-on implementation, discussion, tutorial, written weekly assignments

EXAMINATION INFORMATION
There will be a midterm, either as larger project-like assignment or as an written exam. The final exam will be written. The written weekly assignments will also count for the final grade.

REFERENCES

  • A Multigrid Tutorial; William L. Briggs, Van Emden Henson, and Steve F. McCormick; Second Edition, SIAM, 2000 (book home page), ISBN 0-89871-4621.
  • Multigrid Methods and Applications; Wolfgang Hackbusch; Springer, 1985.
  • An Introduction to Multigrid Methods; Pieter Wesseling; Corrected Reprint. Philadelphia: R.T. Edwards, Inc., 2004. ISBN 1-930217-08-0.
  • Matrix computations; Gene H. Golub and Charles F. Van Loan.
  • Domain Decomposition Methods; Toselli, Widlund
  • Nocedal Wright, Numerical Optimisation; 
  • Trust-Region Methods; Conn Gould Toint.
  • Practical Methods of Optimisation; R. Fletcher.
  • Numerical Optimisation, Series: Springer Series in Operations Research and Financial Engineering, Nocedal, Jorge, Wright, Stephen 2nd ed., 2006, XXII, 664 p. 85 illus., www.springer.com/mathematics/book/978-0-387-30303-1
  • Deep Learning, Ian Goodfellow and Yoshua Bengio and Aaron Courville, MIT Press, http://www.deeplearningbook.org2016

People

 

Krause R.

Course director

Additional information

Semester
Spring
Academic year
2020-2021
ECTS
6
Language
English
Education
Master of Science in Artificial Intelligence, Elective course, Lecture, 2nd year
Master of Science in Artificial Intelligence, Elective course, Lecture, 1st year
Master of Science in Computational Science, Core course, Lecture, 1st year