Optimization Methods

Optimisation algorithms are ubiquitous in all branches of science and technology, starting from shape optimisation in engineering to optimal control problems and data science / machine learning. This course provides an introduction to the most important concepts and methods in continuous optimisation. We will present, analyse, implement, and test some of the most fundamental algorithms. Particular emphasis will be put on the methodology and the underlying mathematical rationale (algorithmic complexity, convergence and convergence speed). We will notably cover optimality conditions, define notions such as convexity and smoothness and review different types of optimisation problems, including unconstrained, least-squares, and linear programming problems.

This course introduces main concepts of optimisation and some of the most used algorithms in data science and machine learning. It covers topics in convex and non-convex optimisation, constrained and unconstrained optimisation, including gradient descent, Newton methods, backpropagation and projection gradient descent.

In presence

Lectures, book chapter reading, self study of slides and lecture notes, exercises (weekly tutorial sessions), problems solving with implementation in Python (bi-weekly home assignments), office hours.

Home assignments (10%), midterm written exam (45%), final written exam (45%).