Looking for online learning materials for this unit?
Online course materials for MATH36061

Convex Optimization


Unit code: MATH36061
Credit Rating: 10
Unit level: Level 3
Teaching period(s): Semester 1
Offered by School of Mathematics
Available as a free choice unit?: N

Requisites

Prerequisite

Additional Requirements

None

Aims

The course aims to introduce students to modern convex optimization and its applications in fields such as machine learning. The course is designed to cover practical modelling aspects, algorithm analysis and design, and the theoretical foundations of the subject.

Overview

Optimization is the art of optimal decision making under constraints. Convex optimization refers to a set of problems and methods that can be formulated using convex functions and sets; countless problems from science, engineering and statistics can be cast as convex optimization problems and solved using efficient algorithms. The course is intended as an introduction to convex optimization, focussing on the theory, the modelling techniques, and the algorithm analysis and design. Recent developments such as convex regularization and compressed sensing will be discussed. The problem sessions will be used to present applications from machine learning, signal processing, and finance.

Learning outcomes

On completion of the course, students should be able to
(1) recognize and formulate convex optimization problems as they arise in practise;
(2) know a range of algorithms for solving linear, quadratic and semidefinite programming problems, and evaluate their performance;
(3) understand the theoretical foundations and be able to use it to characterise optimal solutions to optimization problems;
(4) appreciate the role of convex optimization in machine learning, signal processing, compressed sensing, and finance;
(5) use the software CVX to model and solve convex optimization problems.
 

Assessment methods

  • Other - 20%
  • Written exam - 80%

Assessment Further Information

Coursework: Weighting within unit 20%.

2 hours end of semester examination: Weighting within unit 80%.

Syllabus

The lectures and problem sessions will cover the following topics:
 

(1) Overview and examples of optimization problems;
(2) Least-squares, gradient descent, Newton’s method;
(3) Introduction to CVX;
(4) Fundamentals of convex analysis and geometry: convex sets and functions, subdifferential calculus;
(5) Linear and quadratic programming, semidefinite programming, conic optimization;
(6) Optimality conditions, duality theory, theorems of alternative;
(7) Interior-point methods, augmented Lagrangians, alternating direction method of multipliers;
(8) Applications in machine learning: convex regularization, compressed sensing and matrix completion.

Recommended reading

The main reference is the book

  • Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
    The book is available online at http://www.stanford.edu/~boyd/cvxbook/. The lecture will also make use of the CVX software, which is based on MATLAB.


Other useful references include:

  • J. Nocedal and S.J.Wright. Numerical Optimization. Springer, 2006.
  • A. Ben-Tal and A. Nemirovski. Lectures on Modern Convex Optimization. 2013.
     

CONVEX OPTIMIZATION 3

  • Y. Nesterov. Introductory lectures on convex optimization: A basic course. Springer, 2004.

Feedback methods

Feedback tutorials will provide an opportunity for students' work to be discussed and provide feedback on their understanding.  Coursework or in-class tests (where applicable) also provide an opportunity for students to receive feedback.  Students can also get feedback on their understanding directly from the lecturer, for example during the lecturer's office hour.
 

Study hours

  • Lectures - 22 hours
  • Tutorials - 11 hours
  • Independent study hours - 67 hours

Teaching staff

Martin Lotz - Unit coordinator

▲ Up to the top