Looking for online learning materials for this unit?
Online course materials for MATH69122Stochastic Control with Applications to Finance
Unit code:  MATH69122 
Credit Rating:  15 
Unit level:  Level 6 
Teaching period(s):  Semester 2 
Offered by  School of Mathematics 
Available as a free choice unit?:  N 
Requisites
NoneAims
The aim of the course is to provide students a fundamental background in the optimization of stochastic processes. This is split into three components: 1. Markov Decision Theory, 2. Multiarmed Bandit Problems and the explorationexploitation tradeoff, and 3. Queueing and gametheory in market design. In each area we focus on financial applications.
Overview
Dynamic programming and its extension to Markov processes, Markov Decision Processes, is one of the fundamental algorithmic contributions of the last century. Here one can sequentially optimize a process over time. The theory is extended to diffusion processes and thus is a key tool in optimal investment.
Further, within this framework one can consider aspects of the socalled explorationexploitation tradeoff. Here one can either choose to optimize decisions with all available information or can choose to explore further potential options. This is the Multiarmed bandit framework and literature in this domain has grown rapidly over recent years. We consider aspects of this theory in application to portfolio selection and revenue management.
Further we develop queueing models and consider their application in inventory models and the limit order book model  a model studied in the context of high frequency trading. Finally, we consider aspect of optimal market design covering the famed VCGmechanism and Myerson's optimal auction framework.
Learning outcomes
On successful completion of this course unit students will
 Acquire knowledge required to implement Markov Decision Theory.
 Understand the fundamental limits of explorationexploitation tradeoff.
 Be able to anticipate gametheoretic aspects in market design.
 Have a close working knowledge of how each theoretic component is applied in financial situations.
Assessment methods
 Written exam  100%
Syllabus
 Dynamic Programming & Markov Decision Processes. Applications including shortestpaths, secretary problem, optimal stopping, (s,S) inventory control. [8]
 Control of Diffusions. HamiltonJacobiBellman formulation. Applications including optimal consumption and investment [6]
 Multiarmed Bandits. Gitten's Indices. Lai Robbins, optimal estimation. Applications including revenue management. [5]
 Portfolio selection and sequential investment. Introduction to Hannan consistent algorithms, Kelly Market Vectors, universal portfolio selection, the EG investment strategies. [4]
 Introduction to queueing models and applications: the G/G/1 queue, CramerLundberg model, Economic Order Quantity model, limit order book modeling. [6]

Auction Theory. BayesNash equilibrium. Revenue Equivalence Theorem. Mechanism design. VickeryClarkGroves mechanism. Myerson's revenue optimal auctions. [4]
Recommended reading
 Dynamic Programming and Optimal Control, Dimitri P. Bertsekas, 2nd Edition, Athena, (2012), ISBN: 1886529086
 Arbitrage Theory in Continuous Time, Thomas Bjork, Oxford University Press (2009) ISBN: 9780199574742
 Multiarmed Bandit Allocation Indices, John Gittins, Kevin Glazebrook and Richard Weber, 2nd edition, Wiley, (2011) ISBN: 9780470670026
 Prediction, Learning, and Games, Nicolo CesaBianchi and Gabor Lugosi, Cambridge University Press, (2006), ISBN: 9780521841085
 Applied Probability and Queues, Soeren Asmussen, Springer (2003) ISBN 9780387215259.
 Putting Auction Theory to Work, Paul Milgrom, Cambridge University Press, (2004) ISBN: 9780521536721
Feedback methods
Feedback tutorials will provide an opportunity for students' work to be discussed and provide feedback on their understanding. Coursework or inclass tests (where applicable) also provide an opportunity for students to receive feedback. Students can also get feedback on their understanding directly from the lecturer, for example during the lecturer's office hour.
Study hours
 Lectures  33 hours
 Tutorials  11 hours
 Independent study hours  106 hours
Teaching staff
Neil Walton  Unit coordinator