MATH37012 - 2008/2009

 

Specification

Aims

To develop the idea that processes evolving randomly in time can be modelled mathematically in terms of sequences or families of dependent random variables.

Brief Description of the unit

Markov chains are stochastic processes with the special property of "given the present, the future is independent of the past". Many real-life situations can be modelled by such processes and this course is concerned with their mathematical analysis. It by-passes the measure-theoretic considerations necessary for the development of a general theory of stochastic processes.

Learning Outcomes

On successful completion of the course students will have a good grasp of basic concepts, techniques and results associated with the elementary theory of Markov processes.

Future topics requiring this course unit

The material of this course may be helpful in understanding the 4th year courses on stochastic calculus and Brownian motion.

Syllabus

1. Discrete time Markov chains:
   Review of necessary probability theory. [2]
   Definition of Markov chain. Homogeneity. (1-step) transition probabilities. Transition diagrams. Examples including Ehrenfest diffusion model. The Chapman-Kolmogorov equations. Matrix form. Accessibility. Closed/irreducible sets. Periodicity. Stationary distributions. Positive recurrence, null recurrence, transience. Random walk examples. Convergence to stationary distribution. Discussion of different methods of proof (e.g. Markov's method for finite state space; Doeblin's coupling; renewal type argument). [8]
2. Continuous time Markov chains:
   Theoretical treatment at level of Karlin and Taylor (see below). Revision of Poisson process. Pure birth/birth death processes. [8]
3. Applications:
   Queues. M/M/1. Queue length and waiting time distribution. M/M/s. Variable arrival and service rates. Machine interference. [4]

Textbooks

• D. R. Stirzaker, Stochastic Processes and Models, Oxford University Press, 2005.
• A. N. Shiryaev, Probability, Springer-Verlag, 1996.
• S. Karlin and H. M. Taylor, A First Course in Stochastic Processes, Academic Press, 1975.
• D. R. Stirzaker, Elementary Probability, Cambridge University Press, 2003.
• G. R. Grimmett and D. R. Stirzaker, Probability and Random Processes, Oxford Univ. Press, 1992.

Teaching and learning methods

Two lectures and one examples class each week. In addition students should expect to spend at least four hours each week on private study for this course unit.

Assessment

Coursework: two hours weighting 20%

End of semester examination: two hours weighting 80%

to the top

Arrangements