Dr Len Freeman
Professor John Gurd and members of the Centre for Novel Computing
Level: MSc
Department: Computer Science.
Degree: ACS.
Credit: 15.
Prerequisites: None.
Lectures and Labs: 1 week.
Preliminary work 40 hours.
Assessed practical 40 hours.
Examination 2 hours.
Assessment: 67% lab. work and 33% exam.
Aims and Objectives
The highest performance computers embody substantial amounts of
parallel hardware, to the extent that the latest generation of machines
harness the power of thousands of cooperating processors in
order to achieve numerical processing rates exceeding 1 TeraFlop/s
(i.e. 10^12 floating point operations every second).
The applications of high performance computing (HPC) promise a
revolution in many areas of life, from verification of scientific theory by
computer-based simulation, through integrated interactive design and
manufacturing systems, to "virtual reality" arcade games. However, the
programming of highly parallel hardware has proved to be difficult:
progress has been slow and achieved mostly by "trial-and-error".
Convergence between the competing technologies has taken unusually
long and the HPC market remains highly volatile.
This module studies the base technologies for HPC and allows
"hands-on" experience to be gained of a state-of-the-art
parallel supercomputer. It will explore, through a combination of directed
reading, lectures, laboratories and mini-projects, a framework for the
development, analysis and performance tuning of parallel algorithms for the
solution of numerical problems.
The module tutors are active members of the interdisciplinary Centre for
Novel Computing (CNC), and have extensive research and
development experience in HPC.
Reading List
Balfour, A. and Marwick, D.H., Programming in Standard Fortran 77.
Heinemann, 1979.
Foster, I., Designing and Building Parallel Programs.
Addison-Wesley, 1995.
Hennessy, J.L. and Patterson, D.A., Computer Architecture A
Quantitative Approach.
Morgan Kaufmann, 1996.
Monro, D.M., A Crash Course in Fortran 77.
Edward Arnold, 1989.
Syllabus
- Introduction to HPC
Why is HPC important in Science and Engineering? Introduction to Parallel
Computers and Computational Overheads. Lecture hours (1)
- Levels of Abstraction, Models of Computation and Parallel Overheads.
Levels of Abstraction, Multiple Program Counters in Hardware;
Multi-Thread Execution Models, with Primary Sources of Overhead; Parallel
Languages and Compilers; Task-Parallel versus Data-Parallel Programming
Models; Further Sources of Overhead; Experimentation and Presentation of
Results; Memory Architecture and Memory Access Times and Associated
Sources of Overhead; Multi-Process Execution Model; Performance Tuning
via Overhead Reduction; Task Scheduling; Data Partitioning and its Effect
on Performance. Lecture hours (10)
- Restructuring for Parallel Performance
Parallelising Compilers; Loop Transformations; Data Transformations;
Dependence Analysis; Compiler Strategies.
Lecture hours (3).
- Parallel Algorithms
Examples of Parallel Algorithms: Cyclic Reduction; Iterative Algorithms
(Jacobi, Gauss-Seidel and Red-Black Orderings); Divide-and-Conquer
Algorithms, Adaptive Quadrature, Correct Termination. Lecture hours (3).
- Resume
Review of the course material and the unifying theme of levels of
abstraction. Lecture hours (1)