Computational Steering via Delayed Evaluation,
Self-Optimising Scientific Software Components
Introduction - The DESO Library
One of the recent research efforts in the Software
Performance Optimisation Research Group is a DESO (Delayed
Evaluation, Self-Optimising) library of parallel scientific
software components. The special characteristic of this library
is that rather than library routines being executed immediately,
execution is delayed and a handle for the result (a
"recipe") is returned instead. This is used to
construct a DAG (directed acyclic graph) representing the data
flow of the computation to be performed. The following figure
shows an example for such a DAG (in this case the first
iteration of the Conjugate Gradient method for solving the
equation Ax = b:
Once we have built such a delayed data flow graph, we then
have the opportunity to perform cross-component
optimisations. Our paper here
gives an introduction to the theory behind this library and its
use, whilst the "THEMIS
paper" gives are more big-picture overview of the aims
of this research.
The aim of this project: Computational Steering
A scientific library like the DESO library could be used to
build very long-running complex scientific simulation
programs. The typical mode of operation of such programs is
off-line: a user starts a simulation run on a large
data set overnight (or over a few days), with the program at the
end writing out a large result-file which is then analysed. The
idea behind Computational Steering is that we would
really like something much more interactive:
- We would like to be able to inspect some of the data
while the computation is running. This is an important tool
for making sure that we do not waste valuable time on
simulations that have gone wrong.
- We want to be able to insert such inspection points at
different points in the program's control flow graph
(i.e. for example along the edges in the above sample graph)
and then remove them afterwards.
- We may want to adjust certain simulation parameters. For
example, some linear system solvers converge more quickly or
more slowly based on certain scalar parameters. We want to
be able to adjust these interactively at runtime and watch
the effect.
- We may even want to insert additional computational
components into the data flow graph.
- We may want to insert 3D or 2D visualisation components
into the data flow in order to be able to watch the progress
of a simulation rather than just inspect the final
state.
Another way of explaining this is that we want to take a
library of scientific subroutines (albeit special ones) and
turn it into an interactive parallel service that we can
access and control from a front-end application (probably a
GUI, but a scripting language might be a good first
start).
Key Challenges
- Understand how our DESO Library works. Understand how
existing Steering frameworks are engineered.
- Decide on the appropriate control mechanism for driving
the underlying numerical software components.
- Data placement: We want to minimise the amount of data
that has to be transferred between the front-end workstation
and distributed (parallel) server processors. This is
particularly challenging if the set of server processors we use
changes over time.
- Our library currently uses MPI, which imposes several
restrictions on the way in which we can run parallel
programs. Find ways of overcoming these restrictions.
- Design of an appropriate user interface that does not
constrain the computational scientist in the kind of
computations that can be performed.
Is this project for you?
This is a research-level project. To do this project, you
must not be scared to pick up and work with an existing large
software system. You should enjoy the challenge of getting
software to work and do what you want. If you would like to
discuss this project, please feel free to talk to either Paul Kelly or Olav Beckmann.
Olav Beckmann
Last modified: Tue Oct 4 15:30:25 BST 2005