The Enhance project

Enhancing the Performance Predictability of Grid Applications with Patterns and Process Algebras

Anne Benoit, Murray Cole, Stephen Gilmore and Jane Hillston

Overview

One of the most promising technical innovations in present-day computing is the invention of Grid technologies which harness the computational and storage power of widely distributed collections of computers. Grid technologies are breaking new ground in e-Science where scientists across the globe can collaborate and share data sets, processing power and specialised scientific instruments to accelerate the pace of scientific development. Grid technologies are also opening new doors in e-Business where small or medium-scale businesses can now have access to supercompting power which was once the province of only the most wealthy banks and multi-nationals. Data and resource sharing schemes such as these are changing the basic ground rules of computing.

All such schemes raise difficult issues of resource allocation and scheduling (roughly speaking, how to decide which computer does what, and when, and how they interact) which are made all the more complex by the inherent unpredictability of resource availability and performance. For example, I may forget to switch on my PC, a supercomputer may be required for a more important task, the internet connections I need may be particularly busy.

The Enhance project aims to simplify the effective programming of such systems by exploiting and synthesizing results from two underlying research programmes. Stochastic process algebras such as PEPA are used to model the behaviour of concurrent systems in which some aspects of behaviour are not precisely predictable. Pattern based programming recognizes that many real applications draw from a range of well known solution paradigms and seeks to make it easy for an application developer to tailor such a paradigm to a specific problem without re-inventing the wheel. The choice of a particular paradigm carries with it considerable information about implied scheduling dependencies. By modelling these with PEPA, and thereby being able to include aspects of uncertainty which are inherent to Grid computing, we believe that we will be able to underpin systems which can make better scheduling and dynamic rescheduling decisions than less sophisticated approaches.

About the project

The vision of Grids as both tools and enabling fabrics for distributed collaborative computation continues to excite applications designers and challenge the Computer Science research community. While some of the issues raised are familiar from previous environments in distributed and parallel computing, many are new.

This project seeks to address the allied problems of programmability, cost predictability and scheduling by augmenting a component based Grid HPC framework with a mechanism for constructing applications as instantiations of pre-defined application patterns. Each pattern will encapsulate the logical behaviour of a familiar pattern of multi-component interaction, at a level of abstraction which simplifies the developer's task, without over constraining the selected implementation. The definition and manipulation of semantic and performance behaviour of components and patterns will be facilitated by the use of a performance oriented process algebra as a modelling tool.

Underpinning an application development process of the form illustrated below, our key insights are:

• that the use of patterns will helpfully constrain the implementation challenge, by providing good knowledge of the overall evolution of an application's interactions;
• correspondingly, that patterns allow both more accurate cost prediction (which in turn informs good scheduling decisions) and lighter weight monitoring (because the pattern based model tells us what the crucial aspects are for each instantiation);
• that cost modelling with a formalised, stochastic performance process algebra and associated tools facilitates capture and manipulation of the remaining uncertain aspects of application behaviour.

Ultimately, by construction of a prototype and experimentation with a number of case studies, we will demonstrate that this approach allows simple, intuitive construction of applications, and facilitates run-time scheduling and rescheduling with an effectiveness which surpasses that of less constrained methodologies.

(More information about Enhance.)

Members of the Enhance project

Anne Benoit

Murray Cole

Stephen Gilmore

Horacio Gonzalez-Velez

Jane Hillston

Israel Hernandez

Publications

• Evaluating the Performance of Skeleton-Based High Level Parallel Programs, Anne Benoit, Murray Cole, Stephen Gilmore and Jane Hillston, ICCS 2004: Proceedings of the 4th International Conference on Computational Science, Kraków, Poland, Springer-Verlag Lecture Notes in Computer Science Volume 3038, pp. 289-296, June 6-9, 2004.
• Evaluating the performance of pipeline-structured parallel programs with skeletons and process algebra. Anne Benoit, Murray Cole, Stephen Gilmore and Jane Hillston, To appear in a special issue of Parallel and Distributed Computing Practices PDCP on Practical Aspects of High-level Parallel Programming PAPP 2004, Accepted September 2004.
• Scheduling skeleton-based grid applications using PEPA and NWS. Anne Benoit, Murray Cole, Stephen Gilmore and Jane Hillston, To appear in a special issue of The Computer Journal on Grid Performability Modelling and Measurement, Accepted September 2004.
• Two fundamental concepts in skeletal parallel programming. Anne Benoit and Murray Cole, Accepted to Practical Aspects of High-level Parallel Programming PAPP 2005, part of ICCS 2005. Atlanta, USA, May 2005.
• Enhancing the effective utilisation of Grid clusters by exploiting on-line performability analysis, Anne Benoit, Murray Cole, Stephen Gilmore and Jane Hillston, CCGrid 2005: Proceedings of the 1st International Workshop on Grid Performability, Cardiff, Wales, IEEE Press, May 2005.

Talks and presentations

• Evaluating the performance of skeleton-based grid applications, workshop presentation by Anne Benoit to NeSC Workshop on Grid Performability Modelling and Measurement, Edinburgh, Scotland, March 2004.
• Evaluating the performance of skeleton-based high-level parallel programs, conference presentation by Anne Benoit to ICCS workshop on Practical Aspects of High-level Parallel Programming (ICCS 2004: PAPP 2004), Kraków, Poland, May 2004.
• Enhancing the performance of Grid Applications with Skeletons and Process Algebras, departmental seminar presentation by Anne Benoit in Pisa, December 2004.
• Analyse quantitative de programmes applicatif à base de squelettes algorithmiques, conference presentation by Anne Benoit to Journées Francophones des Langages Applicatifs (JFLA05), Obernai, March 2005.
• Enhancing the effective utilisation of Grid clusters by exploiting on-line performability analysis, presented by Stephen Gilmore at the CCGrid 2005 workshop on Grid Performability, Cardiff, May 2005.
• Evaluation des performances de programmes parallèles haut niveau à base de squelettes algorithmiques, presentation by Anne Benoit to LACL, Paris 12, May 2005.
• The Righteous and the Wicked: efficient high-level methods for performance analysis, departmental colloquium by Stephen Gilmore at the School of Computing Science, University of Newcastle upon Tyne, June 2005.

Software

• The eSkel page

Funding

The Enhance project is funded by the Engineering and Physical Sciences Research council grant number GR/S21717/01. The project commenced on June 1st 2003 and will end on May 31st 2006.