The ECHO project assumes a resource landscape where utility clouds are the norm and aims to experiment with the added value capabilities which the utility resource landscape might offer.

The work will not focus on one cloud provider or one provider’s technology but experiment with applications that are multi-provider and provider technology agnostic. The goal is to use the emerging cloud providers as a marketplace for delivering the best value to researchers at the time they require resources. The work shall look at both small and large scale applications, and investigate how dynamic resource scaling, resilience and quality of service can be achieved and how these might benefit the research community.

The project work focuses on two typical research applications which act as exemplars:

i. a suite of numerical linear algebra applications for solving large data sets. These applications occur throughout HPC and are a well defined exemplar for testing the feasibility of HPC in the cloud.

ii. a suite of bioinformatics algorithms for gene searching. These algorithms also occur frequently in the domain, are well understood and thus are good exemplars for testing service centric applications that are gaining popularity.

The capabilities of cloud resource providers is expanding with multi-core CPUs and GPUs (Amazon EC2), a range of storage, network and security options, and an increasing array of added value services such as automated scaling and resilience. This gives a researcher the capability to create a specialised platform which suits their applications and their user community and to pay for that platform only when they need it.

The output of the project will be a rounded picture of the exemplar research applications run in the cloud which balances speed of execution with cost of use, time to result, cost of long term support, flexibility and support for the dynamic research communities that is the norm in research today.

The project will use the Zeel/i library developed by Belfast e-Science which provides an API for managing resources across many providers. These include private clouds such as the Belfast e-Science cloud, community based clouds such as Eucalyptus and the NGS; and public clouds such as Amazon EC2, Flexiscale and Rackspace. The API allows an application to easily create a dynamic platform tailored towards specific requirements based on cost budgets and resource constraints.

Month 1: A base line of Cloud Applications (Completed):

The work on the project so far has led to the development of a prototype numerical application suite. Each application uses a multi-threaded Java-based orchestration, which interfaces with the cloud through Zeel/i. The orchestration acquires the necessary machines, each one configured with an optimised multi-threaded BLAS/LAPACK library. It then deploys numerical services which use BLAS/LAPACK and orchestrates them in parallel.

In Month 2 the work will focus on expanding the applications in the numerical suite and adding multi-provider features.