Providing flexible resources, no matter what your needs are
About
Cloud computing offers a unique model of computation with its ability to provide “unlimited” compute resources as needed. This provides the flexibility to scale experiments based on needs. Scientific applications can benefit from cloud computing’s “pay-as-you-go” model as it can provide a mid-scale scientist with the resources they need, in addition to their grid resources.
Our process
We identified two categories of geo-science applications based on resource consumption:
- one class of applications demand HPC hardware with strict software requirements
- the second class of applications take very small amount of time to execute a single job on a desktop computer but need ensemble runs with large number of small jobs
Weather Research and Forecast model (WRF), the numerical weather prediction model for meso-scale weather predictions, falls into the first category of applications with its dependence on MPI (Message Passing Interface) to exploit parallelism in the application.
In our lab, we successfully demonstrated running WRF using cloud resources from Windows Azure, namely Virtual Machine roles, by using Windows HPC pack to enable MPI. The probabilistic ensemble execution of the Sea, Lake and Overland Surges from Hurricanes (SLOSH) storm surge prediction model falls into the second class of ensemble applications. SLOSH enables modelers to generate predictions on strength and impact of storms that start over the oceans. SLOSH is run as an ensemble of often up to 15,000 instances.
This high throughput Windows-based application with small data I/O is well suited to Windows Azure execution using Worker roles, which provide a Windows environment to execute .NET code, with very little effort. We used Trident Workflow Workbench as the workflow orchestration engine, scheduling jobs on Windows Azure using Sigiri, as the workflow middleware. We continue to explore efficient scheduling and execution of jobs by middleware on cloud resources.