OpenStack and HPC

The OpenStack and HPC BOF was at SC14

I've spent the last week out in New Orleans at SC14, there seemed to be quite a lot of buzz about HPC and OpenStack, as well as a number of vendors there who didn't really get why OpenStack might be applicable to HPC.

From my perspective, there are two clear use cases for OpenStack with HPC, the first is shared capacity bursting - where a pool of resources is setup into an OpenStack environment and then the job scheduler will dynamically spin-up VMs.

The second is the need to be able to have clean HPC systems, i.e. that data residue isn't left on systems (for example some of our commercial research partners or medical data), and so using an OpenStack VM, which is spun up on demand and destroyed afterwards is of great interest.

At Birmingham, we use Adaptive Moab for our HPC scheduler and Adaptive we demonstrating their scheduler OpenStack integration. To be clear, this isn't a replacement for the Nova VM scheduler but is an add-on for the Moab HPC scheduler. Basically it uses Moab triggers to call the plugin, for example we could set a trigger such that if there is 100 jobs queuing then the trigger will fire. As we're building an OpenStack installation for the CLIMB project, there may be times when that resource is quiet and so we could use some of the resource to fulfil our HPC workload.

Obviously we'd want to be careful about the types of job which ran on the OpenStack environment (big MPI jobs would be bad ;-) ), but it looks of great interest! That's not to say it won't be difficult to build the VM image and I have a lot of question about that (like how to get the GPFS file-system inside a VM image), but its something we'd like to look at.

I also attended the BOF on HPC and OpenStack and was surprised by how many people were there and how much interest there was, I expected a dozen or so, but the room was full and more standing at the back, so clearly OpenStack and HPC is of a lot of interest and vendors really need to sit up and listen!

I'm working with the CLIMB project at Birmingham which spans across Cardiff, Swansea and Warwick as well, and I met up with some of the guys from the sysadmin team on the eMedLab project which is similar. Neither of these are doing the schedule VMs from a traditional scheduler approach, but are there to help get predefined pipelines up and running and to aid with teaching and research. We're planning to catch up in a few weeks either at the Machine Evaluation Workshop or a specific project meeting.

What amazes me about CLIMB and eMedLab is how the proposals were independently developed by the PIs, yet the idea and technology is almost identical. As we are so closely aligned, it makes a lot of sense for us to try and work together on this, we also happen to have both selected the same integrator (OCF) for the OpenStack components backed by SystemX hardware.

If you're doing or planning a similar project in the UK (not necessarily BioInformatics either!), then please get in touch and we'll see if we can get something working collaboratively.