Wiki page for notes on Jan-April 2012 DoC private cloud discussions

Intro

Sometime in early 2012, Susan told DCW that DoC were thinking of hiring someone (Jeremy Cohen) for 6 months into CSG, specifically tasked with building a DoC private cloud [definition unclear].

She explained the core idea was "virtualisation even for research clusters", i.e. research groups currently buy clusters when they have money, CSG set them up, install "linux du jour" on them, configure fileservers (if part of cluster), tape backups (if part), processing node special software etc.

Then the servers age, the OS is essentially frozen (it's often difficult to persuade researchers that we should reinstall their fileservers, webservers and compute nodes). They become "fragile". Sometimes it's hard to even retire them on schedule (4/5/6 years or whatever). Usually, these clusters are only accesible by members of that research group so the resource may not be fully utilised.

Susan's vision: setup a private cloud, researchers add hardware to that cloud's core resources, then create a VM for each cluster node, perhaps tied (1-1 at first) to their own hardware, CSG install that virtual cluster node (VCN)'s OS, researchers work as before - but each node's encapsulated inside a VM. Later, these VMs could share resources - when the group don't need 100% resources, or new more powerful hardware is purchased.

Suppose, for instance, the group needed N nodes x 100% of underlying VM host x M months [and then less thereafter].

Susan also added "and it should just scale, manage itself magically."