DoC's Private IaaS Cloud Service

Background

CSG has been running a DoC Infrastructure-as-a-service private cloud, very like Amazon EC2 ("Elastic Compute Service") over the past four years, allowing all DoC users to specify and create Virtual Machines via a convenient web interface, automatically installing OSes on them and deploying them in a matter of minutes. A longer term goal was to virtualize many research servers, decoupling the OS image from the hardware for greater flexibility, sharing the resources of each physical machine more efficiently.

The web interface for the DoC Private Cloud is now available for departmental users via cloudstack.doc.ic.ac.uk. Please use your normal college user-name and password for authentication; the domain should be "doc". Please note that the web interface is only accessible from within the college network. If you are outside the college network, use VPN or Shell Servers/SSH Tunnelling to access the resource.

We have been allocated a whole 1024-IP address vlan for VMs.

A Cloud user mailing list is available (doc-cloud-users@imperial.ac.uk), to subscribe please email us (help@doc.ic.ac.uk).

Predefined Templates

When creating a new VM (cloud instance), you can either install direct from some ISO image, or install from a predefined template (you can add your own templates later). Here are some of the templates we have created, and when you might use them: Each such VM will still be allocated a globally accessibly public IP address, if you require (with a valid reason) any ports to be accessible externally, please email help@doc.ic.ac.uk

The main "Featured" templates you will see a CSG-controlled Ubuntu-xx.04 templates (version 16.04, 64-bit). Creating a VM from either of these will result in a machine very like a lab machine or physical linux server: understanding DoC users and groups, accepting college passwords, seeing DoC home directories and shared volumes, having a globally accessible public IP address and DNS hostname (such as cloud-vm-XXX-NNN.doc.ic.ac.uk where NNN is the last octet of the IP address), and running CSG's configuration maintenance system (maint) to control services and populate configuration files. For root access on such a VM, the creator will have a pre-logged in root shell in the CloudStack WEB UI console.

If your colleagues need root access on such a VM, email help@doc.ic.ac.uk providing the hostname of the VM - root access for each desired user will be provided via the normal CSG-supported mechanism: kerberized su (su or ksu). CSG control the security and configuration of all such VMs. Hence we suggest that most long term VMs should use one of these templates.

If there are any Non CSG maintained VM available under the "Featured template" tab, it would be using college authentication, wouldn't be using DoC Home directories (would have access to ICT provided home directory services) and a default root login is provided via CloudStack WEB UI console. VMs created in this way will use the Linux standard "sudo" method for becoming root - and you don't need to ask CSG to do this. Root access via ssh can be obtained by providing your username with "sudo" privileges from the CloudStack WEB UI console. The details sections of the template/or the motd (message of the day) visible during login usually provides the method of obtaining root access. In addition, you are responsible for the continuing security of such a VM. You are the systems administrator; not CSG. This is really important. Such VMs are more flexible than CSG-controlled VMs, allowing you to tweak every setting as you like, building kernels from source, hand-tweaking exactly what files and hence services run, and are an excellent method of creating your own templates. Please ensure that IP address is not hardcoded in your VM before creating a template and deploying it.

"Community" templates

Under the "Community" tab, there would be several public customised templates visible, created by other users. We don't have login access to these templates. Please use it at your own risk.

One excellent use of templates created from hand-configured VMs is that you could (say) create a fleet of 5 transient VMs from a hand-built template that automatically start up some distributed service such as Hadoop or ElasticSearch, on booting these nodes can be automatically configured to discover each other and then be ready for perform some parallel/clustered computation - before the whole fleet of VMs is destroyed a few hours later. A cloud guide discussing this is nearly ready.

There are even APIs (called "CloudMonkey" and "Stacker_Bee") for automatically creating one or more VMs.

OFF-SITE NETWORK ACCESS

If you need to run a service that is directly accessible outside of the DoC network, please contact CSG explaining why you need such access.

Discussion list for DoC cloud users

It's a list for interested DoC private cloud users to discuss cloud related things on, spread best practice etc. CSG will also use this list to improve communication about the DoC cloud, for example to announce current problems with, or upcoming upgrades to, the DoC cloud. We'd like to invite you to subscribe to this mailing list if you're interested in doing so. To subscribe, please go to: https://mailman.ic.ac.uk/mailman/listinfo/doc-cloud-users

There is also an email-based interface for users (not administrators) of your list; you can get info about using it by sending a message with just the word `help' as subject or in the body, to:

VM Snaphots/Templates

None of the VMs are backed up by CSG. Please store critical code in gitlab or your home directory. Always prepare for redundancy. You can also use VM/Volume snapshots to backup the complete VM and restore it to the previous configuration.

Cloud Tutorials

More details on the DoC Private Cloud

The cloud uses the following software:

For more details, including the project planning, please see our older project/privatecloud documentation.

services/cloud (last edited 2016-10-06 14:44:11 by tjoseph1)