DoC Computing Support Group


Differences between revisions 4 and 20 (spanning 16 versions)
Revision 4 as of 2010-05-25 21:47:32
Size: 6625
Editor: dcw
Comment:
Revision 20 as of 2010-06-16 18:31:32
Size: 9767
Editor: dcw
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
Line 6: Line 5:
For organised sets of groups associated with a particular lecture course, the lecturer or lab organiser should organise the whole class into numbered groups and provides us with the whole dataset in one go. In this case we provide a standard package of:
Line 7: Line 7:
For organised sets of groups, where a lecturer or lab organiser
organises the whole class/cohort into numbered groups and provides us with
the whole dataset in one go, we provide a standard package of:
 * a new DoC group containing the correct members.
 * a shared group writable directory in /vol/project for the group to store their work.
 * the group project directory is (unless restricted) visible via the web under www.doc.ic.ac.uk/project/XXX..
 * a "fake user" belonging to the group who web CGI scripts in the project dir will run as.
 * a group project Postgres database for the group to use for database-related projects.
 * instructions for setting up Subversion (version control) in the shared directory.
Line 11: Line 14:
  * a new DoC group containing the correct members.
  * a shared group writable directory in /vol/project for the group to store their work.
  * the group project directory is (unless restricted) visible via the web under www.doc.ic.ac.uk/project/XXX..
  * a "fake user" belonging to the group who web CGI scripts in the project dir will run as.
  * a group project Postgres database for the group to use for database-related projects.
  * instructions for setting up Subversion (version control) in the shared directory.

It doesn't matter if you don't need all these facilities. It's easier to create, and delete, the whole
package together.
It doesn't matter if you don't need all these facilities. It's easier to create, and delete, the whole package together.
Line 23: Line 18:
  * are only provided for the particular project and associated with a particular lecture course code,
  * are set up by us (CSG) liasing with lecturers on behalf of whole sets of students,
  * will be archived shortly after the projects are finished.
 * are only provided for the particular project and associated with a particular lecture course code,
 * are set up by us (CSG) liasing with lecturers on behalf of whole sets of students,
 * will be archived shortly after the projects are finished.
Line 28: Line 23:
Many group and individual projects are ''web and database related''. Every DoC user already has a personal Postgres database created when you joined DoC (if you need the password reset and the email resent, just ask us), and we just mentioned above that every group project gets a group Postgres database.
Line 29: Line 25:
Many group and individual projects are ''web and database related''. You already have a personal Postgres database (separate from the group project ones mentioned above) created when you joined DoC,
if you need the password reset and the email resent, just ask us. On the web front, your `~/public_html` is of course available for you to create web and CGI scripts, as is your group project
directory as discussed above. Tomcat is available (we call it Personal Tomcat) for project development, and you can ask us if you want to use ASP.net and Microsoft SQL server.
On the web front, your `~/public_html` is of course available for you to create web and CGI scripts, as is your group project directory as discussed above.
Line 33: Line 27:
We have the following documentation: Tomcat is available (we call it Personal Tomcat) for project development, and you can ask us if you want to use ASP.net and Microsoft SQL server.
Line 35: Line 29:
  * [[guides/web|DoC Web guides covering CGI scripts]]
  * [[guides/databases|DoC database guides covering Postgres and Microsoft SQL server]]
  * [[guides/java|DoC Java guides covering JDBC, Tomcat, Servlets etc]]
We have the following documentation to get you started:

* [[guides/web|DoC Web guides covering CGI scripts]]
 * [[guides/databases|DoC database guides covering Postgres and Microsoft SQL server]]
 * [[guides/java|DoC Java guides covering JDBC, Tomcat, Servlets etc]]
Line 40: Line 36:

Many students want to work with interesting hardware or software for their projects. We understand this is a really interesting part of the course and want to help if possible,
indeed we're very happy to discuss the technical side of things with you or your supervisor, as long as you've done your research and initial planning beforehand..
Many students want to work with interesting hardware or software for their projects. We understand this is a really interesting part of the course and want to help if possible, indeed we're very happy to discuss the technical side of things with you or your supervisor, as long as you've done your research and initial planning beforehand.
Line 46: Line 40:
Short of a project machine, you might need us to install some additional software throughout the lab, in either Linux or Windows. In the Linux case, if it's pre-packaged in the Ubuntu repository,
installing it might be trivial for us. Or, if it's open source software we might be able to download and package it ourselves. Ask and we'll see what we can do.
Short of a project machine, you might need us to install some additional software throughout the lab, in either Linux or Windows. In the Linux case, if it's pre-packaged in the Ubuntu repository, installing it might be trivial for us. Or, if it's open source software we might be able to download and package it ourselves. Ask and we'll see what we can do.
Line 49: Line 42:
Alternatively, many pieces of open source software can be built and installed - by you - in your home directory. If you need more disk quota for building software - or for
storing datasets - please estimate how much you need, run it by your supervisor (especially if it seems on the large size:-)) and email us on help@doc.ic.ac.uk.
If you run into compilation problems, we're very happy to advise you, having considerable expertise in this area..
Alternatively, many pieces of open source software can be built and installed - by you - in your home directory. If you need more disk quota for building software - or for storing datasets - please estimate how much you need, run it by your supervisor (especially if it seems on the large size:-)) and email us on help@doc.ic.ac.uk . If you run into compilation problems, we're very happy to advise you, having considerable expertise in this area.
Line 54: Line 45:
A limited number of projects (group and individual) require dedicated project computers. We provide a small number of such project computers which can be allocated on request where needed.
Line 55: Line 47:
A limited number of projects (group and individual) require dedicated project computers. We provide a small number of such project computers which can be allocated on request after discussion.

However, the starting point is that Academic Committee decreed a few years ago that 95% of all projects (group and individual) should be done on standard shared lab computers, hence only 5% of projects should
require dedicated project machines. There is, therefore, a deliberately high hurdle that people must pass when requesting a project machine. Please discuss your needs with your supervisor first, as we will be
discussing any requests with them as well as you! While most people are very reasonable in their requests, occasionally someone wanders in and requests 5 project machines for a single project. Guess what: we say no!
However, the starting point is that Academic Committee decreed a few years ago that 95% of all projects (group and individual) should be done on standard shared lab computers, hence only 5% of projects should require dedicated project machines. There is, therefore, a deliberately high hurdle that people must pass when requesting a project machine. Please discuss your needs with your supervisor first, as we will be discussing any requests with them as well as you! While most people are very reasonable in their requests, occasionally someone wanders in and requests 5 project machines for a single project. Guess what: we say no!
Line 63: Line 51:
  * We have a small number of project desktop PCs in 210 (currently `project01`..`project11`), refreshed even more regularly than the lab.
  * We have 20 rack-mounted Sunfire X2150 servers in the ICT Data Centre in Mechanical Engineering, called `terra01`..`terra20`. These machines are quite old now, but are very suitable if you need to install and run non-graphical network services on a fixed location.
  * (New!) We can now create virtual machines on demand, essentially these are low power virtualised servers which can be used in a similar fashion to the terras, but of course being virtual you can ask for a particular hardware spec (how much disk space, memory and how many CPU cores the machine pretends to have) and OS release. Note that virtual machines are not suitable for performance measurements as their performance varies depending on what all the other VMs on a particular VMware host are doing!
 * We have a small number of project desktop PCs in 210 (currently `project01`..`project11`), refreshed even more regularly than the lab.
 * We have 20 rack-mounted Sunfire X2150 servers in the ICT Data Centre in Mechanical Engineering, called `terra01`..`terra20`. These machines are quite old now, but are very suitable if you need to install and run non-graphical network services on a fixed location.
 * (New!) We can now create virtual machines on demand, essentially these are low power virtualised servers which can be used in a similar fashion to the terras, but of course being virtual you can ask for a particular hardware spec (how much disk space, memory and how many CPU cores the machine pretends to have) and OS release. Note that virtual machines are not suitable for performance measurements as their performance varies depending on what all the other VMs on a particular VMware host are doing!
Line 69: Line 57:
  * Windows Vista, or
  * Ubuntu Hardy (8.04), or
  * Ubuntu Jaunty (9.04).
 * Windows Vista, or
 * Ubuntu Hardy (8.04), or
 * Ubuntu Jaunty (9.04).
Line 73: Line 61:
If needed, you can have Linux (Windows) root (administrator) privilege on your project machine so that you may reconfigure it, add software, start services, even compile and install a new kernel
or (in extreme cases) Linux distribution. However, in this case you become responsible for the machine's network security.
If needed, you can have Linux root (or Windows administrator) privilege on your project machine so that you may reconfigure it, add software, start services, even compile and install a new kernel or (in extreme cases) Linux distribution. However, in this case you become responsible for the machine's network security.
Line 76: Line 63:
Please email help@doc.ic.ac.uk with any project requests, be as clear as possible what you need, how much you need it, that your supervisor agrees with you, etc. We will discuss project requests
in some detail with you. We may suggest alternative ways and means of tackling the project if your original suggestion is unattainable.
Please email help@doc.ic.ac.uk with any project requests, be as clear as possible what you need, how much you need it, that your supervisor agrees with you, etc. We will discuss project requests in some detail with you. We may suggest alternative ways and means of tackling the project if your original suggestion is unattainable.
Line 79: Line 65:
Note that we will ''never'' allocate ordinary lab PCs in room 219, 202 or 206 to student projects, as these must remain trusted general purpose lab computers - for the other 95% of people, remember!
We cannot, therefore, not give root/admin access to anyone on such lab computers.
Note that we will ''never'' allocate ordinary lab PCs in room 219, 202 or 206 to student projects, as these must remain trusted general purpose lab computers - for the other 95% of people, remember! We cannot, therefore, not give root/admin access to anyone on such lab computers.

Note also that we do not normally move project machines around, for example when you want to give your presentation in a lecture theatre, you should not assume that your project machine can be moved. Plan your presentation and demo accordingly. Can you access your project machine remotely (eg. via ssh or vnc or rdesktop) and demo it from the lecture theatre? Or record a short video demonstration? Or organise a separate demo in room 210?

Project machines will be reclaimed very shortly after the end of your projects and reinstalled for reuse. If you want to archive project materials, do so before the project is over. Do not expect project machines to remain after the end of the project. In exceptional circumstances (eg. your project wins a prize and a demo is required N months later, or your supervisor wishes to deploy your project in a permanent location) contact us to discuss the options for keeping a project machine going for a while.

== Batch Processing and GPGPU ==
You may find that your project requires computational resources in excess of an individual lab computer for reasons other than those given above. If so, you might want to consider the batch-processing options available within the department:

 * The [[services/hpc/condor|Condor]] batch-processing system.
 * [[guides/remoteaccess/ssh|Remote access]] of the batch servers using ssh (see also 'man nohup').
 * Access to the [[http://www.doc.ic.ac.uk/csg-old/planetlab/|PlanetLab]] global research network.

If you are using CUDA, please note that while all the DoC lab computers have a CUDA-capable Nvidia graphics card and driver installed, the CUDA SDK is not installed; you will need to download and install that in your home directory (say) yourself. You should also check that the available graphics cards support the version of CUDA that you require. If not, please consult your supervisor about obtaining the required hardware and then contact CSG.

== Off-site network access ==
If you need to run a service that is accessible outside of the DoC network, ports 55000 to 56000 are open on DoC lab computers for this purpose.

== Colour Printing (Final reports etc) ==
Individual and group projects generate final reports, often in glorious technicolour. CSG have a colour printer called "hue" in room 225, you are allowed to print the required number of copies of your final reports on this printer free of charge. You can either print the whole report, or (if only a small proportion of pages have colour on them) just select pairs of pages and print them. Having printed your reports, come to 225 and ask a member of CSG to release one of your print jobs.

Please make sure that you have finished all your proof reading beforehand, i.e. that what you print is the final copy, as having printed multiple copies we will be seriously unimpressed if you find an error and want to reprint them all! Note that all printers are incredibly busy in the immediate run up to report deadlines, so don't expect printing to take less than 2 hours at that time!

Resources for Group and Individual Projects

We provide various resources for DoC students doing official projects.

Group Project Resources

For organised sets of groups associated with a particular lecture course, the lecturer or lab organiser should organise the whole class into numbered groups and provides us with the whole dataset in one go. In this case we provide a standard package of:

  • a new DoC group containing the correct members.
  • a shared group writable directory in /vol/project for the group to store their work.
  • the group project directory is (unless restricted) visible via the web under www.doc.ic.ac.uk/project/XXX..
  • a "fake user" belonging to the group who web CGI scripts in the project dir will run as.
  • a group project Postgres database for the group to use for database-related projects.
  • instructions for setting up Subversion (version control) in the shared directory.

It doesn't matter if you don't need all these facilities. It's easier to create, and delete, the whole package together.

Please note that these project directories:

  • are only provided for the particular project and associated with a particular lecture course code,
  • are set up by us (CSG) liasing with lecturers on behalf of whole sets of students,
  • will be archived shortly after the projects are finished.

Many group and individual projects are web and database related. Every DoC user already has a personal Postgres database created when you joined DoC (if you need the password reset and the email resent, just ask us), and we just mentioned above that every group project gets a group Postgres database.

On the web front, your ~/public_html is of course available for you to create web and CGI scripts, as is your group project directory as discussed above.

Tomcat is available (we call it Personal Tomcat) for project development, and you can ask us if you want to use ASP.net and Microsoft SQL server.

We have the following documentation to get you started:

Individual Projects

Many students want to work with interesting hardware or software for their projects. We understand this is a really interesting part of the course and want to help if possible, indeed we're very happy to discuss the technical side of things with you or your supervisor, as long as you've done your research and initial planning beforehand.

Now, the single most common request is for a dedicated project machine - please see the next section for a detailed discussion of project machines.

Short of a project machine, you might need us to install some additional software throughout the lab, in either Linux or Windows. In the Linux case, if it's pre-packaged in the Ubuntu repository, installing it might be trivial for us. Or, if it's open source software we might be able to download and package it ourselves. Ask and we'll see what we can do.

Alternatively, many pieces of open source software can be built and installed - by you - in your home directory. If you need more disk quota for building software - or for storing datasets - please estimate how much you need, run it by your supervisor (especially if it seems on the large size:-)) and email us on help@doc.ic.ac.uk . If you run into compilation problems, we're very happy to advise you, having considerable expertise in this area.

Project Machines

A limited number of projects (group and individual) require dedicated project computers. We provide a small number of such project computers which can be allocated on request where needed.

However, the starting point is that Academic Committee decreed a few years ago that 95% of all projects (group and individual) should be done on standard shared lab computers, hence only 5% of projects should require dedicated project machines. There is, therefore, a deliberately high hurdle that people must pass when requesting a project machine. Please discuss your needs with your supervisor first, as we will be discussing any requests with them as well as you! While most people are very reasonable in their requests, occasionally someone wanders in and requests 5 project machines for a single project. Guess what: we say no!

Having said all that, we currently offer three different types of project machines:

  • We have a small number of project desktop PCs in 210 (currently project01..project11), refreshed even more regularly than the lab.

  • We have 20 rack-mounted Sunfire X2150 servers in the ICT Data Centre in Mechanical Engineering, called terra01..terra20. These machines are quite old now, but are very suitable if you need to install and run non-graphical network services on a fixed location.

  • (New!) We can now create virtual machines on demand, essentially these are low power virtualised servers which can be used in a similar fashion to the terras, but of course being virtual you can ask for a particular hardware spec (how much disk space, memory and how many CPU cores the machine pretends to have) and OS release. Note that virtual machines are not suitable for performance measurements as their performance varies depending on what all the other VMs on a particular VMware host are doing!

The project desktops normally come with both Vista and Ubuntu Jaunty installed, the servers (terras and VMs) with only your chosen OS on. We currently offer the following OSes:

  • Windows Vista, or
  • Ubuntu Hardy (8.04), or
  • Ubuntu Jaunty (9.04).

If needed, you can have Linux root (or Windows administrator) privilege on your project machine so that you may reconfigure it, add software, start services, even compile and install a new kernel or (in extreme cases) Linux distribution. However, in this case you become responsible for the machine's network security.

Please email help@doc.ic.ac.uk with any project requests, be as clear as possible what you need, how much you need it, that your supervisor agrees with you, etc. We will discuss project requests in some detail with you. We may suggest alternative ways and means of tackling the project if your original suggestion is unattainable.

Note that we will never allocate ordinary lab PCs in room 219, 202 or 206 to student projects, as these must remain trusted general purpose lab computers - for the other 95% of people, remember! We cannot, therefore, not give root/admin access to anyone on such lab computers.

Note also that we do not normally move project machines around, for example when you want to give your presentation in a lecture theatre, you should not assume that your project machine can be moved. Plan your presentation and demo accordingly. Can you access your project machine remotely (eg. via ssh or vnc or rdesktop) and demo it from the lecture theatre? Or record a short video demonstration? Or organise a separate demo in room 210?

Project machines will be reclaimed very shortly after the end of your projects and reinstalled for reuse. If you want to archive project materials, do so before the project is over. Do not expect project machines to remain after the end of the project. In exceptional circumstances (eg. your project wins a prize and a demo is required N months later, or your supervisor wishes to deploy your project in a permanent location) contact us to discuss the options for keeping a project machine going for a while.

Batch Processing and GPGPU

You may find that your project requires computational resources in excess of an individual lab computer for reasons other than those given above. If so, you might want to consider the batch-processing options available within the department:

  • The Condor batch-processing system.

  • Remote access of the batch servers using ssh (see also 'man nohup').

  • Access to the PlanetLab global research network.

If you are using CUDA, please note that while all the DoC lab computers have a CUDA-capable Nvidia graphics card and driver installed, the CUDA SDK is not installed; you will need to download and install that in your home directory (say) yourself. You should also check that the available graphics cards support the version of CUDA that you require. If not, please consult your supervisor about obtaining the required hardware and then contact CSG.

Off-site network access

If you need to run a service that is accessible outside of the DoC network, ports 55000 to 56000 are open on DoC lab computers for this purpose.

Colour Printing (Final reports etc)

Individual and group projects generate final reports, often in glorious technicolour. CSG have a colour printer called "hue" in room 225, you are allowed to print the required number of copies of your final reports on this printer free of charge. You can either print the whole report, or (if only a small proportion of pages have colour on them) just select pairs of pages and print them. Having printed your reports, come to 225 and ask a member of CSG to release one of your print jobs.

Please make sure that you have finished all your proof reading beforehand, i.e. that what you print is the final copy, as having printed multiple copies we will be seriously unimpressed if you find an error and want to reprint them all! Note that all printers are incredibly busy in the immediate run up to report deadlines, so don't expect printing to take less than 2 hours at that time!

 
 

services/projects (last edited 2015-04-13 15:09:46 by ldk)