https://mrcdata.dide.ic.ac.uk/wiki/api.php?action=feedcontributions&feedformat=atom&user=AdminMRC Centre for Outbreak Analysis and Modelling - User contributions [en]2026-05-24T08:08:24ZUser contributionsMediaWiki 1.43.6https://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Resources_for_Public_Events&diff=14291Resources for Public Events2026-03-23T15:19:37Z<p>Admin: </p>
<hr />
<div>On occasion, we are involved in running educational events for the public. Here, you will find details of some of the events we've done in the past, and the software and hardware we have available for future events. If you're planning a new event, you are really welcome to talk to Wes and see if there are any tools that can help. Everything listed below is designed to be re-used. Also on occasion, being involved in these events has led to some paper-writing, as well as being a lot of fun, so if you get the chance in the future, jump on board if you can.<br />
<br />
== Previous Events ==<br />
* Sep 2019-2025 - Short Course Social. [https://github.com/mrc-ide/public-events-microbit-epidemic/ Microbit Epidemic].<br />
* Mar 2019 - VIMC Annual Meeting. [https://github.com/mrc-ide/public-events-microbit-epidemic/ Microbit Epidemic].<br />
* Sep 2018 - Short Course Social. [https://github.com/mrc-ide/public-events-microbit-epidemic/ Microbit Epidemic].<br />
* Sep 2017 - Short Course Social. [[Barcode Epidemic]].<br />
* May 2017 - Imperial Festival. [[Barcode Epidemic]], [[Herd Immunity]], Epidemic Top Trumps.<br />
* Feb 2017 - Teaching in France. [[Barcode Epidemic]] - French language version!<br />
* September 2016 - Short Course Social. [[Barcode Epidemic]].<br />
* August 2016 - Sutton Trust Summer School. [[Barcode Epidemic]] - thanks Izzy Routledge.<br />
* May 2016 - Imperial Festival. [[Barcode Epidemic]], [[Zombie Sim II]], Epidemic Top Trumps.<br />
* September 2015 - Short Course Social. [[Barcode Epidemic]].<br />
* May 2015 - Imperial Festival. [[Barcode Epidemic]], [[Zombie Sim II]].<br />
* March 2015 - HPRU Open Day. [[Barcode Epidemic]], [[Zombie Sim II]], [[Herd Immunity]].<br />
* September 2014 - Short Course Social. [[Barcode Epidemic]].<br />
* [http://www3.imperial.ac.uk/newsandeventspggrp/imperialcollege/administration/outreach/eventssummary/event_18-12-2012-13-51-41 July 2014 - Imperial Y10 Summer School Workshop]. [[Zombie Sim II]], [[Herd Immunity]] - Harriet Mills and Rafal Mostowy.<br />
* [[May 2014 - Imperial Festival]]. DIDE STI groups, using the tablets for Google Docs surveys.<br />
* September 2013 - Short Course Social. [[Barcode Epidemic]].<br />
* [[June 2013 - MRC Centenary, Science Museum]]. [[Herd Immunity]]. [[Barcode Epidemic]].<br />
* [[March 2013 - Big Bang Fair]]. [[Zombie Sim I]]. [[Live Epidemic]].<br />
* [[October 2012 - Imperial Fringe]]. [[Zombie Sim I]]. [[Live Epidemic]].<br />
<br />
== Hardware ==<br />
<br />
* [[HP Slate 7 2801]] x10 - Android Tablet with various issues that can be worked around.<br />
* [[HP Elitebook 8760w]] - A large heavy 17" quad-core beast of a laptop. Portable in a fairly loose sense.<br />
* [[Lenovo Thinkpad E540]] - A lighter 15.6" quad-core laptop, with only a horrid touchpad to complain about.<br />
* [[Viewsonic VS14935]] (aka PJD8633WS) short-throw projector. Really good bit of kit for massive projections at short range.<br />
* [[Belkin N750DB]] - A supposedly top of the range wireless router with some serious limitations. (Retired)<br />
* [[D-Link Router]] - A new kettle-shaped router that overcame some of the horrors of the Belkin one.<br />
* [[Logitech K750]] - Wireless solar-powered keyboard.<br />
* [[Logitech M185]] - Wireless mouse. (Use single Logitech USB Unified dongle for both)<br />
* 101 micro:bits with lanyards. See the [https://github.com/mrc-ide/public-events-microbit-epidemic/ Microbit Epidemic].<br />
<br />
== Software ==<br />
<br />
* [[Barcode Epidemic]] - A suite of tools for printing unique QR codes, an Android app to scan and send them to a web server running live display tools.<br />
* [[Herd Immunity]] - A snooker-table like analogy where different coloured balls represent sickness, susceptibility, recovery and immunity. R0 and vaccination can be varied. Can be controlled by Android.<br />
* [[Live Epidemic]] - The predecessor to the Barcode Epidemic, with entries typed into Excel, for analysis by R.<br />
* [https://www.github.com/mrc-ide/public-events-barcode-epidemic Microbit Epidemic] - sort-of derived from the [[Barcode Epidemic]] - infections are broadcast by bluetooth radio between micro:bit computers.<br />
* [[Zombie Sim I]] - An interactive UK spatial individual-based gravity-model simulator allowing side-by-side comparison of different parameters. This version has quite a few parameters you can change, so good for a technical, or epi-familiar crowd.<br />
* [[Zombie Sim II]] - This is a cut-down version of [[Zombie Sim I]] with fewer parameters to edit, and a larger map. It's better for a less technical, or younger crowd, or when you have a bit less time.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Cluster_Documentation&diff=14290Cluster Documentation2025-07-14T13:31:17Z<p>Admin: /* Python */</p>
<hr />
<div>* We have two high performance clusters for general use, both running Microsoft HPC 2012 R2, Update 3. The smaller, older cluster is [[fi--dideclusthn]] and the larger is [[fi--didemrchnb]]. <br />
<br />
* Many users in the department write code and packages in R. We have in-house packages that can allow an almost-interactive experience of running jobs on our high performance clusters, so if that sounds like what you'd like, you might as well stop reading here, and go straight to [https://github.com/mrc-ide/didehpc The DIDEHPC package] - or if you need to join the mrc-ide github org first, go to [https://github.com/mrc-ide/welcome The welcome page].<br />
<br />
If you're not using R, or you are interested in other ways, then continue...<br />
<br />
== Windows Users ==<br />
<br />
The cluster nodes are Windows 2012-R2 based. This is good news if you're a Windows user, and especially easy for PCs within the departmental DIDE domain. You have the option of using the MS client for launching jobs (see below), or you can use try the new [[HPC Web Portal]]. If you're preparing to use the cluster, and have doubts about whether it can run what you want - and whether it will be straightforward to develop and run on it, best talk to me and/or the IT guys first!<br />
<br />
== Linux and MAC Users ==<br />
<br />
There are two problems, but both can be at least partially overcome. The first is that there is no client for linux. The second is that it can only run windows binaries.<br />
<br />
=== Launching jobs on Linux/Mac ===<br />
<br />
* The best way might be the [[HPC Web Portal]], which lets you submit jobs through a webpage. Give it a try, and feedback to me any issues! <br />
<br />
* Alternatively, you can either (1) go down the VM route, installing a windows virtual machine, eg VMWare or Parallels, and follow the instructions below. (2) Remote Desktop to your friend's Windows machine. These tend not to be very convenient, but some users have submitted thousands of jobs this way.<br />
<br />
=== What Linux/Mac jobs can I run? ===<br />
<br />
* Many jobs may be platform indepedent. Where compiled code is involved though, you'll have some extra work to make sure the binary parts can be run in windows. For R, [https://github.com/mrc-ide/didehpc the tools] help you sort that out. For others, you may need to run a job on the cluster to do your builds. Also, for many linux applications, there is a windows port. Mileage can vary, but it's worth a try searching for that before you give up hope. Some of these might be installed on the cluster already - see below. Or some you may be able to download and call them from your run script.<br />
<br />
* For C code, again, we can run scripts on the cluster to build the binaries for you, if your code can be compiled with (say) the MinGW port of gcc. Cygwin can also be used (although seems less common these days) with accompanying dlls. Get in touch if you need further ideas; most things can be solved. <br />
<br />
* Matlab is a little more tricky - we have a range of colossal matlab runtimes on the cluster, but you'd need to compile a windows binary to use them, since the matlab platforms are distinct in every way. For this, probably find a friend, or get in touch, and we'll work out how you can compile for it.<br />
<br />
* There is also the option of running a Windows VM, which might solve all of the above problems.<br />
<br />
=== But I really want a unix cluster. ===<br />
<br />
* We have run linux clusters in the past ([[fi--didelxhn]]), but there was very low demand, whereas the other clusters are consistently well used. Accessing DIDE network shares from linux was a little harder in some aspects, but we did not implement the more traditional linux approach of having dedicated separate cluster storage that inputs and outputs would have to be staged on to, in order for the cluster to see them. Generally, the flexibility of the other clusters makes such things quite straightforward, and may even feel a pleasure if your background is with the less domain-integrated style of cluster architecture.<br />
<br />
* It has also become far easier to write software that builds on multiple platforms with good practice. Tools that only run on one platform are rare, and in some circumstances have raised questions about the quality of those packages. Alternatives are often available.<br />
<br />
* Where compilation is the issue - for example if you want to run binaries on the cluster, but your compilation machine runs linux, or is a mac, (Matlab via MCC, or C++ code for example), then get in touch, and there are a few ways we can think of solving that, either by getting someone else, or the cluster itself, to build your code.<br />
<br />
* There are also resources in [https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/computing/high-throughput-computing/ ICT) who run a linux cluster, with the file-staging approach, and also (being shared between a much larger pool of users) have rather stricter rules on job durations and resources than we provide locally in DIDE.<br />
<br />
== Getting Started ==<br />
<br />
=== Getting access to the cluster ===<br />
<br />
Send a mail to Wes (w.hinsley@imperial.ac.uk) requesting access to the cluster. He will add you to the necessary groups, and work out what cluster you should be running on. Unless you are told otherwise, this will be fi--dideclusthn. But if you have been told otherwise, then whenever you see fi--dideclusthn, replace it with the cluster name you’ve been given, either mentally or (less effectively) on your screen with tip-ex.<br />
<br />
=== Windows: Install the HPC client software ===<br />
<br />
* Run \\fi--dideclusthn.dide.ic.ac.uk\REMINST\setup.exe. <br />
* Confirm '''Yes, I really want to run this'''.<br />
* There's an introductory screen, and you click '''Next'''<br />
* You have to tick the '''Accept''', then click '''Next'''<br />
* If it gives you any choice, select '''Install only the client utilities'''<br />
* '''Next''', '''Next''', '''Next'''<br />
* Select '''Use Microsoft Update''' and then '''Next'''<br />
* And then '''Install'''<br />
* And possibly '''Install''' again – if it had to install any pre-requisites first.<br />
* And '''Finish''', and the client installation is done.<br />
* To check everything is alright so far, open a '''NEW command prompt window'''.<br />
* (Because the installer adds things to the path, which only take effect on '''NEW comamnd windows'''! :-)<br />
* (Find it under Accessories, or you can use Start, Run, '''cmd''').<br />
* Type <code>job list /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
And it will list the jobs you have running, which will be none so far!<br />
<br />
<pre><br />
Id Owner Name State Priority Resource Request<br />
---------- ------------- ----------------- ------------ ----------- -------- -------<br />
<br />
0 matching jobs for DIDE\user<br />
</pre><br />
<br />
=== Windows: Using a non-domain machine ===<br />
<br />
If you're using a machine that isn't actually logged into DIDE, then the client software will have a problem working out who you are. In this case, there are two things you need to do.<br />
* Check you've really got a DIDE username. If you don't know what it is, talk to your lovely IT team.<br />
* Connect to the DIDE VPN using ZScaler from https://uafiles.cc.ic.ac.uk/, and login to that with your IC account - but you'll still need your DIDE account to access DIDE servers after the ZScaler connection is up and running.<br />
* Now we'll open a command window using "runas" - which lets you choose which identity the system thinks you are within that command window:-<br />
<pre><br />
runas /netonly /user:DIDE\user cmd<br />
</pre><br />
* (Change user to your own login obviously). It will ask for your password, then open a new command window. In that window, you'll be able to do cluster-related things as your DIDE username.<br />
<br />
=== Windows: Launching and cancelling jobs ===<br />
==== Windows: Command Line ====<br />
<br />
Suppose you have a file called "run.bat" in your home directory, which does what you want to run on the cluster. Let's say it's a single-core, very simple job. We’ll discuss what should be inside "run.bat" later. To submit your job on the cluster, at the command prompt, type this (all on one line) - or put it in a file called "launch.bat", and run it:-<br />
<br />
<code>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1 \\qdrive.dide.ic.ac.uk\homes\user\run.bat</code><br />
<br />
If it's the first time you’ve run a job - or if you've recently changed your password, then it might ask you for your DIDE password and offer to remember it. Otherwise, it will just tell you the ID number of your job.<br />
<br />
<pre>Enter the password for ‘DIDE\user’ to connect to ‘FI—DIDECLUSTHN’:<br />
Remember this password? (Y/N)<br />
job has been submitted. ID: 123<br />
</pre><br />
<br />
If you want to remove the job, then:- <code>job cancel 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
Or view its details with <code>job view 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
==== Windows: Job Manager GUI ====<br />
<br />
Alternatively to the command-line, you can use the job management software, rather than the command-line. The advantage is that it’s a GUI. The disadvantage is, as in all GUIs, you may not feel totally sure you know what it’s up to – where most of the time the things you want to do might not be very complex, as above.<br />
<br />
The job management software will be on your start menu, as above. All the features are under the “Actions” menu, and hopefully it will be self explanatory. Read the details below about launching, and you'll find the interface bits that do it in the Job Manager. However, you may find over time, especially if you run lots of jobs, that learning to do it the scripted way with the command-line can be quicker.<br />
<br />
<br />
== Information for running any job ==<br />
<br />
=== Visibility ===<br />
<br />
Any executable/batch file that the cluster will run must be somewhere on the network that the cluster has access to, when it logs in as you. This amounts to any network accessible drive that you have access to when you login – including network shares, such as the temp drive, your home directory, and any specific network space set up for your project.<br />
<br />
After a job is launched, all paths are relative to the '''/workdir:''' specified on the job submit command-line. Windows does not usually allow a network path to be the current directory, and MS-HPC somehow gets around this. But if in doubt, use fully-qualified paths to specify where files should be read from, or written to. You may want to write code that takes the paths from a parameter file, or as a command-line parameter, to give as much flexibility as possible. <br />
<br />
Your home directory, and the temp drive are on departmental servers that don't have an especially fast connection to the cluster. So avoid having thousands of jobs all reading/writing to those places at once. Additionally, the temp drive is shared between all users, and your home drive has a limited quota. It is also backed up every day, so avoid using it as scratch storage for results you don’t actually want to keep. Generally, the shares you'll want to use from the clusters will be ones containing the word "NAS", which the big cluster has fast access to.<br />
<br />
=== Interactivity ===<br />
<br />
The job must run in an entirely scripted, unattended way. It must not require any key presses or mouse clicks, or other live interactivity while it runs. So jobs generally will read some sort of input (from a file, or from the way you run the job), do some processing, and write the results somewhere for you - all without intervention.<br />
<br />
== Launching jobs ==<br />
<br />
Jobs can be launched either through the Job Manager interface, or through the command line tools, which offer greater flexibility. We'll describe the command line method here; if you want to use the GUI, then it'll be a case of finding the matching boxes... Below are the specifics for our clusters. For more information, simply type <code>job</code> on the commandline, or <code>job submit</code> for the list of submission-related commands.<br />
<br />
=== FI--DIDECLUSTHN vs FI--DIDEMRCHNB ===<br />
<br />
Job submissions, as shown below, must specify a "job template", which sets up a load of default things to make the job work. On fi--dideclusthn, the job templates are called 4Core, 8Core and GeneralNodes, which will respectively force jobs to run on the 4-core nodes, the 8-core nodes, or on any machine available.<br />
<br />
On fi--didemrchnb, you can set the job template to be... 8Core, 12Core, 16Core, 12and16Core, and GeneralNodes - which hopefully are fairly self-explanatory. There are a couple of other job templates, (24Core and Phi), but those are a bit special purpose for now, so don't use them!<br />
<br />
=== Job Submission ===<br />
<br />
Job submissions from the command line can take the following form (all on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /stdout:\\path\to\stdout.txt<br />
/stderr:\\path\to\stderr.txt /numcores:1-1 /jobtemplate:4Core \\path\to\run.bat</pre><br />
<br />
=== The /singlenode argument ===<br />
<br />
In MS HPC (2012), Microsoft finally added a tag to allow you to say that the 'n' cores you requested must be on the same computer. Therefore, if you know precisely how many cores you want, then use the following (on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:true <br />
/jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
However, there is one bug with this, for the specific case where you request a whole node, regardless of how many cores it has. In this case, oddly, you have to disable single node:-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:false <br />
/numnodes:1 /jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
== Languages and libraries supported ==<br />
<br />
A number of languages, and different versions of languages are available on the cluster. The sections below refer to your "run.bat" file - a batch file that you will create which will get run by a cluster node when a suitable one is available. The commands described below are to be put in this "run.bat" file, and they add various directories to the path, so that the software you want will be added to the path. <br />
<br />
=== C/C++ === <br />
<br />
A number of Microsoft C++ and Intel C++ runtimes are installed, but it's usually better to try and avoid using them, and make your executable as stand-alone as possible. If it requires any external libraries that you've had to download, then put the .dll file in the same directory as the .exe file. If you use Microsoft Visual Studio, in Project Preferences, C/C++, Code Generation, make sure the Runtime Library is Multi-threaded (/MT) – the ones with DLL files won’t work. Even so, on recent versions of the Intel and Microsoft C compilers, "static linking" doesn’t really mean static when it comes to OpenMP, and you’ll have to copy some DLLs and put them next to your EXE file. See the OpenMP section below.<br />
<br />
The cluster nodes are all 64-bit, but they will run 32-bit compiled code. Just make sure you provide the right DLLs!<br />
<br />
Here is a document about [[Using the Xeon Phi]]<br />
<br />
Here is a document about [[Using MPI]]<br />
<br />
And if you want to use libraries such as GSL, then try [[C/C++ Libraries for Windows]]<br />
<br />
=== Java ===<br />
<br />
Java doesn't strictly need installing; you can setup any Java Runtime Environment you like, from Oracle, or anywhere. If you want to do this, then, (if you were to use the Server JRE from Oracle), something similar to the following will add java.exe and jvm.dll to the path, and set the JAVA_HOME environment variable, which most Java-using software will appreciate.<br />
<br />
set JAVA_HOME=\\my\path\to\jre1.8.0<br />
set path=%JAVA_HOME%\bin;%JAVA_HOME%\bin\server;%path%<br />
<br />
For convenience, if you're happy with Azul's OpenJDK Java 1.8.0 update 308 (September 2021), then <code>call setJava64</code> in your run script will set this up for you. Incidentally the above will also add what's needed for using the rJava package in R - in which case you'll perhaps also want something like <code>call setR64_4_1_0</code> in your run script.<br />
<br />
=== Perl ===<br />
<br />
Strawberry Perl 64-bit portable, v5.30.1.1. Put <code>call setPerl</code> at the top of your script.<br />
<br />
=== R ===<br />
<br />
'''New tools are coming''': [http://dide-tools.github.io/didehpc/vignettes/quickstart.html see here]. The instructions below are still valid if you want to do this manually.<br />
<br />
<br />
You can R jobs on all the clusters. This is the latest wisdom on how to do so – thanks to James, Jeff, Hannah and others. <br />
<br />
First, if you are wanting to use packages, then set up a package repository on your home directory by running this in R.<br />
<br />
<code>install.packages("<package>",lib="Q:/R")</code><br />
<br />
Now, write your run script. Suppose you have an R script in your home directory: Q:\R-scripts\test.R. And suppose you’ve set up your repository as above. Your run.bat should then be:-<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\test.R<br />
</pre><br />
Various packages require various R versions, and the cluster supports a few versions. To choose which one, change the first line of the script above to one of these – 32-bit or 64-bit versions of R releases. Purely for amusement, R's codenames are shown here too. <br />
<pre><br />
call setr32_2_13_1.bat call setr64_2_13_1.bat (anyone know the codename?)<br />
call setr32_2_14_2.bat call setr64_2_14_2.bat (Gift-getting season)<br />
call setr32_2_15_2.bat call setr64_2_15_2.bat (Trick or treat)<br />
call setr32_3_0_1.bat call setr64_3_0_1.bat (Good Sport)<br />
call setr32_3_0_2.bat call setr64_3_0_2.bat (Frisbee Sailing)<br />
call setr32_3_1_0.bat call setr64_3_1_0.bat (Spring Dance)<br />
call setr32_3_1_2.bat call setr64_3_1_2.bat (Pumpkin Helmet)<br />
call setr32_3_2_2.bat call setr64_3_2_2.bat (Fire Safety)<br />
call setr32_3_2_3.bat call setr64_3_2_3.bat (Wooden Christmas Tree)<br />
call setr32_3_2_4.bat call setr64_3_2_4.bat (Very Secure Dishes - Revised Version - R later renamed this to 3.2.5)<br />
call setr32_3_3_1.bat call setr64_3_3_1.bat (Bug in your hair)<br />
call setr32_3_3_2.bat call setr64_3_3_2.bat (Sincere Pumpkin Patch)<br />
call setr32_3_4_0.bat call setr64_3_4_0.bat (You Stupid Darkness)<br />
call setr32_3_4_2.bat call setr64_3_4_2.bat (Short Summer)<br />
call setr32_3_4_4.bat call setr64_3_4_4.bat (Someone to lean on)<br />
call setr32_3_5_0.bat call setr64_3_5_0.bat (Joy in Playing)<br />
call setr32_3_5_1.bat call setr64_3_5_1.bat (Feather Spray)<br />
call setr32_3_5_3.bat call setr64_3_5_3.bat (Great Truth)<br />
call setr32_3_6_0.bat call setr64_3_6_0.bat (Planting of a Tree)<br />
call setr32_3_6_1.bat call setr64_3_6_1.bat (Action of the Toes)<br />
call setr32_3_6_3.bat call setr64_3_6_3.bat (Holding the Windsock)<br />
call setr32_4_0_2.bat call setr64_4_0_2.bat (Taking Off Again)<br />
call setr32_4_0_3.bat call setr64_4_0_3.bat (Bunny-Wunnies Freak Out)<br />
call setr32_4_0_5.bat call setr64_4_0_5.bat (Shake and Throw)<br />
</pre><br />
<br />
The following versions are *not* available on the cluster at the moment - but here are their names anyway, as it seemed a shame to miss them out. Let me know if you desperately need one of these.<br />
<br />
<pre><br />
2.14.0 Great Pumpkin <br />
2.14.1 December Snowflakes<br />
2.15.0 Easter Beagle<br />
2.15.1 Roasted Marshmallow<br />
2.15.3 Security Blanket<br />
3.0.0 Masked Marvel<br />
3.0.3 Warm Puppy<br />
3.1.1 Sock it to me<br />
3.1.3 Smooth Sidewalk<br />
3.2.0 Full of Ingredients<br />
3.2.1 World famous astronaut<br />
3.2.4 Very Secure Dishes (the non-revised version)<br />
3.3.0 Supposedly Educational<br />
3.3.3 Another Canoe<br />
3.4.1 Single Candle<br />
3.4.3 Kite-Eating Tree<br />
3.5.2 Eggshell Igloo<br />
3.6.2 Dark and Stormy Night<br />
4.0.0 Arbor Day<br />
4.0.1 See Things Now<br />
4.0.4 Lost Library Book<br />
</pre><br />
<br />
It also seems that different R versions put their packages in different structures – sometimes adding "win-library" into there for fun. Basically, R_LIBS and R_LIBS_USER should be paths to a folder that contains a list of other folders, one for each package you've installed.<br />
<br />
IMPORTANT: R_LIBS and R_LIBS_USER paths must NOT contain quotes, nor spaces. If the path to your library contains spaces, you need to use old-fashioned 8-character names. If in your command window, you type <code>dir /x</code>, you’ll see the names – Program Files tends to become PROGRA~1 for instance. <br />
<br />
==== Passing parameters to R scripts ====<br />
Passing parameters to R scripts means you can have fewer versions of your R and bat files and easily run whole sets of jobs. You can get parameters into R using Rscript (but not Rcmd BATCH, I think) as follows. In the run.bat example above, the Rscript statement becomes:<br />
<pre><br />
Rscript Q:\R-scripts\test.R arg1 arg2 arg3<br />
</pre><br />
Within the R code, the arguments can be recovered using <br />
<pre><br />
args <- commandArgs(trailingOnly = TRUE)<br />
<br />
outFileName <- args[1] ## name of output file.<br />
dataFileName <- args[2] ## name of local data file. <br />
currentR0 <- as.numeric(args[3]) ## convert this explicitly to number. <br />
</pre><br />
If you want your arguments to have a particular type, best to explicitly convert (see R0 above). Better still, you can pass parameters directly into the batch file that runs the R script. Command line arguments can be referenced within the batch file using %1, %2, etc. For example, if you have a batch file, runArgs.bat:<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\%1.R %2 %3 %4<br />
</pre><br />
then<br />
<pre><br />
runArgs.bat myRScript arg1 arg2 arg3<br />
</pre><br />
will run the R script myRScript.R and pass it the parameters arg1, arg2 and arg3. The batch file runArgs.bat is now almost a '''generic R script runner'''.<br />
<br />
=== OpenMP ===<br />
<br />
This has been tested with C/C++ - but the same should apply to other languages such as fortran, that achieve multi-threading with a DLL file.<br />
<br />
==== Microsoft's C/C++ Compiler ====<br />
You will need to copy vcomp100.dll (for Visual Studio 2010 – it maybe vcomp90.dll for 2008), into the same directory as your executable. You can usually find the dll file in a directory similar to:- C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist\x64\Microsoft.VC100.OpenMP. Extrapolate for future versions! Also make sure that you've enabled OpenMP in the Properties (C++/Language).<br />
<br />
==== GCC or MinGW ====<br />
Usually, this applies to Eclipse use. In project properties, the C++ command “g++ -fopenmp”, the C command “gcc –fopenmp”, and in the Linker options, under Miscellaneous, Linker flags, put “-fopenmp” too. Copy the OpenMP DLLs from MinGW into the same directory as your final executable. You may find the dlls are in C:\MinGW\Bin\libgomp-1.dll and in the same place, libpthread-2.dll, libstdc++-6.dll and libgcc_s_dw2-1.dll.<br />
<br />
==== Intel C++ Compiler ====<br />
Copy the libiomp5md.dll file from somewhere like C:\Program Files (x86)\Intel\ComposerXE-2011\redist\intel64\compiler\libiomp5md.dll to the same place as your executable. And in Visual Studio, make sure you've enabled OpenMP in the project properties.<br />
<br />
==== How many threads? ====<br />
The OpenMP function omp_max_threads() returns the number of ''physical'' cores on a machine, not the number of cores ''allocated'' to your job. To determine how many cores the scheduler actually allocated to you, use the following code to dig up the environment variable CCP_NUMCPUS, which will be set by the cluster management software:- <br />
<pre><br />
int set_NCores() {<br />
char * val;<br />
char * endchar;<br />
val = getenv("CCP_NUMCPUS");<br />
int cores = omp_get_max_threads();<br />
if (val!=NULL) cores = strtol (val,&endchar,10);<br />
omp_set_num_threads(cores);<br />
return cores;<br />
}<br />
</pre><br />
<br />
=== WinBugs (or OpenBugs) ===<br />
<br />
They are similar, but OpenBugs 3.2.2 is the one we've gone for. Something like this as your run.bat script will work:-<br />
<pre><br />
call setOpenBugs<br />
openbugs \\path\to\script.txt /HEADLESS<br />
</pre><br />
<br />
=== MatLab ===<br />
<br />
There are various ways of producing a non-interactive executable from Matlab. Perhaps the simplest (not necessarily the best performance) way is to use “mcc.exe” – supplied with most full versions of Matlab, including the Imperial site licence version that you've probably got.<br />
<br />
==== Use mcc.exe to compile your code ====<br />
<br />
Use windows explorer to navigate to the folder where the “.m” files are for the project you want to compile. Now use a good text editor to create a file called “compile.bat” in that folder. It should contain something similar to the following:-<br />
<br />
<code>mcc -m file1.m file2.m file3.m -o myexe</code><br />
<br />
Don’t copy/paste the text from this document by the way – Word has a different idea of what a dash is to most other software, and will probably replace the two dashes with funny characters. so you have to list every single .m file that your project needs, after the’-m’. If you save this file, then double-click on it, then it will think for some while, and produce “myexe.exe” in this example. Copy your .exe file into a network accessible place as usual.<br />
<br />
==== Launch on the cluster. ====<br />
<br />
The launch.bat file will be exactly the same as before – see page 2. The run script will then start with a line that tells the cluster which version of Matlab you used to compile the cluster. Below is the table of different versions. The runtimes are huge and cumbersome to install on the cluster, so as a result I haven’t installed every single one. If you need one that’s not listed, get in touch.<br />
<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! Matlab Version<br />
! First line of run script<br />
|- <br />
| R2009b<br />
| call useMatLab79<br />
|- <br />
| R2010a<br />
| call useMatLab713<br />
|- <br />
| R2011a<br />
| call useMatLab715<br />
|- <br />
| R2011b<br />
| call useMatLab716<br />
|- <br />
| R2012a (64-bit)<br />
| call useMatLab717_64<br />
|- <br />
| R2012b (64-bit)<br />
| call useMatLab80_64<br />
|- <br />
| R2013a (64-bit)<br />
| call useMatLab81_64<br />
|- <br />
| R2014a (64-bit)<br />
| call useMatLab83_64<br />
|- <br />
| R2014b (64-bit)<br />
| call useMatLab84_64<br />
|- <br />
| R2015a (64-bit)<br />
| call useMatLab85_64<br />
|- <br />
| R2015b (64-bit)<br />
| call useMatLab90_64<br />
|- <br />
| R2016b (64-bit)<br />
| call useMatLab91_64<br />
|- <br />
| R2017a (64-bit)<br />
| call useMatLab92_64<br />
|- <br />
| R2019a (64-bit)<br />
| call useMatLab96_64<br />
|- <br />
| R2021a (64-bit)<br />
| call useMatLab910_64<br />
|- <br />
| R2022a (64-bit)<br />
| call useMatLab912_64<br />
|}<br />
<br />
=== Python ===<br />
<br />
* The most recent version of each major release is on the cluster. Different python versions within the same release (e.g., 3.5.1 and 3.5.2) do not behave nicely together in automated installation, so if you have a desperate need for a previous minor version, get in touch. While 2.7.18 is available, we '''strongly''' recommend migrating away from Python 2.7, as it has been officially discontinued on 1st January 2020. Experiences may vary, and future success is not guaranteed.<br />
<br />
<pre><br />
2.7.18 64-bit call set_python_27_64.bat<br />
3.5.10 64-bit call set_python_35_64.bat<br />
3.6.15 64-bit call set python_36_64.bat<br />
3.7.17 64-bit call set_python_37_64.bat<br />
3.8.20 64-bit call set python_38_64.bat<br />
3.9.21 64-bit call set_python_39_64.bat<br />
3.10.16 64-bit call set python_310_64.bat<br />
3.11.11 64-bit call set_python_311_64.bat<br />
3.12.11 64-bit call set_python_312_64.bat<br />
3.13.5 64-bit call set_python_313_64.bat<br />
<br />
</pre><br />
<br />
* You'll most likely want to use some packages, so you'll need to have a network-accessible location to put your packages in. If you're running a version of Windows with Python that matches one installed on the cluster, then you'll be able to share that package repository between the two. But if you're running a different version, or a different operating system, or a different Python version, then you'll need to keep your local package repository separate from the one the cluster uses.<br />
* Here comes the basic script to run some python. Suppose I decide to put my Python packages for the cluster in '''\\qdrive.dide.ic.ac.uk\homes\username\PythonCluster37_64''' (which is Q:\PythonCluster37_64 on a DIDE Windows PC). To run some python, something like the following:-<br />
<br />
call set_python_37_64<br />
set PYTHONPATH=\\qdrive\homes\username\PythonCluster37_64<br />
python \\qdrive\homes\username\Python\MyScript.py<br />
<br />
or if you want to do this in a programmatic way...<br />
<br />
import sys<br />
sys.path = ['\\\\qdrive\\homes\\username\\PythonCluster37_64'] + sys.path<br />
<br />
==== Packages ====<br />
<br />
* If you want to install a package like numpy, that's readily available through pip, then you can launch a batch file like this:-<br />
call set_python_36_64<br />
pip install numpy --target \\qdrive\homes\username\PythonCluster36_64<br />
* or alternatively, you could write a python script that looks like this, and launch it as normal:-<br />
import pip<br />
pip.main(['install','numpy','--target','\\\\qdrive\\homes\\username\\PythonCluster36_64'])<br />
<br />
* And if you're running the same Python version on a Windows machine, then you can do these two locally, without submitting a cluster job to do them for you.<br />
<br />
* And next time you submit a python job on the cluster, python will look in the place set by PYTHONPATH, and will find the packages you've just installed when you import them.<br />
<br />
=== Beast and Beagle ===<br />
<br />
We've got a couple of versions available. For BEAST 1.75 with BEAGLE 1.0: (the x y z represents all the command-line arguments you'll want to pass to beast)<br />
<pre><br />
call setbeast<br />
call beast-beagle x y z<br />
</pre><br />
<br />
If you want BEAST 1.82 or 1.83 with BEAGLE 2.1:<br />
<pre><br />
call setbeast_182 call setbeast_183<br />
call beast-beagle_182 x y z call beast-beagle_183 x y z<br />
</pre><br />
<br />
And if you want BEAST 2.3.2 with BEAGLE 2.1 (note that BEAST 2 is a different, er, beast to BEAST 1, regarding packages and compatibilities, so only use BEAST 2 if you know that's what you want to use).<br />
<pre><br />
call setbeast_232<br />
call beast-beagle_232 x y z<br />
</pre><br />
<br />
OpenCL is installed on all the nodes, which means Beagle will pick up the Intel Xeon processors and use what it can on them. It will also detect the Xeon Phis where they're available. If that's something you'd be particularly interested in using, drop me a line.<br />
<br />
=== Blast ===<br />
<br />
Drop me a line to join the party testing this, and to get added to the (large) BlastDB network share.<br />
<br />
<pre><br />
call setblast<br />
</pre><br />
<br />
will set the `BLASTDB` environment variable to point to what we current have of the `v5` database, and the Blast tools (2.9.0, 64-bit) will be added to the path. So you can <br />
then call `blastp`, `blastn`, `blastx` setc.<br />
<br />
=== BOW (BioInformatics on Windows) ===<br />
<br />
A number of Bio-informatics tools are ready to go. These used to be provided together on codeplex, but that has since been retired. Ugene is one project that<br />
maintains windows builds of them. Start your run.bat file with <code>call setBOW</code> to add these to the path:-<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File<br />
! Version<br />
|- <br />
| SamTools.exe<br />
| 0.1.19-44429cd<br />
|- <br />
| BCFTools.exe<br />
| 0.1.19-44428cd<br />
|- <br />
| bgzip<br />
| ?<br />
|- <br />
| razip<br />
| ?<br />
|- <br />
| bwa<br />
| 0.7.17-r1188<br />
|-<br />
| tabix<br />
| 0.2.5 (r1005)<br />
|-<br />
| minimap2<br />
| 2.24 (r1122)<br />
|-<br />
| raxml<br />
| 8.2.12 (best for each node)<br />
|-<br />
| raxml_sse3<br />
| 8.2.12 (pthreads+sse3)<br />
|-<br />
| raxml_avx<br />
| 8.2.12 (pthreads+avx)<br />
|-<br />
| raxml_avx2<br />
| 8.2.12 (pthreads+avx2)<br />
|}<br />
<br />
Note that BWA-MEM is hardwired to use a linux-only feature, so will produce an odd message and not do what you want if you use that mode on Windows. However, it seems that minimap2 may be a faster and more accurate way<br />
of doing what bwa does in any case, and that seems to have no such limitation.<br />
<br />
Also, note that raxml.exe is a copy of the most optimised build for the particular cluster node the job is running on. fi--dideclusthn only supports SSE3;<br />
fi--didemrchnb varies a little; the 32-cores support AVX2, the 24, 20 and 16 core machines support AVX, and only the oldest 12-core nodes only support SSE3. You don't have to worry about, except to expect performance to vary between them.<br />
<br />
=== GATK and Picard ===<br />
<br />
These both rely on Java, so put this in your run script:<br />
<br />
<pre><br />
call setJava64<br />
call setgatk<br />
</pre><br />
<br />
and then <code>gatk</code> in your scripts will use Python 3.7 to call the argument handling wrapper for GATK 4.2.2.0, and <code>picard</code> maps to the picard (2.26.2) jar file.<br />
<br />
=== FastQC ===<br />
<br />
This one has a slight eccentricity that instead of passing normal command-line args, it uses java's -D command-line options to buffer them out of an environment variable<br />
called _JAVA_OPTIONS. Strange but true. The ZIP comes with a bash script, which I'll reimplement if anyone actually wants me to. But in the meantime, it's easy to work <br />
out the arguments by inspection. For example, most likely usage:-<br />
<br />
<pre><br />
call setFastQC<br />
<br />
set _JAVA_OPTIONS='-Dfastqc.output_dir=%OUT%'<br />
call fastqc thing.fq<br />
</pre><br />
<br />
=== MAFFT ===<br />
<br />
For version 7.212 (Win64), write <code>call setMAFFT.bat</code> at the top of your run script.<br />
<br />
=== Applied-Maths Open Source ===<br />
<br />
Write <code>call setAppliedMaths.bat</code> at the top of your run script, to add these to the path:<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File(s)<br />
! Description<br />
! Version<br />
|- <br />
| velvetg_mt_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_mt_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| ray_x64.exe<br />
| Ray 64-bit (for MPI)<br />
| 1.6<br />
|-<br />
| ray_x86.exe<br />
| Ray 32-bit (for MPI)<br />
| 1.6<br />
|-<br />
| Mothur_x64.exe<br />
| Mothur 64-bit (for MPI)<br />
| 1.25.1<br />
|-<br />
| Mothur_x86.exe<br />
| Mothur 32-bit (for MPI)<br />
| 1.25.1<br />
|}<br />
<br />
=== Tensorflow ===<br />
<br />
Here is an example of getting Tensorflow (as a python package) available from an R script, using the cluster to install everything for you. It only works for Tensorflow 2.0 - older versions and newer versions seem to want combinations of DLLs that I can't satisfy. The following are I think minimal steps to get Tensorflow running from Python and R.<br />
<br />
* Create a folder for your python packages in your home directory (so the cluster can see it. I'm calling mine <code>Q:/python_37_repo</code> - and the cluster will call it \\fi--san03.dide.ic.ac.uk\homes\wrh1\python_37_repo - change wrh1 to your DIDE username of course.<br />
* Create a similar folder for R packages, which for me is <code>Q:/R4</code><br />
* And a third folder, Q:/Tensorflow to do this work in.<br />
* Download these DLLs and put them in the Q:/Tensorflow folder - at present they seem to be the ones Python/Tensorflow wants. Let's hope for the best... Download [https://mrcdata.dide.ic.ac.uk/resources/msvcp140.dll msvcp140.dll] and [https://mrcdata.dide.ic.ac.uk/resources/msvcp140_1.dll msvcp140_1.dll] into your <code>Q:/Tensorflow</code> folder.<br />
<br />
* Now, create a script <code>Q:/Tensorflow/prepare.bat</code>. We're going to launch this on the cluster, and get it to wire up the packages we want. Replace USER with your username...<br />
<br />
<pre><br />
net delete Q: /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set TMPDIR=%TEMP%<br />
set PYTHONPATH=\\qdrive.dide.ic.ac.uk\homes\USER\python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
pip install --no-cache-dir tensorflow==2.00 --target Q:\python_37_repo<br />
RScript Q:\Tensorflow\install_tensorflow.R<br />
</pre><br />
<br />
* Create also <code>Q:/Tensorflow/install_tensorflow.R</code> contains:<br />
<pre><br />
install.packages(c("reticulate", "tensorflow"),lib="Q:/R4",repos="https://cran.ma.imperial.ac.uk/")<br />
</pre><br />
<br />
* Now submit the prepare job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> in the big window will do it. That will take a while, but you should notice lots of new files in your Q:/python_37_repo folder. You'll only need to do this once (unless you delete your Python repo folder, or it randomly breaks and you start again...)<br />
<br />
* Next, create a file <code>Q:/Tensorflow/run.bat</code> containing the following the two "call" lines near the end are the ones that do some "work" in python or R, so this script basically a template for running your future jobs. Just change those two calls to set off the python or R code you want to run that uses Tensorflow.<br />
<br />
<pre><br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
net use Q: /delete /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set PYTHONPATH=Q:/python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call python Q:\tensorflow\test_tensorflow.py<br />
call RScript Q:\tensorflow\test_tensorflow.R<br />
</pre><br />
<br />
* And finally, you'll need <code>Q:\tensorflow\test_tensorflow.py</code> and <code>Q:\tensorflow\test_tensorflow.R</code> to exist. Here are minimal tests that everything is wired up. <code.test_tensorflow.py</code> first:-<br />
<br />
<pre><br />
import tensorflow as tf;<br />
print(tf.reduce_sum(tf.random.normal([1000, 1000])));<br />
</pre><br />
<br />
and <code>Q:\Tensorflow\test_tensorflow.R</code> contains:<br />
<br />
<pre><br />
library(reticulate)<br />
reticulate::py_config()<br />
library(tensorflow)<br />
tf$constant('Hello, TensorFlow!')<br />
tf$Variable(tf$zeros(shape(1L)))<br />
</pre><br />
<br />
* Now submit the run job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> in the big window will do it - and perhaps paste <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\output.txt</code> in the stdout box to collect the input and see how it gets on. You should see sensible output from both tests.<br />
<br />
== Launching a job ==<br />
<br />
<br />
== Submitting many jobs ==<br />
<br />
Suppose you write an exe that you might run with… Mysim.exe init.txt 1 2 3, and you want to run it many times with a range of parameters. One way of many, is to write a launch.bat file that will run “job submit” separately, for example (thanks Tini/James!):-<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set initFile=\\networkshare\job\init.txt<br />
set exeFile=\\networkshare\job\mysim.exe<br />
<br />
%SubDetails% %exeFile% %initfile% 1 2 3<br />
%SubDetails% %exeFile% %initfile% 4 5 6<br />
%SubDetails% %exeFile% %initfile% 7 8 9<br />
</pre><br />
<br />
Suppose the job you want to run is an R script? To specify arguments to an R script, you have to add ′--args a=1 b=2′ - so… you might make launch.bat like this:-<br />
<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set rbatFile=\\networkshare\R-scripts\run.bat<br />
<br />
%SubDetails% %rbatFile% 1 2<br />
%SubDetails% %rbatFile% 3 4<br />
</pre><br />
And make the significant line of your run.bat:-<br />
<code><br />
Rcmd BATCH ′--args a=%1 b=%2′ U:\R-scripts\test.R<br />
</code><br />
%1 and %2 will map to the first and second thing after the batch file. You can go all the way up to %9.<br />
<br />
IMPORTANT NOTE<br />
<br />
Make sure you get the apostrophe character right in the above example – NEVER copy and paste from a word document into a script. It will go hideously wrong. Type the apostrophes (and dashes for that matter) in a good text editor – you want just the standard old-fashioned characters.<br />
<br />
== Requesting resources ==<br />
<br />
The following modifiers next to the “/scheduler:” part of the job submit line (before your app.exe 1 2 3 part), will request things you might want…<br />
<br />
/numcores:8 - number of cores you want<br />
<br />
/numcores:8-12 - minimum and maximum cores appropriate for your job<br />
<br />
/memorypernode:1024 - amount of mem in MegaBytes needed.<br />
<br />
/workdir:\\networkshare\work - set working directory<br />
<br />
/stdout:\\networkshare\out.txt - divert stdout to a file<br />
<br />
/stderr: or /stdin: - similar for stderr and stdin<br />
<br />
== Troubleshooting / Miscellany / Q & A ==<br />
<br />
* My job doesn’t work.<br />
** Run the HPC Job Manager application. Find your job id, possible under “My Jobs, Failed”. Double click on it, then on “View All Tasks”. Perhaps something in the output section will help.<br />
<br />
* Check that the path to your job is visible everywhere.<br />
** Avoid spaces in your paths - “job submit” doesn’t like them very much. If you must have them, they’ll be ok in the “run.bat” batch file that the cluster will run – in which case, put the path in standard double-quotes ("). But avoid them in your “launch.bat” file – you may have to relocate your run.bat to a simple non-space-containing directory.<br />
** Rather than putting the full application and parameters on the job submit line, you might want to write a batch-file to do all that, and submit the batch file to the cluster. (See section 6 about R for example). But make sure the batch file is somewhere visible to the cluster.<br />
<br />
* My job seems to work, but reports as having failed.<br />
** The success/failure depends on the error code returned. If you’re running C code, end with “return 0;” for success.<br />
<br />
* job submit ..blah blah.. app.exe >out.txt doesn’t work!<br />
** The contents of out.txt will be the result of “job submit”, not the result of “app.exe”. You meant to say this job submit ..blah blah.. /stdout:out.txt app.exe correcting out.txt and app.exe to network paths of course.<br />
<br />
* My C/C++ Executable works perfectly on my desktop, but doesn't work on the cluster. What's wrong with the cluster?<br />
** Your EXE file probably relies on some DLL files that the cluster doesn't have. If you use Visual Studio, check that in Code Generation, you have "Multi-Threaded (/MT)" as your target - with no mention of DLLs. If your code uses OpenMP, check you've copied the right OpenMP DLL(s) (go back to the C section above). As a last resort, give your EXE to someone who doesn't have Visual Studio installed, and get them to run it, and see what DLL files it seems to ask for.<br />
<br />
== Contributing Authors ==<br />
<br />
Wes Hinsley, James Truscott, Tini Garske, Jeff Eaton, Hannah Clapham</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Fi--didemrchnb&diff=14289Fi--didemrchnb2025-03-17T15:30:34Z<p>Admin: </p>
<hr />
<div>The fi--didemrchnb cluster is the larger cluster, suited to very high performance, multi-core or high memory applications.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Head Node<br />
! Model<br />
! Processor (Intel Xeon)<br />
! Cores<br />
! Mem (Gb)<br />
! Mem Type<br />
! Infiniband<br />
|-<br />
| FI--DIDEMRCHNB<br />
| Dell R320<br />
| E5-2430 2.20GHz<br />
| 6 (HT->12)<br />
| 16<br />
| 1333<br />
| MT_1090110018<br />
|}<br />
<br />
<br />
{| class="wikitable"<br />
|-<br />
! Node<br />
! Model<br />
! Processor (Intel Xeon)<br />
! Cores<br />
! Mem (Gb)<br />
! Mem Type<br />
! Mem slots<br />
! Mem part<br />
! Infiniband <br />
|-<br />
| FI--DIDEMRC09<br />
| Dell C1100<br />
| 2 x X5670 2.93GHz<br />
| 12<br />
| 144<br />
| 1333<br />
| 18/18<br />
| HMT31GR7BFR4C-H9<br />
| MT_0BB0120003<br />
|-<br />
| FI--DIDEMRC10<br />
| Dell C1100<br />
| 2 x X5670 2.93GHz<br />
| 12<br />
| 144<br />
| 1333<br />
| 18/18<br />
| HMT31GR7BFR4C-H9<br />
| MT_0BB0120003<br />
|-<br />
| FI--DIDEMRC11<br />
| Dell C1100<br />
| 2 x X5670 2.93GHz<br />
| 12<br />
| 144<br />
| 1333<br />
| 18/18<br />
| HMT31GR7BFR4C-H9<br />
| MT_0BB0120003<br />
|-<br />
| FI--DIDEMRC12<br />
| Dell C1100<br />
| 2 x X5670 2.93GHz<br />
| 12<br />
| 144<br />
| 1333<br />
| 18/18<br />
| HMT31GR7BFR4C-H9<br />
| MT_0BB0120003<br />
|-<br />
| FI--DIDEMRC17<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC18<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC19<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC20<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC21<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC22<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC23<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC24<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC25<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC26<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC27<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC28<br />
| Dell C6100<br />
| 2 x X5675 3.07GHz<br />
| 12<br />
| 96<br />
| 1333<br />
| 12/12<br />
| M393B1K70CHD-YH9<br />
| DEL09A0000009<br />
|-<br />
| FI--DIDEMRC38<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC39<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC40<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC41<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC42<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC43<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC44<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_0BB0120003<br />
|-<br />
| FI--DIDEMRC45<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC46<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC47<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC48<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_0BB0120003<br />
|-<br />
| FI--DIDEMRC49<br />
| Dell R620<br />
| 2 x E5-2690 0 2.90GHz<br />
| 16<br />
| 128<br />
| 1600<br />
| 8/24<br />
| HMT42GR7MFR4C-PB<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC50<br />
| Dell R720<br />
| 2 x E5-2690 0 2.90GHz+PHI<br />
| 16<br />
| 128<br />
| 1333<br />
| 8/24<br />
| 36KSF2G72PZ-1G4E1 / 002C04B3002C<br />
| MT_1090110018<br />
|-<br />
| FI--DIDEMRC67<br />
| Dell C8220X<br />
| 2 x E5-2695 v2 2.40GHz + PHI <br />
| 24<br />
| 256<br />
| 1866<br />
| 8/16<br />
| M386B4G70DM0-CMA3<br />
| DEL0A30000019<br />
|-<br />
| FI--DIDEMRC68<br />
| Dell C8220X<br />
| 2 x E5-2695 v2 2.40GHz + PHI <br />
| 24<br />
| 256<br />
| 1866<br />
| 8/16<br />
| M386B4G70DM0-CMA3<br />
| DEL0A30000019<br />
|-<br />
| FI--DIDEMRC69<br />
| Dell C8220X<br />
| 2 x E5-2695 v2 2.40GHz + PHI <br />
| 24<br />
| 256<br />
| 1866<br />
| 8/16<br />
| M386B4G70DM0-CMA3<br />
| DEL0A30000019<br />
|-<br />
| FI--DIDEMRC70<br />
| Dell C8220X<br />
| 2 x E5-2695 v2 2.40GHz + PHI <br />
| 24<br />
| 256<br />
| 1866<br />
| 8/16<br />
| M386B4G70DM0-CMA3<br />
| DEL0A30000019<br />
|-<br />
| FI--DIDEMRC71<br />
| Dell C8220X<br />
| 2 x E5-2695 v2 2.40GHz + PHI <br />
| 24<br />
| 256<br />
| 1866<br />
| 8/16<br />
| M386B4G70DM0-CMA3<br />
| DEL0A30000019<br />
|-<br />
| FI--DIDEMRC72<br />
| Dell C8220X<br />
| 2 x E5-2695 v2 2.40GHz + PHI <br />
| 24<br />
| 256<br />
| 1866<br />
| 8/16<br />
| M386B4G70DM0-CMA3<br />
| DEL0A30000019<br />
|-<br />
| FI--DIDEMRC73<br />
| Dell C8220X<br />
| 2 x E5-2695 v2 2.40GHz + PHI <br />
| 24<br />
| 256<br />
| 1866<br />
| 8/16<br />
| M386B4G70DM0-CMA3<br />
| DEL0A30000019<br />
|-<br />
| FI--DIDEMRC74<br />
| Dell C8220X<br />
| 2 x E5-2695 v2 2.40GHz + PHI <br />
| 24<br />
| 256<br />
| 1866<br />
| 8/16<br />
| M386B4G70DM0-CMA3<br />
| DEL0A30000019<br />
|}</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Cluster_Documentation&diff=14288Cluster Documentation2025-03-06T22:00:19Z<p>Admin: /* Python */</p>
<hr />
<div>* We have two high performance clusters for general use, both running Microsoft HPC 2012 R2, Update 3. The smaller, older cluster is [[fi--dideclusthn]] and the larger is [[fi--didemrchnb]]. <br />
<br />
* Many users in the department write code and packages in R. We have in-house packages that can allow an almost-interactive experience of running jobs on our high performance clusters, so if that sounds like what you'd like, you might as well stop reading here, and go straight to [https://github.com/mrc-ide/didehpc The DIDEHPC package] - or if you need to join the mrc-ide github org first, go to [https://github.com/mrc-ide/welcome The welcome page].<br />
<br />
If you're not using R, or you are interested in other ways, then continue...<br />
<br />
== Windows Users ==<br />
<br />
The cluster nodes are Windows 2012-R2 based. This is good news if you're a Windows user, and especially easy for PCs within the departmental DIDE domain. You have the option of using the MS client for launching jobs (see below), or you can use try the new [[HPC Web Portal]]. If you're preparing to use the cluster, and have doubts about whether it can run what you want - and whether it will be straightforward to develop and run on it, best talk to me and/or the IT guys first!<br />
<br />
== Linux and MAC Users ==<br />
<br />
There are two problems, but both can be at least partially overcome. The first is that there is no client for linux. The second is that it can only run windows binaries.<br />
<br />
=== Launching jobs on Linux/Mac ===<br />
<br />
* The best way might be the [[HPC Web Portal]], which lets you submit jobs through a webpage. Give it a try, and feedback to me any issues! <br />
<br />
* Alternatively, you can either (1) go down the VM route, installing a windows virtual machine, eg VMWare or Parallels, and follow the instructions below. (2) Remote Desktop to your friend's Windows machine. These tend not to be very convenient, but some users have submitted thousands of jobs this way.<br />
<br />
=== What Linux/Mac jobs can I run? ===<br />
<br />
* Many jobs may be platform indepedent. Where compiled code is involved though, you'll have some extra work to make sure the binary parts can be run in windows. For R, [https://github.com/mrc-ide/didehpc the tools] help you sort that out. For others, you may need to run a job on the cluster to do your builds. Also, for many linux applications, there is a windows port. Mileage can vary, but it's worth a try searching for that before you give up hope. Some of these might be installed on the cluster already - see below. Or some you may be able to download and call them from your run script.<br />
<br />
* For C code, again, we can run scripts on the cluster to build the binaries for you, if your code can be compiled with (say) the MinGW port of gcc. Cygwin can also be used (although seems less common these days) with accompanying dlls. Get in touch if you need further ideas; most things can be solved. <br />
<br />
* Matlab is a little more tricky - we have a range of colossal matlab runtimes on the cluster, but you'd need to compile a windows binary to use them, since the matlab platforms are distinct in every way. For this, probably find a friend, or get in touch, and we'll work out how you can compile for it.<br />
<br />
* There is also the option of running a Windows VM, which might solve all of the above problems.<br />
<br />
=== But I really want a unix cluster. ===<br />
<br />
* We have run linux clusters in the past ([[fi--didelxhn]]), but there was very low demand, whereas the other clusters are consistently well used. Accessing DIDE network shares from linux was a little harder in some aspects, but we did not implement the more traditional linux approach of having dedicated separate cluster storage that inputs and outputs would have to be staged on to, in order for the cluster to see them. Generally, the flexibility of the other clusters makes such things quite straightforward, and may even feel a pleasure if your background is with the less domain-integrated style of cluster architecture.<br />
<br />
* It has also become far easier to write software that builds on multiple platforms with good practice. Tools that only run on one platform are rare, and in some circumstances have raised questions about the quality of those packages. Alternatives are often available.<br />
<br />
* Where compilation is the issue - for example if you want to run binaries on the cluster, but your compilation machine runs linux, or is a mac, (Matlab via MCC, or C++ code for example), then get in touch, and there are a few ways we can think of solving that, either by getting someone else, or the cluster itself, to build your code.<br />
<br />
* There are also resources in [https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/computing/high-throughput-computing/ ICT) who run a linux cluster, with the file-staging approach, and also (being shared between a much larger pool of users) have rather stricter rules on job durations and resources than we provide locally in DIDE.<br />
<br />
== Getting Started ==<br />
<br />
=== Getting access to the cluster ===<br />
<br />
Send a mail to Wes (w.hinsley@imperial.ac.uk) requesting access to the cluster. He will add you to the necessary groups, and work out what cluster you should be running on. Unless you are told otherwise, this will be fi--dideclusthn. But if you have been told otherwise, then whenever you see fi--dideclusthn, replace it with the cluster name you’ve been given, either mentally or (less effectively) on your screen with tip-ex.<br />
<br />
=== Windows: Install the HPC client software ===<br />
<br />
* Run \\fi--dideclusthn.dide.ic.ac.uk\REMINST\setup.exe. <br />
* Confirm '''Yes, I really want to run this'''.<br />
* There's an introductory screen, and you click '''Next'''<br />
* You have to tick the '''Accept''', then click '''Next'''<br />
* If it gives you any choice, select '''Install only the client utilities'''<br />
* '''Next''', '''Next''', '''Next'''<br />
* Select '''Use Microsoft Update''' and then '''Next'''<br />
* And then '''Install'''<br />
* And possibly '''Install''' again – if it had to install any pre-requisites first.<br />
* And '''Finish''', and the client installation is done.<br />
* To check everything is alright so far, open a '''NEW command prompt window'''.<br />
* (Because the installer adds things to the path, which only take effect on '''NEW comamnd windows'''! :-)<br />
* (Find it under Accessories, or you can use Start, Run, '''cmd''').<br />
* Type <code>job list /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
And it will list the jobs you have running, which will be none so far!<br />
<br />
<pre><br />
Id Owner Name State Priority Resource Request<br />
---------- ------------- ----------------- ------------ ----------- -------- -------<br />
<br />
0 matching jobs for DIDE\user<br />
</pre><br />
<br />
=== Windows: Using a non-domain machine ===<br />
<br />
If you're using a machine that isn't actually logged into DIDE, then the client software will have a problem working out who you are. In this case, there are two things you need to do.<br />
* Check you've really got a DIDE username. If you don't know what it is, talk to your lovely IT team.<br />
* Connect to the DIDE VPN using ZScaler from https://uafiles.cc.ic.ac.uk/, and login to that with your IC account - but you'll still need your DIDE account to access DIDE servers after the ZScaler connection is up and running.<br />
* Now we'll open a command window using "runas" - which lets you choose which identity the system thinks you are within that command window:-<br />
<pre><br />
runas /netonly /user:DIDE\user cmd<br />
</pre><br />
* (Change user to your own login obviously). It will ask for your password, then open a new command window. In that window, you'll be able to do cluster-related things as your DIDE username.<br />
<br />
=== Windows: Launching and cancelling jobs ===<br />
==== Windows: Command Line ====<br />
<br />
Suppose you have a file called "run.bat" in your home directory, which does what you want to run on the cluster. Let's say it's a single-core, very simple job. We’ll discuss what should be inside "run.bat" later. To submit your job on the cluster, at the command prompt, type this (all on one line) - or put it in a file called "launch.bat", and run it:-<br />
<br />
<code>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1 \\qdrive.dide.ic.ac.uk\homes\user\run.bat</code><br />
<br />
If it's the first time you’ve run a job - or if you've recently changed your password, then it might ask you for your DIDE password and offer to remember it. Otherwise, it will just tell you the ID number of your job.<br />
<br />
<pre>Enter the password for ‘DIDE\user’ to connect to ‘FI—DIDECLUSTHN’:<br />
Remember this password? (Y/N)<br />
job has been submitted. ID: 123<br />
</pre><br />
<br />
If you want to remove the job, then:- <code>job cancel 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
Or view its details with <code>job view 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
==== Windows: Job Manager GUI ====<br />
<br />
Alternatively to the command-line, you can use the job management software, rather than the command-line. The advantage is that it’s a GUI. The disadvantage is, as in all GUIs, you may not feel totally sure you know what it’s up to – where most of the time the things you want to do might not be very complex, as above.<br />
<br />
The job management software will be on your start menu, as above. All the features are under the “Actions” menu, and hopefully it will be self explanatory. Read the details below about launching, and you'll find the interface bits that do it in the Job Manager. However, you may find over time, especially if you run lots of jobs, that learning to do it the scripted way with the command-line can be quicker.<br />
<br />
<br />
== Information for running any job ==<br />
<br />
=== Visibility ===<br />
<br />
Any executable/batch file that the cluster will run must be somewhere on the network that the cluster has access to, when it logs in as you. This amounts to any network accessible drive that you have access to when you login – including network shares, such as the temp drive, your home directory, and any specific network space set up for your project.<br />
<br />
After a job is launched, all paths are relative to the '''/workdir:''' specified on the job submit command-line. Windows does not usually allow a network path to be the current directory, and MS-HPC somehow gets around this. But if in doubt, use fully-qualified paths to specify where files should be read from, or written to. You may want to write code that takes the paths from a parameter file, or as a command-line parameter, to give as much flexibility as possible. <br />
<br />
Your home directory, and the temp drive are on departmental servers that don't have an especially fast connection to the cluster. So avoid having thousands of jobs all reading/writing to those places at once. Additionally, the temp drive is shared between all users, and your home drive has a limited quota. It is also backed up every day, so avoid using it as scratch storage for results you don’t actually want to keep. Generally, the shares you'll want to use from the clusters will be ones containing the word "NAS", which the big cluster has fast access to.<br />
<br />
=== Interactivity ===<br />
<br />
The job must run in an entirely scripted, unattended way. It must not require any key presses or mouse clicks, or other live interactivity while it runs. So jobs generally will read some sort of input (from a file, or from the way you run the job), do some processing, and write the results somewhere for you - all without intervention.<br />
<br />
== Launching jobs ==<br />
<br />
Jobs can be launched either through the Job Manager interface, or through the command line tools, which offer greater flexibility. We'll describe the command line method here; if you want to use the GUI, then it'll be a case of finding the matching boxes... Below are the specifics for our clusters. For more information, simply type <code>job</code> on the commandline, or <code>job submit</code> for the list of submission-related commands.<br />
<br />
=== FI--DIDECLUSTHN vs FI--DIDEMRCHNB ===<br />
<br />
Job submissions, as shown below, must specify a "job template", which sets up a load of default things to make the job work. On fi--dideclusthn, the job templates are called 4Core, 8Core and GeneralNodes, which will respectively force jobs to run on the 4-core nodes, the 8-core nodes, or on any machine available.<br />
<br />
On fi--didemrchnb, you can set the job template to be... 8Core, 12Core, 16Core, 12and16Core, and GeneralNodes - which hopefully are fairly self-explanatory. There are a couple of other job templates, (24Core and Phi), but those are a bit special purpose for now, so don't use them!<br />
<br />
=== Job Submission ===<br />
<br />
Job submissions from the command line can take the following form (all on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /stdout:\\path\to\stdout.txt<br />
/stderr:\\path\to\stderr.txt /numcores:1-1 /jobtemplate:4Core \\path\to\run.bat</pre><br />
<br />
=== The /singlenode argument ===<br />
<br />
In MS HPC (2012), Microsoft finally added a tag to allow you to say that the 'n' cores you requested must be on the same computer. Therefore, if you know precisely how many cores you want, then use the following (on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:true <br />
/jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
However, there is one bug with this, for the specific case where you request a whole node, regardless of how many cores it has. In this case, oddly, you have to disable single node:-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:false <br />
/numnodes:1 /jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
== Languages and libraries supported ==<br />
<br />
A number of languages, and different versions of languages are available on the cluster. The sections below refer to your "run.bat" file - a batch file that you will create which will get run by a cluster node when a suitable one is available. The commands described below are to be put in this "run.bat" file, and they add various directories to the path, so that the software you want will be added to the path. <br />
<br />
=== C/C++ === <br />
<br />
A number of Microsoft C++ and Intel C++ runtimes are installed, but it's usually better to try and avoid using them, and make your executable as stand-alone as possible. If it requires any external libraries that you've had to download, then put the .dll file in the same directory as the .exe file. If you use Microsoft Visual Studio, in Project Preferences, C/C++, Code Generation, make sure the Runtime Library is Multi-threaded (/MT) – the ones with DLL files won’t work. Even so, on recent versions of the Intel and Microsoft C compilers, "static linking" doesn’t really mean static when it comes to OpenMP, and you’ll have to copy some DLLs and put them next to your EXE file. See the OpenMP section below.<br />
<br />
The cluster nodes are all 64-bit, but they will run 32-bit compiled code. Just make sure you provide the right DLLs!<br />
<br />
Here is a document about [[Using the Xeon Phi]]<br />
<br />
Here is a document about [[Using MPI]]<br />
<br />
And if you want to use libraries such as GSL, then try [[C/C++ Libraries for Windows]]<br />
<br />
=== Java ===<br />
<br />
Java doesn't strictly need installing; you can setup any Java Runtime Environment you like, from Oracle, or anywhere. If you want to do this, then, (if you were to use the Server JRE from Oracle), something similar to the following will add java.exe and jvm.dll to the path, and set the JAVA_HOME environment variable, which most Java-using software will appreciate.<br />
<br />
set JAVA_HOME=\\my\path\to\jre1.8.0<br />
set path=%JAVA_HOME%\bin;%JAVA_HOME%\bin\server;%path%<br />
<br />
For convenience, if you're happy with Azul's OpenJDK Java 1.8.0 update 308 (September 2021), then <code>call setJava64</code> in your run script will set this up for you. Incidentally the above will also add what's needed for using the rJava package in R - in which case you'll perhaps also want something like <code>call setR64_4_1_0</code> in your run script.<br />
<br />
=== Perl ===<br />
<br />
Strawberry Perl 64-bit portable, v5.30.1.1. Put <code>call setPerl</code> at the top of your script.<br />
<br />
=== R ===<br />
<br />
'''New tools are coming''': [http://dide-tools.github.io/didehpc/vignettes/quickstart.html see here]. The instructions below are still valid if you want to do this manually.<br />
<br />
<br />
You can R jobs on all the clusters. This is the latest wisdom on how to do so – thanks to James, Jeff, Hannah and others. <br />
<br />
First, if you are wanting to use packages, then set up a package repository on your home directory by running this in R.<br />
<br />
<code>install.packages("<package>",lib="Q:/R")</code><br />
<br />
Now, write your run script. Suppose you have an R script in your home directory: Q:\R-scripts\test.R. And suppose you’ve set up your repository as above. Your run.bat should then be:-<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\test.R<br />
</pre><br />
Various packages require various R versions, and the cluster supports a few versions. To choose which one, change the first line of the script above to one of these – 32-bit or 64-bit versions of R releases. Purely for amusement, R's codenames are shown here too. <br />
<pre><br />
call setr32_2_13_1.bat call setr64_2_13_1.bat (anyone know the codename?)<br />
call setr32_2_14_2.bat call setr64_2_14_2.bat (Gift-getting season)<br />
call setr32_2_15_2.bat call setr64_2_15_2.bat (Trick or treat)<br />
call setr32_3_0_1.bat call setr64_3_0_1.bat (Good Sport)<br />
call setr32_3_0_2.bat call setr64_3_0_2.bat (Frisbee Sailing)<br />
call setr32_3_1_0.bat call setr64_3_1_0.bat (Spring Dance)<br />
call setr32_3_1_2.bat call setr64_3_1_2.bat (Pumpkin Helmet)<br />
call setr32_3_2_2.bat call setr64_3_2_2.bat (Fire Safety)<br />
call setr32_3_2_3.bat call setr64_3_2_3.bat (Wooden Christmas Tree)<br />
call setr32_3_2_4.bat call setr64_3_2_4.bat (Very Secure Dishes - Revised Version - R later renamed this to 3.2.5)<br />
call setr32_3_3_1.bat call setr64_3_3_1.bat (Bug in your hair)<br />
call setr32_3_3_2.bat call setr64_3_3_2.bat (Sincere Pumpkin Patch)<br />
call setr32_3_4_0.bat call setr64_3_4_0.bat (You Stupid Darkness)<br />
call setr32_3_4_2.bat call setr64_3_4_2.bat (Short Summer)<br />
call setr32_3_4_4.bat call setr64_3_4_4.bat (Someone to lean on)<br />
call setr32_3_5_0.bat call setr64_3_5_0.bat (Joy in Playing)<br />
call setr32_3_5_1.bat call setr64_3_5_1.bat (Feather Spray)<br />
call setr32_3_5_3.bat call setr64_3_5_3.bat (Great Truth)<br />
call setr32_3_6_0.bat call setr64_3_6_0.bat (Planting of a Tree)<br />
call setr32_3_6_1.bat call setr64_3_6_1.bat (Action of the Toes)<br />
call setr32_3_6_3.bat call setr64_3_6_3.bat (Holding the Windsock)<br />
call setr32_4_0_2.bat call setr64_4_0_2.bat (Taking Off Again)<br />
call setr32_4_0_3.bat call setr64_4_0_3.bat (Bunny-Wunnies Freak Out)<br />
call setr32_4_0_5.bat call setr64_4_0_5.bat (Shake and Throw)<br />
</pre><br />
<br />
The following versions are *not* available on the cluster at the moment - but here are their names anyway, as it seemed a shame to miss them out. Let me know if you desperately need one of these.<br />
<br />
<pre><br />
2.14.0 Great Pumpkin <br />
2.14.1 December Snowflakes<br />
2.15.0 Easter Beagle<br />
2.15.1 Roasted Marshmallow<br />
2.15.3 Security Blanket<br />
3.0.0 Masked Marvel<br />
3.0.3 Warm Puppy<br />
3.1.1 Sock it to me<br />
3.1.3 Smooth Sidewalk<br />
3.2.0 Full of Ingredients<br />
3.2.1 World famous astronaut<br />
3.2.4 Very Secure Dishes (the non-revised version)<br />
3.3.0 Supposedly Educational<br />
3.3.3 Another Canoe<br />
3.4.1 Single Candle<br />
3.4.3 Kite-Eating Tree<br />
3.5.2 Eggshell Igloo<br />
3.6.2 Dark and Stormy Night<br />
4.0.0 Arbor Day<br />
4.0.1 See Things Now<br />
4.0.4 Lost Library Book<br />
</pre><br />
<br />
It also seems that different R versions put their packages in different structures – sometimes adding "win-library" into there for fun. Basically, R_LIBS and R_LIBS_USER should be paths to a folder that contains a list of other folders, one for each package you've installed.<br />
<br />
IMPORTANT: R_LIBS and R_LIBS_USER paths must NOT contain quotes, nor spaces. If the path to your library contains spaces, you need to use old-fashioned 8-character names. If in your command window, you type <code>dir /x</code>, you’ll see the names – Program Files tends to become PROGRA~1 for instance. <br />
<br />
==== Passing parameters to R scripts ====<br />
Passing parameters to R scripts means you can have fewer versions of your R and bat files and easily run whole sets of jobs. You can get parameters into R using Rscript (but not Rcmd BATCH, I think) as follows. In the run.bat example above, the Rscript statement becomes:<br />
<pre><br />
Rscript Q:\R-scripts\test.R arg1 arg2 arg3<br />
</pre><br />
Within the R code, the arguments can be recovered using <br />
<pre><br />
args <- commandArgs(trailingOnly = TRUE)<br />
<br />
outFileName <- args[1] ## name of output file.<br />
dataFileName <- args[2] ## name of local data file. <br />
currentR0 <- as.numeric(args[3]) ## convert this explicitly to number. <br />
</pre><br />
If you want your arguments to have a particular type, best to explicitly convert (see R0 above). Better still, you can pass parameters directly into the batch file that runs the R script. Command line arguments can be referenced within the batch file using %1, %2, etc. For example, if you have a batch file, runArgs.bat:<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\%1.R %2 %3 %4<br />
</pre><br />
then<br />
<pre><br />
runArgs.bat myRScript arg1 arg2 arg3<br />
</pre><br />
will run the R script myRScript.R and pass it the parameters arg1, arg2 and arg3. The batch file runArgs.bat is now almost a '''generic R script runner'''.<br />
<br />
=== OpenMP ===<br />
<br />
This has been tested with C/C++ - but the same should apply to other languages such as fortran, that achieve multi-threading with a DLL file.<br />
<br />
==== Microsoft's C/C++ Compiler ====<br />
You will need to copy vcomp100.dll (for Visual Studio 2010 – it maybe vcomp90.dll for 2008), into the same directory as your executable. You can usually find the dll file in a directory similar to:- C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist\x64\Microsoft.VC100.OpenMP. Extrapolate for future versions! Also make sure that you've enabled OpenMP in the Properties (C++/Language).<br />
<br />
==== GCC or MinGW ====<br />
Usually, this applies to Eclipse use. In project properties, the C++ command “g++ -fopenmp”, the C command “gcc –fopenmp”, and in the Linker options, under Miscellaneous, Linker flags, put “-fopenmp” too. Copy the OpenMP DLLs from MinGW into the same directory as your final executable. You may find the dlls are in C:\MinGW\Bin\libgomp-1.dll and in the same place, libpthread-2.dll, libstdc++-6.dll and libgcc_s_dw2-1.dll.<br />
<br />
==== Intel C++ Compiler ====<br />
Copy the libiomp5md.dll file from somewhere like C:\Program Files (x86)\Intel\ComposerXE-2011\redist\intel64\compiler\libiomp5md.dll to the same place as your executable. And in Visual Studio, make sure you've enabled OpenMP in the project properties.<br />
<br />
==== How many threads? ====<br />
The OpenMP function omp_max_threads() returns the number of ''physical'' cores on a machine, not the number of cores ''allocated'' to your job. To determine how many cores the scheduler actually allocated to you, use the following code to dig up the environment variable CCP_NUMCPUS, which will be set by the cluster management software:- <br />
<pre><br />
int set_NCores() {<br />
char * val;<br />
char * endchar;<br />
val = getenv("CCP_NUMCPUS");<br />
int cores = omp_get_max_threads();<br />
if (val!=NULL) cores = strtol (val,&endchar,10);<br />
omp_set_num_threads(cores);<br />
return cores;<br />
}<br />
</pre><br />
<br />
=== WinBugs (or OpenBugs) ===<br />
<br />
They are similar, but OpenBugs 3.2.2 is the one we've gone for. Something like this as your run.bat script will work:-<br />
<pre><br />
call setOpenBugs<br />
openbugs \\path\to\script.txt /HEADLESS<br />
</pre><br />
<br />
=== MatLab ===<br />
<br />
There are various ways of producing a non-interactive executable from Matlab. Perhaps the simplest (not necessarily the best performance) way is to use “mcc.exe” – supplied with most full versions of Matlab, including the Imperial site licence version that you've probably got.<br />
<br />
==== Use mcc.exe to compile your code ====<br />
<br />
Use windows explorer to navigate to the folder where the “.m” files are for the project you want to compile. Now use a good text editor to create a file called “compile.bat” in that folder. It should contain something similar to the following:-<br />
<br />
<code>mcc -m file1.m file2.m file3.m -o myexe</code><br />
<br />
Don’t copy/paste the text from this document by the way – Word has a different idea of what a dash is to most other software, and will probably replace the two dashes with funny characters. so you have to list every single .m file that your project needs, after the’-m’. If you save this file, then double-click on it, then it will think for some while, and produce “myexe.exe” in this example. Copy your .exe file into a network accessible place as usual.<br />
<br />
==== Launch on the cluster. ====<br />
<br />
The launch.bat file will be exactly the same as before – see page 2. The run script will then start with a line that tells the cluster which version of Matlab you used to compile the cluster. Below is the table of different versions. The runtimes are huge and cumbersome to install on the cluster, so as a result I haven’t installed every single one. If you need one that’s not listed, get in touch.<br />
<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! Matlab Version<br />
! First line of run script<br />
|- <br />
| R2009b<br />
| call useMatLab79<br />
|- <br />
| R2010a<br />
| call useMatLab713<br />
|- <br />
| R2011a<br />
| call useMatLab715<br />
|- <br />
| R2011b<br />
| call useMatLab716<br />
|- <br />
| R2012a (64-bit)<br />
| call useMatLab717_64<br />
|- <br />
| R2012b (64-bit)<br />
| call useMatLab80_64<br />
|- <br />
| R2013a (64-bit)<br />
| call useMatLab81_64<br />
|- <br />
| R2014a (64-bit)<br />
| call useMatLab83_64<br />
|- <br />
| R2014b (64-bit)<br />
| call useMatLab84_64<br />
|- <br />
| R2015a (64-bit)<br />
| call useMatLab85_64<br />
|- <br />
| R2015b (64-bit)<br />
| call useMatLab90_64<br />
|- <br />
| R2016b (64-bit)<br />
| call useMatLab91_64<br />
|- <br />
| R2017a (64-bit)<br />
| call useMatLab92_64<br />
|- <br />
| R2019a (64-bit)<br />
| call useMatLab96_64<br />
|- <br />
| R2021a (64-bit)<br />
| call useMatLab910_64<br />
|- <br />
| R2022a (64-bit)<br />
| call useMatLab912_64<br />
|}<br />
<br />
=== Python ===<br />
<br />
* The most recent version of each major release is on the cluster. Different python versions within the same release (e.g., 3.5.1 and 3.5.2) do not behave nicely together in automated installation, so if you have a desperate need for a previous minor version, get in touch. While 2.7.18 is available, we '''strongly''' recommend migrating away from Python 2.7, as it has been officially discontinued on 1st January 2020. Experiences may vary, and future success is not guaranteed.<br />
<br />
<pre><br />
2.7.18 64-bit call set_python_27_64.bat<br />
3.5.10 64-bit call set_python_35_64.bat<br />
3.6.15 64-bit call set python_36_64.bat<br />
3.7.17 64-bit call set_python_37_64.bat<br />
3.8.20 64-bit call set python_38_64.bat<br />
3.9.21 64-bit call set_python_39_64.bat<br />
3.10.16 64-bit call set python_310_64.bat<br />
3.11.11 64-bit call set_python_311_64.bat<br />
3.12.8 64-bit call set_python_312_64.bat<br />
3.13.2 64-bit call set_python_313_64.bat<br />
<br />
</pre><br />
<br />
* You'll most likely want to use some packages, so you'll need to have a network-accessible location to put your packages in. If you're running a version of Windows with Python that matches one installed on the cluster, then you'll be able to share that package repository between the two. But if you're running a different version, or a different operating system, or a different Python version, then you'll need to keep your local package repository separate from the one the cluster uses.<br />
* Here comes the basic script to run some python. Suppose I decide to put my Python packages for the cluster in '''\\qdrive.dide.ic.ac.uk\homes\username\PythonCluster37_64''' (which is Q:\PythonCluster37_64 on a DIDE Windows PC). To run some python, something like the following:-<br />
<br />
call set_python_37_64<br />
set PYTHONPATH=\\qdrive\homes\username\PythonCluster37_64<br />
python \\qdrive\homes\username\Python\MyScript.py<br />
<br />
or if you want to do this in a programmatic way...<br />
<br />
import sys<br />
sys.path = ['\\\\qdrive\\homes\\username\\PythonCluster37_64'] + sys.path<br />
<br />
==== Packages ====<br />
<br />
* If you want to install a package like numpy, that's readily available through pip, then you can launch a batch file like this:-<br />
call set_python_36_64<br />
pip install numpy --target \\qdrive\homes\username\PythonCluster36_64<br />
* or alternatively, you could write a python script that looks like this, and launch it as normal:-<br />
import pip<br />
pip.main(['install','numpy','--target','\\\\qdrive\\homes\\username\\PythonCluster36_64'])<br />
<br />
* And if you're running the same Python version on a Windows machine, then you can do these two locally, without submitting a cluster job to do them for you.<br />
<br />
* And next time you submit a python job on the cluster, python will look in the place set by PYTHONPATH, and will find the packages you've just installed when you import them.<br />
<br />
=== Beast and Beagle ===<br />
<br />
We've got a couple of versions available. For BEAST 1.75 with BEAGLE 1.0: (the x y z represents all the command-line arguments you'll want to pass to beast)<br />
<pre><br />
call setbeast<br />
call beast-beagle x y z<br />
</pre><br />
<br />
If you want BEAST 1.82 or 1.83 with BEAGLE 2.1:<br />
<pre><br />
call setbeast_182 call setbeast_183<br />
call beast-beagle_182 x y z call beast-beagle_183 x y z<br />
</pre><br />
<br />
And if you want BEAST 2.3.2 with BEAGLE 2.1 (note that BEAST 2 is a different, er, beast to BEAST 1, regarding packages and compatibilities, so only use BEAST 2 if you know that's what you want to use).<br />
<pre><br />
call setbeast_232<br />
call beast-beagle_232 x y z<br />
</pre><br />
<br />
OpenCL is installed on all the nodes, which means Beagle will pick up the Intel Xeon processors and use what it can on them. It will also detect the Xeon Phis where they're available. If that's something you'd be particularly interested in using, drop me a line.<br />
<br />
=== Blast ===<br />
<br />
Drop me a line to join the party testing this, and to get added to the (large) BlastDB network share.<br />
<br />
<pre><br />
call setblast<br />
</pre><br />
<br />
will set the `BLASTDB` environment variable to point to what we current have of the `v5` database, and the Blast tools (2.9.0, 64-bit) will be added to the path. So you can <br />
then call `blastp`, `blastn`, `blastx` setc.<br />
<br />
=== BOW (BioInformatics on Windows) ===<br />
<br />
A number of Bio-informatics tools are ready to go. These used to be provided together on codeplex, but that has since been retired. Ugene is one project that<br />
maintains windows builds of them. Start your run.bat file with <code>call setBOW</code> to add these to the path:-<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File<br />
! Version<br />
|- <br />
| SamTools.exe<br />
| 0.1.19-44429cd<br />
|- <br />
| BCFTools.exe<br />
| 0.1.19-44428cd<br />
|- <br />
| bgzip<br />
| ?<br />
|- <br />
| razip<br />
| ?<br />
|- <br />
| bwa<br />
| 0.7.17-r1188<br />
|-<br />
| tabix<br />
| 0.2.5 (r1005)<br />
|-<br />
| minimap2<br />
| 2.24 (r1122)<br />
|-<br />
| raxml<br />
| 8.2.12 (best for each node)<br />
|-<br />
| raxml_sse3<br />
| 8.2.12 (pthreads+sse3)<br />
|-<br />
| raxml_avx<br />
| 8.2.12 (pthreads+avx)<br />
|-<br />
| raxml_avx2<br />
| 8.2.12 (pthreads+avx2)<br />
|}<br />
<br />
Note that BWA-MEM is hardwired to use a linux-only feature, so will produce an odd message and not do what you want if you use that mode on Windows. However, it seems that minimap2 may be a faster and more accurate way<br />
of doing what bwa does in any case, and that seems to have no such limitation.<br />
<br />
Also, note that raxml.exe is a copy of the most optimised build for the particular cluster node the job is running on. fi--dideclusthn only supports SSE3;<br />
fi--didemrchnb varies a little; the 32-cores support AVX2, the 24, 20 and 16 core machines support AVX, and only the oldest 12-core nodes only support SSE3. You don't have to worry about, except to expect performance to vary between them.<br />
<br />
=== GATK and Picard ===<br />
<br />
These both rely on Java, so put this in your run script:<br />
<br />
<pre><br />
call setJava64<br />
call setgatk<br />
</pre><br />
<br />
and then <code>gatk</code> in your scripts will use Python 3.7 to call the argument handling wrapper for GATK 4.2.2.0, and <code>picard</code> maps to the picard (2.26.2) jar file.<br />
<br />
=== FastQC ===<br />
<br />
This one has a slight eccentricity that instead of passing normal command-line args, it uses java's -D command-line options to buffer them out of an environment variable<br />
called _JAVA_OPTIONS. Strange but true. The ZIP comes with a bash script, which I'll reimplement if anyone actually wants me to. But in the meantime, it's easy to work <br />
out the arguments by inspection. For example, most likely usage:-<br />
<br />
<pre><br />
call setFastQC<br />
<br />
set _JAVA_OPTIONS='-Dfastqc.output_dir=%OUT%'<br />
call fastqc thing.fq<br />
</pre><br />
<br />
=== MAFFT ===<br />
<br />
For version 7.212 (Win64), write <code>call setMAFFT.bat</code> at the top of your run script.<br />
<br />
=== Applied-Maths Open Source ===<br />
<br />
Write <code>call setAppliedMaths.bat</code> at the top of your run script, to add these to the path:<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File(s)<br />
! Description<br />
! Version<br />
|- <br />
| velvetg_mt_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_mt_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| ray_x64.exe<br />
| Ray 64-bit (for MPI)<br />
| 1.6<br />
|-<br />
| ray_x86.exe<br />
| Ray 32-bit (for MPI)<br />
| 1.6<br />
|-<br />
| Mothur_x64.exe<br />
| Mothur 64-bit (for MPI)<br />
| 1.25.1<br />
|-<br />
| Mothur_x86.exe<br />
| Mothur 32-bit (for MPI)<br />
| 1.25.1<br />
|}<br />
<br />
=== Tensorflow ===<br />
<br />
Here is an example of getting Tensorflow (as a python package) available from an R script, using the cluster to install everything for you. It only works for Tensorflow 2.0 - older versions and newer versions seem to want combinations of DLLs that I can't satisfy. The following are I think minimal steps to get Tensorflow running from Python and R.<br />
<br />
* Create a folder for your python packages in your home directory (so the cluster can see it. I'm calling mine <code>Q:/python_37_repo</code> - and the cluster will call it \\fi--san03.dide.ic.ac.uk\homes\wrh1\python_37_repo - change wrh1 to your DIDE username of course.<br />
* Create a similar folder for R packages, which for me is <code>Q:/R4</code><br />
* And a third folder, Q:/Tensorflow to do this work in.<br />
* Download these DLLs and put them in the Q:/Tensorflow folder - at present they seem to be the ones Python/Tensorflow wants. Let's hope for the best... Download [https://mrcdata.dide.ic.ac.uk/resources/msvcp140.dll msvcp140.dll] and [https://mrcdata.dide.ic.ac.uk/resources/msvcp140_1.dll msvcp140_1.dll] into your <code>Q:/Tensorflow</code> folder.<br />
<br />
* Now, create a script <code>Q:/Tensorflow/prepare.bat</code>. We're going to launch this on the cluster, and get it to wire up the packages we want. Replace USER with your username...<br />
<br />
<pre><br />
net delete Q: /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set TMPDIR=%TEMP%<br />
set PYTHONPATH=\\qdrive.dide.ic.ac.uk\homes\USER\python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
pip install --no-cache-dir tensorflow==2.00 --target Q:\python_37_repo<br />
RScript Q:\Tensorflow\install_tensorflow.R<br />
</pre><br />
<br />
* Create also <code>Q:/Tensorflow/install_tensorflow.R</code> contains:<br />
<pre><br />
install.packages(c("reticulate", "tensorflow"),lib="Q:/R4",repos="https://cran.ma.imperial.ac.uk/")<br />
</pre><br />
<br />
* Now submit the prepare job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> in the big window will do it. That will take a while, but you should notice lots of new files in your Q:/python_37_repo folder. You'll only need to do this once (unless you delete your Python repo folder, or it randomly breaks and you start again...)<br />
<br />
* Next, create a file <code>Q:/Tensorflow/run.bat</code> containing the following the two "call" lines near the end are the ones that do some "work" in python or R, so this script basically a template for running your future jobs. Just change those two calls to set off the python or R code you want to run that uses Tensorflow.<br />
<br />
<pre><br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
net use Q: /delete /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set PYTHONPATH=Q:/python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call python Q:\tensorflow\test_tensorflow.py<br />
call RScript Q:\tensorflow\test_tensorflow.R<br />
</pre><br />
<br />
* And finally, you'll need <code>Q:\tensorflow\test_tensorflow.py</code> and <code>Q:\tensorflow\test_tensorflow.R</code> to exist. Here are minimal tests that everything is wired up. <code.test_tensorflow.py</code> first:-<br />
<br />
<pre><br />
import tensorflow as tf;<br />
print(tf.reduce_sum(tf.random.normal([1000, 1000])));<br />
</pre><br />
<br />
and <code>Q:\Tensorflow\test_tensorflow.R</code> contains:<br />
<br />
<pre><br />
library(reticulate)<br />
reticulate::py_config()<br />
library(tensorflow)<br />
tf$constant('Hello, TensorFlow!')<br />
tf$Variable(tf$zeros(shape(1L)))<br />
</pre><br />
<br />
* Now submit the run job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> in the big window will do it - and perhaps paste <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\output.txt</code> in the stdout box to collect the input and see how it gets on. You should see sensible output from both tests.<br />
<br />
== Launching a job ==<br />
<br />
<br />
== Submitting many jobs ==<br />
<br />
Suppose you write an exe that you might run with… Mysim.exe init.txt 1 2 3, and you want to run it many times with a range of parameters. One way of many, is to write a launch.bat file that will run “job submit” separately, for example (thanks Tini/James!):-<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set initFile=\\networkshare\job\init.txt<br />
set exeFile=\\networkshare\job\mysim.exe<br />
<br />
%SubDetails% %exeFile% %initfile% 1 2 3<br />
%SubDetails% %exeFile% %initfile% 4 5 6<br />
%SubDetails% %exeFile% %initfile% 7 8 9<br />
</pre><br />
<br />
Suppose the job you want to run is an R script? To specify arguments to an R script, you have to add ′--args a=1 b=2′ - so… you might make launch.bat like this:-<br />
<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set rbatFile=\\networkshare\R-scripts\run.bat<br />
<br />
%SubDetails% %rbatFile% 1 2<br />
%SubDetails% %rbatFile% 3 4<br />
</pre><br />
And make the significant line of your run.bat:-<br />
<code><br />
Rcmd BATCH ′--args a=%1 b=%2′ U:\R-scripts\test.R<br />
</code><br />
%1 and %2 will map to the first and second thing after the batch file. You can go all the way up to %9.<br />
<br />
IMPORTANT NOTE<br />
<br />
Make sure you get the apostrophe character right in the above example – NEVER copy and paste from a word document into a script. It will go hideously wrong. Type the apostrophes (and dashes for that matter) in a good text editor – you want just the standard old-fashioned characters.<br />
<br />
== Requesting resources ==<br />
<br />
The following modifiers next to the “/scheduler:” part of the job submit line (before your app.exe 1 2 3 part), will request things you might want…<br />
<br />
/numcores:8 - number of cores you want<br />
<br />
/numcores:8-12 - minimum and maximum cores appropriate for your job<br />
<br />
/memorypernode:1024 - amount of mem in MegaBytes needed.<br />
<br />
/workdir:\\networkshare\work - set working directory<br />
<br />
/stdout:\\networkshare\out.txt - divert stdout to a file<br />
<br />
/stderr: or /stdin: - similar for stderr and stdin<br />
<br />
== Troubleshooting / Miscellany / Q & A ==<br />
<br />
* My job doesn’t work.<br />
** Run the HPC Job Manager application. Find your job id, possible under “My Jobs, Failed”. Double click on it, then on “View All Tasks”. Perhaps something in the output section will help.<br />
<br />
* Check that the path to your job is visible everywhere.<br />
** Avoid spaces in your paths - “job submit” doesn’t like them very much. If you must have them, they’ll be ok in the “run.bat” batch file that the cluster will run – in which case, put the path in standard double-quotes ("). But avoid them in your “launch.bat” file – you may have to relocate your run.bat to a simple non-space-containing directory.<br />
** Rather than putting the full application and parameters on the job submit line, you might want to write a batch-file to do all that, and submit the batch file to the cluster. (See section 6 about R for example). But make sure the batch file is somewhere visible to the cluster.<br />
<br />
* My job seems to work, but reports as having failed.<br />
** The success/failure depends on the error code returned. If you’re running C code, end with “return 0;” for success.<br />
<br />
* job submit ..blah blah.. app.exe >out.txt doesn’t work!<br />
** The contents of out.txt will be the result of “job submit”, not the result of “app.exe”. You meant to say this job submit ..blah blah.. /stdout:out.txt app.exe correcting out.txt and app.exe to network paths of course.<br />
<br />
* My C/C++ Executable works perfectly on my desktop, but doesn't work on the cluster. What's wrong with the cluster?<br />
** Your EXE file probably relies on some DLL files that the cluster doesn't have. If you use Visual Studio, check that in Code Generation, you have "Multi-Threaded (/MT)" as your target - with no mention of DLLs. If your code uses OpenMP, check you've copied the right OpenMP DLL(s) (go back to the C section above). As a last resort, give your EXE to someone who doesn't have Visual Studio installed, and get them to run it, and see what DLL files it seems to ask for.<br />
<br />
== Contributing Authors ==<br />
<br />
Wes Hinsley, James Truscott, Tini Garske, Jeff Eaton, Hannah Clapham</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Printing&diff=14287Printing2025-01-27T14:36:14Z<p>Admin: </p>
<hr />
<div>= Printing at Imperial College =<br />
The print system is handled by HP themselves on behalf of the college. You should follow [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Imperial's Mobile Print instructions]. This page is a guide to follow while you're working through the ICT Mobile Print page.<br />
<br />
== Advice for when following the Mobile Print instrucions ==<br />
Most of you are Windows users so I'll go over the key steps here. The main ICT instructions are quite good if you pay close attention, but they are from an old version of Windows 10. My page here is a supplement in case you get stuck.<br />
<br />
Log in to the [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Mobile Print site] with your Imperial credentials and then select '''Driver Print'''. Note that this page will only work on campus.<br />
<br />
[[file:mobileprint1.jpeg|border]]<br />
<br />
Now select your operating system with the large icons at the bottom (you are most likely Windows):<br />
<br />
[[file:mobileprint2os.jpeg|border]]<br />
<br />
After this you get a couple of pages with only one option each (there used to be more). Go through these formalities and select the only option you have:<br />
<br />
[[File:Mobileprint3option.jpeg|border]]<br />
<br />
We now get to a mildly tricky bit as we have to download the actual HP software for talking to the printers. There is a handy link on the Mobile Print page to the [https://ftp.hp.com/pub/softlib/software13/COL40842/ds-99374-24/upd-pcl6-x64-7.0.1.24923.exe HP FTP site] so click on that to download it.<br />
<br />
[[File:Mobileprint4drivers.jpeg|border]]<br />
<br />
Once downloaded, double click on it to run it and press the '''Unzip''' button. Go through the installer as normal but make sure you select '''USB Mode - Plug and Play''' as your printer type. Leave the two text boxes at the bottom ticked.<br />
<br />
[[File:Mobileprint5usb.png|border]]<br />
<br />
Now we get to add the actual print queue to your computer. From your start menu type '''printers''' and you'll see a menu item to add printers. Click on that and then select '''add a printer'''. It will have a think for a while and then show you some completely incorrect printers. As soon as you see the option '''my printer is not listed here''' click on that.<br />
<br />
[[File:Printernotlisted.png|border]]<br />
<br />
Now you can copy and paste the name of your print queue, which is listed in Step 4 of the Mobile Print page guide (below is a screenshot for your reference - you can't copy and paste this):<br />
<br />
[[File:Mobileprint6queue.jpeg|border]]<br />
<br />
You do need to copy and paste the whole address line, including the bit I've blurred out (I've blurred my exact address out or you would be printing as me with my print credit!). Remember that you're not copying and pasting this line, you're using the line from the Mobile Print web page!<br />
<br />
The rest of the ICT instructions should work ok, you merely have to select that you're adding an '''HP''' printer and you want to use '''HP Universal Printing PCL 6'''. There is no need to print a test page.<br />
<br />
== I don't have time for this and just want to print something! ==<br />
If you're in a rush and have to print right now, then email your document to [mailto:print@imperial.ac.uk print@imperial.ac.uk] from your Imperial email address. You may then go to any swipe printer and collect your document. Remember to delete the email from your Sent Items folder afterwards or you may be wasting mailbox space.<br />
<br />
== It won't print in colour! ==<br />
This is because the printers always autodetect as mono. Go in Printer properties -> Device Settings tab -> scroll down to Device Type and change it to say '''color'''.<br />
<br />
== Gotchas ==<br />
If you are in the habit of using Remote Desktop and the VPN to connect to your DIDE PC, then there is a setting which you should '''untick''' in your Remote Desktop connection settings screen:<br />
<br />
[[File:Printer3.jpg]]<br />
<br />
Otherwise you'll end up with your office printers connected to your home computer and it has been known to mess things up.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Cluster_Documentation&diff=14286Cluster Documentation2025-01-20T21:33:00Z<p>Admin: /* Python */</p>
<hr />
<div>* We have two high performance clusters for general use, both running Microsoft HPC 2012 R2, Update 3. The smaller, older cluster is [[fi--dideclusthn]] and the larger is [[fi--didemrchnb]]. <br />
<br />
* Many users in the department write code and packages in R. We have in-house packages that can allow an almost-interactive experience of running jobs on our high performance clusters, so if that sounds like what you'd like, you might as well stop reading here, and go straight to [https://github.com/mrc-ide/didehpc The DIDEHPC package] - or if you need to join the mrc-ide github org first, go to [https://github.com/mrc-ide/welcome The welcome page].<br />
<br />
If you're not using R, or you are interested in other ways, then continue...<br />
<br />
== Windows Users ==<br />
<br />
The cluster nodes are Windows 2012-R2 based. This is good news if you're a Windows user, and especially easy for PCs within the departmental DIDE domain. You have the option of using the MS client for launching jobs (see below), or you can use try the new [[HPC Web Portal]]. If you're preparing to use the cluster, and have doubts about whether it can run what you want - and whether it will be straightforward to develop and run on it, best talk to me and/or the IT guys first!<br />
<br />
== Linux and MAC Users ==<br />
<br />
There are two problems, but both can be at least partially overcome. The first is that there is no client for linux. The second is that it can only run windows binaries.<br />
<br />
=== Launching jobs on Linux/Mac ===<br />
<br />
* The best way might be the [[HPC Web Portal]], which lets you submit jobs through a webpage. Give it a try, and feedback to me any issues! <br />
<br />
* Alternatively, you can either (1) go down the VM route, installing a windows virtual machine, eg VMWare or Parallels, and follow the instructions below. (2) Remote Desktop to your friend's Windows machine. These tend not to be very convenient, but some users have submitted thousands of jobs this way.<br />
<br />
=== What Linux/Mac jobs can I run? ===<br />
<br />
* Many jobs may be platform indepedent. Where compiled code is involved though, you'll have some extra work to make sure the binary parts can be run in windows. For R, [https://github.com/mrc-ide/didehpc the tools] help you sort that out. For others, you may need to run a job on the cluster to do your builds. Also, for many linux applications, there is a windows port. Mileage can vary, but it's worth a try searching for that before you give up hope. Some of these might be installed on the cluster already - see below. Or some you may be able to download and call them from your run script.<br />
<br />
* For C code, again, we can run scripts on the cluster to build the binaries for you, if your code can be compiled with (say) the MinGW port of gcc. Cygwin can also be used (although seems less common these days) with accompanying dlls. Get in touch if you need further ideas; most things can be solved. <br />
<br />
* Matlab is a little more tricky - we have a range of colossal matlab runtimes on the cluster, but you'd need to compile a windows binary to use them, since the matlab platforms are distinct in every way. For this, probably find a friend, or get in touch, and we'll work out how you can compile for it.<br />
<br />
* There is also the option of running a Windows VM, which might solve all of the above problems.<br />
<br />
=== But I really want a unix cluster. ===<br />
<br />
* We have run linux clusters in the past ([[fi--didelxhn]]), but there was very low demand, whereas the other clusters are consistently well used. Accessing DIDE network shares from linux was a little harder in some aspects, but we did not implement the more traditional linux approach of having dedicated separate cluster storage that inputs and outputs would have to be staged on to, in order for the cluster to see them. Generally, the flexibility of the other clusters makes such things quite straightforward, and may even feel a pleasure if your background is with the less domain-integrated style of cluster architecture.<br />
<br />
* It has also become far easier to write software that builds on multiple platforms with good practice. Tools that only run on one platform are rare, and in some circumstances have raised questions about the quality of those packages. Alternatives are often available.<br />
<br />
* Where compilation is the issue - for example if you want to run binaries on the cluster, but your compilation machine runs linux, or is a mac, (Matlab via MCC, or C++ code for example), then get in touch, and there are a few ways we can think of solving that, either by getting someone else, or the cluster itself, to build your code.<br />
<br />
* There are also resources in [https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/computing/high-throughput-computing/ ICT) who run a linux cluster, with the file-staging approach, and also (being shared between a much larger pool of users) have rather stricter rules on job durations and resources than we provide locally in DIDE.<br />
<br />
== Getting Started ==<br />
<br />
=== Getting access to the cluster ===<br />
<br />
Send a mail to Wes (w.hinsley@imperial.ac.uk) requesting access to the cluster. He will add you to the necessary groups, and work out what cluster you should be running on. Unless you are told otherwise, this will be fi--dideclusthn. But if you have been told otherwise, then whenever you see fi--dideclusthn, replace it with the cluster name you’ve been given, either mentally or (less effectively) on your screen with tip-ex.<br />
<br />
=== Windows: Install the HPC client software ===<br />
<br />
* Run \\fi--dideclusthn.dide.ic.ac.uk\REMINST\setup.exe. <br />
* Confirm '''Yes, I really want to run this'''.<br />
* There's an introductory screen, and you click '''Next'''<br />
* You have to tick the '''Accept''', then click '''Next'''<br />
* If it gives you any choice, select '''Install only the client utilities'''<br />
* '''Next''', '''Next''', '''Next'''<br />
* Select '''Use Microsoft Update''' and then '''Next'''<br />
* And then '''Install'''<br />
* And possibly '''Install''' again – if it had to install any pre-requisites first.<br />
* And '''Finish''', and the client installation is done.<br />
* To check everything is alright so far, open a '''NEW command prompt window'''.<br />
* (Because the installer adds things to the path, which only take effect on '''NEW comamnd windows'''! :-)<br />
* (Find it under Accessories, or you can use Start, Run, '''cmd''').<br />
* Type <code>job list /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
And it will list the jobs you have running, which will be none so far!<br />
<br />
<pre><br />
Id Owner Name State Priority Resource Request<br />
---------- ------------- ----------------- ------------ ----------- -------- -------<br />
<br />
0 matching jobs for DIDE\user<br />
</pre><br />
<br />
=== Windows: Using a non-domain machine ===<br />
<br />
If you're using a machine that isn't actually logged into DIDE, then the client software will have a problem working out who you are. In this case, there are two things you need to do.<br />
* Check you've really got a DIDE username. If you don't know what it is, talk to your lovely IT team.<br />
* Connect to the DIDE VPN using ZScaler from https://uafiles.cc.ic.ac.uk/, and login to that with your IC account - but you'll still need your DIDE account to access DIDE servers after the ZScaler connection is up and running.<br />
* Now we'll open a command window using "runas" - which lets you choose which identity the system thinks you are within that command window:-<br />
<pre><br />
runas /netonly /user:DIDE\user cmd<br />
</pre><br />
* (Change user to your own login obviously). It will ask for your password, then open a new command window. In that window, you'll be able to do cluster-related things as your DIDE username.<br />
<br />
=== Windows: Launching and cancelling jobs ===<br />
==== Windows: Command Line ====<br />
<br />
Suppose you have a file called "run.bat" in your home directory, which does what you want to run on the cluster. Let's say it's a single-core, very simple job. We’ll discuss what should be inside "run.bat" later. To submit your job on the cluster, at the command prompt, type this (all on one line) - or put it in a file called "launch.bat", and run it:-<br />
<br />
<code>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1 \\qdrive.dide.ic.ac.uk\homes\user\run.bat</code><br />
<br />
If it's the first time you’ve run a job - or if you've recently changed your password, then it might ask you for your DIDE password and offer to remember it. Otherwise, it will just tell you the ID number of your job.<br />
<br />
<pre>Enter the password for ‘DIDE\user’ to connect to ‘FI—DIDECLUSTHN’:<br />
Remember this password? (Y/N)<br />
job has been submitted. ID: 123<br />
</pre><br />
<br />
If you want to remove the job, then:- <code>job cancel 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
Or view its details with <code>job view 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
==== Windows: Job Manager GUI ====<br />
<br />
Alternatively to the command-line, you can use the job management software, rather than the command-line. The advantage is that it’s a GUI. The disadvantage is, as in all GUIs, you may not feel totally sure you know what it’s up to – where most of the time the things you want to do might not be very complex, as above.<br />
<br />
The job management software will be on your start menu, as above. All the features are under the “Actions” menu, and hopefully it will be self explanatory. Read the details below about launching, and you'll find the interface bits that do it in the Job Manager. However, you may find over time, especially if you run lots of jobs, that learning to do it the scripted way with the command-line can be quicker.<br />
<br />
<br />
== Information for running any job ==<br />
<br />
=== Visibility ===<br />
<br />
Any executable/batch file that the cluster will run must be somewhere on the network that the cluster has access to, when it logs in as you. This amounts to any network accessible drive that you have access to when you login – including network shares, such as the temp drive, your home directory, and any specific network space set up for your project.<br />
<br />
After a job is launched, all paths are relative to the '''/workdir:''' specified on the job submit command-line. Windows does not usually allow a network path to be the current directory, and MS-HPC somehow gets around this. But if in doubt, use fully-qualified paths to specify where files should be read from, or written to. You may want to write code that takes the paths from a parameter file, or as a command-line parameter, to give as much flexibility as possible. <br />
<br />
Your home directory, and the temp drive are on departmental servers that don't have an especially fast connection to the cluster. So avoid having thousands of jobs all reading/writing to those places at once. Additionally, the temp drive is shared between all users, and your home drive has a limited quota. It is also backed up every day, so avoid using it as scratch storage for results you don’t actually want to keep. Generally, the shares you'll want to use from the clusters will be ones containing the word "NAS", which the big cluster has fast access to.<br />
<br />
=== Interactivity ===<br />
<br />
The job must run in an entirely scripted, unattended way. It must not require any key presses or mouse clicks, or other live interactivity while it runs. So jobs generally will read some sort of input (from a file, or from the way you run the job), do some processing, and write the results somewhere for you - all without intervention.<br />
<br />
== Launching jobs ==<br />
<br />
Jobs can be launched either through the Job Manager interface, or through the command line tools, which offer greater flexibility. We'll describe the command line method here; if you want to use the GUI, then it'll be a case of finding the matching boxes... Below are the specifics for our clusters. For more information, simply type <code>job</code> on the commandline, or <code>job submit</code> for the list of submission-related commands.<br />
<br />
=== FI--DIDECLUSTHN vs FI--DIDEMRCHNB ===<br />
<br />
Job submissions, as shown below, must specify a "job template", which sets up a load of default things to make the job work. On fi--dideclusthn, the job templates are called 4Core, 8Core and GeneralNodes, which will respectively force jobs to run on the 4-core nodes, the 8-core nodes, or on any machine available.<br />
<br />
On fi--didemrchnb, you can set the job template to be... 8Core, 12Core, 16Core, 12and16Core, and GeneralNodes - which hopefully are fairly self-explanatory. There are a couple of other job templates, (24Core and Phi), but those are a bit special purpose for now, so don't use them!<br />
<br />
=== Job Submission ===<br />
<br />
Job submissions from the command line can take the following form (all on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /stdout:\\path\to\stdout.txt<br />
/stderr:\\path\to\stderr.txt /numcores:1-1 /jobtemplate:4Core \\path\to\run.bat</pre><br />
<br />
=== The /singlenode argument ===<br />
<br />
In MS HPC (2012), Microsoft finally added a tag to allow you to say that the 'n' cores you requested must be on the same computer. Therefore, if you know precisely how many cores you want, then use the following (on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:true <br />
/jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
However, there is one bug with this, for the specific case where you request a whole node, regardless of how many cores it has. In this case, oddly, you have to disable single node:-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:false <br />
/numnodes:1 /jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
== Languages and libraries supported ==<br />
<br />
A number of languages, and different versions of languages are available on the cluster. The sections below refer to your "run.bat" file - a batch file that you will create which will get run by a cluster node when a suitable one is available. The commands described below are to be put in this "run.bat" file, and they add various directories to the path, so that the software you want will be added to the path. <br />
<br />
=== C/C++ === <br />
<br />
A number of Microsoft C++ and Intel C++ runtimes are installed, but it's usually better to try and avoid using them, and make your executable as stand-alone as possible. If it requires any external libraries that you've had to download, then put the .dll file in the same directory as the .exe file. If you use Microsoft Visual Studio, in Project Preferences, C/C++, Code Generation, make sure the Runtime Library is Multi-threaded (/MT) – the ones with DLL files won’t work. Even so, on recent versions of the Intel and Microsoft C compilers, "static linking" doesn’t really mean static when it comes to OpenMP, and you’ll have to copy some DLLs and put them next to your EXE file. See the OpenMP section below.<br />
<br />
The cluster nodes are all 64-bit, but they will run 32-bit compiled code. Just make sure you provide the right DLLs!<br />
<br />
Here is a document about [[Using the Xeon Phi]]<br />
<br />
Here is a document about [[Using MPI]]<br />
<br />
And if you want to use libraries such as GSL, then try [[C/C++ Libraries for Windows]]<br />
<br />
=== Java ===<br />
<br />
Java doesn't strictly need installing; you can setup any Java Runtime Environment you like, from Oracle, or anywhere. If you want to do this, then, (if you were to use the Server JRE from Oracle), something similar to the following will add java.exe and jvm.dll to the path, and set the JAVA_HOME environment variable, which most Java-using software will appreciate.<br />
<br />
set JAVA_HOME=\\my\path\to\jre1.8.0<br />
set path=%JAVA_HOME%\bin;%JAVA_HOME%\bin\server;%path%<br />
<br />
For convenience, if you're happy with Azul's OpenJDK Java 1.8.0 update 308 (September 2021), then <code>call setJava64</code> in your run script will set this up for you. Incidentally the above will also add what's needed for using the rJava package in R - in which case you'll perhaps also want something like <code>call setR64_4_1_0</code> in your run script.<br />
<br />
=== Perl ===<br />
<br />
Strawberry Perl 64-bit portable, v5.30.1.1. Put <code>call setPerl</code> at the top of your script.<br />
<br />
=== R ===<br />
<br />
'''New tools are coming''': [http://dide-tools.github.io/didehpc/vignettes/quickstart.html see here]. The instructions below are still valid if you want to do this manually.<br />
<br />
<br />
You can R jobs on all the clusters. This is the latest wisdom on how to do so – thanks to James, Jeff, Hannah and others. <br />
<br />
First, if you are wanting to use packages, then set up a package repository on your home directory by running this in R.<br />
<br />
<code>install.packages("<package>",lib="Q:/R")</code><br />
<br />
Now, write your run script. Suppose you have an R script in your home directory: Q:\R-scripts\test.R. And suppose you’ve set up your repository as above. Your run.bat should then be:-<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\test.R<br />
</pre><br />
Various packages require various R versions, and the cluster supports a few versions. To choose which one, change the first line of the script above to one of these – 32-bit or 64-bit versions of R releases. Purely for amusement, R's codenames are shown here too. <br />
<pre><br />
call setr32_2_13_1.bat call setr64_2_13_1.bat (anyone know the codename?)<br />
call setr32_2_14_2.bat call setr64_2_14_2.bat (Gift-getting season)<br />
call setr32_2_15_2.bat call setr64_2_15_2.bat (Trick or treat)<br />
call setr32_3_0_1.bat call setr64_3_0_1.bat (Good Sport)<br />
call setr32_3_0_2.bat call setr64_3_0_2.bat (Frisbee Sailing)<br />
call setr32_3_1_0.bat call setr64_3_1_0.bat (Spring Dance)<br />
call setr32_3_1_2.bat call setr64_3_1_2.bat (Pumpkin Helmet)<br />
call setr32_3_2_2.bat call setr64_3_2_2.bat (Fire Safety)<br />
call setr32_3_2_3.bat call setr64_3_2_3.bat (Wooden Christmas Tree)<br />
call setr32_3_2_4.bat call setr64_3_2_4.bat (Very Secure Dishes - Revised Version - R later renamed this to 3.2.5)<br />
call setr32_3_3_1.bat call setr64_3_3_1.bat (Bug in your hair)<br />
call setr32_3_3_2.bat call setr64_3_3_2.bat (Sincere Pumpkin Patch)<br />
call setr32_3_4_0.bat call setr64_3_4_0.bat (You Stupid Darkness)<br />
call setr32_3_4_2.bat call setr64_3_4_2.bat (Short Summer)<br />
call setr32_3_4_4.bat call setr64_3_4_4.bat (Someone to lean on)<br />
call setr32_3_5_0.bat call setr64_3_5_0.bat (Joy in Playing)<br />
call setr32_3_5_1.bat call setr64_3_5_1.bat (Feather Spray)<br />
call setr32_3_5_3.bat call setr64_3_5_3.bat (Great Truth)<br />
call setr32_3_6_0.bat call setr64_3_6_0.bat (Planting of a Tree)<br />
call setr32_3_6_1.bat call setr64_3_6_1.bat (Action of the Toes)<br />
call setr32_3_6_3.bat call setr64_3_6_3.bat (Holding the Windsock)<br />
call setr32_4_0_2.bat call setr64_4_0_2.bat (Taking Off Again)<br />
call setr32_4_0_3.bat call setr64_4_0_3.bat (Bunny-Wunnies Freak Out)<br />
call setr32_4_0_5.bat call setr64_4_0_5.bat (Shake and Throw)<br />
</pre><br />
<br />
The following versions are *not* available on the cluster at the moment - but here are their names anyway, as it seemed a shame to miss them out. Let me know if you desperately need one of these.<br />
<br />
<pre><br />
2.14.0 Great Pumpkin <br />
2.14.1 December Snowflakes<br />
2.15.0 Easter Beagle<br />
2.15.1 Roasted Marshmallow<br />
2.15.3 Security Blanket<br />
3.0.0 Masked Marvel<br />
3.0.3 Warm Puppy<br />
3.1.1 Sock it to me<br />
3.1.3 Smooth Sidewalk<br />
3.2.0 Full of Ingredients<br />
3.2.1 World famous astronaut<br />
3.2.4 Very Secure Dishes (the non-revised version)<br />
3.3.0 Supposedly Educational<br />
3.3.3 Another Canoe<br />
3.4.1 Single Candle<br />
3.4.3 Kite-Eating Tree<br />
3.5.2 Eggshell Igloo<br />
3.6.2 Dark and Stormy Night<br />
4.0.0 Arbor Day<br />
4.0.1 See Things Now<br />
4.0.4 Lost Library Book<br />
</pre><br />
<br />
It also seems that different R versions put their packages in different structures – sometimes adding "win-library" into there for fun. Basically, R_LIBS and R_LIBS_USER should be paths to a folder that contains a list of other folders, one for each package you've installed.<br />
<br />
IMPORTANT: R_LIBS and R_LIBS_USER paths must NOT contain quotes, nor spaces. If the path to your library contains spaces, you need to use old-fashioned 8-character names. If in your command window, you type <code>dir /x</code>, you’ll see the names – Program Files tends to become PROGRA~1 for instance. <br />
<br />
==== Passing parameters to R scripts ====<br />
Passing parameters to R scripts means you can have fewer versions of your R and bat files and easily run whole sets of jobs. You can get parameters into R using Rscript (but not Rcmd BATCH, I think) as follows. In the run.bat example above, the Rscript statement becomes:<br />
<pre><br />
Rscript Q:\R-scripts\test.R arg1 arg2 arg3<br />
</pre><br />
Within the R code, the arguments can be recovered using <br />
<pre><br />
args <- commandArgs(trailingOnly = TRUE)<br />
<br />
outFileName <- args[1] ## name of output file.<br />
dataFileName <- args[2] ## name of local data file. <br />
currentR0 <- as.numeric(args[3]) ## convert this explicitly to number. <br />
</pre><br />
If you want your arguments to have a particular type, best to explicitly convert (see R0 above). Better still, you can pass parameters directly into the batch file that runs the R script. Command line arguments can be referenced within the batch file using %1, %2, etc. For example, if you have a batch file, runArgs.bat:<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\%1.R %2 %3 %4<br />
</pre><br />
then<br />
<pre><br />
runArgs.bat myRScript arg1 arg2 arg3<br />
</pre><br />
will run the R script myRScript.R and pass it the parameters arg1, arg2 and arg3. The batch file runArgs.bat is now almost a '''generic R script runner'''.<br />
<br />
=== OpenMP ===<br />
<br />
This has been tested with C/C++ - but the same should apply to other languages such as fortran, that achieve multi-threading with a DLL file.<br />
<br />
==== Microsoft's C/C++ Compiler ====<br />
You will need to copy vcomp100.dll (for Visual Studio 2010 – it maybe vcomp90.dll for 2008), into the same directory as your executable. You can usually find the dll file in a directory similar to:- C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist\x64\Microsoft.VC100.OpenMP. Extrapolate for future versions! Also make sure that you've enabled OpenMP in the Properties (C++/Language).<br />
<br />
==== GCC or MinGW ====<br />
Usually, this applies to Eclipse use. In project properties, the C++ command “g++ -fopenmp”, the C command “gcc –fopenmp”, and in the Linker options, under Miscellaneous, Linker flags, put “-fopenmp” too. Copy the OpenMP DLLs from MinGW into the same directory as your final executable. You may find the dlls are in C:\MinGW\Bin\libgomp-1.dll and in the same place, libpthread-2.dll, libstdc++-6.dll and libgcc_s_dw2-1.dll.<br />
<br />
==== Intel C++ Compiler ====<br />
Copy the libiomp5md.dll file from somewhere like C:\Program Files (x86)\Intel\ComposerXE-2011\redist\intel64\compiler\libiomp5md.dll to the same place as your executable. And in Visual Studio, make sure you've enabled OpenMP in the project properties.<br />
<br />
==== How many threads? ====<br />
The OpenMP function omp_max_threads() returns the number of ''physical'' cores on a machine, not the number of cores ''allocated'' to your job. To determine how many cores the scheduler actually allocated to you, use the following code to dig up the environment variable CCP_NUMCPUS, which will be set by the cluster management software:- <br />
<pre><br />
int set_NCores() {<br />
char * val;<br />
char * endchar;<br />
val = getenv("CCP_NUMCPUS");<br />
int cores = omp_get_max_threads();<br />
if (val!=NULL) cores = strtol (val,&endchar,10);<br />
omp_set_num_threads(cores);<br />
return cores;<br />
}<br />
</pre><br />
<br />
=== WinBugs (or OpenBugs) ===<br />
<br />
They are similar, but OpenBugs 3.2.2 is the one we've gone for. Something like this as your run.bat script will work:-<br />
<pre><br />
call setOpenBugs<br />
openbugs \\path\to\script.txt /HEADLESS<br />
</pre><br />
<br />
=== MatLab ===<br />
<br />
There are various ways of producing a non-interactive executable from Matlab. Perhaps the simplest (not necessarily the best performance) way is to use “mcc.exe” – supplied with most full versions of Matlab, including the Imperial site licence version that you've probably got.<br />
<br />
==== Use mcc.exe to compile your code ====<br />
<br />
Use windows explorer to navigate to the folder where the “.m” files are for the project you want to compile. Now use a good text editor to create a file called “compile.bat” in that folder. It should contain something similar to the following:-<br />
<br />
<code>mcc -m file1.m file2.m file3.m -o myexe</code><br />
<br />
Don’t copy/paste the text from this document by the way – Word has a different idea of what a dash is to most other software, and will probably replace the two dashes with funny characters. so you have to list every single .m file that your project needs, after the’-m’. If you save this file, then double-click on it, then it will think for some while, and produce “myexe.exe” in this example. Copy your .exe file into a network accessible place as usual.<br />
<br />
==== Launch on the cluster. ====<br />
<br />
The launch.bat file will be exactly the same as before – see page 2. The run script will then start with a line that tells the cluster which version of Matlab you used to compile the cluster. Below is the table of different versions. The runtimes are huge and cumbersome to install on the cluster, so as a result I haven’t installed every single one. If you need one that’s not listed, get in touch.<br />
<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! Matlab Version<br />
! First line of run script<br />
|- <br />
| R2009b<br />
| call useMatLab79<br />
|- <br />
| R2010a<br />
| call useMatLab713<br />
|- <br />
| R2011a<br />
| call useMatLab715<br />
|- <br />
| R2011b<br />
| call useMatLab716<br />
|- <br />
| R2012a (64-bit)<br />
| call useMatLab717_64<br />
|- <br />
| R2012b (64-bit)<br />
| call useMatLab80_64<br />
|- <br />
| R2013a (64-bit)<br />
| call useMatLab81_64<br />
|- <br />
| R2014a (64-bit)<br />
| call useMatLab83_64<br />
|- <br />
| R2014b (64-bit)<br />
| call useMatLab84_64<br />
|- <br />
| R2015a (64-bit)<br />
| call useMatLab85_64<br />
|- <br />
| R2015b (64-bit)<br />
| call useMatLab90_64<br />
|- <br />
| R2016b (64-bit)<br />
| call useMatLab91_64<br />
|- <br />
| R2017a (64-bit)<br />
| call useMatLab92_64<br />
|- <br />
| R2019a (64-bit)<br />
| call useMatLab96_64<br />
|- <br />
| R2021a (64-bit)<br />
| call useMatLab910_64<br />
|- <br />
| R2022a (64-bit)<br />
| call useMatLab912_64<br />
|}<br />
<br />
=== Python ===<br />
<br />
* The most recent version of each major release is on the cluster. Different python versions within the same release (e.g., 3.5.1 and 3.5.2) do not behave nicely together in automated installation, so if you have a desperate need for a previous minor version, get in touch. While 2.7.18 is available, we '''strongly''' recommend migrating away from Python 2.7, as it has been officially discontinued on 1st January 2020. Experiences may vary, and future success is not guaranteed.<br />
<br />
<pre><br />
2.7.18 64-bit call set_python_27_64.bat<br />
3.5.10 64-bit call set_python_35_64.bat<br />
3.6.15 64-bit call set python_36_64.bat<br />
3.7.17 64-bit call set_python_37_64.bat<br />
3.8.20 64-bit call set python_38_64.bat<br />
3.9.21 64-bit call set_python_39_64.bat<br />
3.10.16 64-bit call set python_310_64.bat<br />
3.11.11 64-bit call set_python_311_64.bat<br />
3.12.8 64-bit call set_python_312_64.bat<br />
3.13.1 64-bit call set_python_313_64.bat<br />
<br />
</pre><br />
<br />
* You'll most likely want to use some packages, so you'll need to have a network-accessible location to put your packages in. If you're running a version of Windows with Python that matches one installed on the cluster, then you'll be able to share that package repository between the two. But if you're running a different version, or a different operating system, or a different Python version, then you'll need to keep your local package repository separate from the one the cluster uses.<br />
* Here comes the basic script to run some python. Suppose I decide to put my Python packages for the cluster in '''\\qdrive.dide.ic.ac.uk\homes\username\PythonCluster37_64''' (which is Q:\PythonCluster37_64 on a DIDE Windows PC). To run some python, something like the following:-<br />
<br />
call set_python_37_64<br />
set PYTHONPATH=\\qdrive\homes\username\PythonCluster37_64<br />
python \\qdrive\homes\username\Python\MyScript.py<br />
<br />
or if you want to do this in a programmatic way...<br />
<br />
import sys<br />
sys.path = ['\\\\qdrive\\homes\\username\\PythonCluster37_64'] + sys.path<br />
<br />
==== Packages ====<br />
<br />
* If you want to install a package like numpy, that's readily available through pip, then you can launch a batch file like this:-<br />
call set_python_36_64<br />
pip install numpy --target \\qdrive\homes\username\PythonCluster36_64<br />
* or alternatively, you could write a python script that looks like this, and launch it as normal:-<br />
import pip<br />
pip.main(['install','numpy','--target','\\\\qdrive\\homes\\username\\PythonCluster36_64'])<br />
<br />
* And if you're running the same Python version on a Windows machine, then you can do these two locally, without submitting a cluster job to do them for you.<br />
<br />
* And next time you submit a python job on the cluster, python will look in the place set by PYTHONPATH, and will find the packages you've just installed when you import them.<br />
<br />
=== Beast and Beagle ===<br />
<br />
We've got a couple of versions available. For BEAST 1.75 with BEAGLE 1.0: (the x y z represents all the command-line arguments you'll want to pass to beast)<br />
<pre><br />
call setbeast<br />
call beast-beagle x y z<br />
</pre><br />
<br />
If you want BEAST 1.82 or 1.83 with BEAGLE 2.1:<br />
<pre><br />
call setbeast_182 call setbeast_183<br />
call beast-beagle_182 x y z call beast-beagle_183 x y z<br />
</pre><br />
<br />
And if you want BEAST 2.3.2 with BEAGLE 2.1 (note that BEAST 2 is a different, er, beast to BEAST 1, regarding packages and compatibilities, so only use BEAST 2 if you know that's what you want to use).<br />
<pre><br />
call setbeast_232<br />
call beast-beagle_232 x y z<br />
</pre><br />
<br />
OpenCL is installed on all the nodes, which means Beagle will pick up the Intel Xeon processors and use what it can on them. It will also detect the Xeon Phis where they're available. If that's something you'd be particularly interested in using, drop me a line.<br />
<br />
=== Blast ===<br />
<br />
Drop me a line to join the party testing this, and to get added to the (large) BlastDB network share.<br />
<br />
<pre><br />
call setblast<br />
</pre><br />
<br />
will set the `BLASTDB` environment variable to point to what we current have of the `v5` database, and the Blast tools (2.9.0, 64-bit) will be added to the path. So you can <br />
then call `blastp`, `blastn`, `blastx` setc.<br />
<br />
=== BOW (BioInformatics on Windows) ===<br />
<br />
A number of Bio-informatics tools are ready to go. These used to be provided together on codeplex, but that has since been retired. Ugene is one project that<br />
maintains windows builds of them. Start your run.bat file with <code>call setBOW</code> to add these to the path:-<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File<br />
! Version<br />
|- <br />
| SamTools.exe<br />
| 0.1.19-44429cd<br />
|- <br />
| BCFTools.exe<br />
| 0.1.19-44428cd<br />
|- <br />
| bgzip<br />
| ?<br />
|- <br />
| razip<br />
| ?<br />
|- <br />
| bwa<br />
| 0.7.17-r1188<br />
|-<br />
| tabix<br />
| 0.2.5 (r1005)<br />
|-<br />
| minimap2<br />
| 2.24 (r1122)<br />
|-<br />
| raxml<br />
| 8.2.12 (best for each node)<br />
|-<br />
| raxml_sse3<br />
| 8.2.12 (pthreads+sse3)<br />
|-<br />
| raxml_avx<br />
| 8.2.12 (pthreads+avx)<br />
|-<br />
| raxml_avx2<br />
| 8.2.12 (pthreads+avx2)<br />
|}<br />
<br />
Note that BWA-MEM is hardwired to use a linux-only feature, so will produce an odd message and not do what you want if you use that mode on Windows. However, it seems that minimap2 may be a faster and more accurate way<br />
of doing what bwa does in any case, and that seems to have no such limitation.<br />
<br />
Also, note that raxml.exe is a copy of the most optimised build for the particular cluster node the job is running on. fi--dideclusthn only supports SSE3;<br />
fi--didemrchnb varies a little; the 32-cores support AVX2, the 24, 20 and 16 core machines support AVX, and only the oldest 12-core nodes only support SSE3. You don't have to worry about, except to expect performance to vary between them.<br />
<br />
=== GATK and Picard ===<br />
<br />
These both rely on Java, so put this in your run script:<br />
<br />
<pre><br />
call setJava64<br />
call setgatk<br />
</pre><br />
<br />
and then <code>gatk</code> in your scripts will use Python 3.7 to call the argument handling wrapper for GATK 4.2.2.0, and <code>picard</code> maps to the picard (2.26.2) jar file.<br />
<br />
=== FastQC ===<br />
<br />
This one has a slight eccentricity that instead of passing normal command-line args, it uses java's -D command-line options to buffer them out of an environment variable<br />
called _JAVA_OPTIONS. Strange but true. The ZIP comes with a bash script, which I'll reimplement if anyone actually wants me to. But in the meantime, it's easy to work <br />
out the arguments by inspection. For example, most likely usage:-<br />
<br />
<pre><br />
call setFastQC<br />
<br />
set _JAVA_OPTIONS='-Dfastqc.output_dir=%OUT%'<br />
call fastqc thing.fq<br />
</pre><br />
<br />
=== MAFFT ===<br />
<br />
For version 7.212 (Win64), write <code>call setMAFFT.bat</code> at the top of your run script.<br />
<br />
=== Applied-Maths Open Source ===<br />
<br />
Write <code>call setAppliedMaths.bat</code> at the top of your run script, to add these to the path:<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File(s)<br />
! Description<br />
! Version<br />
|- <br />
| velvetg_mt_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_mt_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| ray_x64.exe<br />
| Ray 64-bit (for MPI)<br />
| 1.6<br />
|-<br />
| ray_x86.exe<br />
| Ray 32-bit (for MPI)<br />
| 1.6<br />
|-<br />
| Mothur_x64.exe<br />
| Mothur 64-bit (for MPI)<br />
| 1.25.1<br />
|-<br />
| Mothur_x86.exe<br />
| Mothur 32-bit (for MPI)<br />
| 1.25.1<br />
|}<br />
<br />
=== Tensorflow ===<br />
<br />
Here is an example of getting Tensorflow (as a python package) available from an R script, using the cluster to install everything for you. It only works for Tensorflow 2.0 - older versions and newer versions seem to want combinations of DLLs that I can't satisfy. The following are I think minimal steps to get Tensorflow running from Python and R.<br />
<br />
* Create a folder for your python packages in your home directory (so the cluster can see it. I'm calling mine <code>Q:/python_37_repo</code> - and the cluster will call it \\fi--san03.dide.ic.ac.uk\homes\wrh1\python_37_repo - change wrh1 to your DIDE username of course.<br />
* Create a similar folder for R packages, which for me is <code>Q:/R4</code><br />
* And a third folder, Q:/Tensorflow to do this work in.<br />
* Download these DLLs and put them in the Q:/Tensorflow folder - at present they seem to be the ones Python/Tensorflow wants. Let's hope for the best... Download [https://mrcdata.dide.ic.ac.uk/resources/msvcp140.dll msvcp140.dll] and [https://mrcdata.dide.ic.ac.uk/resources/msvcp140_1.dll msvcp140_1.dll] into your <code>Q:/Tensorflow</code> folder.<br />
<br />
* Now, create a script <code>Q:/Tensorflow/prepare.bat</code>. We're going to launch this on the cluster, and get it to wire up the packages we want. Replace USER with your username...<br />
<br />
<pre><br />
net delete Q: /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set TMPDIR=%TEMP%<br />
set PYTHONPATH=\\qdrive.dide.ic.ac.uk\homes\USER\python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
pip install --no-cache-dir tensorflow==2.00 --target Q:\python_37_repo<br />
RScript Q:\Tensorflow\install_tensorflow.R<br />
</pre><br />
<br />
* Create also <code>Q:/Tensorflow/install_tensorflow.R</code> contains:<br />
<pre><br />
install.packages(c("reticulate", "tensorflow"),lib="Q:/R4",repos="https://cran.ma.imperial.ac.uk/")<br />
</pre><br />
<br />
* Now submit the prepare job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> in the big window will do it. That will take a while, but you should notice lots of new files in your Q:/python_37_repo folder. You'll only need to do this once (unless you delete your Python repo folder, or it randomly breaks and you start again...)<br />
<br />
* Next, create a file <code>Q:/Tensorflow/run.bat</code> containing the following the two "call" lines near the end are the ones that do some "work" in python or R, so this script basically a template for running your future jobs. Just change those two calls to set off the python or R code you want to run that uses Tensorflow.<br />
<br />
<pre><br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
net use Q: /delete /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set PYTHONPATH=Q:/python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call python Q:\tensorflow\test_tensorflow.py<br />
call RScript Q:\tensorflow\test_tensorflow.R<br />
</pre><br />
<br />
* And finally, you'll need <code>Q:\tensorflow\test_tensorflow.py</code> and <code>Q:\tensorflow\test_tensorflow.R</code> to exist. Here are minimal tests that everything is wired up. <code.test_tensorflow.py</code> first:-<br />
<br />
<pre><br />
import tensorflow as tf;<br />
print(tf.reduce_sum(tf.random.normal([1000, 1000])));<br />
</pre><br />
<br />
and <code>Q:\Tensorflow\test_tensorflow.R</code> contains:<br />
<br />
<pre><br />
library(reticulate)<br />
reticulate::py_config()<br />
library(tensorflow)<br />
tf$constant('Hello, TensorFlow!')<br />
tf$Variable(tf$zeros(shape(1L)))<br />
</pre><br />
<br />
* Now submit the run job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> in the big window will do it - and perhaps paste <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\output.txt</code> in the stdout box to collect the input and see how it gets on. You should see sensible output from both tests.<br />
<br />
== Launching a job ==<br />
<br />
<br />
== Submitting many jobs ==<br />
<br />
Suppose you write an exe that you might run with… Mysim.exe init.txt 1 2 3, and you want to run it many times with a range of parameters. One way of many, is to write a launch.bat file that will run “job submit” separately, for example (thanks Tini/James!):-<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set initFile=\\networkshare\job\init.txt<br />
set exeFile=\\networkshare\job\mysim.exe<br />
<br />
%SubDetails% %exeFile% %initfile% 1 2 3<br />
%SubDetails% %exeFile% %initfile% 4 5 6<br />
%SubDetails% %exeFile% %initfile% 7 8 9<br />
</pre><br />
<br />
Suppose the job you want to run is an R script? To specify arguments to an R script, you have to add ′--args a=1 b=2′ - so… you might make launch.bat like this:-<br />
<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set rbatFile=\\networkshare\R-scripts\run.bat<br />
<br />
%SubDetails% %rbatFile% 1 2<br />
%SubDetails% %rbatFile% 3 4<br />
</pre><br />
And make the significant line of your run.bat:-<br />
<code><br />
Rcmd BATCH ′--args a=%1 b=%2′ U:\R-scripts\test.R<br />
</code><br />
%1 and %2 will map to the first and second thing after the batch file. You can go all the way up to %9.<br />
<br />
IMPORTANT NOTE<br />
<br />
Make sure you get the apostrophe character right in the above example – NEVER copy and paste from a word document into a script. It will go hideously wrong. Type the apostrophes (and dashes for that matter) in a good text editor – you want just the standard old-fashioned characters.<br />
<br />
== Requesting resources ==<br />
<br />
The following modifiers next to the “/scheduler:” part of the job submit line (before your app.exe 1 2 3 part), will request things you might want…<br />
<br />
/numcores:8 - number of cores you want<br />
<br />
/numcores:8-12 - minimum and maximum cores appropriate for your job<br />
<br />
/memorypernode:1024 - amount of mem in MegaBytes needed.<br />
<br />
/workdir:\\networkshare\work - set working directory<br />
<br />
/stdout:\\networkshare\out.txt - divert stdout to a file<br />
<br />
/stderr: or /stdin: - similar for stderr and stdin<br />
<br />
== Troubleshooting / Miscellany / Q & A ==<br />
<br />
* My job doesn’t work.<br />
** Run the HPC Job Manager application. Find your job id, possible under “My Jobs, Failed”. Double click on it, then on “View All Tasks”. Perhaps something in the output section will help.<br />
<br />
* Check that the path to your job is visible everywhere.<br />
** Avoid spaces in your paths - “job submit” doesn’t like them very much. If you must have them, they’ll be ok in the “run.bat” batch file that the cluster will run – in which case, put the path in standard double-quotes ("). But avoid them in your “launch.bat” file – you may have to relocate your run.bat to a simple non-space-containing directory.<br />
** Rather than putting the full application and parameters on the job submit line, you might want to write a batch-file to do all that, and submit the batch file to the cluster. (See section 6 about R for example). But make sure the batch file is somewhere visible to the cluster.<br />
<br />
* My job seems to work, but reports as having failed.<br />
** The success/failure depends on the error code returned. If you’re running C code, end with “return 0;” for success.<br />
<br />
* job submit ..blah blah.. app.exe >out.txt doesn’t work!<br />
** The contents of out.txt will be the result of “job submit”, not the result of “app.exe”. You meant to say this job submit ..blah blah.. /stdout:out.txt app.exe correcting out.txt and app.exe to network paths of course.<br />
<br />
* My C/C++ Executable works perfectly on my desktop, but doesn't work on the cluster. What's wrong with the cluster?<br />
** Your EXE file probably relies on some DLL files that the cluster doesn't have. If you use Visual Studio, check that in Code Generation, you have "Multi-Threaded (/MT)" as your target - with no mention of DLLs. If your code uses OpenMP, check you've copied the right OpenMP DLL(s) (go back to the C section above). As a last resort, give your EXE to someone who doesn't have Visual Studio installed, and get them to run it, and see what DLL files it seems to ask for.<br />
<br />
== Contributing Authors ==<br />
<br />
Wes Hinsley, James Truscott, Tini Garske, Jeff Eaton, Hannah Clapham</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14285Working in the new School of Public Health building2024-07-05T10:25:05Z<p>Admin: /* My laptop is so old that I don’t have any USB-C ports. */</p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).<br />
<br />
== I want to use the keyboard and mouse with my laptop, but they’re connected to the screen. ==<br />
This is correct and by design. The keyboard and mouse are shared between the PC on the desk (if present) and the laptop connected by the USB-C cable. Control of the keyboard and mouse is passed through the USB-C cable to the laptop. Do not disconnect the keyboard and mouse to connect it to your laptop.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 8-9 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2025 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== My screens work fine but they're in the wrong arrangement ==<br />
This is normal and depends on the order in which your PC detects the screens. Your laptop will always be Screen 1 but the other two displays could be 2 and 3, or 3 and 2.<br />
<br />
[https://support.microsoft.com/en-us/windows/how-to-use-multiple-monitors-in-windows-329c6962-5a4d-b481-7baa-bec9671f728a#Category=Windows_10 See this page for how to rearrange your screens. ]<br />
<br />
== My Apple laptop will only disply to one screen ==<br />
This is by design on Apple M1 and M2-based systems. They will only output to one display. If you want an Apple laptop which displays to two screens, you need an M3, or an old Intel one. Or buy a USB-C dongle.<br />
<br />
[https://support.apple.com/en-us/117373 See here for details.]<br />
<br />
== I have a question not listed here. ==<br />
Either visit your local IT staff, or if not on site email [mailto:dide-it@imperial.ac.uk DIDE-IT], or send them a message on Teams.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14284Working in the new School of Public Health building2024-04-05T19:15:27Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).<br />
<br />
== I want to use the keyboard and mouse with my laptop, but they’re connected to the screen. ==<br />
This is correct and by design. The keyboard and mouse are shared between the PC on the desk (if present) and the laptop connected by the USB-C cable. Control of the keyboard and mouse is passed through the USB-C cable to the laptop. Do not disconnect the keyboard and mouse to connect it to your laptop.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 8-9 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== My screens work fine but they're in the wrong arrangement ==<br />
This is normal and depends on the order in which your PC detects the screens. Your laptop will always be Screen 1 but the other two displays could be 2 and 3, or 3 and 2.<br />
<br />
[https://support.microsoft.com/en-us/windows/how-to-use-multiple-monitors-in-windows-329c6962-5a4d-b481-7baa-bec9671f728a#Category=Windows_10 See this page for how to rearrange your screens. ]<br />
<br />
== My Apple laptop will only disply to one screen ==<br />
This is by design on Apple M1 and M2-based systems. They will only output to one display. If you want an Apple laptop which displays to two screens, you need an M3, or an old Intel one. Or buy a USB-C dongle.<br />
<br />
[https://support.apple.com/en-us/117373 See here for details.]<br />
<br />
== I have a question not listed here. ==<br />
Either visit your local IT staff, or if not on site email [mailto:dide-it@imperial.ac.uk DIDE-IT], or send them a message on Teams.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14283Working in the new School of Public Health building2024-04-05T19:12:55Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).<br />
<br />
== I want to use the keyboard and mouse with my laptop, but they’re connected to the screen. ==<br />
This is correct and by design. The keyboard and mouse are shared between the PC on the desk (if present) and the laptop connected by the USB-C cable. Control of the keyboard and mouse is passed through the USB-C cable to the laptop. Do not disconnect the keyboard and mouse to connect it to your laptop.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 8-9 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== My screens work fine but they're in the wrong arrangement ==<br />
This is normal and depends on the order in which your PC detects the screens. Your laptop will always be Screen 1 but the other two displays could be 2 and 3, or 3 and 2.<br />
<br />
[https://support.microsoft.com/en-us/windows/how-to-use-multiple-monitors-in-windows-329c6962-5a4d-b481-7baa-bec9671f728a#Category=Windows_10 See this page for how to rearrange your screens. ]<br />
<br />
== My Apple laptop will only disply to one screen ==<br />
This is by design on Apple M1 and M2-based systems. They will only output to one display. If you want an Apple laptop which displays to two screens, you need an M3, or an old Intel one. Or buy a USB-C dongle.<br />
<br />
[https://support.apple.com/en-us/117373 See here for details.]<br />
<br />
== I have a question not listed here. ==<br />
Either visit your local IT staff, or if not on site email [mailto:dide-it@imperial.ac.uk DIDE-IT].</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14282Working in the new School of Public Health building2024-04-05T19:11:26Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).<br />
<br />
== I want to use the keyboard and mouse with my laptop, but they’re connected to the screen. ==<br />
This is correct and by design. The keyboard and mouse are shared between the PC on the desk (if present) and the laptop connected by the USB-C cable. Control of the keyboard and mouse is passed through the USB-C cable to the laptop. Do not disconnect the keyboard and mouse to connect it to your laptop.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 8-9 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== My screens work fine but they're in the wrong arrangement ==<br />
This is normal and depends on the order in which your PC detects the screens. Your laptop will always be Screen 1 but the other two displays could be 2 and 3, or 3 and 2.<br />
<br />
[https://support.microsoft.com/en-us/windows/how-to-use-multiple-monitors-in-windows-329c6962-5a4d-b481-7baa-bec9671f728a#Category=Windows_10 See this page for how to rearrange your screens. ]<br />
<br />
== My Apple laptop will only disply to one screen ==<br />
This is by design on Apple M1 and M2-based systems. They will only output to one display. If you want an Apple laptop which displays to two screens, you need an M3, or an old Intel one. Or buy a USB-C dongle.<br />
<br />
== I have a question not listed here. ==<br />
Either visit your local IT staff, or if not on site email [mailto:dide-it@imperial.ac.uk DIDE-IT].</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14281Working in the new School of Public Health building2024-03-25T10:29:40Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).<br />
<br />
== I want to use the keyboard and mouse with my laptop, but they’re connected to the screen. ==<br />
This is correct and by design. The keyboard and mouse are shared between the PC on the desk (if present) and the laptop connected by the USB-C cable. Control of the keyboard and mouse is passed through the USB-C cable to the laptop. Do not disconnect the keyboard and mouse to connect it to your laptop.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 8-9 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== My screens work fine but they're in the wrong arrangement ==<br />
This is normal and depends on the order in which your PC detects the screens. Your laptop will always be Screen 1 but the other two displays could be 2 and 3, or 3 and 2.<br />
<br />
[https://support.microsoft.com/en-us/windows/how-to-use-multiple-monitors-in-windows-329c6962-5a4d-b481-7baa-bec9671f728a#Category=Windows_10 See this page for how to rearrange your screens. ]<br />
<br />
== I have a question not listed here. ==<br />
Either visit your local IT staff, or if not on site email [mailto:dide-it@imperial.ac.uk DIDE-IT].</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14280Working in the new School of Public Health building2024-03-25T10:29:07Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).<br />
<br />
== I want to use the keyboard and mouse with my laptop, but they’re connected to the screen. ==<br />
This is correct and by design. The keyboard and mouse are shared between the PC on the desk (if present) and the laptop connected by the USB-C cable. Control of the keyboard and mouse is passed through the USB-C cable to the laptop. Do not disconnect the keyboard and mouse to connect it to your laptop.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 8-9 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== My screens work fine but they're in the wrong arrangement ==<br />
This is normal and depends on the order in which your monitors detect the screens. Your laptop will always be Screen 1 but the other two displays could be 2 and 3, or 3 and 2.<br />
<br />
[https://support.microsoft.com/en-us/windows/how-to-use-multiple-monitors-in-windows-329c6962-5a4d-b481-7baa-bec9671f728a#Category=Windows_10 See this page for how to rearrange your screens. ]<br />
<br />
== I have a question not listed here. ==<br />
Either visit your local IT staff, or if not on site email [mailto:dide-it@imperial.ac.uk DIDE-IT].</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14279Working in the new School of Public Health building2024-03-23T16:04:23Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).<br />
<br />
== I want to use the keyboard and mouse with my laptop, but they’re connected to the screen. ==<br />
This is correct and by design. The keyboard and mouse are shared between the PC on the desk (if present) and the laptop connected by the USB-C cable. Control of the keyboard and mouse is passed through the USB-C cable to the laptop. Do not disconnect the keyboard and mouse to connect it to your laptop.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 8-9 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I have a question not listed here. ==<br />
Either visit your local IT staff, or if not on site email [mailto:dide-it@imperial.ac.uk DIDE-IT].</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14278Working in the new School of Public Health building2024-03-23T15:58:31Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).<br />
<br />
== I want to use the keyboard and mouse with my laptop, but they’re connected to the screen. ==<br />
This is correct and by design. The keyboard and mouse are shared between the PC on the desk (if present) and the laptop connected by the USB-C cable. Control of the keyboard and mouse is passed through the USB-C cable to the laptop. Do not disconnect the keyboard and mouse to connect it to your laptop.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I have a question not listed here. ==<br />
Either visit your local IT staff, or if not on site email [mailto:dide-it@imperial.ac.uk DIDE-IT].</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14277Working in the new School of Public Health building2024-03-23T15:55:09Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).<br />
<br />
== I want to use the keyboard and mouse with my laptop, but they’re connected to the screen. ==<br />
This is correct and by design. The keyboard and mouse are shared between the PC on the desk (if present) and the laptop connected by the USB-C cable. Control of the keyboard and mouse is passed through the USB-C cable to the laptop. Do not disconnect the keyboard and mouse to connect it to your laptop.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14276Working in the new School of Public Health building2024-03-23T15:52:21Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).<br />
<br />
== I want to use the keyboard and mouse with my laptop, but they’re connected to the screen. ==<br />
This is correct and by design. The keyboard and mouse are shared between the PC on the desk (if present) and the laptop connected by the USB-C cable. Control of the keyboard and mouse is passed through the USB-C cable to the laptop. Do not disconnect the keyboard and mouse to connect it to your laptop.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14275Working in the new School of Public Health building2024-03-23T15:28:27Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff (DIDE IT are in 512; EBS IT are in 714).<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14274Working in the new School of Public Health building2024-03-23T15:26:57Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, ''e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff.<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14273Working in the new School of Public Health building2024-03-23T15:26:29Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, '’e.g.'' to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff.<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14272Working in the new School of Public Health building2024-03-23T15:24:26Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports. ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens. ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, e.g. to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports. ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software. ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff.<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14271Working in the new School of Public Health building2024-03-23T15:22:09Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, e.g. to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff.<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14270Working in the new School of Public Health building2024-03-23T15:21:37Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link [https://imperial.officespacesoftware.com/visual-directory/floors/75 is here]. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ==<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, e.g. to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff.<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14269Working in the new School of Public Health building2024-03-23T14:27:49Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol, which curiously denotes it as a Thunderbolt port - not to be confused with a Lightning port, which you would expect from the logo).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ===<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, e.g. to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve an Apple laptop and I don’t have USB-C ports ==<br />
Yes you do, but Apple calls them Thunderbolt ports. You can identify the ports on your Apple laptop [https://support.apple.com/en-gb/109523 on this page]. If you want to read about the subtle differences between USB-C and Thunderbolt, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].<br />
<br />
== Nothing’s working. I’ve tried different ports and I’ve installed the software ==<br />
The most likely explanation is that the USB-C cable has come out of the back of the monitor. Carefully push it back in, or see your local IT staff.<br />
<br />
The next most likely explanation is that the main Dell screen (the one with the USB-C cable connected) is set to not display from the USB-C cable. Change the monitor source to be USB-C (it will be labelled as ‘USB-C (PD)’ - the PD means Power Delivery, meaning it will also charge your laptop).</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14268Working in the new School of Public Health building2024-03-23T14:17:43Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ===<br />
It’s a little socket in the side of your laptop, about the size of the charging socket on your phone.<br />
The role of a USB-C port is mostly for data transfer, e.g. to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.<br />
<br />
== I’ve got an Apple laptop and I don’t have USB-C ports ==<br />
Yes you do, but Apple calls them Thunderbolt ports. If you want to read about the subtle differences, [https://www.intel.com/content/www/us/en/architecture-and-technology/thunderbolt/thunderbolt-4-vs-usb-c.html read here].</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14267Working in the new School of Public Health building2024-03-23T14:12:48Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.<br />
<br />
== What’s a USB-C port and what does it do? ===<br />
It’s a little socket in the side of your laptop, about the size of the socket on your phone charging socket.<br />
The role of a USB-C port is mostly for data transfer, e.g. to memory sticks, but these days is also capable of charging and carrying video, audio and network traffic.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14266Working in the new School of Public Health building2024-03-23T14:09:30Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends. ICT will be blocking all access from Windows 10 computers once support expires.<br />
<br />
== I plugged into my USB-C port and nothing happens ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.] Once these packages are installed (installed by default on Dell laptops), restart your laptop and try again.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14265Working in the new School of Public Health building2024-03-23T14:06:51Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.<br />
<br />
= Troubleshooting =<br />
Sometimes, especially with older laptops, things might not go exactly to plan. Read on.<br />
<br />
== My laptop is so old that I don’t have any USB-C ports ==<br />
This has been a standard port for 7-8 years. If your laptop is older than this then it’s really time to upgrade. You also won’t be able to upgrade to Windows 11 in 2024 when all support for Windows 10 ends.<br />
<br />
== I plugged into my USB-C port and nothing happens ==<br />
First of all, try another port. Usually one of them is designated as a port supporting external displays (often indicated by a lightning symbol).<br />
<br />
If that fails, especially with HP laptops, then you’ll need to install the software to talk to the dock in the main Dell screen. [https://www.dell.com/support/home/en-uk/product-support/product/u2424he-monitor/drivers Go here and install the software.]</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14264Working in the new School of Public Health building2024-03-23T13:20:12Z<p>Admin: </p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work (laptop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.<br />
# '''(Optional): Network'''. In some cases, such as the private offices used exclusively by one staff member, the cable also handles their network connection.<br />
<br />
= How to work (desktop users) =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, you may sit at the keyboard and mouse and use the PC exactly as you have always done.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Working_in_the_new_School_of_Public_Health_building&diff=14263Working in the new School of Public Health building2024-03-23T13:16:55Z<p>Admin: Created page with "= Introduction = In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop. Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner. ''' Do not disconnect any cables. If you are unsure of how things work, speak direc..."</p>
<hr />
<div>= Introduction =<br />
In the new building, desks in the common areas are set up to allow any member of the SPH to work from any desk with their departmental laptop.<br />
<br />
Some desks have PCs on them. These are to allow remote users a fast wired presence to our DIDE HPC cluster, or else they are powerful workstations for GPU work. Do not interfere with these PCs unless you are the registered owner.<br />
<br />
''' Do not disconnect any cables. If you are unsure of how things work, speak directly to your local IT staff. '''<br />
<br />
= How to work =<br />
The first step is to book a desk. You may not use a desk unless you have booked it for the time period which you are sat at the desk. The booking system link has been emailed to each of you. Log in with your Imperial account.<br />
<br />
Once you have booked a desk, connect your laptop using the supplied USB-C cable. This single cable handles the following functions:<br />
<br />
# '''Power'''. Your laptop does not need its charger if it’s a USB-C type. Obviously if it’s the old barrel type you need to connect your charger to the white sockets on the desk.<br />
# '''Screens'''. The cable connects you to the two screens. If it does not, check that the main Dell screen which the cable connects to is set to have the input source as USB-C.<br />
# '''Keyboard and mouse'''. These accessories are connected to the screen and are used by both the desktop PC (if any) and the laptop. Do not disconnect these to connect them directly to your laptop.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=The_DIDE_IT_Team_Documentation&diff=14262The DIDE IT Team Documentation2024-03-23T13:03:04Z<p>Admin: </p>
<hr />
<div>= Common Topics in DIDE IT =<br />
These pages are for internal DIDE IT documentation. None of this information is sensitive, it is simply a list of procedures already present in the main ICT help pages but modified to be more useful to DIDE.<br />
<br />
== [[Working in the new School of Public Health building]] ==<br />
This section describes the new working practices for the department since the move in March 2024.<br />
<br />
== [[Printing]] ==<br />
This section describes how to install the mono and colour queues for the managed swipe printers.<br />
== [[Office 365]] ==<br />
This section explains the features of your Office 365 subscription. Note that all Office 365 services are managed by your Imperial account (username@ic.ac.uk), not your DIDE one.<br />
== [[Wi-Fi]] ==<br />
How to access Imperial's Wi-Fi network, whether you are a member of Imperial College (using Imperial-WPA), an academic guest (using EduRoam) or a visitor (using _The Cloud).<br />
== [[Remote access to DIDE]] ==<br />
In order to access DIDE resources from anywhere other than our wired internal network, you must either access via the zScaler Unified Access app.<br />
<br />
== [[Disk quotas]] ==<br />
All DIDE members have a default personal disk quota of 15GB. All research groups also have a research group share quota.<br />
== [[Hardware orders]] ==<br />
A guide to our stock equipment from HP and Dell.<br />
== [[Multi-Factor Authentication (MFA)]] ==<br />
How to enable extra security features on your Imperial account.<br />
== [[Anti-Virus Upgrades (Autumn 2022)]] ==<br />
Here we discuss how to move from our old security software to our new software.<br />
== [[Troubleshooting]] ==<br />
Sometimes things can go wrong. Here we list the most common problems not listed on the main Imperial ICT help pages and how to resolve them.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Remote_access_to_DIDE&diff=14261Remote access to DIDE2024-03-23T13:01:27Z<p>Admin: /* Remote Access to your Windows DIDE Desktop via ZScaler */</p>
<hr />
<div>These instructions are now obsolete. All remote access to DIDE is now via [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/unified-access/ Zscaler]. '''ZScaler replaces both the Pulse Secure VPN and the Remote Desktop Gateway'''.<br />
<br />
== Remote Access to your Windows DIDE Desktop via ZScaler ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
# Ensure that you've uninstalled Pulse Secure VPN, or have never installed it.<br />
# Ensure that you've installed ZScaler via above link.<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
== OBSOLETE SINCE ZSCALER - Remote Access to your Windows DIDE Desktop via Terminal Services Gateway ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
[[File:TS connection2.png|thumb|Steps 4 to 9]]<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Select the '''Advanced''' tab on the far right and click the '''Settings...''' button under Connect from anywhere.<br />
# Select the '''Use these RD Gateway server settings''' button.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the ''Server name'' field.<br />
# Select '''Allow me to select later''' in the ''Log-on method'' field. We'll add this later (see after these steps).<br />
# Ensure that '''Bypass RD Gateway server for local addresses''' is ''ticked''.<br />
# Ensure that '''Use my RD Gateway credentials for the remote computer''' at the bottom is ''unticked''. <br />
# Press the OK button<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
You will now have a new icon on your desktop and you never need to do the above again. To connect to your DIDE desktop, click the '''Connect''' button in your new Remote Desktop shortcut.<br />
<br />
You will now be asked for your Imperial credentials in order to access the Terminal Services Gateway. Enter your details as '''IC\username''' with your Imperial password.<br />
<br />
Once that allows you through you will be prompted for your DIDE credentials in order to access your PC. Enter your details as '''DIDE\username''' with your DIDE password. If this is the first time that you've connected to your DIDE PC for a while you'll get a certificate warning message. Tick the box and select '''Yes'''.<br />
<br />
If you don't get prompted for your DIDE account that probably means your PC is asleep. To wake it up [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/remotely-access-my-college-computer/wake-my-pc/ click here].<br />
<br />
'''N.B.''' If all the above seems too difficult, I've made a template file with all the settings. It is in the '''T:''' drive in folder '''IT\VPN''' and is called '''DIDE-Remote.RDP'''. Right-click and edit that to put your correct hostname and username in and all the other settings are done for you.<br />
<br />
== Remote Access to your Ubuntu DIDE Desktop via Terminal Services Gateway ==<br />
First of all, you will need to have enabled remote desktop in advance. [https://linuxconfig.org/ubuntu-remote-desktop-18-04-bionic-beaver-linux See here for example]. You should also check you have the Remmina app installed.<br />
<br />
# Open Remmina<br />
# Click the “+” button to add a new configuration, and give it a meaningful name.<br />
# Protocol: RDP - Remote Desktop Protocol<br />
# Open Remmina and on the Basic tab fill in the following details: <br />
#* Server: Your desktop PC FQDN (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''')<br />
#* User Name: '''Your Ubuntu username'''<br />
#* Password: '''Your Ubuntu password'''<br />
#* Domain: '''DIDE''' (only if your workstation is on the DIDE domain - very likely it's not).<br />
# Adjust colour depth to '''16 bit'''.<br />
# Go to to the Advanced Tab and set up the ICT RDP Gateway with the following details:<br />
#* RDG Server: '''ictsgw.cc.ic.ac.uk'''<br />
#* RDG username: '''Your IC username'''<br />
#* RDG password: '''Your IC password'''<br />
#* RDG domain: '''IC'''<br />
# Click '''Save''' and you will have a new configuration with the name you initially specified. <br />
# Click on this new entry to connect to your server.<br />
# Accept the certificate warnings. These should only appear the very first time you connect.<br />
# Your Ubuntu remote session should now be active.<br />
<br />
<br />
== Remote Access to your DIDE Desktop via Terminal Services Gateway on a Mac ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
<br />
# Start the '''Microsoft Remote Desktop 10''' app from the App Store and select '''Add desktop''' to add a new profile entry.<br />
# Enter a name for the remote desktop profile.<br />
# Enter the full name of your DIDE machine (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''').<br />
# Change the '''User Account''' drop down to Add User Account<br />
# Enter '''DIDE\username''' in the User name field.<br />
# Enter your DIDE password in the Password field.<br />
# Enter a friendly name if desired (optional)<br />
# Click '''Add''' and '''Save''' again.<br />
# Click the Cog in the top left of the screen then Preferences.<br />
# Click Gateways.<br />
# Click on the '''+''' symbol on the bottom left of the window.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the Server name field.<br />
# Select your Imperial '''username''' in the User name field.<br />
# Enter a name for the gateway in the Friendly Name field.<br />
# Click '''Save'''.<br />
# Close the screen.<br />
# Now choose the gateway you created from the Gateway dropdown list.<br />
# Click the pencil icon.<br />
# Under '''General''' select the gateway name in the Gateway field.<br />
# Click '''Save'''.<br />
# The main screen will display your new profile. Double click this to start a remote desktop session.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=The_DIDE_IT_Team_Documentation&diff=14260The DIDE IT Team Documentation2024-03-23T13:00:21Z<p>Admin: /* Remote access to DIDE */</p>
<hr />
<div>= Common Topics in DIDE IT =<br />
These pages are for internal DIDE IT documentation. None of this information is sensitive, it is simply a list of procedures already present in the main ICT help pages but modified to be more useful to DIDE.<br />
<br />
== [[Printing]] ==<br />
This section describes how to install the mono and colour queues for the managed swipe printers.<br />
== [[Office 365]] ==<br />
This section explains the features of your Office 365 subscription. Note that all Office 365 services are managed by your Imperial account (username@ic.ac.uk), not your DIDE one.<br />
== [[Wi-Fi]] ==<br />
How to access Imperial's Wi-Fi network, whether you are a member of Imperial College (using Imperial-WPA), an academic guest (using EduRoam) or a visitor (using _The Cloud).<br />
== [[Remote access to DIDE]] ==<br />
In order to access DIDE resources from anywhere other than our wired internal network, you must either access via the zScaler Unified Access app.<br />
<br />
== [[Disk quotas]] ==<br />
All DIDE members have a default personal disk quota of 15GB. All research groups also have a research group share quota.<br />
== [[Hardware orders]] ==<br />
A guide to our stock equipment from HP and Dell.<br />
== [[Multi-Factor Authentication (MFA)]] ==<br />
How to enable extra security features on your Imperial account.<br />
== [[Anti-Virus Upgrades (Autumn 2022)]] ==<br />
Here we discuss how to move from our old security software to our new software.<br />
== [[Troubleshooting]] ==<br />
Sometimes things can go wrong. Here we list the most common problems not listed on the main Imperial ICT help pages and how to resolve them.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Cluster_Documentation&diff=14259Cluster Documentation2024-02-28T16:58:58Z<p>Admin: </p>
<hr />
<div>* We have two high performance clusters for general use, both running Microsoft HPC 2012 R2, Update 3. The smaller, older cluster is [[fi--dideclusthn]] and the larger is [[fi--didemrchnb]]. <br />
<br />
* Many users in the department write code and packages in R. We have in-house packages that can allow an almost-interactive experience of running jobs on our high performance clusters, so if that sounds like what you'd like, you might as well stop reading here, and go straight to [https://github.com/mrc-ide/didehpc The DIDEHPC package] - or if you need to join the mrc-ide github org first, go to [https://github.com/mrc-ide/welcome The welcome page].<br />
<br />
If you're not using R, or you are interested in other ways, then continue...<br />
<br />
== Windows Users ==<br />
<br />
The cluster nodes are Windows 2012-R2 based. This is good news if you're a Windows user, and especially easy for PCs within the departmental DIDE domain. You have the option of using the MS client for launching jobs (see below), or you can use try the new [[HPC Web Portal]]. If you're preparing to use the cluster, and have doubts about whether it can run what you want - and whether it will be straightforward to develop and run on it, best talk to me and/or the IT guys first!<br />
<br />
== Linux and MAC Users ==<br />
<br />
There are two problems, but both can be at least partially overcome. The first is that there is no client for linux. The second is that it can only run windows binaries.<br />
<br />
=== Launching jobs on Linux/Mac ===<br />
<br />
* The best way might be the [[HPC Web Portal]], which lets you submit jobs through a webpage. Give it a try, and feedback to me any issues! <br />
<br />
* Alternatively, you can either (1) go down the VM route, installing a windows virtual machine, eg VMWare or Parallels, and follow the instructions below. (2) Remote Desktop to your friend's Windows machine. These tend not to be very convenient, but some users have submitted thousands of jobs this way.<br />
<br />
=== What Linux/Mac jobs can I run? ===<br />
<br />
* Many jobs may be platform indepedent. Where compiled code is involved though, you'll have some extra work to make sure the binary parts can be run in windows. For R, [https://github.com/mrc-ide/didehpc the tools] help you sort that out. For others, you may need to run a job on the cluster to do your builds. Also, for many linux applications, there is a windows port. Mileage can vary, but it's worth a try searching for that before you give up hope. Some of these might be installed on the cluster already - see below. Or some you may be able to download and call them from your run script.<br />
<br />
* For C code, again, we can run scripts on the cluster to build the binaries for you, if your code can be compiled with (say) the MinGW port of gcc. Cygwin can also be used (although seems less common these days) with accompanying dlls. Get in touch if you need further ideas; most things can be solved. <br />
<br />
* Matlab is a little more tricky - we have a range of colossal matlab runtimes on the cluster, but you'd need to compile a windows binary to use them, since the matlab platforms are distinct in every way. For this, probably find a friend, or get in touch, and we'll work out how you can compile for it.<br />
<br />
* There is also the option of running a Windows VM, which might solve all of the above problems.<br />
<br />
=== But I really want a unix cluster. ===<br />
<br />
* We have run linux clusters in the past ([[fi--didelxhn]]), but there was very low demand, whereas the other clusters are consistently well used. Accessing DIDE network shares from linux was a little harder in some aspects, but we did not implement the more traditional linux approach of having dedicated separate cluster storage that inputs and outputs would have to be staged on to, in order for the cluster to see them. Generally, the flexibility of the other clusters makes such things quite straightforward, and may even feel a pleasure if your background is with the less domain-integrated style of cluster architecture.<br />
<br />
* It has also become far easier to write software that builds on multiple platforms with good practice. Tools that only run on one platform are rare, and in some circumstances have raised questions about the quality of those packages. Alternatives are often available.<br />
<br />
* Where compilation is the issue - for example if you want to run binaries on the cluster, but your compilation machine runs linux, or is a mac, (Matlab via MCC, or C++ code for example), then get in touch, and there are a few ways we can think of solving that, either by getting someone else, or the cluster itself, to build your code.<br />
<br />
* There are also resources in [https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/computing/high-throughput-computing/ ICT) who run a linux cluster, with the file-staging approach, and also (being shared between a much larger pool of users) have rather stricter rules on job durations and resources than we provide locally in DIDE.<br />
<br />
== Getting Started ==<br />
<br />
=== Getting access to the cluster ===<br />
<br />
Send a mail to Wes (w.hinsley@imperial.ac.uk) requesting access to the cluster. He will add you to the necessary groups, and work out what cluster you should be running on. Unless you are told otherwise, this will be fi--dideclusthn. But if you have been told otherwise, then whenever you see fi--dideclusthn, replace it with the cluster name you’ve been given, either mentally or (less effectively) on your screen with tip-ex.<br />
<br />
=== Windows: Install the HPC client software ===<br />
<br />
* Run \\fi--dideclusthn.dide.ic.ac.uk\REMINST\setup.exe. <br />
* Confirm '''Yes, I really want to run this'''.<br />
* There's an introductory screen, and you click '''Next'''<br />
* You have to tick the '''Accept''', then click '''Next'''<br />
* If it gives you any choice, select '''Install only the client utilities'''<br />
* '''Next''', '''Next''', '''Next'''<br />
* Select '''Use Microsoft Update''' and then '''Next'''<br />
* And then '''Install'''<br />
* And possibly '''Install''' again – if it had to install any pre-requisites first.<br />
* And '''Finish''', and the client installation is done.<br />
* To check everything is alright so far, open a '''NEW command prompt window'''.<br />
* (Because the installer adds things to the path, which only take effect on '''NEW comamnd windows'''! :-)<br />
* (Find it under Accessories, or you can use Start, Run, '''cmd''').<br />
* Type <code>job list /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
And it will list the jobs you have running, which will be none so far!<br />
<br />
<pre><br />
Id Owner Name State Priority Resource Request<br />
---------- ------------- ----------------- ------------ ----------- -------- -------<br />
<br />
0 matching jobs for DIDE\user<br />
</pre><br />
<br />
=== Windows: Using a non-domain machine ===<br />
<br />
If you're using a machine that isn't actually logged into DIDE, then the client software will have a problem working out who you are. In this case, there are two things you need to do.<br />
* Check you've really got a DIDE username. If you don't know what it is, talk to your lovely IT team.<br />
* Connect to the DIDE VPN using ZScaler from https://uafiles.cc.ic.ac.uk/, and login to that with your IC account - but you'll still need your DIDE account to access DIDE servers after the ZScaler connection is up and running.<br />
* Now we'll open a command window using "runas" - which lets you choose which identity the system thinks you are within that command window:-<br />
<pre><br />
runas /netonly /user:DIDE\user cmd<br />
</pre><br />
* (Change user to your own login obviously). It will ask for your password, then open a new command window. In that window, you'll be able to do cluster-related things as your DIDE username.<br />
<br />
=== Windows: Launching and cancelling jobs ===<br />
==== Windows: Command Line ====<br />
<br />
Suppose you have a file called "run.bat" in your home directory, which does what you want to run on the cluster. Let's say it's a single-core, very simple job. We’ll discuss what should be inside "run.bat" later. To submit your job on the cluster, at the command prompt, type this (all on one line) - or put it in a file called "launch.bat", and run it:-<br />
<br />
<code>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1 \\qdrive.dide.ic.ac.uk\homes\user\run.bat</code><br />
<br />
If it's the first time you’ve run a job - or if you've recently changed your password, then it might ask you for your DIDE password and offer to remember it. Otherwise, it will just tell you the ID number of your job.<br />
<br />
<pre>Enter the password for ‘DIDE\user’ to connect to ‘FI—DIDECLUSTHN’:<br />
Remember this password? (Y/N)<br />
job has been submitted. ID: 123<br />
</pre><br />
<br />
If you want to remove the job, then:- <code>job cancel 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
Or view its details with <code>job view 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
==== Windows: Job Manager GUI ====<br />
<br />
Alternatively to the command-line, you can use the job management software, rather than the command-line. The advantage is that it’s a GUI. The disadvantage is, as in all GUIs, you may not feel totally sure you know what it’s up to – where most of the time the things you want to do might not be very complex, as above.<br />
<br />
The job management software will be on your start menu, as above. All the features are under the “Actions” menu, and hopefully it will be self explanatory. Read the details below about launching, and you'll find the interface bits that do it in the Job Manager. However, you may find over time, especially if you run lots of jobs, that learning to do it the scripted way with the command-line can be quicker.<br />
<br />
<br />
== Information for running any job ==<br />
<br />
=== Visibility ===<br />
<br />
Any executable/batch file that the cluster will run must be somewhere on the network that the cluster has access to, when it logs in as you. This amounts to any network accessible drive that you have access to when you login – including network shares, such as the temp drive, your home directory, and any specific network space set up for your project.<br />
<br />
After a job is launched, all paths are relative to the '''/workdir:''' specified on the job submit command-line. Windows does not usually allow a network path to be the current directory, and MS-HPC somehow gets around this. But if in doubt, use fully-qualified paths to specify where files should be read from, or written to. You may want to write code that takes the paths from a parameter file, or as a command-line parameter, to give as much flexibility as possible. <br />
<br />
Your home directory, and the temp drive are on departmental servers that don't have an especially fast connection to the cluster. So avoid having thousands of jobs all reading/writing to those places at once. Additionally, the temp drive is shared between all users, and your home drive has a limited quota. It is also backed up every day, so avoid using it as scratch storage for results you don’t actually want to keep. Generally, the shares you'll want to use from the clusters will be ones containing the word "NAS", which the big cluster has fast access to.<br />
<br />
=== Interactivity ===<br />
<br />
The job must run in an entirely scripted, unattended way. It must not require any key presses or mouse clicks, or other live interactivity while it runs. So jobs generally will read some sort of input (from a file, or from the way you run the job), do some processing, and write the results somewhere for you - all without intervention.<br />
<br />
== Launching jobs ==<br />
<br />
Jobs can be launched either through the Job Manager interface, or through the command line tools, which offer greater flexibility. We'll describe the command line method here; if you want to use the GUI, then it'll be a case of finding the matching boxes... Below are the specifics for our clusters. For more information, simply type <code>job</code> on the commandline, or <code>job submit</code> for the list of submission-related commands.<br />
<br />
=== FI--DIDECLUSTHN vs FI--DIDEMRCHNB ===<br />
<br />
Job submissions, as shown below, must specify a "job template", which sets up a load of default things to make the job work. On fi--dideclusthn, the job templates are called 4Core, 8Core and GeneralNodes, which will respectively force jobs to run on the 4-core nodes, the 8-core nodes, or on any machine available.<br />
<br />
On fi--didemrchnb, you can set the job template to be... 8Core, 12Core, 16Core, 12and16Core, and GeneralNodes - which hopefully are fairly self-explanatory. There are a couple of other job templates, (24Core and Phi), but those are a bit special purpose for now, so don't use them!<br />
<br />
=== Job Submission ===<br />
<br />
Job submissions from the command line can take the following form (all on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /stdout:\\path\to\stdout.txt<br />
/stderr:\\path\to\stderr.txt /numcores:1-1 /jobtemplate:4Core \\path\to\run.bat</pre><br />
<br />
=== The /singlenode argument ===<br />
<br />
In MS HPC (2012), Microsoft finally added a tag to allow you to say that the 'n' cores you requested must be on the same computer. Therefore, if you know precisely how many cores you want, then use the following (on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:true <br />
/jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
However, there is one bug with this, for the specific case where you request a whole node, regardless of how many cores it has. In this case, oddly, you have to disable single node:-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:false <br />
/numnodes:1 /jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
== Languages and libraries supported ==<br />
<br />
A number of languages, and different versions of languages are available on the cluster. The sections below refer to your "run.bat" file - a batch file that you will create which will get run by a cluster node when a suitable one is available. The commands described below are to be put in this "run.bat" file, and they add various directories to the path, so that the software you want will be added to the path. <br />
<br />
=== C/C++ === <br />
<br />
A number of Microsoft C++ and Intel C++ runtimes are installed, but it's usually better to try and avoid using them, and make your executable as stand-alone as possible. If it requires any external libraries that you've had to download, then put the .dll file in the same directory as the .exe file. If you use Microsoft Visual Studio, in Project Preferences, C/C++, Code Generation, make sure the Runtime Library is Multi-threaded (/MT) – the ones with DLL files won’t work. Even so, on recent versions of the Intel and Microsoft C compilers, "static linking" doesn’t really mean static when it comes to OpenMP, and you’ll have to copy some DLLs and put them next to your EXE file. See the OpenMP section below.<br />
<br />
The cluster nodes are all 64-bit, but they will run 32-bit compiled code. Just make sure you provide the right DLLs!<br />
<br />
Here is a document about [[Using the Xeon Phi]]<br />
<br />
Here is a document about [[Using MPI]]<br />
<br />
And if you want to use libraries such as GSL, then try [[C/C++ Libraries for Windows]]<br />
<br />
=== Java ===<br />
<br />
Java doesn't strictly need installing; you can setup any Java Runtime Environment you like, from Oracle, or anywhere. If you want to do this, then, (if you were to use the Server JRE from Oracle), something similar to the following will add java.exe and jvm.dll to the path, and set the JAVA_HOME environment variable, which most Java-using software will appreciate.<br />
<br />
set JAVA_HOME=\\my\path\to\jre1.8.0<br />
set path=%JAVA_HOME%\bin;%JAVA_HOME%\bin\server;%path%<br />
<br />
For convenience, if you're happy with Azul's OpenJDK Java 1.8.0 update 308 (September 2021), then <code>call setJava64</code> in your run script will set this up for you. Incidentally the above will also add what's needed for using the rJava package in R - in which case you'll perhaps also want something like <code>call setR64_4_1_0</code> in your run script.<br />
<br />
=== Perl ===<br />
<br />
Strawberry Perl 64-bit portable, v5.30.1.1. Put <code>call setPerl</code> at the top of your script.<br />
<br />
=== R ===<br />
<br />
'''New tools are coming''': [http://dide-tools.github.io/didehpc/vignettes/quickstart.html see here]. The instructions below are still valid if you want to do this manually.<br />
<br />
<br />
You can R jobs on all the clusters. This is the latest wisdom on how to do so – thanks to James, Jeff, Hannah and others. <br />
<br />
First, if you are wanting to use packages, then set up a package repository on your home directory by running this in R.<br />
<br />
<code>install.packages("<package>",lib="Q:/R")</code><br />
<br />
Now, write your run script. Suppose you have an R script in your home directory: Q:\R-scripts\test.R. And suppose you’ve set up your repository as above. Your run.bat should then be:-<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\test.R<br />
</pre><br />
Various packages require various R versions, and the cluster supports a few versions. To choose which one, change the first line of the script above to one of these – 32-bit or 64-bit versions of R releases. Purely for amusement, R's codenames are shown here too. <br />
<pre><br />
call setr32_2_13_1.bat call setr64_2_13_1.bat (anyone know the codename?)<br />
call setr32_2_14_2.bat call setr64_2_14_2.bat (Gift-getting season)<br />
call setr32_2_15_2.bat call setr64_2_15_2.bat (Trick or treat)<br />
call setr32_3_0_1.bat call setr64_3_0_1.bat (Good Sport)<br />
call setr32_3_0_2.bat call setr64_3_0_2.bat (Frisbee Sailing)<br />
call setr32_3_1_0.bat call setr64_3_1_0.bat (Spring Dance)<br />
call setr32_3_1_2.bat call setr64_3_1_2.bat (Pumpkin Helmet)<br />
call setr32_3_2_2.bat call setr64_3_2_2.bat (Fire Safety)<br />
call setr32_3_2_3.bat call setr64_3_2_3.bat (Wooden Christmas Tree)<br />
call setr32_3_2_4.bat call setr64_3_2_4.bat (Very Secure Dishes - Revised Version - R later renamed this to 3.2.5)<br />
call setr32_3_3_1.bat call setr64_3_3_1.bat (Bug in your hair)<br />
call setr32_3_3_2.bat call setr64_3_3_2.bat (Sincere Pumpkin Patch)<br />
call setr32_3_4_0.bat call setr64_3_4_0.bat (You Stupid Darkness)<br />
call setr32_3_4_2.bat call setr64_3_4_2.bat (Short Summer)<br />
call setr32_3_4_4.bat call setr64_3_4_4.bat (Someone to lean on)<br />
call setr32_3_5_0.bat call setr64_3_5_0.bat (Joy in Playing)<br />
call setr32_3_5_1.bat call setr64_3_5_1.bat (Feather Spray)<br />
call setr32_3_5_3.bat call setr64_3_5_3.bat (Great Truth)<br />
call setr32_3_6_0.bat call setr64_3_6_0.bat (Planting of a Tree)<br />
call setr32_3_6_1.bat call setr64_3_6_1.bat (Action of the Toes)<br />
call setr32_3_6_3.bat call setr64_3_6_3.bat (Holding the Windsock)<br />
call setr32_4_0_2.bat call setr64_4_0_2.bat (Taking Off Again)<br />
call setr32_4_0_3.bat call setr64_4_0_3.bat (Bunny-Wunnies Freak Out)<br />
call setr32_4_0_5.bat call setr64_4_0_5.bat (Shake and Throw)<br />
</pre><br />
<br />
The following versions are *not* available on the cluster at the moment - but here are their names anyway, as it seemed a shame to miss them out. Let me know if you desperately need one of these.<br />
<br />
<pre><br />
2.14.0 Great Pumpkin <br />
2.14.1 December Snowflakes<br />
2.15.0 Easter Beagle<br />
2.15.1 Roasted Marshmallow<br />
2.15.3 Security Blanket<br />
3.0.0 Masked Marvel<br />
3.0.3 Warm Puppy<br />
3.1.1 Sock it to me<br />
3.1.3 Smooth Sidewalk<br />
3.2.0 Full of Ingredients<br />
3.2.1 World famous astronaut<br />
3.2.4 Very Secure Dishes (the non-revised version)<br />
3.3.0 Supposedly Educational<br />
3.3.3 Another Canoe<br />
3.4.1 Single Candle<br />
3.4.3 Kite-Eating Tree<br />
3.5.2 Eggshell Igloo<br />
3.6.2 Dark and Stormy Night<br />
4.0.0 Arbor Day<br />
4.0.1 See Things Now<br />
4.0.4 Lost Library Book<br />
</pre><br />
<br />
It also seems that different R versions put their packages in different structures – sometimes adding "win-library" into there for fun. Basically, R_LIBS and R_LIBS_USER should be paths to a folder that contains a list of other folders, one for each package you've installed.<br />
<br />
IMPORTANT: R_LIBS and R_LIBS_USER paths must NOT contain quotes, nor spaces. If the path to your library contains spaces, you need to use old-fashioned 8-character names. If in your command window, you type <code>dir /x</code>, you’ll see the names – Program Files tends to become PROGRA~1 for instance. <br />
<br />
==== Passing parameters to R scripts ====<br />
Passing parameters to R scripts means you can have fewer versions of your R and bat files and easily run whole sets of jobs. You can get parameters into R using Rscript (but not Rcmd BATCH, I think) as follows. In the run.bat example above, the Rscript statement becomes:<br />
<pre><br />
Rscript Q:\R-scripts\test.R arg1 arg2 arg3<br />
</pre><br />
Within the R code, the arguments can be recovered using <br />
<pre><br />
args <- commandArgs(trailingOnly = TRUE)<br />
<br />
outFileName <- args[1] ## name of output file.<br />
dataFileName <- args[2] ## name of local data file. <br />
currentR0 <- as.numeric(args[3]) ## convert this explicitly to number. <br />
</pre><br />
If you want your arguments to have a particular type, best to explicitly convert (see R0 above). Better still, you can pass parameters directly into the batch file that runs the R script. Command line arguments can be referenced within the batch file using %1, %2, etc. For example, if you have a batch file, runArgs.bat:<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\%1.R %2 %3 %4<br />
</pre><br />
then<br />
<pre><br />
runArgs.bat myRScript arg1 arg2 arg3<br />
</pre><br />
will run the R script myRScript.R and pass it the parameters arg1, arg2 and arg3. The batch file runArgs.bat is now almost a '''generic R script runner'''.<br />
<br />
=== OpenMP ===<br />
<br />
This has been tested with C/C++ - but the same should apply to other languages such as fortran, that achieve multi-threading with a DLL file.<br />
<br />
==== Microsoft's C/C++ Compiler ====<br />
You will need to copy vcomp100.dll (for Visual Studio 2010 – it maybe vcomp90.dll for 2008), into the same directory as your executable. You can usually find the dll file in a directory similar to:- C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist\x64\Microsoft.VC100.OpenMP. Extrapolate for future versions! Also make sure that you've enabled OpenMP in the Properties (C++/Language).<br />
<br />
==== GCC or MinGW ====<br />
Usually, this applies to Eclipse use. In project properties, the C++ command “g++ -fopenmp”, the C command “gcc –fopenmp”, and in the Linker options, under Miscellaneous, Linker flags, put “-fopenmp” too. Copy the OpenMP DLLs from MinGW into the same directory as your final executable. You may find the dlls are in C:\MinGW\Bin\libgomp-1.dll and in the same place, libpthread-2.dll, libstdc++-6.dll and libgcc_s_dw2-1.dll.<br />
<br />
==== Intel C++ Compiler ====<br />
Copy the libiomp5md.dll file from somewhere like C:\Program Files (x86)\Intel\ComposerXE-2011\redist\intel64\compiler\libiomp5md.dll to the same place as your executable. And in Visual Studio, make sure you've enabled OpenMP in the project properties.<br />
<br />
==== How many threads? ====<br />
The OpenMP function omp_max_threads() returns the number of ''physical'' cores on a machine, not the number of cores ''allocated'' to your job. To determine how many cores the scheduler actually allocated to you, use the following code to dig up the environment variable CCP_NUMCPUS, which will be set by the cluster management software:- <br />
<pre><br />
int set_NCores() {<br />
char * val;<br />
char * endchar;<br />
val = getenv("CCP_NUMCPUS");<br />
int cores = omp_get_max_threads();<br />
if (val!=NULL) cores = strtol (val,&endchar,10);<br />
omp_set_num_threads(cores);<br />
return cores;<br />
}<br />
</pre><br />
<br />
=== WinBugs (or OpenBugs) ===<br />
<br />
They are similar, but OpenBugs 3.2.2 is the one we've gone for. Something like this as your run.bat script will work:-<br />
<pre><br />
call setOpenBugs<br />
openbugs \\path\to\script.txt /HEADLESS<br />
</pre><br />
<br />
=== MatLab ===<br />
<br />
There are various ways of producing a non-interactive executable from Matlab. Perhaps the simplest (not necessarily the best performance) way is to use “mcc.exe” – supplied with most full versions of Matlab, including the Imperial site licence version that you've probably got.<br />
<br />
==== Use mcc.exe to compile your code ====<br />
<br />
Use windows explorer to navigate to the folder where the “.m” files are for the project you want to compile. Now use a good text editor to create a file called “compile.bat” in that folder. It should contain something similar to the following:-<br />
<br />
<code>mcc -m file1.m file2.m file3.m -o myexe</code><br />
<br />
Don’t copy/paste the text from this document by the way – Word has a different idea of what a dash is to most other software, and will probably replace the two dashes with funny characters. so you have to list every single .m file that your project needs, after the’-m’. If you save this file, then double-click on it, then it will think for some while, and produce “myexe.exe” in this example. Copy your .exe file into a network accessible place as usual.<br />
<br />
==== Launch on the cluster. ====<br />
<br />
The launch.bat file will be exactly the same as before – see page 2. The run script will then start with a line that tells the cluster which version of Matlab you used to compile the cluster. Below is the table of different versions. The runtimes are huge and cumbersome to install on the cluster, so as a result I haven’t installed every single one. If you need one that’s not listed, get in touch.<br />
<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! Matlab Version<br />
! First line of run script<br />
|- <br />
| R2009b<br />
| call useMatLab79<br />
|- <br />
| R2010a<br />
| call useMatLab713<br />
|- <br />
| R2011a<br />
| call useMatLab715<br />
|- <br />
| R2011b<br />
| call useMatLab716<br />
|- <br />
| R2012a (64-bit)<br />
| call useMatLab717_64<br />
|- <br />
| R2012b (64-bit)<br />
| call useMatLab80_64<br />
|- <br />
| R2013a (64-bit)<br />
| call useMatLab81_64<br />
|- <br />
| R2014a (64-bit)<br />
| call useMatLab83_64<br />
|- <br />
| R2014b (64-bit)<br />
| call useMatLab84_64<br />
|- <br />
| R2015a (64-bit)<br />
| call useMatLab85_64<br />
|- <br />
| R2015b (64-bit)<br />
| call useMatLab90_64<br />
|- <br />
| R2016b (64-bit)<br />
| call useMatLab91_64<br />
|- <br />
| R2017a (64-bit)<br />
| call useMatLab92_64<br />
|- <br />
| R2019a (64-bit)<br />
| call useMatLab96_64<br />
|- <br />
| R2021a (64-bit)<br />
| call useMatLab910_64<br />
|- <br />
| R2022a (64-bit)<br />
| call useMatLab912_64<br />
|}<br />
<br />
=== Python ===<br />
<br />
* The most recent version of each major release is on the cluster. Different python versions within the same release (e.g., 3.5.1 and 3.5.2) do not behave nicely together in automated installation, so if you have a desperate need for a previous minor version, get in touch. While 2.7.18 is available, we '''strongly''' recommend migrating away from Python 2.7, as it has been officially discontinued on 1st January 2020. Experiences may vary, and future success is not guaranteed.<br />
<br />
<pre><br />
2.7.18 64-bit call set_python_27_64.bat<br />
3.5.9 64-bit call set_python_35_64.bat<br />
3.6.15 64-bit call set python_36_64.bat<br />
3.7.17 64-bit call set_python_37_64.bat<br />
3.8.17 64-bit call set python_38_64.bat<br />
3.9.17 64-bit call set_python_39_64.bat<br />
3.10.12 64-bit call set python_310_64.bat<br />
3.11.4 64-bit call set_python_311_64.bat<br />
<br />
</pre><br />
<br />
* You'll most likely want to use some packages, so you'll need to have a network-accessible location to put your packages in. If you're running a version of Windows with Python that matches one installed on the cluster, then you'll be able to share that package repository between the two. But if you're running a different version, or a different operating system, or a different Python version, then you'll need to keep your local package repository separate from the one the cluster uses.<br />
* Here comes the basic script to run some python. Suppose I decide to put my Python packages for the cluster in '''\\qdrive.dide.ic.ac.uk\homes\username\PythonCluster37_64''' (which is Q:\PythonCluster37_64 on a DIDE Windows PC). To run some python, something like the following:-<br />
<br />
call set_python_37_64<br />
set PYTHONPATH=\\qdrive\homes\username\PythonCluster37_64<br />
python \\qdrive\homes\username\Python\MyScript.py<br />
<br />
or if you want to do this in a programmatic way...<br />
<br />
import sys<br />
sys.path = ['\\\\qdrive\\homes\\username\\PythonCluster37_64'] + sys.path<br />
<br />
==== Packages ====<br />
<br />
* If you want to install a package like numpy, that's readily available through pip, then you can launch a batch file like this:-<br />
call set_python_36_64<br />
pip install numpy --target \\qdrive\homes\username\PythonCluster36_64<br />
* or alternatively, you could write a python script that looks like this, and launch it as normal:-<br />
import pip<br />
pip.main(['install','numpy','--target','\\\\qdrive\\homes\\username\\PythonCluster36_64'])<br />
<br />
* And if you're running the same Python version on a Windows machine, then you can do these two locally, without submitting a cluster job to do them for you.<br />
<br />
* And next time you submit a python job on the cluster, python will look in the place set by PYTHONPATH, and will find the packages you've just installed when you import them.<br />
<br />
=== Beast and Beagle ===<br />
<br />
We've got a couple of versions available. For BEAST 1.75 with BEAGLE 1.0: (the x y z represents all the command-line arguments you'll want to pass to beast)<br />
<pre><br />
call setbeast<br />
call beast-beagle x y z<br />
</pre><br />
<br />
If you want BEAST 1.82 or 1.83 with BEAGLE 2.1:<br />
<pre><br />
call setbeast_182 call setbeast_183<br />
call beast-beagle_182 x y z call beast-beagle_183 x y z<br />
</pre><br />
<br />
And if you want BEAST 2.3.2 with BEAGLE 2.1 (note that BEAST 2 is a different, er, beast to BEAST 1, regarding packages and compatibilities, so only use BEAST 2 if you know that's what you want to use).<br />
<pre><br />
call setbeast_232<br />
call beast-beagle_232 x y z<br />
</pre><br />
<br />
OpenCL is installed on all the nodes, which means Beagle will pick up the Intel Xeon processors and use what it can on them. It will also detect the Xeon Phis where they're available. If that's something you'd be particularly interested in using, drop me a line.<br />
<br />
=== Blast ===<br />
<br />
Drop me a line to join the party testing this, and to get added to the (large) BlastDB network share.<br />
<br />
<pre><br />
call setblast<br />
</pre><br />
<br />
will set the `BLASTDB` environment variable to point to what we current have of the `v5` database, and the Blast tools (2.9.0, 64-bit) will be added to the path. So you can <br />
then call `blastp`, `blastn`, `blastx` setc.<br />
<br />
=== BOW (BioInformatics on Windows) ===<br />
<br />
A number of Bio-informatics tools are ready to go. These used to be provided together on codeplex, but that has since been retired. Ugene is one project that<br />
maintains windows builds of them. Start your run.bat file with <code>call setBOW</code> to add these to the path:-<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File<br />
! Version<br />
|- <br />
| SamTools.exe<br />
| 0.1.19-44429cd<br />
|- <br />
| BCFTools.exe<br />
| 0.1.19-44428cd<br />
|- <br />
| bgzip<br />
| ?<br />
|- <br />
| razip<br />
| ?<br />
|- <br />
| bwa<br />
| 0.7.17-r1188<br />
|-<br />
| tabix<br />
| 0.2.5 (r1005)<br />
|-<br />
| minimap2<br />
| 2.24 (r1122)<br />
|-<br />
| raxml<br />
| 8.2.12 (best for each node)<br />
|-<br />
| raxml_sse3<br />
| 8.2.12 (pthreads+sse3)<br />
|-<br />
| raxml_avx<br />
| 8.2.12 (pthreads+avx)<br />
|-<br />
| raxml_avx2<br />
| 8.2.12 (pthreads+avx2)<br />
|}<br />
<br />
Note that BWA-MEM is hardwired to use a linux-only feature, so will produce an odd message and not do what you want if you use that mode on Windows. However, it seems that minimap2 may be a faster and more accurate way<br />
of doing what bwa does in any case, and that seems to have no such limitation.<br />
<br />
Also, note that raxml.exe is a copy of the most optimised build for the particular cluster node the job is running on. fi--dideclusthn only supports SSE3;<br />
fi--didemrchnb varies a little; the 32-cores support AVX2, the 24, 20 and 16 core machines support AVX, and only the oldest 12-core nodes only support SSE3. You don't have to worry about, except to expect performance to vary between them.<br />
<br />
=== GATK and Picard ===<br />
<br />
These both rely on Java, so put this in your run script:<br />
<br />
<pre><br />
call setJava64<br />
call setgatk<br />
</pre><br />
<br />
and then <code>gatk</code> in your scripts will use Python 3.7 to call the argument handling wrapper for GATK 4.2.2.0, and <code>picard</code> maps to the picard (2.26.2) jar file.<br />
<br />
=== FastQC ===<br />
<br />
This one has a slight eccentricity that instead of passing normal command-line args, it uses java's -D command-line options to buffer them out of an environment variable<br />
called _JAVA_OPTIONS. Strange but true. The ZIP comes with a bash script, which I'll reimplement if anyone actually wants me to. But in the meantime, it's easy to work <br />
out the arguments by inspection. For example, most likely usage:-<br />
<br />
<pre><br />
call setFastQC<br />
<br />
set _JAVA_OPTIONS='-Dfastqc.output_dir=%OUT%'<br />
call fastqc thing.fq<br />
</pre><br />
<br />
=== MAFFT ===<br />
<br />
For version 7.212 (Win64), write <code>call setMAFFT.bat</code> at the top of your run script.<br />
<br />
=== Applied-Maths Open Source ===<br />
<br />
Write <code>call setAppliedMaths.bat</code> at the top of your run script, to add these to the path:<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File(s)<br />
! Description<br />
! Version<br />
|- <br />
| velvetg_mt_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_mt_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| ray_x64.exe<br />
| Ray 64-bit (for MPI)<br />
| 1.6<br />
|-<br />
| ray_x86.exe<br />
| Ray 32-bit (for MPI)<br />
| 1.6<br />
|-<br />
| Mothur_x64.exe<br />
| Mothur 64-bit (for MPI)<br />
| 1.25.1<br />
|-<br />
| Mothur_x86.exe<br />
| Mothur 32-bit (for MPI)<br />
| 1.25.1<br />
|}<br />
<br />
=== Tensorflow ===<br />
<br />
Here is an example of getting Tensorflow (as a python package) available from an R script, using the cluster to install everything for you. It only works for Tensorflow 2.0 - older versions and newer versions seem to want combinations of DLLs that I can't satisfy. The following are I think minimal steps to get Tensorflow running from Python and R.<br />
<br />
* Create a folder for your python packages in your home directory (so the cluster can see it. I'm calling mine <code>Q:/python_37_repo</code> - and the cluster will call it \\fi--san03.dide.ic.ac.uk\homes\wrh1\python_37_repo - change wrh1 to your DIDE username of course.<br />
* Create a similar folder for R packages, which for me is <code>Q:/R4</code><br />
* And a third folder, Q:/Tensorflow to do this work in.<br />
* Download these DLLs and put them in the Q:/Tensorflow folder - at present they seem to be the ones Python/Tensorflow wants. Let's hope for the best... Download [https://mrcdata.dide.ic.ac.uk/resources/msvcp140.dll msvcp140.dll] and [https://mrcdata.dide.ic.ac.uk/resources/msvcp140_1.dll msvcp140_1.dll] into your <code>Q:/Tensorflow</code> folder.<br />
<br />
* Now, create a script <code>Q:/Tensorflow/prepare.bat</code>. We're going to launch this on the cluster, and get it to wire up the packages we want. Replace USER with your username...<br />
<br />
<pre><br />
net delete Q: /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set TMPDIR=%TEMP%<br />
set PYTHONPATH=\\qdrive.dide.ic.ac.uk\homes\USER\python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
pip install --no-cache-dir tensorflow==2.00 --target Q:\python_37_repo<br />
RScript Q:\Tensorflow\install_tensorflow.R<br />
</pre><br />
<br />
* Create also <code>Q:/Tensorflow/install_tensorflow.R</code> contains:<br />
<pre><br />
install.packages(c("reticulate", "tensorflow"),lib="Q:/R4",repos="https://cran.ma.imperial.ac.uk/")<br />
</pre><br />
<br />
* Now submit the prepare job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> in the big window will do it. That will take a while, but you should notice lots of new files in your Q:/python_37_repo folder. You'll only need to do this once (unless you delete your Python repo folder, or it randomly breaks and you start again...)<br />
<br />
* Next, create a file <code>Q:/Tensorflow/run.bat</code> containing the following the two "call" lines near the end are the ones that do some "work" in python or R, so this script basically a template for running your future jobs. Just change those two calls to set off the python or R code you want to run that uses Tensorflow.<br />
<br />
<pre><br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
net use Q: /delete /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set PYTHONPATH=Q:/python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call python Q:\tensorflow\test_tensorflow.py<br />
call RScript Q:\tensorflow\test_tensorflow.R<br />
</pre><br />
<br />
* And finally, you'll need <code>Q:\tensorflow\test_tensorflow.py</code> and <code>Q:\tensorflow\test_tensorflow.R</code> to exist. Here are minimal tests that everything is wired up. <code.test_tensorflow.py</code> first:-<br />
<br />
<pre><br />
import tensorflow as tf;<br />
print(tf.reduce_sum(tf.random.normal([1000, 1000])));<br />
</pre><br />
<br />
and <code>Q:\Tensorflow\test_tensorflow.R</code> contains:<br />
<br />
<pre><br />
library(reticulate)<br />
reticulate::py_config()<br />
library(tensorflow)<br />
tf$constant('Hello, TensorFlow!')<br />
tf$Variable(tf$zeros(shape(1L)))<br />
</pre><br />
<br />
* Now submit the run job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> in the big window will do it - and perhaps paste <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\output.txt</code> in the stdout box to collect the input and see how it gets on. You should see sensible output from both tests.<br />
<br />
== Launching a job ==<br />
<br />
<br />
== Submitting many jobs ==<br />
<br />
Suppose you write an exe that you might run with… Mysim.exe init.txt 1 2 3, and you want to run it many times with a range of parameters. One way of many, is to write a launch.bat file that will run “job submit” separately, for example (thanks Tini/James!):-<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set initFile=\\networkshare\job\init.txt<br />
set exeFile=\\networkshare\job\mysim.exe<br />
<br />
%SubDetails% %exeFile% %initfile% 1 2 3<br />
%SubDetails% %exeFile% %initfile% 4 5 6<br />
%SubDetails% %exeFile% %initfile% 7 8 9<br />
</pre><br />
<br />
Suppose the job you want to run is an R script? To specify arguments to an R script, you have to add ′--args a=1 b=2′ - so… you might make launch.bat like this:-<br />
<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set rbatFile=\\networkshare\R-scripts\run.bat<br />
<br />
%SubDetails% %rbatFile% 1 2<br />
%SubDetails% %rbatFile% 3 4<br />
</pre><br />
And make the significant line of your run.bat:-<br />
<code><br />
Rcmd BATCH ′--args a=%1 b=%2′ U:\R-scripts\test.R<br />
</code><br />
%1 and %2 will map to the first and second thing after the batch file. You can go all the way up to %9.<br />
<br />
IMPORTANT NOTE<br />
<br />
Make sure you get the apostrophe character right in the above example – NEVER copy and paste from a word document into a script. It will go hideously wrong. Type the apostrophes (and dashes for that matter) in a good text editor – you want just the standard old-fashioned characters.<br />
<br />
== Requesting resources ==<br />
<br />
The following modifiers next to the “/scheduler:” part of the job submit line (before your app.exe 1 2 3 part), will request things you might want…<br />
<br />
/numcores:8 - number of cores you want<br />
<br />
/numcores:8-12 - minimum and maximum cores appropriate for your job<br />
<br />
/memorypernode:1024 - amount of mem in MegaBytes needed.<br />
<br />
/workdir:\\networkshare\work - set working directory<br />
<br />
/stdout:\\networkshare\out.txt - divert stdout to a file<br />
<br />
/stderr: or /stdin: - similar for stderr and stdin<br />
<br />
== Troubleshooting / Miscellany / Q & A ==<br />
<br />
* My job doesn’t work.<br />
** Run the HPC Job Manager application. Find your job id, possible under “My Jobs, Failed”. Double click on it, then on “View All Tasks”. Perhaps something in the output section will help.<br />
<br />
* Check that the path to your job is visible everywhere.<br />
** Avoid spaces in your paths - “job submit” doesn’t like them very much. If you must have them, they’ll be ok in the “run.bat” batch file that the cluster will run – in which case, put the path in standard double-quotes ("). But avoid them in your “launch.bat” file – you may have to relocate your run.bat to a simple non-space-containing directory.<br />
** Rather than putting the full application and parameters on the job submit line, you might want to write a batch-file to do all that, and submit the batch file to the cluster. (See section 6 about R for example). But make sure the batch file is somewhere visible to the cluster.<br />
<br />
* My job seems to work, but reports as having failed.<br />
** The success/failure depends on the error code returned. If you’re running C code, end with “return 0;” for success.<br />
<br />
* job submit ..blah blah.. app.exe >out.txt doesn’t work!<br />
** The contents of out.txt will be the result of “job submit”, not the result of “app.exe”. You meant to say this job submit ..blah blah.. /stdout:out.txt app.exe correcting out.txt and app.exe to network paths of course.<br />
<br />
* My C/C++ Executable works perfectly on my desktop, but doesn't work on the cluster. What's wrong with the cluster?<br />
** Your EXE file probably relies on some DLL files that the cluster doesn't have. If you use Visual Studio, check that in Code Generation, you have "Multi-Threaded (/MT)" as your target - with no mention of DLLs. If your code uses OpenMP, check you've copied the right OpenMP DLL(s) (go back to the C section above). As a last resort, give your EXE to someone who doesn't have Visual Studio installed, and get them to run it, and see what DLL files it seems to ask for.<br />
<br />
== Contributing Authors ==<br />
<br />
Wes Hinsley, James Truscott, Tini Garske, Jeff Eaton, Hannah Clapham</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Cluster_Documentation&diff=14258Cluster Documentation2024-02-28T16:58:09Z<p>Admin: </p>
<hr />
<div>* We have two high performance clusters for general use, both running Microsoft HPC 2012 R2, Update 3. The smaller, older cluster is [[fi--dideclusthn]] and the larger is [[fi--didemrchnb]]. <br />
<br />
* Many users in the department write code and packages in R. We have in-house packages that can allow an almost-interactive experience of running jobs on our high performance clusters, so if that sounds like what you'd like, you might as well stop reading here, and go straight to [https://github.com/mrc-ide/didehpc The DIDEHPC package] - or if you need to join the mrc-ide github org first, go to [https://github.com/mrc-ide/welcome The welcome page].<br />
<br />
If you're not using R, or you are interested in other ways, then continue...<br />
<br />
== Windows Users ==<br />
<br />
The cluster nodes are Windows 2012-R2 based. This is good news if you're a Windows user, and especially easy for PCs within the departmental DIDE domain. You have the option of using the MS client for launching jobs (see below), or you can use try the new [[HPC Web Portal]]. If you're preparing to use the cluster, and have doubts about whether it can run what you want - and whether it will be straightforward to develop and run on it, best talk to me and/or the IT guys first!<br />
<br />
== Linux and MAC Users ==<br />
<br />
There are two problems, but both can be at least partially overcome. The first is that there is no client for linux. The second is that it can only run windows binaries.<br />
<br />
=== Launching jobs on Linux/Mac ===<br />
<br />
* The best way might be the [[HPC Web Portal]], which lets you submit jobs through a webpage. Give it a try, and feedback to me any issues! <br />
<br />
* Alternatively, you can either (1) go down the VM route, installing a windows virtual machine, eg VMWare or Parallels, and follow the instructions below. (2) Remote Desktop to your friend's Windows machine. These tend not to be very convenient, but some users have submitted thousands of jobs this way.<br />
<br />
=== What Linux/Mac jobs can I run? ===<br />
<br />
* Many jobs may be platform indepedent. Where compiled code is involved though, you'll have some extra work to make sure the binary parts can be run in windows. For R, [https://github.com/mrc-ide/didehpc the tools] help you sort that out. For others, you may need to run a job on the cluster to do your builds. Also, for many linux applications, there is a windows port. Mileage can vary, but it's worth a try searching for that before you give up hope. Some of these might be installed on the cluster already - see below. Or some you may be able to download and call them from your run script.<br />
<br />
* For C code, again, we can run scripts on the cluster to build the binaries for you, if your code can be compiled with (say) the MinGW port of gcc. Cygwin can also be used (although seems less common these days) with accompanying dlls. Get in touch if you need further ideas; most things can be solved. <br />
<br />
* Matlab is a little more tricky - we have a range of colossal matlab runtimes on the cluster, but you'd need to compile a windows binary to use them, since the matlab platforms are distinct in every way. For this, probably find a friend, or get in touch, and we'll work out how you can compile for it.<br />
<br />
* There is also the option of running a Windows VM, which might solve all of the above problems.<br />
<br />
=== But I really want a unix cluster. ===<br />
<br />
* We have run linux clusters in the past ([[fi--didelxhn]]), but there was very low demand, whereas the other clusters are consistently well used. Accessing DIDE network shares from linux was a little harder in some aspects, but we did not implement the more traditional linux approach of having dedicated separate cluster storage that inputs and outputs would have to be staged on to, in order for the cluster to see them. Generally, the flexibility of the other clusters makes such things quite straightforward, and may even feel a pleasure if your background is with the less domain-integrated style of cluster architecture.<br />
<br />
* It has also become far easier to write software that builds on multiple platforms with good practice. Tools that only run on one platform are rare, and in some circumstances have raised questions about the quality of those packages. Alternatives are often available.<br />
<br />
* Where compilation is the issue - for example if you want to run binaries on the cluster, but your compilation machine runs linux, or is a mac, (Matlab via MCC, or C++ code for example), then get in touch, and there are a few ways we can think of solving that, either by getting someone else, or the cluster itself, to build your code.<br />
<br />
* There are also resources in [https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/computing/high-throughput-computing/ ICT) who run a linux cluster, with the file-staging approach, and also (being shared between a much larger pool of users) have rather stricter rules on job durations and resources than we provide locally in DIDE.<br />
<br />
== Getting Started ==<br />
<br />
=== Getting access to the cluster ===<br />
<br />
Send a mail to Wes (w.hinsley@imperial.ac.uk) requesting access to the cluster. He will add you to the necessary groups, and work out what cluster you should be running on. Unless you are told otherwise, this will be fi--dideclusthn. But if you have been told otherwise, then whenever you see fi--dideclusthn, replace it with the cluster name you’ve been given, either mentally or (less effectively) on your screen with tip-ex.<br />
<br />
=== Windows: Install the HPC client software ===<br />
<br />
* Run \\fi--dideclusthn.dide.ic.ac.uk\REMINST\setup.exe. <br />
* Confirm '''Yes, I really want to run this'''.<br />
* There's an introductory screen, and you click '''Next'''<br />
* You have to tick the '''Accept''', then click '''Next'''<br />
* If it gives you any choice, select '''Install only the client utilities'''<br />
* '''Next''', '''Next''', '''Next'''<br />
* Select '''Use Microsoft Update''' and then '''Next'''<br />
* And then '''Install'''<br />
* And possibly '''Install''' again – if it had to install any pre-requisites first.<br />
* And '''Finish''', and the client installation is done.<br />
* To check everything is alright so far, open a '''NEW command prompt window'''.<br />
* (Because the installer adds things to the path, which only take effect on '''NEW comamnd windows'''! :-)<br />
* (Find it under Accessories, or you can use Start, Run, '''cmd''').<br />
* Type <code>job list /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
And it will list the jobs you have running, which will be none so far!<br />
<br />
<pre><br />
Id Owner Name State Priority Resource Request<br />
---------- ------------- ----------------- ------------ ----------- -------- -------<br />
<br />
0 matching jobs for DIDE\user<br />
</pre><br />
<br />
=== Windows: Using a non-domain machine ===<br />
<br />
If you're using a machine that isn't actually logged into DIDE, then the client software will have a problem working out who you are. In this case, there are two things you need to do.<br />
* Check you've really got a DIDE username. If you don't know what it is, talk to your lovely IT team.<br />
* Connect to the DIDE VPN (see http://www1.imperial.ac.uk/publichealth/departments/ide/it/remote/) - login to that with your DIDE account.<br />
* Alternatively to the DIDE VPN, you can install ICT's ZScaler from https://uafiles.cc.ic.ac.uk/, and login to that with your IC account - but you'll still need your DIDE account to access DIDE servers after the ZScaler connection is up and running.<br />
* Now we'll open a command window using "runas" - which lets you choose which identity the system thinks you are within that command window:-<br />
<pre><br />
runas /netonly /user:DIDE\user cmd<br />
</pre><br />
* (Change user to your own login obviously). It will ask for your password, then open a new command window. In that window, you'll be able to do cluster-related things as your DIDE username.<br />
<br />
=== Windows: Launching and cancelling jobs ===<br />
==== Windows: Command Line ====<br />
<br />
Suppose you have a file called "run.bat" in your home directory, which does what you want to run on the cluster. Let's say it's a single-core, very simple job. We’ll discuss what should be inside "run.bat" later. To submit your job on the cluster, at the command prompt, type this (all on one line) - or put it in a file called "launch.bat", and run it:-<br />
<br />
<code>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1 \\qdrive.dide.ic.ac.uk\homes\user\run.bat</code><br />
<br />
If it's the first time you’ve run a job - or if you've recently changed your password, then it might ask you for your DIDE password and offer to remember it. Otherwise, it will just tell you the ID number of your job.<br />
<br />
<pre>Enter the password for ‘DIDE\user’ to connect to ‘FI—DIDECLUSTHN’:<br />
Remember this password? (Y/N)<br />
job has been submitted. ID: 123<br />
</pre><br />
<br />
If you want to remove the job, then:- <code>job cancel 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
Or view its details with <code>job view 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
==== Windows: Job Manager GUI ====<br />
<br />
Alternatively to the command-line, you can use the job management software, rather than the command-line. The advantage is that it’s a GUI. The disadvantage is, as in all GUIs, you may not feel totally sure you know what it’s up to – where most of the time the things you want to do might not be very complex, as above.<br />
<br />
The job management software will be on your start menu, as above. All the features are under the “Actions” menu, and hopefully it will be self explanatory. Read the details below about launching, and you'll find the interface bits that do it in the Job Manager. However, you may find over time, especially if you run lots of jobs, that learning to do it the scripted way with the command-line can be quicker.<br />
<br />
<br />
== Information for running any job ==<br />
<br />
=== Visibility ===<br />
<br />
Any executable/batch file that the cluster will run must be somewhere on the network that the cluster has access to, when it logs in as you. This amounts to any network accessible drive that you have access to when you login – including network shares, such as the temp drive, your home directory, and any specific network space set up for your project.<br />
<br />
After a job is launched, all paths are relative to the '''/workdir:''' specified on the job submit command-line. Windows does not usually allow a network path to be the current directory, and MS-HPC somehow gets around this. But if in doubt, use fully-qualified paths to specify where files should be read from, or written to. You may want to write code that takes the paths from a parameter file, or as a command-line parameter, to give as much flexibility as possible. <br />
<br />
Your home directory, and the temp drive are on departmental servers that don't have an especially fast connection to the cluster. So avoid having thousands of jobs all reading/writing to those places at once. Additionally, the temp drive is shared between all users, and your home drive has a limited quota. It is also backed up every day, so avoid using it as scratch storage for results you don’t actually want to keep. Generally, the shares you'll want to use from the clusters will be ones containing the word "NAS", which the big cluster has fast access to.<br />
<br />
=== Interactivity ===<br />
<br />
The job must run in an entirely scripted, unattended way. It must not require any key presses or mouse clicks, or other live interactivity while it runs. So jobs generally will read some sort of input (from a file, or from the way you run the job), do some processing, and write the results somewhere for you - all without intervention.<br />
<br />
== Launching jobs ==<br />
<br />
Jobs can be launched either through the Job Manager interface, or through the command line tools, which offer greater flexibility. We'll describe the command line method here; if you want to use the GUI, then it'll be a case of finding the matching boxes... Below are the specifics for our clusters. For more information, simply type <code>job</code> on the commandline, or <code>job submit</code> for the list of submission-related commands.<br />
<br />
=== FI--DIDECLUSTHN vs FI--DIDEMRCHNB ===<br />
<br />
Job submissions, as shown below, must specify a "job template", which sets up a load of default things to make the job work. On fi--dideclusthn, the job templates are called 4Core, 8Core and GeneralNodes, which will respectively force jobs to run on the 4-core nodes, the 8-core nodes, or on any machine available.<br />
<br />
On fi--didemrchnb, you can set the job template to be... 8Core, 12Core, 16Core, 12and16Core, and GeneralNodes - which hopefully are fairly self-explanatory. There are a couple of other job templates, (24Core and Phi), but those are a bit special purpose for now, so don't use them!<br />
<br />
=== Job Submission ===<br />
<br />
Job submissions from the command line can take the following form (all on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /stdout:\\path\to\stdout.txt<br />
/stderr:\\path\to\stderr.txt /numcores:1-1 /jobtemplate:4Core \\path\to\run.bat</pre><br />
<br />
=== The /singlenode argument ===<br />
<br />
In MS HPC (2012), Microsoft finally added a tag to allow you to say that the 'n' cores you requested must be on the same computer. Therefore, if you know precisely how many cores you want, then use the following (on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:true <br />
/jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
However, there is one bug with this, for the specific case where you request a whole node, regardless of how many cores it has. In this case, oddly, you have to disable single node:-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:false <br />
/numnodes:1 /jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
== Languages and libraries supported ==<br />
<br />
A number of languages, and different versions of languages are available on the cluster. The sections below refer to your "run.bat" file - a batch file that you will create which will get run by a cluster node when a suitable one is available. The commands described below are to be put in this "run.bat" file, and they add various directories to the path, so that the software you want will be added to the path. <br />
<br />
=== C/C++ === <br />
<br />
A number of Microsoft C++ and Intel C++ runtimes are installed, but it's usually better to try and avoid using them, and make your executable as stand-alone as possible. If it requires any external libraries that you've had to download, then put the .dll file in the same directory as the .exe file. If you use Microsoft Visual Studio, in Project Preferences, C/C++, Code Generation, make sure the Runtime Library is Multi-threaded (/MT) – the ones with DLL files won’t work. Even so, on recent versions of the Intel and Microsoft C compilers, "static linking" doesn’t really mean static when it comes to OpenMP, and you’ll have to copy some DLLs and put them next to your EXE file. See the OpenMP section below.<br />
<br />
The cluster nodes are all 64-bit, but they will run 32-bit compiled code. Just make sure you provide the right DLLs!<br />
<br />
Here is a document about [[Using the Xeon Phi]]<br />
<br />
Here is a document about [[Using MPI]]<br />
<br />
And if you want to use libraries such as GSL, then try [[C/C++ Libraries for Windows]]<br />
<br />
=== Java ===<br />
<br />
Java doesn't strictly need installing; you can setup any Java Runtime Environment you like, from Oracle, or anywhere. If you want to do this, then, (if you were to use the Server JRE from Oracle), something similar to the following will add java.exe and jvm.dll to the path, and set the JAVA_HOME environment variable, which most Java-using software will appreciate.<br />
<br />
set JAVA_HOME=\\my\path\to\jre1.8.0<br />
set path=%JAVA_HOME%\bin;%JAVA_HOME%\bin\server;%path%<br />
<br />
For convenience, if you're happy with Azul's OpenJDK Java 1.8.0 update 308 (September 2021), then <code>call setJava64</code> in your run script will set this up for you. Incidentally the above will also add what's needed for using the rJava package in R - in which case you'll perhaps also want something like <code>call setR64_4_1_0</code> in your run script.<br />
<br />
=== Perl ===<br />
<br />
Strawberry Perl 64-bit portable, v5.30.1.1. Put <code>call setPerl</code> at the top of your script.<br />
<br />
=== R ===<br />
<br />
'''New tools are coming''': [http://dide-tools.github.io/didehpc/vignettes/quickstart.html see here]. The instructions below are still valid if you want to do this manually.<br />
<br />
<br />
You can R jobs on all the clusters. This is the latest wisdom on how to do so – thanks to James, Jeff, Hannah and others. <br />
<br />
First, if you are wanting to use packages, then set up a package repository on your home directory by running this in R.<br />
<br />
<code>install.packages("<package>",lib="Q:/R")</code><br />
<br />
Now, write your run script. Suppose you have an R script in your home directory: Q:\R-scripts\test.R. And suppose you’ve set up your repository as above. Your run.bat should then be:-<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\test.R<br />
</pre><br />
Various packages require various R versions, and the cluster supports a few versions. To choose which one, change the first line of the script above to one of these – 32-bit or 64-bit versions of R releases. Purely for amusement, R's codenames are shown here too. <br />
<pre><br />
call setr32_2_13_1.bat call setr64_2_13_1.bat (anyone know the codename?)<br />
call setr32_2_14_2.bat call setr64_2_14_2.bat (Gift-getting season)<br />
call setr32_2_15_2.bat call setr64_2_15_2.bat (Trick or treat)<br />
call setr32_3_0_1.bat call setr64_3_0_1.bat (Good Sport)<br />
call setr32_3_0_2.bat call setr64_3_0_2.bat (Frisbee Sailing)<br />
call setr32_3_1_0.bat call setr64_3_1_0.bat (Spring Dance)<br />
call setr32_3_1_2.bat call setr64_3_1_2.bat (Pumpkin Helmet)<br />
call setr32_3_2_2.bat call setr64_3_2_2.bat (Fire Safety)<br />
call setr32_3_2_3.bat call setr64_3_2_3.bat (Wooden Christmas Tree)<br />
call setr32_3_2_4.bat call setr64_3_2_4.bat (Very Secure Dishes - Revised Version - R later renamed this to 3.2.5)<br />
call setr32_3_3_1.bat call setr64_3_3_1.bat (Bug in your hair)<br />
call setr32_3_3_2.bat call setr64_3_3_2.bat (Sincere Pumpkin Patch)<br />
call setr32_3_4_0.bat call setr64_3_4_0.bat (You Stupid Darkness)<br />
call setr32_3_4_2.bat call setr64_3_4_2.bat (Short Summer)<br />
call setr32_3_4_4.bat call setr64_3_4_4.bat (Someone to lean on)<br />
call setr32_3_5_0.bat call setr64_3_5_0.bat (Joy in Playing)<br />
call setr32_3_5_1.bat call setr64_3_5_1.bat (Feather Spray)<br />
call setr32_3_5_3.bat call setr64_3_5_3.bat (Great Truth)<br />
call setr32_3_6_0.bat call setr64_3_6_0.bat (Planting of a Tree)<br />
call setr32_3_6_1.bat call setr64_3_6_1.bat (Action of the Toes)<br />
call setr32_3_6_3.bat call setr64_3_6_3.bat (Holding the Windsock)<br />
call setr32_4_0_2.bat call setr64_4_0_2.bat (Taking Off Again)<br />
call setr32_4_0_3.bat call setr64_4_0_3.bat (Bunny-Wunnies Freak Out)<br />
call setr32_4_0_5.bat call setr64_4_0_5.bat (Shake and Throw)<br />
</pre><br />
<br />
The following versions are *not* available on the cluster at the moment - but here are their names anyway, as it seemed a shame to miss them out. Let me know if you desperately need one of these.<br />
<br />
<pre><br />
2.14.0 Great Pumpkin <br />
2.14.1 December Snowflakes<br />
2.15.0 Easter Beagle<br />
2.15.1 Roasted Marshmallow<br />
2.15.3 Security Blanket<br />
3.0.0 Masked Marvel<br />
3.0.3 Warm Puppy<br />
3.1.1 Sock it to me<br />
3.1.3 Smooth Sidewalk<br />
3.2.0 Full of Ingredients<br />
3.2.1 World famous astronaut<br />
3.2.4 Very Secure Dishes (the non-revised version)<br />
3.3.0 Supposedly Educational<br />
3.3.3 Another Canoe<br />
3.4.1 Single Candle<br />
3.4.3 Kite-Eating Tree<br />
3.5.2 Eggshell Igloo<br />
3.6.2 Dark and Stormy Night<br />
4.0.0 Arbor Day<br />
4.0.1 See Things Now<br />
4.0.4 Lost Library Book<br />
</pre><br />
<br />
It also seems that different R versions put their packages in different structures – sometimes adding "win-library" into there for fun. Basically, R_LIBS and R_LIBS_USER should be paths to a folder that contains a list of other folders, one for each package you've installed.<br />
<br />
IMPORTANT: R_LIBS and R_LIBS_USER paths must NOT contain quotes, nor spaces. If the path to your library contains spaces, you need to use old-fashioned 8-character names. If in your command window, you type <code>dir /x</code>, you’ll see the names – Program Files tends to become PROGRA~1 for instance. <br />
<br />
==== Passing parameters to R scripts ====<br />
Passing parameters to R scripts means you can have fewer versions of your R and bat files and easily run whole sets of jobs. You can get parameters into R using Rscript (but not Rcmd BATCH, I think) as follows. In the run.bat example above, the Rscript statement becomes:<br />
<pre><br />
Rscript Q:\R-scripts\test.R arg1 arg2 arg3<br />
</pre><br />
Within the R code, the arguments can be recovered using <br />
<pre><br />
args <- commandArgs(trailingOnly = TRUE)<br />
<br />
outFileName <- args[1] ## name of output file.<br />
dataFileName <- args[2] ## name of local data file. <br />
currentR0 <- as.numeric(args[3]) ## convert this explicitly to number. <br />
</pre><br />
If you want your arguments to have a particular type, best to explicitly convert (see R0 above). Better still, you can pass parameters directly into the batch file that runs the R script. Command line arguments can be referenced within the batch file using %1, %2, etc. For example, if you have a batch file, runArgs.bat:<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\%1.R %2 %3 %4<br />
</pre><br />
then<br />
<pre><br />
runArgs.bat myRScript arg1 arg2 arg3<br />
</pre><br />
will run the R script myRScript.R and pass it the parameters arg1, arg2 and arg3. The batch file runArgs.bat is now almost a '''generic R script runner'''.<br />
<br />
=== OpenMP ===<br />
<br />
This has been tested with C/C++ - but the same should apply to other languages such as fortran, that achieve multi-threading with a DLL file.<br />
<br />
==== Microsoft's C/C++ Compiler ====<br />
You will need to copy vcomp100.dll (for Visual Studio 2010 – it maybe vcomp90.dll for 2008), into the same directory as your executable. You can usually find the dll file in a directory similar to:- C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist\x64\Microsoft.VC100.OpenMP. Extrapolate for future versions! Also make sure that you've enabled OpenMP in the Properties (C++/Language).<br />
<br />
==== GCC or MinGW ====<br />
Usually, this applies to Eclipse use. In project properties, the C++ command “g++ -fopenmp”, the C command “gcc –fopenmp”, and in the Linker options, under Miscellaneous, Linker flags, put “-fopenmp” too. Copy the OpenMP DLLs from MinGW into the same directory as your final executable. You may find the dlls are in C:\MinGW\Bin\libgomp-1.dll and in the same place, libpthread-2.dll, libstdc++-6.dll and libgcc_s_dw2-1.dll.<br />
<br />
==== Intel C++ Compiler ====<br />
Copy the libiomp5md.dll file from somewhere like C:\Program Files (x86)\Intel\ComposerXE-2011\redist\intel64\compiler\libiomp5md.dll to the same place as your executable. And in Visual Studio, make sure you've enabled OpenMP in the project properties.<br />
<br />
==== How many threads? ====<br />
The OpenMP function omp_max_threads() returns the number of ''physical'' cores on a machine, not the number of cores ''allocated'' to your job. To determine how many cores the scheduler actually allocated to you, use the following code to dig up the environment variable CCP_NUMCPUS, which will be set by the cluster management software:- <br />
<pre><br />
int set_NCores() {<br />
char * val;<br />
char * endchar;<br />
val = getenv("CCP_NUMCPUS");<br />
int cores = omp_get_max_threads();<br />
if (val!=NULL) cores = strtol (val,&endchar,10);<br />
omp_set_num_threads(cores);<br />
return cores;<br />
}<br />
</pre><br />
<br />
=== WinBugs (or OpenBugs) ===<br />
<br />
They are similar, but OpenBugs 3.2.2 is the one we've gone for. Something like this as your run.bat script will work:-<br />
<pre><br />
call setOpenBugs<br />
openbugs \\path\to\script.txt /HEADLESS<br />
</pre><br />
<br />
=== MatLab ===<br />
<br />
There are various ways of producing a non-interactive executable from Matlab. Perhaps the simplest (not necessarily the best performance) way is to use “mcc.exe” – supplied with most full versions of Matlab, including the Imperial site licence version that you've probably got.<br />
<br />
==== Use mcc.exe to compile your code ====<br />
<br />
Use windows explorer to navigate to the folder where the “.m” files are for the project you want to compile. Now use a good text editor to create a file called “compile.bat” in that folder. It should contain something similar to the following:-<br />
<br />
<code>mcc -m file1.m file2.m file3.m -o myexe</code><br />
<br />
Don’t copy/paste the text from this document by the way – Word has a different idea of what a dash is to most other software, and will probably replace the two dashes with funny characters. so you have to list every single .m file that your project needs, after the’-m’. If you save this file, then double-click on it, then it will think for some while, and produce “myexe.exe” in this example. Copy your .exe file into a network accessible place as usual.<br />
<br />
==== Launch on the cluster. ====<br />
<br />
The launch.bat file will be exactly the same as before – see page 2. The run script will then start with a line that tells the cluster which version of Matlab you used to compile the cluster. Below is the table of different versions. The runtimes are huge and cumbersome to install on the cluster, so as a result I haven’t installed every single one. If you need one that’s not listed, get in touch.<br />
<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! Matlab Version<br />
! First line of run script<br />
|- <br />
| R2009b<br />
| call useMatLab79<br />
|- <br />
| R2010a<br />
| call useMatLab713<br />
|- <br />
| R2011a<br />
| call useMatLab715<br />
|- <br />
| R2011b<br />
| call useMatLab716<br />
|- <br />
| R2012a (64-bit)<br />
| call useMatLab717_64<br />
|- <br />
| R2012b (64-bit)<br />
| call useMatLab80_64<br />
|- <br />
| R2013a (64-bit)<br />
| call useMatLab81_64<br />
|- <br />
| R2014a (64-bit)<br />
| call useMatLab83_64<br />
|- <br />
| R2014b (64-bit)<br />
| call useMatLab84_64<br />
|- <br />
| R2015a (64-bit)<br />
| call useMatLab85_64<br />
|- <br />
| R2015b (64-bit)<br />
| call useMatLab90_64<br />
|- <br />
| R2016b (64-bit)<br />
| call useMatLab91_64<br />
|- <br />
| R2017a (64-bit)<br />
| call useMatLab92_64<br />
|- <br />
| R2019a (64-bit)<br />
| call useMatLab96_64<br />
|- <br />
| R2021a (64-bit)<br />
| call useMatLab910_64<br />
|- <br />
| R2022a (64-bit)<br />
| call useMatLab912_64<br />
|}<br />
<br />
=== Python ===<br />
<br />
* The most recent version of each major release is on the cluster. Different python versions within the same release (e.g., 3.5.1 and 3.5.2) do not behave nicely together in automated installation, so if you have a desperate need for a previous minor version, get in touch. While 2.7.18 is available, we '''strongly''' recommend migrating away from Python 2.7, as it has been officially discontinued on 1st January 2020. Experiences may vary, and future success is not guaranteed.<br />
<br />
<pre><br />
2.7.18 64-bit call set_python_27_64.bat<br />
3.5.9 64-bit call set_python_35_64.bat<br />
3.6.15 64-bit call set python_36_64.bat<br />
3.7.17 64-bit call set_python_37_64.bat<br />
3.8.17 64-bit call set python_38_64.bat<br />
3.9.17 64-bit call set_python_39_64.bat<br />
3.10.12 64-bit call set python_310_64.bat<br />
3.11.4 64-bit call set_python_311_64.bat<br />
<br />
</pre><br />
<br />
* You'll most likely want to use some packages, so you'll need to have a network-accessible location to put your packages in. If you're running a version of Windows with Python that matches one installed on the cluster, then you'll be able to share that package repository between the two. But if you're running a different version, or a different operating system, or a different Python version, then you'll need to keep your local package repository separate from the one the cluster uses.<br />
* Here comes the basic script to run some python. Suppose I decide to put my Python packages for the cluster in '''\\qdrive.dide.ic.ac.uk\homes\username\PythonCluster37_64''' (which is Q:\PythonCluster37_64 on a DIDE Windows PC). To run some python, something like the following:-<br />
<br />
call set_python_37_64<br />
set PYTHONPATH=\\qdrive\homes\username\PythonCluster37_64<br />
python \\qdrive\homes\username\Python\MyScript.py<br />
<br />
or if you want to do this in a programmatic way...<br />
<br />
import sys<br />
sys.path = ['\\\\qdrive\\homes\\username\\PythonCluster37_64'] + sys.path<br />
<br />
==== Packages ====<br />
<br />
* If you want to install a package like numpy, that's readily available through pip, then you can launch a batch file like this:-<br />
call set_python_36_64<br />
pip install numpy --target \\qdrive\homes\username\PythonCluster36_64<br />
* or alternatively, you could write a python script that looks like this, and launch it as normal:-<br />
import pip<br />
pip.main(['install','numpy','--target','\\\\qdrive\\homes\\username\\PythonCluster36_64'])<br />
<br />
* And if you're running the same Python version on a Windows machine, then you can do these two locally, without submitting a cluster job to do them for you.<br />
<br />
* And next time you submit a python job on the cluster, python will look in the place set by PYTHONPATH, and will find the packages you've just installed when you import them.<br />
<br />
=== Beast and Beagle ===<br />
<br />
We've got a couple of versions available. For BEAST 1.75 with BEAGLE 1.0: (the x y z represents all the command-line arguments you'll want to pass to beast)<br />
<pre><br />
call setbeast<br />
call beast-beagle x y z<br />
</pre><br />
<br />
If you want BEAST 1.82 or 1.83 with BEAGLE 2.1:<br />
<pre><br />
call setbeast_182 call setbeast_183<br />
call beast-beagle_182 x y z call beast-beagle_183 x y z<br />
</pre><br />
<br />
And if you want BEAST 2.3.2 with BEAGLE 2.1 (note that BEAST 2 is a different, er, beast to BEAST 1, regarding packages and compatibilities, so only use BEAST 2 if you know that's what you want to use).<br />
<pre><br />
call setbeast_232<br />
call beast-beagle_232 x y z<br />
</pre><br />
<br />
OpenCL is installed on all the nodes, which means Beagle will pick up the Intel Xeon processors and use what it can on them. It will also detect the Xeon Phis where they're available. If that's something you'd be particularly interested in using, drop me a line.<br />
<br />
=== Blast ===<br />
<br />
Drop me a line to join the party testing this, and to get added to the (large) BlastDB network share.<br />
<br />
<pre><br />
call setblast<br />
</pre><br />
<br />
will set the `BLASTDB` environment variable to point to what we current have of the `v5` database, and the Blast tools (2.9.0, 64-bit) will be added to the path. So you can <br />
then call `blastp`, `blastn`, `blastx` setc.<br />
<br />
=== BOW (BioInformatics on Windows) ===<br />
<br />
A number of Bio-informatics tools are ready to go. These used to be provided together on codeplex, but that has since been retired. Ugene is one project that<br />
maintains windows builds of them. Start your run.bat file with <code>call setBOW</code> to add these to the path:-<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File<br />
! Version<br />
|- <br />
| SamTools.exe<br />
| 0.1.19-44429cd<br />
|- <br />
| BCFTools.exe<br />
| 0.1.19-44428cd<br />
|- <br />
| bgzip<br />
| ?<br />
|- <br />
| razip<br />
| ?<br />
|- <br />
| bwa<br />
| 0.7.17-r1188<br />
|-<br />
| tabix<br />
| 0.2.5 (r1005)<br />
|-<br />
| minimap2<br />
| 2.24 (r1122)<br />
|-<br />
| raxml<br />
| 8.2.12 (best for each node)<br />
|-<br />
| raxml_sse3<br />
| 8.2.12 (pthreads+sse3)<br />
|-<br />
| raxml_avx<br />
| 8.2.12 (pthreads+avx)<br />
|-<br />
| raxml_avx2<br />
| 8.2.12 (pthreads+avx2)<br />
|}<br />
<br />
Note that BWA-MEM is hardwired to use a linux-only feature, so will produce an odd message and not do what you want if you use that mode on Windows. However, it seems that minimap2 may be a faster and more accurate way<br />
of doing what bwa does in any case, and that seems to have no such limitation.<br />
<br />
Also, note that raxml.exe is a copy of the most optimised build for the particular cluster node the job is running on. fi--dideclusthn only supports SSE3;<br />
fi--didemrchnb varies a little; the 32-cores support AVX2, the 24, 20 and 16 core machines support AVX, and only the oldest 12-core nodes only support SSE3. You don't have to worry about, except to expect performance to vary between them.<br />
<br />
=== GATK and Picard ===<br />
<br />
These both rely on Java, so put this in your run script:<br />
<br />
<pre><br />
call setJava64<br />
call setgatk<br />
</pre><br />
<br />
and then <code>gatk</code> in your scripts will use Python 3.7 to call the argument handling wrapper for GATK 4.2.2.0, and <code>picard</code> maps to the picard (2.26.2) jar file.<br />
<br />
=== FastQC ===<br />
<br />
This one has a slight eccentricity that instead of passing normal command-line args, it uses java's -D command-line options to buffer them out of an environment variable<br />
called _JAVA_OPTIONS. Strange but true. The ZIP comes with a bash script, which I'll reimplement if anyone actually wants me to. But in the meantime, it's easy to work <br />
out the arguments by inspection. For example, most likely usage:-<br />
<br />
<pre><br />
call setFastQC<br />
<br />
set _JAVA_OPTIONS='-Dfastqc.output_dir=%OUT%'<br />
call fastqc thing.fq<br />
</pre><br />
<br />
=== MAFFT ===<br />
<br />
For version 7.212 (Win64), write <code>call setMAFFT.bat</code> at the top of your run script.<br />
<br />
=== Applied-Maths Open Source ===<br />
<br />
Write <code>call setAppliedMaths.bat</code> at the top of your run script, to add these to the path:<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File(s)<br />
! Description<br />
! Version<br />
|- <br />
| velvetg_mt_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_mt_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| ray_x64.exe<br />
| Ray 64-bit (for MPI)<br />
| 1.6<br />
|-<br />
| ray_x86.exe<br />
| Ray 32-bit (for MPI)<br />
| 1.6<br />
|-<br />
| Mothur_x64.exe<br />
| Mothur 64-bit (for MPI)<br />
| 1.25.1<br />
|-<br />
| Mothur_x86.exe<br />
| Mothur 32-bit (for MPI)<br />
| 1.25.1<br />
|}<br />
<br />
=== Tensorflow ===<br />
<br />
Here is an example of getting Tensorflow (as a python package) available from an R script, using the cluster to install everything for you. It only works for Tensorflow 2.0 - older versions and newer versions seem to want combinations of DLLs that I can't satisfy. The following are I think minimal steps to get Tensorflow running from Python and R.<br />
<br />
* Create a folder for your python packages in your home directory (so the cluster can see it. I'm calling mine <code>Q:/python_37_repo</code> - and the cluster will call it \\fi--san03.dide.ic.ac.uk\homes\wrh1\python_37_repo - change wrh1 to your DIDE username of course.<br />
* Create a similar folder for R packages, which for me is <code>Q:/R4</code><br />
* And a third folder, Q:/Tensorflow to do this work in.<br />
* Download these DLLs and put them in the Q:/Tensorflow folder - at present they seem to be the ones Python/Tensorflow wants. Let's hope for the best... Download [https://mrcdata.dide.ic.ac.uk/resources/msvcp140.dll msvcp140.dll] and [https://mrcdata.dide.ic.ac.uk/resources/msvcp140_1.dll msvcp140_1.dll] into your <code>Q:/Tensorflow</code> folder.<br />
<br />
* Now, create a script <code>Q:/Tensorflow/prepare.bat</code>. We're going to launch this on the cluster, and get it to wire up the packages we want. Replace USER with your username...<br />
<br />
<pre><br />
net delete Q: /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set TMPDIR=%TEMP%<br />
set PYTHONPATH=\\qdrive.dide.ic.ac.uk\homes\USER\python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
pip install --no-cache-dir tensorflow==2.00 --target Q:\python_37_repo<br />
RScript Q:\Tensorflow\install_tensorflow.R<br />
</pre><br />
<br />
* Create also <code>Q:/Tensorflow/install_tensorflow.R</code> contains:<br />
<pre><br />
install.packages(c("reticulate", "tensorflow"),lib="Q:/R4",repos="https://cran.ma.imperial.ac.uk/")<br />
</pre><br />
<br />
* Now submit the prepare job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> in the big window will do it. That will take a while, but you should notice lots of new files in your Q:/python_37_repo folder. You'll only need to do this once (unless you delete your Python repo folder, or it randomly breaks and you start again...)<br />
<br />
* Next, create a file <code>Q:/Tensorflow/run.bat</code> containing the following the two "call" lines near the end are the ones that do some "work" in python or R, so this script basically a template for running your future jobs. Just change those two calls to set off the python or R code you want to run that uses Tensorflow.<br />
<br />
<pre><br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
net use Q: /delete /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set PYTHONPATH=Q:/python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call python Q:\tensorflow\test_tensorflow.py<br />
call RScript Q:\tensorflow\test_tensorflow.R<br />
</pre><br />
<br />
* And finally, you'll need <code>Q:\tensorflow\test_tensorflow.py</code> and <code>Q:\tensorflow\test_tensorflow.R</code> to exist. Here are minimal tests that everything is wired up. <code.test_tensorflow.py</code> first:-<br />
<br />
<pre><br />
import tensorflow as tf;<br />
print(tf.reduce_sum(tf.random.normal([1000, 1000])));<br />
</pre><br />
<br />
and <code>Q:\Tensorflow\test_tensorflow.R</code> contains:<br />
<br />
<pre><br />
library(reticulate)<br />
reticulate::py_config()<br />
library(tensorflow)<br />
tf$constant('Hello, TensorFlow!')<br />
tf$Variable(tf$zeros(shape(1L)))<br />
</pre><br />
<br />
* Now submit the run job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> in the big window will do it - and perhaps paste <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\output.txt</code> in the stdout box to collect the input and see how it gets on. You should see sensible output from both tests.<br />
<br />
== Launching a job ==<br />
<br />
<br />
== Submitting many jobs ==<br />
<br />
Suppose you write an exe that you might run with… Mysim.exe init.txt 1 2 3, and you want to run it many times with a range of parameters. One way of many, is to write a launch.bat file that will run “job submit” separately, for example (thanks Tini/James!):-<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set initFile=\\networkshare\job\init.txt<br />
set exeFile=\\networkshare\job\mysim.exe<br />
<br />
%SubDetails% %exeFile% %initfile% 1 2 3<br />
%SubDetails% %exeFile% %initfile% 4 5 6<br />
%SubDetails% %exeFile% %initfile% 7 8 9<br />
</pre><br />
<br />
Suppose the job you want to run is an R script? To specify arguments to an R script, you have to add ′--args a=1 b=2′ - so… you might make launch.bat like this:-<br />
<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set rbatFile=\\networkshare\R-scripts\run.bat<br />
<br />
%SubDetails% %rbatFile% 1 2<br />
%SubDetails% %rbatFile% 3 4<br />
</pre><br />
And make the significant line of your run.bat:-<br />
<code><br />
Rcmd BATCH ′--args a=%1 b=%2′ U:\R-scripts\test.R<br />
</code><br />
%1 and %2 will map to the first and second thing after the batch file. You can go all the way up to %9.<br />
<br />
IMPORTANT NOTE<br />
<br />
Make sure you get the apostrophe character right in the above example – NEVER copy and paste from a word document into a script. It will go hideously wrong. Type the apostrophes (and dashes for that matter) in a good text editor – you want just the standard old-fashioned characters.<br />
<br />
== Requesting resources ==<br />
<br />
The following modifiers next to the “/scheduler:” part of the job submit line (before your app.exe 1 2 3 part), will request things you might want…<br />
<br />
/numcores:8 - number of cores you want<br />
<br />
/numcores:8-12 - minimum and maximum cores appropriate for your job<br />
<br />
/memorypernode:1024 - amount of mem in MegaBytes needed.<br />
<br />
/workdir:\\networkshare\work - set working directory<br />
<br />
/stdout:\\networkshare\out.txt - divert stdout to a file<br />
<br />
/stderr: or /stdin: - similar for stderr and stdin<br />
<br />
== Troubleshooting / Miscellany / Q & A ==<br />
<br />
* My job doesn’t work.<br />
** Run the HPC Job Manager application. Find your job id, possible under “My Jobs, Failed”. Double click on it, then on “View All Tasks”. Perhaps something in the output section will help.<br />
<br />
* Check that the path to your job is visible everywhere.<br />
** Avoid spaces in your paths - “job submit” doesn’t like them very much. If you must have them, they’ll be ok in the “run.bat” batch file that the cluster will run – in which case, put the path in standard double-quotes ("). But avoid them in your “launch.bat” file – you may have to relocate your run.bat to a simple non-space-containing directory.<br />
** Rather than putting the full application and parameters on the job submit line, you might want to write a batch-file to do all that, and submit the batch file to the cluster. (See section 6 about R for example). But make sure the batch file is somewhere visible to the cluster.<br />
<br />
* My job seems to work, but reports as having failed.<br />
** The success/failure depends on the error code returned. If you’re running C code, end with “return 0;” for success.<br />
<br />
* job submit ..blah blah.. app.exe >out.txt doesn’t work!<br />
** The contents of out.txt will be the result of “job submit”, not the result of “app.exe”. You meant to say this job submit ..blah blah.. /stdout:out.txt app.exe correcting out.txt and app.exe to network paths of course.<br />
<br />
* My C/C++ Executable works perfectly on my desktop, but doesn't work on the cluster. What's wrong with the cluster?<br />
** Your EXE file probably relies on some DLL files that the cluster doesn't have. If you use Visual Studio, check that in Code Generation, you have "Multi-Threaded (/MT)" as your target - with no mention of DLLs. If your code uses OpenMP, check you've copied the right OpenMP DLL(s) (go back to the C section above). As a last resort, give your EXE to someone who doesn't have Visual Studio installed, and get them to run it, and see what DLL files it seems to ask for.<br />
<br />
== Contributing Authors ==<br />
<br />
Wes Hinsley, James Truscott, Tini Garske, Jeff Eaton, Hannah Clapham</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Remote_access_to_DIDE&diff=14257Remote access to DIDE2023-12-11T11:00:30Z<p>Admin: </p>
<hr />
<div>These instructions are now obsolete. All remote access to DIDE is now via [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/unified-access/ Zscaler]. '''ZScaler replaces both the Pulse Secure VPN and the Remote Desktop Gateway'''.<br />
<br />
== Remote Access to your Windows DIDE Desktop via ZScaler ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
# Ensure that you've uninstalled Pulse Secure VPN.<br />
# Ensure that you've installed ZScaler via above link.<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
== OBSOLETE SINCE ZSCALER - Remote Access to your Windows DIDE Desktop via Terminal Services Gateway ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
[[File:TS connection2.png|thumb|Steps 4 to 9]]<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Select the '''Advanced''' tab on the far right and click the '''Settings...''' button under Connect from anywhere.<br />
# Select the '''Use these RD Gateway server settings''' button.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the ''Server name'' field.<br />
# Select '''Allow me to select later''' in the ''Log-on method'' field. We'll add this later (see after these steps).<br />
# Ensure that '''Bypass RD Gateway server for local addresses''' is ''ticked''.<br />
# Ensure that '''Use my RD Gateway credentials for the remote computer''' at the bottom is ''unticked''. <br />
# Press the OK button<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
You will now have a new icon on your desktop and you never need to do the above again. To connect to your DIDE desktop, click the '''Connect''' button in your new Remote Desktop shortcut.<br />
<br />
You will now be asked for your Imperial credentials in order to access the Terminal Services Gateway. Enter your details as '''IC\username''' with your Imperial password.<br />
<br />
Once that allows you through you will be prompted for your DIDE credentials in order to access your PC. Enter your details as '''DIDE\username''' with your DIDE password. If this is the first time that you've connected to your DIDE PC for a while you'll get a certificate warning message. Tick the box and select '''Yes'''.<br />
<br />
If you don't get prompted for your DIDE account that probably means your PC is asleep. To wake it up [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/remotely-access-my-college-computer/wake-my-pc/ click here].<br />
<br />
'''N.B.''' If all the above seems too difficult, I've made a template file with all the settings. It is in the '''T:''' drive in folder '''IT\VPN''' and is called '''DIDE-Remote.RDP'''. Right-click and edit that to put your correct hostname and username in and all the other settings are done for you.<br />
<br />
== Remote Access to your Ubuntu DIDE Desktop via Terminal Services Gateway ==<br />
First of all, you will need to have enabled remote desktop in advance. [https://linuxconfig.org/ubuntu-remote-desktop-18-04-bionic-beaver-linux See here for example]. You should also check you have the Remmina app installed.<br />
<br />
# Open Remmina<br />
# Click the “+” button to add a new configuration, and give it a meaningful name.<br />
# Protocol: RDP - Remote Desktop Protocol<br />
# Open Remmina and on the Basic tab fill in the following details: <br />
#* Server: Your desktop PC FQDN (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''')<br />
#* User Name: '''Your Ubuntu username'''<br />
#* Password: '''Your Ubuntu password'''<br />
#* Domain: '''DIDE''' (only if your workstation is on the DIDE domain - very likely it's not).<br />
# Adjust colour depth to '''16 bit'''.<br />
# Go to to the Advanced Tab and set up the ICT RDP Gateway with the following details:<br />
#* RDG Server: '''ictsgw.cc.ic.ac.uk'''<br />
#* RDG username: '''Your IC username'''<br />
#* RDG password: '''Your IC password'''<br />
#* RDG domain: '''IC'''<br />
# Click '''Save''' and you will have a new configuration with the name you initially specified. <br />
# Click on this new entry to connect to your server.<br />
# Accept the certificate warnings. These should only appear the very first time you connect.<br />
# Your Ubuntu remote session should now be active.<br />
<br />
<br />
== Remote Access to your DIDE Desktop via Terminal Services Gateway on a Mac ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
<br />
# Start the '''Microsoft Remote Desktop 10''' app from the App Store and select '''Add desktop''' to add a new profile entry.<br />
# Enter a name for the remote desktop profile.<br />
# Enter the full name of your DIDE machine (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''').<br />
# Change the '''User Account''' drop down to Add User Account<br />
# Enter '''DIDE\username''' in the User name field.<br />
# Enter your DIDE password in the Password field.<br />
# Enter a friendly name if desired (optional)<br />
# Click '''Add''' and '''Save''' again.<br />
# Click the Cog in the top left of the screen then Preferences.<br />
# Click Gateways.<br />
# Click on the '''+''' symbol on the bottom left of the window.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the Server name field.<br />
# Select your Imperial '''username''' in the User name field.<br />
# Enter a name for the gateway in the Friendly Name field.<br />
# Click '''Save'''.<br />
# Close the screen.<br />
# Now choose the gateway you created from the Gateway dropdown list.<br />
# Click the pencil icon.<br />
# Under '''General''' select the gateway name in the Gateway field.<br />
# Click '''Save'''.<br />
# The main screen will display your new profile. Double click this to start a remote desktop session.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Remote_access_to_DIDE&diff=14256Remote access to DIDE2023-12-11T11:00:13Z<p>Admin: </p>
<hr />
<div>These instructions are now obsolete. All remote access to DIDE is now via [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/unified-access/ Zscaler]. '''ZScaler replaces both the Pulse Secure VPN and the Remote Desktop Gatway'''.<br />
<br />
== Remote Access to your Windows DIDE Desktop via ZScaler ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
# Ensure that you've uninstalled Pulse Secure VPN.<br />
# Ensure that you've installed ZScaler via above link.<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
== OBSOLETE SINCE ZSCALER - Remote Access to your Windows DIDE Desktop via Terminal Services Gateway ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
[[File:TS connection2.png|thumb|Steps 4 to 9]]<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Select the '''Advanced''' tab on the far right and click the '''Settings...''' button under Connect from anywhere.<br />
# Select the '''Use these RD Gateway server settings''' button.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the ''Server name'' field.<br />
# Select '''Allow me to select later''' in the ''Log-on method'' field. We'll add this later (see after these steps).<br />
# Ensure that '''Bypass RD Gateway server for local addresses''' is ''ticked''.<br />
# Ensure that '''Use my RD Gateway credentials for the remote computer''' at the bottom is ''unticked''. <br />
# Press the OK button<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
You will now have a new icon on your desktop and you never need to do the above again. To connect to your DIDE desktop, click the '''Connect''' button in your new Remote Desktop shortcut.<br />
<br />
You will now be asked for your Imperial credentials in order to access the Terminal Services Gateway. Enter your details as '''IC\username''' with your Imperial password.<br />
<br />
Once that allows you through you will be prompted for your DIDE credentials in order to access your PC. Enter your details as '''DIDE\username''' with your DIDE password. If this is the first time that you've connected to your DIDE PC for a while you'll get a certificate warning message. Tick the box and select '''Yes'''.<br />
<br />
If you don't get prompted for your DIDE account that probably means your PC is asleep. To wake it up [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/remotely-access-my-college-computer/wake-my-pc/ click here].<br />
<br />
'''N.B.''' If all the above seems too difficult, I've made a template file with all the settings. It is in the '''T:''' drive in folder '''IT\VPN''' and is called '''DIDE-Remote.RDP'''. Right-click and edit that to put your correct hostname and username in and all the other settings are done for you.<br />
<br />
== Remote Access to your Ubuntu DIDE Desktop via Terminal Services Gateway ==<br />
First of all, you will need to have enabled remote desktop in advance. [https://linuxconfig.org/ubuntu-remote-desktop-18-04-bionic-beaver-linux See here for example]. You should also check you have the Remmina app installed.<br />
<br />
# Open Remmina<br />
# Click the “+” button to add a new configuration, and give it a meaningful name.<br />
# Protocol: RDP - Remote Desktop Protocol<br />
# Open Remmina and on the Basic tab fill in the following details: <br />
#* Server: Your desktop PC FQDN (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''')<br />
#* User Name: '''Your Ubuntu username'''<br />
#* Password: '''Your Ubuntu password'''<br />
#* Domain: '''DIDE''' (only if your workstation is on the DIDE domain - very likely it's not).<br />
# Adjust colour depth to '''16 bit'''.<br />
# Go to to the Advanced Tab and set up the ICT RDP Gateway with the following details:<br />
#* RDG Server: '''ictsgw.cc.ic.ac.uk'''<br />
#* RDG username: '''Your IC username'''<br />
#* RDG password: '''Your IC password'''<br />
#* RDG domain: '''IC'''<br />
# Click '''Save''' and you will have a new configuration with the name you initially specified. <br />
# Click on this new entry to connect to your server.<br />
# Accept the certificate warnings. These should only appear the very first time you connect.<br />
# Your Ubuntu remote session should now be active.<br />
<br />
<br />
== Remote Access to your DIDE Desktop via Terminal Services Gateway on a Mac ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
<br />
# Start the '''Microsoft Remote Desktop 10''' app from the App Store and select '''Add desktop''' to add a new profile entry.<br />
# Enter a name for the remote desktop profile.<br />
# Enter the full name of your DIDE machine (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''').<br />
# Change the '''User Account''' drop down to Add User Account<br />
# Enter '''DIDE\username''' in the User name field.<br />
# Enter your DIDE password in the Password field.<br />
# Enter a friendly name if desired (optional)<br />
# Click '''Add''' and '''Save''' again.<br />
# Click the Cog in the top left of the screen then Preferences.<br />
# Click Gateways.<br />
# Click on the '''+''' symbol on the bottom left of the window.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the Server name field.<br />
# Select your Imperial '''username''' in the User name field.<br />
# Enter a name for the gateway in the Friendly Name field.<br />
# Click '''Save'''.<br />
# Close the screen.<br />
# Now choose the gateway you created from the Gateway dropdown list.<br />
# Click the pencil icon.<br />
# Under '''General''' select the gateway name in the Gateway field.<br />
# Click '''Save'''.<br />
# The main screen will display your new profile. Double click this to start a remote desktop session.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Cluster_Documentation&diff=14255Cluster Documentation2023-08-24T07:20:18Z<p>Admin: /* Python */</p>
<hr />
<div>* We have two high performance clusters for general use, both running Microsoft HPC 2012 R2, Update 3. The smaller, older cluster is [[fi--dideclusthn]] and the larger is [[fi--didemrchnb]]. <br />
<br />
* Many users in the department write code and packages in R. We have in-house packages that can allow an almost-interactive experience of running jobs on our high performance clusters, so if that sounds like what you'd like, you might as well stop reading here, and go straight to [https://github.com/mrc-ide/didehpc The DIDEHPC package] - or if you need to join the mrc-ide github org first, go to [https://github.com/mrc-ide/welcome The welcome page].<br />
<br />
If you're not using R, or you are interested in other ways, then continue...<br />
<br />
== Windows Users ==<br />
<br />
The cluster nodes are Windows 2012-R2 based. This is good news if you're a Windows user, and especially easy for PCs within the departmental DIDE domain. You have the option of using the MS client for launching jobs (see below), or you can use try the new [[HPC Web Portal]]. If you're preparing to use the cluster, and have doubts about whether it can run what you want - and whether it will be straightforward to develop and run on it, best talk to me and/or the IT guys first!<br />
<br />
== Linux and MAC Users ==<br />
<br />
There are two problems, but both can be at least partially overcome. The first is that there is no client for linux. The second is that it can only run windows binaries.<br />
<br />
=== Launching jobs on Linux/Mac ===<br />
<br />
* The best way might be the [[HPC Web Portal]], which lets you submit jobs through a webpage. Give it a try, and feedback to me any issues! <br />
<br />
* Alternatively, you can either (1) go down the VM route, installing a windows virtual machine, eg VMWare or Parallels, and follow the instructions below. (2) Remote Desktop to your friend's Windows machine. These tend not to be very convenient, but some users have submitted thousands of jobs this way.<br />
<br />
=== What Linux/Mac jobs can I run? ===<br />
<br />
* Many jobs may be platform indepedent. Where compiled code is involved though, you'll have some extra work to make sure the binary parts can be run in windows. For R, [https://github.com/mrc-ide/didehpc the tools] help you sort that out. For others, you may need to run a job on the cluster to do your builds. Also, for many linux applications, there is a windows port. Mileage can vary, but it's worth a try searching for that before you give up hope. Some of these might be installed on the cluster already - see below. Or some you may be able to download and call them from your run script.<br />
<br />
* For C code, again, we can run scripts on the cluster to build the binaries for you, if your code can be compiled with (say) the MinGW port of gcc. Cygwin can also be used (although seems less common these days) with accompanying dlls. Get in touch if you need further ideas; most things can be solved. <br />
<br />
* Matlab is a little more tricky - we have a range of colossal matlab runtimes on the cluster, but you'd need to compile a windows binary to use them, since the matlab platforms are distinct in every way. For this, probably find a friend, or get in touch, and we'll work out how you can compile for it.<br />
<br />
* There is also the option of running a Windows VM, which might solve all of the above problems.<br />
<br />
=== But I really want a unix cluster. ===<br />
<br />
* We have run linux clusters in the past ([[fi--didelxhn]]), but there was very low demand, whereas the other clusters are consistently well used. Accessing DIDE network shares from linux was a little harder in some aspects, but we did not implement the more traditional linux approach of having dedicated separate cluster storage that inputs and outputs would have to be staged on to, in order for the cluster to see them. Generally, the flexibility of the other clusters makes such things quite straightforward, and may even feel a pleasure if your background is with the less domain-integrated style of cluster architecture.<br />
<br />
* It has also become far easier to write software that builds on multiple platforms with good practice. Tools that only run on one platform are rare, and in some circumstances have raised questions about the quality of those packages. Alternatives are often available.<br />
<br />
* Where compilation is the issue - for example if you want to run binaries on the cluster, but your compilation machine runs linux, or is a mac, (Matlab via MCC, or C++ code for example), then get in touch, and there are a few ways we can think of solving that, either by getting someone else, or the cluster itself, to build your code.<br />
<br />
* There are also resources in [https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/computing/high-throughput-computing/ ICT) who run a linux cluster, with the file-staging approach, and also (being shared between a much larger pool of users) have rather stricter rules on job durations and resources than we provide locally in DIDE.<br />
<br />
== Getting Started ==<br />
<br />
=== Getting access to the cluster ===<br />
<br />
Send a mail to Wes (w.hinsley@imperial.ac.uk) requesting access to the cluster. He will add you to the necessary groups, and work out what cluster you should be running on. Unless you are told otherwise, this will be fi--dideclusthn. But if you have been told otherwise, then whenever you see fi--dideclusthn, replace it with the cluster name you’ve been given, either mentally or (less effectively) on your screen with tip-ex.<br />
<br />
=== Windows: Install the HPC client software ===<br />
<br />
* Run \\fi--dideclusthn.dide.ic.ac.uk\REMINST\setup.exe. <br />
* Confirm '''Yes, I really want to run this'''.<br />
* There's an introductory screen, and you click '''Next'''<br />
* You have to tick the '''Accept''', then click '''Next'''<br />
* If it gives you any choice, select '''Install only the client utilities'''<br />
* '''Next''', '''Next''', '''Next'''<br />
* Select '''Use Microsoft Update''' and then '''Next'''<br />
* And then '''Install'''<br />
* And possibly '''Install''' again – if it had to install any pre-requisites first.<br />
* And '''Finish''', and the client installation is done.<br />
* To check everything is alright so far, open a '''NEW command prompt window'''.<br />
* (Because the installer adds things to the path, which only take effect on '''NEW comamnd windows'''! :-)<br />
* (Find it under Accessories, or you can use Start, Run, '''cmd''').<br />
* Type <code>job list /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
And it will list the jobs you have running, which will be none so far!<br />
<br />
<pre><br />
Id Owner Name State Priority Resource Request<br />
---------- ------------- ----------------- ------------ ----------- -------- -------<br />
<br />
0 matching jobs for DIDE\user<br />
</pre><br />
<br />
=== Windows: Using a non-domain machine ===<br />
<br />
If you're using a machine that isn't actually logged into DIDE, then the client software will have a problem working out who you are. In this case, there are two things you need to do.<br />
* Check you've really got a DIDE username. If you don't know what it is, talk to your lovely IT team.<br />
* Connect to the DIDE VPN (see http://www1.imperial.ac.uk/publichealth/departments/ide/it/remote/) - login to that with your DIDE account.<br />
* Alternatively to the DIDE VPN, you can install ICT's Juniper client from http://secureaccess.imperial.ac.uk, and login to that with your IC account - but you'll still need your DIDE account to access DIDE servers after the Juniper connection is up and running.<br />
* Now we'll open a command window using "runas" - which lets you choose which identity the system thinks you are within that command window:-<br />
<pre><br />
runas /netonly /user:DIDE\user cmd<br />
</pre><br />
* (Change user to your own login obviously). It will ask for your password, then open a new command window. In that window, you'll be able to do cluster-related things as your DIDE username.<br />
<br />
=== Windows: Launching and cancelling jobs ===<br />
==== Windows: Command Line ====<br />
<br />
Suppose you have a file called "run.bat" in your home directory, which does what you want to run on the cluster. Let's say it's a single-core, very simple job. We’ll discuss what should be inside "run.bat" later. To submit your job on the cluster, at the command prompt, type this (all on one line) - or put it in a file called "launch.bat", and run it:-<br />
<br />
<code>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1 \\qdrive.dide.ic.ac.uk\homes\user\run.bat</code><br />
<br />
If it's the first time you’ve run a job - or if you've recently changed your password, then it might ask you for your DIDE password and offer to remember it. Otherwise, it will just tell you the ID number of your job.<br />
<br />
<pre>Enter the password for ‘DIDE\user’ to connect to ‘FI—DIDECLUSTHN’:<br />
Remember this password? (Y/N)<br />
job has been submitted. ID: 123<br />
</pre><br />
<br />
If you want to remove the job, then:- <code>job cancel 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
Or view its details with <code>job view 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
==== Windows: Job Manager GUI ====<br />
<br />
Alternatively to the command-line, you can use the job management software, rather than the command-line. The advantage is that it’s a GUI. The disadvantage is, as in all GUIs, you may not feel totally sure you know what it’s up to – where most of the time the things you want to do might not be very complex, as above.<br />
<br />
The job management software will be on your start menu, as above. All the features are under the “Actions” menu, and hopefully it will be self explanatory. Read the details below about launching, and you'll find the interface bits that do it in the Job Manager. However, you may find over time, especially if you run lots of jobs, that learning to do it the scripted way with the command-line can be quicker.<br />
<br />
<br />
== Information for running any job ==<br />
<br />
=== Visibility ===<br />
<br />
Any executable/batch file that the cluster will run must be somewhere on the network that the cluster has access to, when it logs in as you. This amounts to any network accessible drive that you have access to when you login – including network shares, such as the temp drive, your home directory, and any specific network space set up for your project.<br />
<br />
After a job is launched, all paths are relative to the '''/workdir:''' specified on the job submit command-line. Windows does not usually allow a network path to be the current directory, and MS-HPC somehow gets around this. But if in doubt, use fully-qualified paths to specify where files should be read from, or written to. You may want to write code that takes the paths from a parameter file, or as a command-line parameter, to give as much flexibility as possible. <br />
<br />
Your home directory, and the temp drive are on departmental servers that don't have an especially fast connection to the cluster. So avoid having thousands of jobs all reading/writing to those places at once. Additionally, the temp drive is shared between all users, and your home drive has a limited quota. It is also backed up every day, so avoid using it as scratch storage for results you don’t actually want to keep. Generally, the shares you'll want to use from the clusters will be ones containing the word "NAS", which the big cluster has fast access to.<br />
<br />
=== Interactivity ===<br />
<br />
The job must run in an entirely scripted, unattended way. It must not require any key presses or mouse clicks, or other live interactivity while it runs. So jobs generally will read some sort of input (from a file, or from the way you run the job), do some processing, and write the results somewhere for you - all without intervention.<br />
<br />
== Launching jobs ==<br />
<br />
Jobs can be launched either through the Job Manager interface, or through the command line tools, which offer greater flexibility. We'll describe the command line method here; if you want to use the GUI, then it'll be a case of finding the matching boxes... Below are the specifics for our clusters. For more information, simply type <code>job</code> on the commandline, or <code>job submit</code> for the list of submission-related commands.<br />
<br />
=== FI--DIDECLUSTHN vs FI--DIDEMRCHNB ===<br />
<br />
Job submissions, as shown below, must specify a "job template", which sets up a load of default things to make the job work. On fi--dideclusthn, the job templates are called 4Core, 8Core and GeneralNodes, which will respectively force jobs to run on the 4-core nodes, the 8-core nodes, or on any machine available.<br />
<br />
On fi--didemrchnb, you can set the job template to be... 8Core, 12Core, 16Core, 12and16Core, and GeneralNodes - which hopefully are fairly self-explanatory. There are a couple of other job templates, (24Core and Phi), but those are a bit special purpose for now, so don't use them!<br />
<br />
=== Job Submission ===<br />
<br />
Job submissions from the command line can take the following form (all on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /stdout:\\path\to\stdout.txt<br />
/stderr:\\path\to\stderr.txt /numcores:1-1 /jobtemplate:4Core \\path\to\run.bat</pre><br />
<br />
=== The /singlenode argument ===<br />
<br />
In MS HPC (2012), Microsoft finally added a tag to allow you to say that the 'n' cores you requested must be on the same computer. Therefore, if you know precisely how many cores you want, then use the following (on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:true <br />
/jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
However, there is one bug with this, for the specific case where you request a whole node, regardless of how many cores it has. In this case, oddly, you have to disable single node:-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:false <br />
/numnodes:1 /jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
== Languages and libraries supported ==<br />
<br />
A number of languages, and different versions of languages are available on the cluster. The sections below refer to your "run.bat" file - a batch file that you will create which will get run by a cluster node when a suitable one is available. The commands described below are to be put in this "run.bat" file, and they add various directories to the path, so that the software you want will be added to the path. <br />
<br />
=== C/C++ === <br />
<br />
A number of Microsoft C++ and Intel C++ runtimes are installed, but it's usually better to try and avoid using them, and make your executable as stand-alone as possible. If it requires any external libraries that you've had to download, then put the .dll file in the same directory as the .exe file. If you use Microsoft Visual Studio, in Project Preferences, C/C++, Code Generation, make sure the Runtime Library is Multi-threaded (/MT) – the ones with DLL files won’t work. Even so, on recent versions of the Intel and Microsoft C compilers, "static linking" doesn’t really mean static when it comes to OpenMP, and you’ll have to copy some DLLs and put them next to your EXE file. See the OpenMP section below.<br />
<br />
The cluster nodes are all 64-bit, but they will run 32-bit compiled code. Just make sure you provide the right DLLs!<br />
<br />
Here is a document about [[Using the Xeon Phi]]<br />
<br />
Here is a document about [[Using MPI]]<br />
<br />
And if you want to use libraries such as GSL, then try [[C/C++ Libraries for Windows]]<br />
<br />
=== Java ===<br />
<br />
Java doesn't strictly need installing; you can setup any Java Runtime Environment you like, from Oracle, or anywhere. If you want to do this, then, (if you were to use the Server JRE from Oracle), something similar to the following will add java.exe and jvm.dll to the path, and set the JAVA_HOME environment variable, which most Java-using software will appreciate.<br />
<br />
set JAVA_HOME=\\my\path\to\jre1.8.0<br />
set path=%JAVA_HOME%\bin;%JAVA_HOME%\bin\server;%path%<br />
<br />
For convenience, if you're happy with Azul's OpenJDK Java 1.8.0 update 308 (September 2021), then <code>call setJava64</code> in your run script will set this up for you. Incidentally the above will also add what's needed for using the rJava package in R - in which case you'll perhaps also want something like <code>call setR64_4_1_0</code> in your run script.<br />
<br />
=== Perl ===<br />
<br />
Strawberry Perl 64-bit portable, v5.30.1.1. Put <code>call setPerl</code> at the top of your script.<br />
<br />
=== R ===<br />
<br />
'''New tools are coming''': [http://dide-tools.github.io/didehpc/vignettes/quickstart.html see here]. The instructions below are still valid if you want to do this manually.<br />
<br />
<br />
You can R jobs on all the clusters. This is the latest wisdom on how to do so – thanks to James, Jeff, Hannah and others. <br />
<br />
First, if you are wanting to use packages, then set up a package repository on your home directory by running this in R.<br />
<br />
<code>install.packages("<package>",lib="Q:/R")</code><br />
<br />
Now, write your run script. Suppose you have an R script in your home directory: Q:\R-scripts\test.R. And suppose you’ve set up your repository as above. Your run.bat should then be:-<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\test.R<br />
</pre><br />
Various packages require various R versions, and the cluster supports a few versions. To choose which one, change the first line of the script above to one of these – 32-bit or 64-bit versions of R releases. Purely for amusement, R's codenames are shown here too. <br />
<pre><br />
call setr32_2_13_1.bat call setr64_2_13_1.bat (anyone know the codename?)<br />
call setr32_2_14_2.bat call setr64_2_14_2.bat (Gift-getting season)<br />
call setr32_2_15_2.bat call setr64_2_15_2.bat (Trick or treat)<br />
call setr32_3_0_1.bat call setr64_3_0_1.bat (Good Sport)<br />
call setr32_3_0_2.bat call setr64_3_0_2.bat (Frisbee Sailing)<br />
call setr32_3_1_0.bat call setr64_3_1_0.bat (Spring Dance)<br />
call setr32_3_1_2.bat call setr64_3_1_2.bat (Pumpkin Helmet)<br />
call setr32_3_2_2.bat call setr64_3_2_2.bat (Fire Safety)<br />
call setr32_3_2_3.bat call setr64_3_2_3.bat (Wooden Christmas Tree)<br />
call setr32_3_2_4.bat call setr64_3_2_4.bat (Very Secure Dishes - Revised Version - R later renamed this to 3.2.5)<br />
call setr32_3_3_1.bat call setr64_3_3_1.bat (Bug in your hair)<br />
call setr32_3_3_2.bat call setr64_3_3_2.bat (Sincere Pumpkin Patch)<br />
call setr32_3_4_0.bat call setr64_3_4_0.bat (You Stupid Darkness)<br />
call setr32_3_4_2.bat call setr64_3_4_2.bat (Short Summer)<br />
call setr32_3_4_4.bat call setr64_3_4_4.bat (Someone to lean on)<br />
call setr32_3_5_0.bat call setr64_3_5_0.bat (Joy in Playing)<br />
call setr32_3_5_1.bat call setr64_3_5_1.bat (Feather Spray)<br />
call setr32_3_5_3.bat call setr64_3_5_3.bat (Great Truth)<br />
call setr32_3_6_0.bat call setr64_3_6_0.bat (Planting of a Tree)<br />
call setr32_3_6_1.bat call setr64_3_6_1.bat (Action of the Toes)<br />
call setr32_3_6_3.bat call setr64_3_6_3.bat (Holding the Windsock)<br />
call setr32_4_0_2.bat call setr64_4_0_2.bat (Taking Off Again)<br />
call setr32_4_0_3.bat call setr64_4_0_3.bat (Bunny-Wunnies Freak Out)<br />
call setr32_4_0_5.bat call setr64_4_0_5.bat (Shake and Throw)<br />
</pre><br />
<br />
The following versions are *not* available on the cluster at the moment - but here are their names anyway, as it seemed a shame to miss them out. Let me know if you desperately need one of these.<br />
<br />
<pre><br />
2.14.0 Great Pumpkin <br />
2.14.1 December Snowflakes<br />
2.15.0 Easter Beagle<br />
2.15.1 Roasted Marshmallow<br />
2.15.3 Security Blanket<br />
3.0.0 Masked Marvel<br />
3.0.3 Warm Puppy<br />
3.1.1 Sock it to me<br />
3.1.3 Smooth Sidewalk<br />
3.2.0 Full of Ingredients<br />
3.2.1 World famous astronaut<br />
3.2.4 Very Secure Dishes (the non-revised version)<br />
3.3.0 Supposedly Educational<br />
3.3.3 Another Canoe<br />
3.4.1 Single Candle<br />
3.4.3 Kite-Eating Tree<br />
3.5.2 Eggshell Igloo<br />
3.6.2 Dark and Stormy Night<br />
4.0.0 Arbor Day<br />
4.0.1 See Things Now<br />
4.0.4 Lost Library Book<br />
</pre><br />
<br />
It also seems that different R versions put their packages in different structures – sometimes adding "win-library" into there for fun. Basically, R_LIBS and R_LIBS_USER should be paths to a folder that contains a list of other folders, one for each package you've installed.<br />
<br />
IMPORTANT: R_LIBS and R_LIBS_USER paths must NOT contain quotes, nor spaces. If the path to your library contains spaces, you need to use old-fashioned 8-character names. If in your command window, you type <code>dir /x</code>, you’ll see the names – Program Files tends to become PROGRA~1 for instance. <br />
<br />
==== Passing parameters to R scripts ====<br />
Passing parameters to R scripts means you can have fewer versions of your R and bat files and easily run whole sets of jobs. You can get parameters into R using Rscript (but not Rcmd BATCH, I think) as follows. In the run.bat example above, the Rscript statement becomes:<br />
<pre><br />
Rscript Q:\R-scripts\test.R arg1 arg2 arg3<br />
</pre><br />
Within the R code, the arguments can be recovered using <br />
<pre><br />
args <- commandArgs(trailingOnly = TRUE)<br />
<br />
outFileName <- args[1] ## name of output file.<br />
dataFileName <- args[2] ## name of local data file. <br />
currentR0 <- as.numeric(args[3]) ## convert this explicitly to number. <br />
</pre><br />
If you want your arguments to have a particular type, best to explicitly convert (see R0 above). Better still, you can pass parameters directly into the batch file that runs the R script. Command line arguments can be referenced within the batch file using %1, %2, etc. For example, if you have a batch file, runArgs.bat:<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\%1.R %2 %3 %4<br />
</pre><br />
then<br />
<pre><br />
runArgs.bat myRScript arg1 arg2 arg3<br />
</pre><br />
will run the R script myRScript.R and pass it the parameters arg1, arg2 and arg3. The batch file runArgs.bat is now almost a '''generic R script runner'''.<br />
<br />
=== OpenMP ===<br />
<br />
This has been tested with C/C++ - but the same should apply to other languages such as fortran, that achieve multi-threading with a DLL file.<br />
<br />
==== Microsoft's C/C++ Compiler ====<br />
You will need to copy vcomp100.dll (for Visual Studio 2010 – it maybe vcomp90.dll for 2008), into the same directory as your executable. You can usually find the dll file in a directory similar to:- C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist\x64\Microsoft.VC100.OpenMP. Extrapolate for future versions! Also make sure that you've enabled OpenMP in the Properties (C++/Language).<br />
<br />
==== GCC or MinGW ====<br />
Usually, this applies to Eclipse use. In project properties, the C++ command “g++ -fopenmp”, the C command “gcc –fopenmp”, and in the Linker options, under Miscellaneous, Linker flags, put “-fopenmp” too. Copy the OpenMP DLLs from MinGW into the same directory as your final executable. You may find the dlls are in C:\MinGW\Bin\libgomp-1.dll and in the same place, libpthread-2.dll, libstdc++-6.dll and libgcc_s_dw2-1.dll.<br />
<br />
==== Intel C++ Compiler ====<br />
Copy the libiomp5md.dll file from somewhere like C:\Program Files (x86)\Intel\ComposerXE-2011\redist\intel64\compiler\libiomp5md.dll to the same place as your executable. And in Visual Studio, make sure you've enabled OpenMP in the project properties.<br />
<br />
==== How many threads? ====<br />
The OpenMP function omp_max_threads() returns the number of ''physical'' cores on a machine, not the number of cores ''allocated'' to your job. To determine how many cores the scheduler actually allocated to you, use the following code to dig up the environment variable CCP_NUMCPUS, which will be set by the cluster management software:- <br />
<pre><br />
int set_NCores() {<br />
char * val;<br />
char * endchar;<br />
val = getenv("CCP_NUMCPUS");<br />
int cores = omp_get_max_threads();<br />
if (val!=NULL) cores = strtol (val,&endchar,10);<br />
omp_set_num_threads(cores);<br />
return cores;<br />
}<br />
</pre><br />
<br />
=== WinBugs (or OpenBugs) ===<br />
<br />
They are similar, but OpenBugs 3.2.2 is the one we've gone for. Something like this as your run.bat script will work:-<br />
<pre><br />
call setOpenBugs<br />
openbugs \\path\to\script.txt /HEADLESS<br />
</pre><br />
<br />
=== MatLab ===<br />
<br />
There are various ways of producing a non-interactive executable from Matlab. Perhaps the simplest (not necessarily the best performance) way is to use “mcc.exe” – supplied with most full versions of Matlab, including the Imperial site licence version that you've probably got.<br />
<br />
==== Use mcc.exe to compile your code ====<br />
<br />
Use windows explorer to navigate to the folder where the “.m” files are for the project you want to compile. Now use a good text editor to create a file called “compile.bat” in that folder. It should contain something similar to the following:-<br />
<br />
<code>mcc -m file1.m file2.m file3.m -o myexe</code><br />
<br />
Don’t copy/paste the text from this document by the way – Word has a different idea of what a dash is to most other software, and will probably replace the two dashes with funny characters. so you have to list every single .m file that your project needs, after the’-m’. If you save this file, then double-click on it, then it will think for some while, and produce “myexe.exe” in this example. Copy your .exe file into a network accessible place as usual.<br />
<br />
==== Launch on the cluster. ====<br />
<br />
The launch.bat file will be exactly the same as before – see page 2. The run script will then start with a line that tells the cluster which version of Matlab you used to compile the cluster. Below is the table of different versions. The runtimes are huge and cumbersome to install on the cluster, so as a result I haven’t installed every single one. If you need one that’s not listed, get in touch.<br />
<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! Matlab Version<br />
! First line of run script<br />
|- <br />
| R2009b<br />
| call useMatLab79<br />
|- <br />
| R2010a<br />
| call useMatLab713<br />
|- <br />
| R2011a<br />
| call useMatLab715<br />
|- <br />
| R2011b<br />
| call useMatLab716<br />
|- <br />
| R2012a (64-bit)<br />
| call useMatLab717_64<br />
|- <br />
| R2012b (64-bit)<br />
| call useMatLab80_64<br />
|- <br />
| R2013a (64-bit)<br />
| call useMatLab81_64<br />
|- <br />
| R2014a (64-bit)<br />
| call useMatLab83_64<br />
|- <br />
| R2014b (64-bit)<br />
| call useMatLab84_64<br />
|- <br />
| R2015a (64-bit)<br />
| call useMatLab85_64<br />
|- <br />
| R2015b (64-bit)<br />
| call useMatLab90_64<br />
|- <br />
| R2016b (64-bit)<br />
| call useMatLab91_64<br />
|- <br />
| R2017a (64-bit)<br />
| call useMatLab92_64<br />
|- <br />
| R2019a (64-bit)<br />
| call useMatLab96_64<br />
|- <br />
| R2021a (64-bit)<br />
| call useMatLab910_64<br />
|- <br />
| R2022a (64-bit)<br />
| call useMatLab912_64<br />
|}<br />
<br />
=== Python ===<br />
<br />
* The most recent version of each major release is on the cluster. Different python versions within the same release (e.g., 3.5.1 and 3.5.2) do not behave nicely together in automated installation, so if you have a desperate need for a previous minor version, get in touch. While 2.7.18 is available, we '''strongly''' recommend migrating away from Python 2.7, as it has been officially discontinued on 1st January 2020. Experiences may vary, and future success is not guaranteed.<br />
<br />
<pre><br />
2.7.18 64-bit call set_python_27_64.bat<br />
3.5.9 64-bit call set_python_35_64.bat<br />
3.6.15 64-bit call set python_36_64.bat<br />
3.7.17 64-bit call set_python_37_64.bat<br />
3.8.17 64-bit call set python_38_64.bat<br />
3.9.17 64-bit call set_python_39_64.bat<br />
3.10.12 64-bit call set python_310_64.bat<br />
3.11.4 64-bit call set_python_311_64.bat<br />
<br />
</pre><br />
<br />
* You'll most likely want to use some packages, so you'll need to have a network-accessible location to put your packages in. If you're running a version of Windows with Python that matches one installed on the cluster, then you'll be able to share that package repository between the two. But if you're running a different version, or a different operating system, or a different Python version, then you'll need to keep your local package repository separate from the one the cluster uses.<br />
* Here comes the basic script to run some python. Suppose I decide to put my Python packages for the cluster in '''\\qdrive.dide.ic.ac.uk\homes\username\PythonCluster37_64''' (which is Q:\PythonCluster37_64 on a DIDE Windows PC). To run some python, something like the following:-<br />
<br />
call set_python_37_64<br />
set PYTHONPATH=\\qdrive\homes\username\PythonCluster37_64<br />
python \\qdrive\homes\username\Python\MyScript.py<br />
<br />
or if you want to do this in a programmatic way...<br />
<br />
import sys<br />
sys.path = ['\\\\qdrive\\homes\\username\\PythonCluster37_64'] + sys.path<br />
<br />
==== Packages ====<br />
<br />
* If you want to install a package like numpy, that's readily available through pip, then you can launch a batch file like this:-<br />
call set_python_36_64<br />
pip install numpy --target \\qdrive\homes\username\PythonCluster36_64<br />
* or alternatively, you could write a python script that looks like this, and launch it as normal:-<br />
import pip<br />
pip.main(['install','numpy','--target','\\\\qdrive\\homes\\username\\PythonCluster36_64'])<br />
<br />
* And if you're running the same Python version on a Windows machine, then you can do these two locally, without submitting a cluster job to do them for you.<br />
<br />
* And next time you submit a python job on the cluster, python will look in the place set by PYTHONPATH, and will find the packages you've just installed when you import them.<br />
<br />
=== Beast and Beagle ===<br />
<br />
We've got a couple of versions available. For BEAST 1.75 with BEAGLE 1.0: (the x y z represents all the command-line arguments you'll want to pass to beast)<br />
<pre><br />
call setbeast<br />
call beast-beagle x y z<br />
</pre><br />
<br />
If you want BEAST 1.82 or 1.83 with BEAGLE 2.1:<br />
<pre><br />
call setbeast_182 call setbeast_183<br />
call beast-beagle_182 x y z call beast-beagle_183 x y z<br />
</pre><br />
<br />
And if you want BEAST 2.3.2 with BEAGLE 2.1 (note that BEAST 2 is a different, er, beast to BEAST 1, regarding packages and compatibilities, so only use BEAST 2 if you know that's what you want to use).<br />
<pre><br />
call setbeast_232<br />
call beast-beagle_232 x y z<br />
</pre><br />
<br />
OpenCL is installed on all the nodes, which means Beagle will pick up the Intel Xeon processors and use what it can on them. It will also detect the Xeon Phis where they're available. If that's something you'd be particularly interested in using, drop me a line.<br />
<br />
=== Blast ===<br />
<br />
Drop me a line to join the party testing this, and to get added to the (large) BlastDB network share.<br />
<br />
<pre><br />
call setblast<br />
</pre><br />
<br />
will set the `BLASTDB` environment variable to point to what we current have of the `v5` database, and the Blast tools (2.9.0, 64-bit) will be added to the path. So you can <br />
then call `blastp`, `blastn`, `blastx` setc.<br />
<br />
=== BOW (BioInformatics on Windows) ===<br />
<br />
A number of Bio-informatics tools are ready to go. These used to be provided together on codeplex, but that has since been retired. Ugene is one project that<br />
maintains windows builds of them. Start your run.bat file with <code>call setBOW</code> to add these to the path:-<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File<br />
! Version<br />
|- <br />
| SamTools.exe<br />
| 0.1.19-44429cd<br />
|- <br />
| BCFTools.exe<br />
| 0.1.19-44428cd<br />
|- <br />
| bgzip<br />
| ?<br />
|- <br />
| razip<br />
| ?<br />
|- <br />
| bwa<br />
| 0.7.17-r1188<br />
|-<br />
| tabix<br />
| 0.2.5 (r1005)<br />
|-<br />
| minimap2<br />
| 2.24 (r1122)<br />
|-<br />
| raxml<br />
| 8.2.12 (best for each node)<br />
|-<br />
| raxml_sse3<br />
| 8.2.12 (pthreads+sse3)<br />
|-<br />
| raxml_avx<br />
| 8.2.12 (pthreads+avx)<br />
|-<br />
| raxml_avx2<br />
| 8.2.12 (pthreads+avx2)<br />
|}<br />
<br />
Note that BWA-MEM is hardwired to use a linux-only feature, so will produce an odd message and not do what you want if you use that mode on Windows. However, it seems that minimap2 may be a faster and more accurate way<br />
of doing what bwa does in any case, and that seems to have no such limitation.<br />
<br />
Also, note that raxml.exe is a copy of the most optimised build for the particular cluster node the job is running on. fi--dideclusthn only supports SSE3;<br />
fi--didemrchnb varies a little; the 32-cores support AVX2, the 24, 20 and 16 core machines support AVX, and only the oldest 12-core nodes only support SSE3. You don't have to worry about, except to expect performance to vary between them.<br />
<br />
=== GATK and Picard ===<br />
<br />
These both rely on Java, so put this in your run script:<br />
<br />
<pre><br />
call setJava64<br />
call setgatk<br />
</pre><br />
<br />
and then <code>gatk</code> in your scripts will use Python 3.7 to call the argument handling wrapper for GATK 4.2.2.0, and <code>picard</code> maps to the picard (2.26.2) jar file.<br />
<br />
=== FastQC ===<br />
<br />
This one has a slight eccentricity that instead of passing normal command-line args, it uses java's -D command-line options to buffer them out of an environment variable<br />
called _JAVA_OPTIONS. Strange but true. The ZIP comes with a bash script, which I'll reimplement if anyone actually wants me to. But in the meantime, it's easy to work <br />
out the arguments by inspection. For example, most likely usage:-<br />
<br />
<pre><br />
call setFastQC<br />
<br />
set _JAVA_OPTIONS='-Dfastqc.output_dir=%OUT%'<br />
call fastqc thing.fq<br />
</pre><br />
<br />
=== MAFFT ===<br />
<br />
For version 7.212 (Win64), write <code>call setMAFFT.bat</code> at the top of your run script.<br />
<br />
=== Applied-Maths Open Source ===<br />
<br />
Write <code>call setAppliedMaths.bat</code> at the top of your run script, to add these to the path:<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File(s)<br />
! Description<br />
! Version<br />
|- <br />
| velvetg_mt_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_mt_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| ray_x64.exe<br />
| Ray 64-bit (for MPI)<br />
| 1.6<br />
|-<br />
| ray_x86.exe<br />
| Ray 32-bit (for MPI)<br />
| 1.6<br />
|-<br />
| Mothur_x64.exe<br />
| Mothur 64-bit (for MPI)<br />
| 1.25.1<br />
|-<br />
| Mothur_x86.exe<br />
| Mothur 32-bit (for MPI)<br />
| 1.25.1<br />
|}<br />
<br />
=== Tensorflow ===<br />
<br />
Here is an example of getting Tensorflow (as a python package) available from an R script, using the cluster to install everything for you. It only works for Tensorflow 2.0 - older versions and newer versions seem to want combinations of DLLs that I can't satisfy. The following are I think minimal steps to get Tensorflow running from Python and R.<br />
<br />
* Create a folder for your python packages in your home directory (so the cluster can see it. I'm calling mine <code>Q:/python_37_repo</code> - and the cluster will call it \\fi--san03.dide.ic.ac.uk\homes\wrh1\python_37_repo - change wrh1 to your DIDE username of course.<br />
* Create a similar folder for R packages, which for me is <code>Q:/R4</code><br />
* And a third folder, Q:/Tensorflow to do this work in.<br />
* Download these DLLs and put them in the Q:/Tensorflow folder - at present they seem to be the ones Python/Tensorflow wants. Let's hope for the best... Download [https://mrcdata.dide.ic.ac.uk/resources/msvcp140.dll msvcp140.dll] and [https://mrcdata.dide.ic.ac.uk/resources/msvcp140_1.dll msvcp140_1.dll] into your <code>Q:/Tensorflow</code> folder.<br />
<br />
* Now, create a script <code>Q:/Tensorflow/prepare.bat</code>. We're going to launch this on the cluster, and get it to wire up the packages we want. Replace USER with your username...<br />
<br />
<pre><br />
net delete Q: /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set TMPDIR=%TEMP%<br />
set PYTHONPATH=\\qdrive.dide.ic.ac.uk\homes\USER\python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
pip install --no-cache-dir tensorflow==2.00 --target Q:\python_37_repo<br />
RScript Q:\Tensorflow\install_tensorflow.R<br />
</pre><br />
<br />
* Create also <code>Q:/Tensorflow/install_tensorflow.R</code> contains:<br />
<pre><br />
install.packages(c("reticulate", "tensorflow"),lib="Q:/R4",repos="https://cran.ma.imperial.ac.uk/")<br />
</pre><br />
<br />
* Now submit the prepare job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> in the big window will do it. That will take a while, but you should notice lots of new files in your Q:/python_37_repo folder. You'll only need to do this once (unless you delete your Python repo folder, or it randomly breaks and you start again...)<br />
<br />
* Next, create a file <code>Q:/Tensorflow/run.bat</code> containing the following the two "call" lines near the end are the ones that do some "work" in python or R, so this script basically a template for running your future jobs. Just change those two calls to set off the python or R code you want to run that uses Tensorflow.<br />
<br />
<pre><br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
net use Q: /delete /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set PYTHONPATH=Q:/python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call python Q:\tensorflow\test_tensorflow.py<br />
call RScript Q:\tensorflow\test_tensorflow.R<br />
</pre><br />
<br />
* And finally, you'll need <code>Q:\tensorflow\test_tensorflow.py</code> and <code>Q:\tensorflow\test_tensorflow.R</code> to exist. Here are minimal tests that everything is wired up. <code.test_tensorflow.py</code> first:-<br />
<br />
<pre><br />
import tensorflow as tf;<br />
print(tf.reduce_sum(tf.random.normal([1000, 1000])));<br />
</pre><br />
<br />
and <code>Q:\Tensorflow\test_tensorflow.R</code> contains:<br />
<br />
<pre><br />
library(reticulate)<br />
reticulate::py_config()<br />
library(tensorflow)<br />
tf$constant('Hello, TensorFlow!')<br />
tf$Variable(tf$zeros(shape(1L)))<br />
</pre><br />
<br />
* Now submit the run job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> in the big window will do it - and perhaps paste <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\output.txt</code> in the stdout box to collect the input and see how it gets on. You should see sensible output from both tests.<br />
<br />
== Launching a job ==<br />
<br />
<br />
== Submitting many jobs ==<br />
<br />
Suppose you write an exe that you might run with… Mysim.exe init.txt 1 2 3, and you want to run it many times with a range of parameters. One way of many, is to write a launch.bat file that will run “job submit” separately, for example (thanks Tini/James!):-<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set initFile=\\networkshare\job\init.txt<br />
set exeFile=\\networkshare\job\mysim.exe<br />
<br />
%SubDetails% %exeFile% %initfile% 1 2 3<br />
%SubDetails% %exeFile% %initfile% 4 5 6<br />
%SubDetails% %exeFile% %initfile% 7 8 9<br />
</pre><br />
<br />
Suppose the job you want to run is an R script? To specify arguments to an R script, you have to add ′--args a=1 b=2′ - so… you might make launch.bat like this:-<br />
<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set rbatFile=\\networkshare\R-scripts\run.bat<br />
<br />
%SubDetails% %rbatFile% 1 2<br />
%SubDetails% %rbatFile% 3 4<br />
</pre><br />
And make the significant line of your run.bat:-<br />
<code><br />
Rcmd BATCH ′--args a=%1 b=%2′ U:\R-scripts\test.R<br />
</code><br />
%1 and %2 will map to the first and second thing after the batch file. You can go all the way up to %9.<br />
<br />
IMPORTANT NOTE<br />
<br />
Make sure you get the apostrophe character right in the above example – NEVER copy and paste from a word document into a script. It will go hideously wrong. Type the apostrophes (and dashes for that matter) in a good text editor – you want just the standard old-fashioned characters.<br />
<br />
== Requesting resources ==<br />
<br />
The following modifiers next to the “/scheduler:” part of the job submit line (before your app.exe 1 2 3 part), will request things you might want…<br />
<br />
/numcores:8 - number of cores you want<br />
<br />
/numcores:8-12 - minimum and maximum cores appropriate for your job<br />
<br />
/memorypernode:1024 - amount of mem in MegaBytes needed.<br />
<br />
/workdir:\\networkshare\work - set working directory<br />
<br />
/stdout:\\networkshare\out.txt - divert stdout to a file<br />
<br />
/stderr: or /stdin: - similar for stderr and stdin<br />
<br />
== Troubleshooting / Miscellany / Q & A ==<br />
<br />
* My job doesn’t work.<br />
** Run the HPC Job Manager application. Find your job id, possible under “My Jobs, Failed”. Double click on it, then on “View All Tasks”. Perhaps something in the output section will help.<br />
<br />
* Check that the path to your job is visible everywhere.<br />
** Avoid spaces in your paths - “job submit” doesn’t like them very much. If you must have them, they’ll be ok in the “run.bat” batch file that the cluster will run – in which case, put the path in standard double-quotes ("). But avoid them in your “launch.bat” file – you may have to relocate your run.bat to a simple non-space-containing directory.<br />
** Rather than putting the full application and parameters on the job submit line, you might want to write a batch-file to do all that, and submit the batch file to the cluster. (See section 6 about R for example). But make sure the batch file is somewhere visible to the cluster.<br />
<br />
* My job seems to work, but reports as having failed.<br />
** The success/failure depends on the error code returned. If you’re running C code, end with “return 0;” for success.<br />
<br />
* job submit ..blah blah.. app.exe >out.txt doesn’t work!<br />
** The contents of out.txt will be the result of “job submit”, not the result of “app.exe”. You meant to say this job submit ..blah blah.. /stdout:out.txt app.exe correcting out.txt and app.exe to network paths of course.<br />
<br />
* My C/C++ Executable works perfectly on my desktop, but doesn't work on the cluster. What's wrong with the cluster?<br />
** Your EXE file probably relies on some DLL files that the cluster doesn't have. If you use Visual Studio, check that in Code Generation, you have "Multi-Threaded (/MT)" as your target - with no mention of DLLs. If your code uses OpenMP, check you've copied the right OpenMP DLL(s) (go back to the C section above). As a last resort, give your EXE to someone who doesn't have Visual Studio installed, and get them to run it, and see what DLL files it seems to ask for.<br />
<br />
== Contributing Authors ==<br />
<br />
Wes Hinsley, James Truscott, Tini Garske, Jeff Eaton, Hannah Clapham</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Cluster_Documentation&diff=14254Cluster Documentation2023-08-24T06:50:51Z<p>Admin: /* Python */</p>
<hr />
<div>* We have two high performance clusters for general use, both running Microsoft HPC 2012 R2, Update 3. The smaller, older cluster is [[fi--dideclusthn]] and the larger is [[fi--didemrchnb]]. <br />
<br />
* Many users in the department write code and packages in R. We have in-house packages that can allow an almost-interactive experience of running jobs on our high performance clusters, so if that sounds like what you'd like, you might as well stop reading here, and go straight to [https://github.com/mrc-ide/didehpc The DIDEHPC package] - or if you need to join the mrc-ide github org first, go to [https://github.com/mrc-ide/welcome The welcome page].<br />
<br />
If you're not using R, or you are interested in other ways, then continue...<br />
<br />
== Windows Users ==<br />
<br />
The cluster nodes are Windows 2012-R2 based. This is good news if you're a Windows user, and especially easy for PCs within the departmental DIDE domain. You have the option of using the MS client for launching jobs (see below), or you can use try the new [[HPC Web Portal]]. If you're preparing to use the cluster, and have doubts about whether it can run what you want - and whether it will be straightforward to develop and run on it, best talk to me and/or the IT guys first!<br />
<br />
== Linux and MAC Users ==<br />
<br />
There are two problems, but both can be at least partially overcome. The first is that there is no client for linux. The second is that it can only run windows binaries.<br />
<br />
=== Launching jobs on Linux/Mac ===<br />
<br />
* The best way might be the [[HPC Web Portal]], which lets you submit jobs through a webpage. Give it a try, and feedback to me any issues! <br />
<br />
* Alternatively, you can either (1) go down the VM route, installing a windows virtual machine, eg VMWare or Parallels, and follow the instructions below. (2) Remote Desktop to your friend's Windows machine. These tend not to be very convenient, but some users have submitted thousands of jobs this way.<br />
<br />
=== What Linux/Mac jobs can I run? ===<br />
<br />
* Many jobs may be platform indepedent. Where compiled code is involved though, you'll have some extra work to make sure the binary parts can be run in windows. For R, [https://github.com/mrc-ide/didehpc the tools] help you sort that out. For others, you may need to run a job on the cluster to do your builds. Also, for many linux applications, there is a windows port. Mileage can vary, but it's worth a try searching for that before you give up hope. Some of these might be installed on the cluster already - see below. Or some you may be able to download and call them from your run script.<br />
<br />
* For C code, again, we can run scripts on the cluster to build the binaries for you, if your code can be compiled with (say) the MinGW port of gcc. Cygwin can also be used (although seems less common these days) with accompanying dlls. Get in touch if you need further ideas; most things can be solved. <br />
<br />
* Matlab is a little more tricky - we have a range of colossal matlab runtimes on the cluster, but you'd need to compile a windows binary to use them, since the matlab platforms are distinct in every way. For this, probably find a friend, or get in touch, and we'll work out how you can compile for it.<br />
<br />
* There is also the option of running a Windows VM, which might solve all of the above problems.<br />
<br />
=== But I really want a unix cluster. ===<br />
<br />
* We have run linux clusters in the past ([[fi--didelxhn]]), but there was very low demand, whereas the other clusters are consistently well used. Accessing DIDE network shares from linux was a little harder in some aspects, but we did not implement the more traditional linux approach of having dedicated separate cluster storage that inputs and outputs would have to be staged on to, in order for the cluster to see them. Generally, the flexibility of the other clusters makes such things quite straightforward, and may even feel a pleasure if your background is with the less domain-integrated style of cluster architecture.<br />
<br />
* It has also become far easier to write software that builds on multiple platforms with good practice. Tools that only run on one platform are rare, and in some circumstances have raised questions about the quality of those packages. Alternatives are often available.<br />
<br />
* Where compilation is the issue - for example if you want to run binaries on the cluster, but your compilation machine runs linux, or is a mac, (Matlab via MCC, or C++ code for example), then get in touch, and there are a few ways we can think of solving that, either by getting someone else, or the cluster itself, to build your code.<br />
<br />
* There are also resources in [https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/computing/high-throughput-computing/ ICT) who run a linux cluster, with the file-staging approach, and also (being shared between a much larger pool of users) have rather stricter rules on job durations and resources than we provide locally in DIDE.<br />
<br />
== Getting Started ==<br />
<br />
=== Getting access to the cluster ===<br />
<br />
Send a mail to Wes (w.hinsley@imperial.ac.uk) requesting access to the cluster. He will add you to the necessary groups, and work out what cluster you should be running on. Unless you are told otherwise, this will be fi--dideclusthn. But if you have been told otherwise, then whenever you see fi--dideclusthn, replace it with the cluster name you’ve been given, either mentally or (less effectively) on your screen with tip-ex.<br />
<br />
=== Windows: Install the HPC client software ===<br />
<br />
* Run \\fi--dideclusthn.dide.ic.ac.uk\REMINST\setup.exe. <br />
* Confirm '''Yes, I really want to run this'''.<br />
* There's an introductory screen, and you click '''Next'''<br />
* You have to tick the '''Accept''', then click '''Next'''<br />
* If it gives you any choice, select '''Install only the client utilities'''<br />
* '''Next''', '''Next''', '''Next'''<br />
* Select '''Use Microsoft Update''' and then '''Next'''<br />
* And then '''Install'''<br />
* And possibly '''Install''' again – if it had to install any pre-requisites first.<br />
* And '''Finish''', and the client installation is done.<br />
* To check everything is alright so far, open a '''NEW command prompt window'''.<br />
* (Because the installer adds things to the path, which only take effect on '''NEW comamnd windows'''! :-)<br />
* (Find it under Accessories, or you can use Start, Run, '''cmd''').<br />
* Type <code>job list /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
And it will list the jobs you have running, which will be none so far!<br />
<br />
<pre><br />
Id Owner Name State Priority Resource Request<br />
---------- ------------- ----------------- ------------ ----------- -------- -------<br />
<br />
0 matching jobs for DIDE\user<br />
</pre><br />
<br />
=== Windows: Using a non-domain machine ===<br />
<br />
If you're using a machine that isn't actually logged into DIDE, then the client software will have a problem working out who you are. In this case, there are two things you need to do.<br />
* Check you've really got a DIDE username. If you don't know what it is, talk to your lovely IT team.<br />
* Connect to the DIDE VPN (see http://www1.imperial.ac.uk/publichealth/departments/ide/it/remote/) - login to that with your DIDE account.<br />
* Alternatively to the DIDE VPN, you can install ICT's Juniper client from http://secureaccess.imperial.ac.uk, and login to that with your IC account - but you'll still need your DIDE account to access DIDE servers after the Juniper connection is up and running.<br />
* Now we'll open a command window using "runas" - which lets you choose which identity the system thinks you are within that command window:-<br />
<pre><br />
runas /netonly /user:DIDE\user cmd<br />
</pre><br />
* (Change user to your own login obviously). It will ask for your password, then open a new command window. In that window, you'll be able to do cluster-related things as your DIDE username.<br />
<br />
=== Windows: Launching and cancelling jobs ===<br />
==== Windows: Command Line ====<br />
<br />
Suppose you have a file called "run.bat" in your home directory, which does what you want to run on the cluster. Let's say it's a single-core, very simple job. We’ll discuss what should be inside "run.bat" later. To submit your job on the cluster, at the command prompt, type this (all on one line) - or put it in a file called "launch.bat", and run it:-<br />
<br />
<code>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1 \\qdrive.dide.ic.ac.uk\homes\user\run.bat</code><br />
<br />
If it's the first time you’ve run a job - or if you've recently changed your password, then it might ask you for your DIDE password and offer to remember it. Otherwise, it will just tell you the ID number of your job.<br />
<br />
<pre>Enter the password for ‘DIDE\user’ to connect to ‘FI—DIDECLUSTHN’:<br />
Remember this password? (Y/N)<br />
job has been submitted. ID: 123<br />
</pre><br />
<br />
If you want to remove the job, then:- <code>job cancel 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
Or view its details with <code>job view 123 /scheduler:fi--dideclusthn.dide.ic.ac.uk</code><br />
<br />
==== Windows: Job Manager GUI ====<br />
<br />
Alternatively to the command-line, you can use the job management software, rather than the command-line. The advantage is that it’s a GUI. The disadvantage is, as in all GUIs, you may not feel totally sure you know what it’s up to – where most of the time the things you want to do might not be very complex, as above.<br />
<br />
The job management software will be on your start menu, as above. All the features are under the “Actions” menu, and hopefully it will be self explanatory. Read the details below about launching, and you'll find the interface bits that do it in the Job Manager. However, you may find over time, especially if you run lots of jobs, that learning to do it the scripted way with the command-line can be quicker.<br />
<br />
<br />
== Information for running any job ==<br />
<br />
=== Visibility ===<br />
<br />
Any executable/batch file that the cluster will run must be somewhere on the network that the cluster has access to, when it logs in as you. This amounts to any network accessible drive that you have access to when you login – including network shares, such as the temp drive, your home directory, and any specific network space set up for your project.<br />
<br />
After a job is launched, all paths are relative to the '''/workdir:''' specified on the job submit command-line. Windows does not usually allow a network path to be the current directory, and MS-HPC somehow gets around this. But if in doubt, use fully-qualified paths to specify where files should be read from, or written to. You may want to write code that takes the paths from a parameter file, or as a command-line parameter, to give as much flexibility as possible. <br />
<br />
Your home directory, and the temp drive are on departmental servers that don't have an especially fast connection to the cluster. So avoid having thousands of jobs all reading/writing to those places at once. Additionally, the temp drive is shared between all users, and your home drive has a limited quota. It is also backed up every day, so avoid using it as scratch storage for results you don’t actually want to keep. Generally, the shares you'll want to use from the clusters will be ones containing the word "NAS", which the big cluster has fast access to.<br />
<br />
=== Interactivity ===<br />
<br />
The job must run in an entirely scripted, unattended way. It must not require any key presses or mouse clicks, or other live interactivity while it runs. So jobs generally will read some sort of input (from a file, or from the way you run the job), do some processing, and write the results somewhere for you - all without intervention.<br />
<br />
== Launching jobs ==<br />
<br />
Jobs can be launched either through the Job Manager interface, or through the command line tools, which offer greater flexibility. We'll describe the command line method here; if you want to use the GUI, then it'll be a case of finding the matching boxes... Below are the specifics for our clusters. For more information, simply type <code>job</code> on the commandline, or <code>job submit</code> for the list of submission-related commands.<br />
<br />
=== FI--DIDECLUSTHN vs FI--DIDEMRCHNB ===<br />
<br />
Job submissions, as shown below, must specify a "job template", which sets up a load of default things to make the job work. On fi--dideclusthn, the job templates are called 4Core, 8Core and GeneralNodes, which will respectively force jobs to run on the 4-core nodes, the 8-core nodes, or on any machine available.<br />
<br />
On fi--didemrchnb, you can set the job template to be... 8Core, 12Core, 16Core, 12and16Core, and GeneralNodes - which hopefully are fairly self-explanatory. There are a couple of other job templates, (24Core and Phi), but those are a bit special purpose for now, so don't use them!<br />
<br />
=== Job Submission ===<br />
<br />
Job submissions from the command line can take the following form (all on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /stdout:\\path\to\stdout.txt<br />
/stderr:\\path\to\stderr.txt /numcores:1-1 /jobtemplate:4Core \\path\to\run.bat</pre><br />
<br />
=== The /singlenode argument ===<br />
<br />
In MS HPC (2012), Microsoft finally added a tag to allow you to say that the 'n' cores you requested must be on the same computer. Therefore, if you know precisely how many cores you want, then use the following (on one line):-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:true <br />
/jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
However, there is one bug with this, for the specific case where you request a whole node, regardless of how many cores it has. In this case, oddly, you have to disable single node:-<br />
<br />
<pre>job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /singlenode:false <br />
/numnodes:1 /jobtemplate:8Core \\path\to\run.bat</pre><br />
<br />
== Languages and libraries supported ==<br />
<br />
A number of languages, and different versions of languages are available on the cluster. The sections below refer to your "run.bat" file - a batch file that you will create which will get run by a cluster node when a suitable one is available. The commands described below are to be put in this "run.bat" file, and they add various directories to the path, so that the software you want will be added to the path. <br />
<br />
=== C/C++ === <br />
<br />
A number of Microsoft C++ and Intel C++ runtimes are installed, but it's usually better to try and avoid using them, and make your executable as stand-alone as possible. If it requires any external libraries that you've had to download, then put the .dll file in the same directory as the .exe file. If you use Microsoft Visual Studio, in Project Preferences, C/C++, Code Generation, make sure the Runtime Library is Multi-threaded (/MT) – the ones with DLL files won’t work. Even so, on recent versions of the Intel and Microsoft C compilers, "static linking" doesn’t really mean static when it comes to OpenMP, and you’ll have to copy some DLLs and put them next to your EXE file. See the OpenMP section below.<br />
<br />
The cluster nodes are all 64-bit, but they will run 32-bit compiled code. Just make sure you provide the right DLLs!<br />
<br />
Here is a document about [[Using the Xeon Phi]]<br />
<br />
Here is a document about [[Using MPI]]<br />
<br />
And if you want to use libraries such as GSL, then try [[C/C++ Libraries for Windows]]<br />
<br />
=== Java ===<br />
<br />
Java doesn't strictly need installing; you can setup any Java Runtime Environment you like, from Oracle, or anywhere. If you want to do this, then, (if you were to use the Server JRE from Oracle), something similar to the following will add java.exe and jvm.dll to the path, and set the JAVA_HOME environment variable, which most Java-using software will appreciate.<br />
<br />
set JAVA_HOME=\\my\path\to\jre1.8.0<br />
set path=%JAVA_HOME%\bin;%JAVA_HOME%\bin\server;%path%<br />
<br />
For convenience, if you're happy with Azul's OpenJDK Java 1.8.0 update 308 (September 2021), then <code>call setJava64</code> in your run script will set this up for you. Incidentally the above will also add what's needed for using the rJava package in R - in which case you'll perhaps also want something like <code>call setR64_4_1_0</code> in your run script.<br />
<br />
=== Perl ===<br />
<br />
Strawberry Perl 64-bit portable, v5.30.1.1. Put <code>call setPerl</code> at the top of your script.<br />
<br />
=== R ===<br />
<br />
'''New tools are coming''': [http://dide-tools.github.io/didehpc/vignettes/quickstart.html see here]. The instructions below are still valid if you want to do this manually.<br />
<br />
<br />
You can R jobs on all the clusters. This is the latest wisdom on how to do so – thanks to James, Jeff, Hannah and others. <br />
<br />
First, if you are wanting to use packages, then set up a package repository on your home directory by running this in R.<br />
<br />
<code>install.packages("<package>",lib="Q:/R")</code><br />
<br />
Now, write your run script. Suppose you have an R script in your home directory: Q:\R-scripts\test.R. And suppose you’ve set up your repository as above. Your run.bat should then be:-<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\test.R<br />
</pre><br />
Various packages require various R versions, and the cluster supports a few versions. To choose which one, change the first line of the script above to one of these – 32-bit or 64-bit versions of R releases. Purely for amusement, R's codenames are shown here too. <br />
<pre><br />
call setr32_2_13_1.bat call setr64_2_13_1.bat (anyone know the codename?)<br />
call setr32_2_14_2.bat call setr64_2_14_2.bat (Gift-getting season)<br />
call setr32_2_15_2.bat call setr64_2_15_2.bat (Trick or treat)<br />
call setr32_3_0_1.bat call setr64_3_0_1.bat (Good Sport)<br />
call setr32_3_0_2.bat call setr64_3_0_2.bat (Frisbee Sailing)<br />
call setr32_3_1_0.bat call setr64_3_1_0.bat (Spring Dance)<br />
call setr32_3_1_2.bat call setr64_3_1_2.bat (Pumpkin Helmet)<br />
call setr32_3_2_2.bat call setr64_3_2_2.bat (Fire Safety)<br />
call setr32_3_2_3.bat call setr64_3_2_3.bat (Wooden Christmas Tree)<br />
call setr32_3_2_4.bat call setr64_3_2_4.bat (Very Secure Dishes - Revised Version - R later renamed this to 3.2.5)<br />
call setr32_3_3_1.bat call setr64_3_3_1.bat (Bug in your hair)<br />
call setr32_3_3_2.bat call setr64_3_3_2.bat (Sincere Pumpkin Patch)<br />
call setr32_3_4_0.bat call setr64_3_4_0.bat (You Stupid Darkness)<br />
call setr32_3_4_2.bat call setr64_3_4_2.bat (Short Summer)<br />
call setr32_3_4_4.bat call setr64_3_4_4.bat (Someone to lean on)<br />
call setr32_3_5_0.bat call setr64_3_5_0.bat (Joy in Playing)<br />
call setr32_3_5_1.bat call setr64_3_5_1.bat (Feather Spray)<br />
call setr32_3_5_3.bat call setr64_3_5_3.bat (Great Truth)<br />
call setr32_3_6_0.bat call setr64_3_6_0.bat (Planting of a Tree)<br />
call setr32_3_6_1.bat call setr64_3_6_1.bat (Action of the Toes)<br />
call setr32_3_6_3.bat call setr64_3_6_3.bat (Holding the Windsock)<br />
call setr32_4_0_2.bat call setr64_4_0_2.bat (Taking Off Again)<br />
call setr32_4_0_3.bat call setr64_4_0_3.bat (Bunny-Wunnies Freak Out)<br />
call setr32_4_0_5.bat call setr64_4_0_5.bat (Shake and Throw)<br />
</pre><br />
<br />
The following versions are *not* available on the cluster at the moment - but here are their names anyway, as it seemed a shame to miss them out. Let me know if you desperately need one of these.<br />
<br />
<pre><br />
2.14.0 Great Pumpkin <br />
2.14.1 December Snowflakes<br />
2.15.0 Easter Beagle<br />
2.15.1 Roasted Marshmallow<br />
2.15.3 Security Blanket<br />
3.0.0 Masked Marvel<br />
3.0.3 Warm Puppy<br />
3.1.1 Sock it to me<br />
3.1.3 Smooth Sidewalk<br />
3.2.0 Full of Ingredients<br />
3.2.1 World famous astronaut<br />
3.2.4 Very Secure Dishes (the non-revised version)<br />
3.3.0 Supposedly Educational<br />
3.3.3 Another Canoe<br />
3.4.1 Single Candle<br />
3.4.3 Kite-Eating Tree<br />
3.5.2 Eggshell Igloo<br />
3.6.2 Dark and Stormy Night<br />
4.0.0 Arbor Day<br />
4.0.1 See Things Now<br />
4.0.4 Lost Library Book<br />
</pre><br />
<br />
It also seems that different R versions put their packages in different structures – sometimes adding "win-library" into there for fun. Basically, R_LIBS and R_LIBS_USER should be paths to a folder that contains a list of other folders, one for each package you've installed.<br />
<br />
IMPORTANT: R_LIBS and R_LIBS_USER paths must NOT contain quotes, nor spaces. If the path to your library contains spaces, you need to use old-fashioned 8-character names. If in your command window, you type <code>dir /x</code>, you’ll see the names – Program Files tends to become PROGRA~1 for instance. <br />
<br />
==== Passing parameters to R scripts ====<br />
Passing parameters to R scripts means you can have fewer versions of your R and bat files and easily run whole sets of jobs. You can get parameters into R using Rscript (but not Rcmd BATCH, I think) as follows. In the run.bat example above, the Rscript statement becomes:<br />
<pre><br />
Rscript Q:\R-scripts\test.R arg1 arg2 arg3<br />
</pre><br />
Within the R code, the arguments can be recovered using <br />
<pre><br />
args <- commandArgs(trailingOnly = TRUE)<br />
<br />
outFileName <- args[1] ## name of output file.<br />
dataFileName <- args[2] ## name of local data file. <br />
currentR0 <- as.numeric(args[3]) ## convert this explicitly to number. <br />
</pre><br />
If you want your arguments to have a particular type, best to explicitly convert (see R0 above). Better still, you can pass parameters directly into the batch file that runs the R script. Command line arguments can be referenced within the batch file using %1, %2, etc. For example, if you have a batch file, runArgs.bat:<br />
<pre><br />
call <script for the R version you want – see below><br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\user<br />
set R_LIBS=Q:\R<br />
set R_LIBS_USER=Q:\R<br />
Rscript Q:\R-scripts\%1.R %2 %3 %4<br />
</pre><br />
then<br />
<pre><br />
runArgs.bat myRScript arg1 arg2 arg3<br />
</pre><br />
will run the R script myRScript.R and pass it the parameters arg1, arg2 and arg3. The batch file runArgs.bat is now almost a '''generic R script runner'''.<br />
<br />
=== OpenMP ===<br />
<br />
This has been tested with C/C++ - but the same should apply to other languages such as fortran, that achieve multi-threading with a DLL file.<br />
<br />
==== Microsoft's C/C++ Compiler ====<br />
You will need to copy vcomp100.dll (for Visual Studio 2010 – it maybe vcomp90.dll for 2008), into the same directory as your executable. You can usually find the dll file in a directory similar to:- C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\redist\x64\Microsoft.VC100.OpenMP. Extrapolate for future versions! Also make sure that you've enabled OpenMP in the Properties (C++/Language).<br />
<br />
==== GCC or MinGW ====<br />
Usually, this applies to Eclipse use. In project properties, the C++ command “g++ -fopenmp”, the C command “gcc –fopenmp”, and in the Linker options, under Miscellaneous, Linker flags, put “-fopenmp” too. Copy the OpenMP DLLs from MinGW into the same directory as your final executable. You may find the dlls are in C:\MinGW\Bin\libgomp-1.dll and in the same place, libpthread-2.dll, libstdc++-6.dll and libgcc_s_dw2-1.dll.<br />
<br />
==== Intel C++ Compiler ====<br />
Copy the libiomp5md.dll file from somewhere like C:\Program Files (x86)\Intel\ComposerXE-2011\redist\intel64\compiler\libiomp5md.dll to the same place as your executable. And in Visual Studio, make sure you've enabled OpenMP in the project properties.<br />
<br />
==== How many threads? ====<br />
The OpenMP function omp_max_threads() returns the number of ''physical'' cores on a machine, not the number of cores ''allocated'' to your job. To determine how many cores the scheduler actually allocated to you, use the following code to dig up the environment variable CCP_NUMCPUS, which will be set by the cluster management software:- <br />
<pre><br />
int set_NCores() {<br />
char * val;<br />
char * endchar;<br />
val = getenv("CCP_NUMCPUS");<br />
int cores = omp_get_max_threads();<br />
if (val!=NULL) cores = strtol (val,&endchar,10);<br />
omp_set_num_threads(cores);<br />
return cores;<br />
}<br />
</pre><br />
<br />
=== WinBugs (or OpenBugs) ===<br />
<br />
They are similar, but OpenBugs 3.2.2 is the one we've gone for. Something like this as your run.bat script will work:-<br />
<pre><br />
call setOpenBugs<br />
openbugs \\path\to\script.txt /HEADLESS<br />
</pre><br />
<br />
=== MatLab ===<br />
<br />
There are various ways of producing a non-interactive executable from Matlab. Perhaps the simplest (not necessarily the best performance) way is to use “mcc.exe” – supplied with most full versions of Matlab, including the Imperial site licence version that you've probably got.<br />
<br />
==== Use mcc.exe to compile your code ====<br />
<br />
Use windows explorer to navigate to the folder where the “.m” files are for the project you want to compile. Now use a good text editor to create a file called “compile.bat” in that folder. It should contain something similar to the following:-<br />
<br />
<code>mcc -m file1.m file2.m file3.m -o myexe</code><br />
<br />
Don’t copy/paste the text from this document by the way – Word has a different idea of what a dash is to most other software, and will probably replace the two dashes with funny characters. so you have to list every single .m file that your project needs, after the’-m’. If you save this file, then double-click on it, then it will think for some while, and produce “myexe.exe” in this example. Copy your .exe file into a network accessible place as usual.<br />
<br />
==== Launch on the cluster. ====<br />
<br />
The launch.bat file will be exactly the same as before – see page 2. The run script will then start with a line that tells the cluster which version of Matlab you used to compile the cluster. Below is the table of different versions. The runtimes are huge and cumbersome to install on the cluster, so as a result I haven’t installed every single one. If you need one that’s not listed, get in touch.<br />
<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! Matlab Version<br />
! First line of run script<br />
|- <br />
| R2009b<br />
| call useMatLab79<br />
|- <br />
| R2010a<br />
| call useMatLab713<br />
|- <br />
| R2011a<br />
| call useMatLab715<br />
|- <br />
| R2011b<br />
| call useMatLab716<br />
|- <br />
| R2012a (64-bit)<br />
| call useMatLab717_64<br />
|- <br />
| R2012b (64-bit)<br />
| call useMatLab80_64<br />
|- <br />
| R2013a (64-bit)<br />
| call useMatLab81_64<br />
|- <br />
| R2014a (64-bit)<br />
| call useMatLab83_64<br />
|- <br />
| R2014b (64-bit)<br />
| call useMatLab84_64<br />
|- <br />
| R2015a (64-bit)<br />
| call useMatLab85_64<br />
|- <br />
| R2015b (64-bit)<br />
| call useMatLab90_64<br />
|- <br />
| R2016b (64-bit)<br />
| call useMatLab91_64<br />
|- <br />
| R2017a (64-bit)<br />
| call useMatLab92_64<br />
|- <br />
| R2019a (64-bit)<br />
| call useMatLab96_64<br />
|- <br />
| R2021a (64-bit)<br />
| call useMatLab910_64<br />
|- <br />
| R2022a (64-bit)<br />
| call useMatLab912_64<br />
|}<br />
<br />
=== Python ===<br />
<br />
* Many Python versions are available on the cluster. Different python versions within the same release (e.g., 3.5.1 and 3.5.2) do not behave nicely together in automated installation, so if you have a desperate need for a previous minor version, get in touch. While 2.7.18 is available, we '''strongly''' recommend migrating away from Python 2.7, as it has been officially discontinued on 1st January 2020. Experiences may vary, and future success is not guaranteed.<br />
<br />
<pre><br />
2.7.18 64-bit call set_python_27_64.bat<br />
3.5.9 64-bit call set_python_35_64.bat<br />
3.6.15 64-bit call set python_36_64.bat<br />
3.7.17 64-bit call set_python_37_64.bat<br />
3.8.17 64-bit call set python_38_64.bat<br />
3.9.17 64-bit call set_python_39_64.bat<br />
3.10.12 64-bit call set python_310_64.bat<br />
3.11.4 64-bit call set_python_311_64.bat<br />
<br />
</pre><br />
<br />
* You'll most likely want to use some packages, so you'll need to have a network-accessible location to put your packages in. If you're running a version of Windows with Python that matches one installed on the cluster, then you'll be able to share that package repository between the two. But if you're running a different version, or a different operating system, or a different Python version, then you'll need to keep your local package repository separate from the one the cluster uses.<br />
* Here comes the basic script to run some python. Suppose I decide to put my Python packages for the cluster in '''\\qdrive.dide.ic.ac.uk\homes\username\PythonCluster37_64''' (which is Q:\PythonCluster37_64 on a DIDE Windows PC). To run some python, something like the following:-<br />
<br />
call set_python_37_64<br />
set PYTHONPATH=\\qdrive\homes\username\PythonCluster37_64<br />
python \\qdrive\homes\username\Python\MyScript.py<br />
<br />
or if you want to do this in a programmatic way...<br />
<br />
import sys<br />
sys.path = ['\\\\qdrive\\homes\\username\\PythonCluster37_64'] + sys.path<br />
<br />
==== Packages ====<br />
<br />
* If you want to install a package like numpy, that's readily available through pip, then you can launch a batch file like this:-<br />
call set_python_36_64<br />
pip install numpy --target \\qdrive\homes\username\PythonCluster36_64<br />
* or alternatively, you could write a python script that looks like this, and launch it as normal:-<br />
import pip<br />
pip.main(['install','numpy','--target','\\\\qdrive\\homes\\username\\PythonCluster36_64'])<br />
<br />
* And if you're running the same Python version on a Windows machine, then you can do these two locally, without submitting a cluster job to do them for you.<br />
<br />
* And next time you submit a python job on the cluster, python will look in the place set by PYTHONPATH, and will find the packages you've just installed when you import them.<br />
<br />
=== Beast and Beagle ===<br />
<br />
We've got a couple of versions available. For BEAST 1.75 with BEAGLE 1.0: (the x y z represents all the command-line arguments you'll want to pass to beast)<br />
<pre><br />
call setbeast<br />
call beast-beagle x y z<br />
</pre><br />
<br />
If you want BEAST 1.82 or 1.83 with BEAGLE 2.1:<br />
<pre><br />
call setbeast_182 call setbeast_183<br />
call beast-beagle_182 x y z call beast-beagle_183 x y z<br />
</pre><br />
<br />
And if you want BEAST 2.3.2 with BEAGLE 2.1 (note that BEAST 2 is a different, er, beast to BEAST 1, regarding packages and compatibilities, so only use BEAST 2 if you know that's what you want to use).<br />
<pre><br />
call setbeast_232<br />
call beast-beagle_232 x y z<br />
</pre><br />
<br />
OpenCL is installed on all the nodes, which means Beagle will pick up the Intel Xeon processors and use what it can on them. It will also detect the Xeon Phis where they're available. If that's something you'd be particularly interested in using, drop me a line.<br />
<br />
=== Blast ===<br />
<br />
Drop me a line to join the party testing this, and to get added to the (large) BlastDB network share.<br />
<br />
<pre><br />
call setblast<br />
</pre><br />
<br />
will set the `BLASTDB` environment variable to point to what we current have of the `v5` database, and the Blast tools (2.9.0, 64-bit) will be added to the path. So you can <br />
then call `blastp`, `blastn`, `blastx` setc.<br />
<br />
=== BOW (BioInformatics on Windows) ===<br />
<br />
A number of Bio-informatics tools are ready to go. These used to be provided together on codeplex, but that has since been retired. Ugene is one project that<br />
maintains windows builds of them. Start your run.bat file with <code>call setBOW</code> to add these to the path:-<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File<br />
! Version<br />
|- <br />
| SamTools.exe<br />
| 0.1.19-44429cd<br />
|- <br />
| BCFTools.exe<br />
| 0.1.19-44428cd<br />
|- <br />
| bgzip<br />
| ?<br />
|- <br />
| razip<br />
| ?<br />
|- <br />
| bwa<br />
| 0.7.17-r1188<br />
|-<br />
| tabix<br />
| 0.2.5 (r1005)<br />
|-<br />
| minimap2<br />
| 2.24 (r1122)<br />
|-<br />
| raxml<br />
| 8.2.12 (best for each node)<br />
|-<br />
| raxml_sse3<br />
| 8.2.12 (pthreads+sse3)<br />
|-<br />
| raxml_avx<br />
| 8.2.12 (pthreads+avx)<br />
|-<br />
| raxml_avx2<br />
| 8.2.12 (pthreads+avx2)<br />
|}<br />
<br />
Note that BWA-MEM is hardwired to use a linux-only feature, so will produce an odd message and not do what you want if you use that mode on Windows. However, it seems that minimap2 may be a faster and more accurate way<br />
of doing what bwa does in any case, and that seems to have no such limitation.<br />
<br />
Also, note that raxml.exe is a copy of the most optimised build for the particular cluster node the job is running on. fi--dideclusthn only supports SSE3;<br />
fi--didemrchnb varies a little; the 32-cores support AVX2, the 24, 20 and 16 core machines support AVX, and only the oldest 12-core nodes only support SSE3. You don't have to worry about, except to expect performance to vary between them.<br />
<br />
=== GATK and Picard ===<br />
<br />
These both rely on Java, so put this in your run script:<br />
<br />
<pre><br />
call setJava64<br />
call setgatk<br />
</pre><br />
<br />
and then <code>gatk</code> in your scripts will use Python 3.7 to call the argument handling wrapper for GATK 4.2.2.0, and <code>picard</code> maps to the picard (2.26.2) jar file.<br />
<br />
=== FastQC ===<br />
<br />
This one has a slight eccentricity that instead of passing normal command-line args, it uses java's -D command-line options to buffer them out of an environment variable<br />
called _JAVA_OPTIONS. Strange but true. The ZIP comes with a bash script, which I'll reimplement if anyone actually wants me to. But in the meantime, it's easy to work <br />
out the arguments by inspection. For example, most likely usage:-<br />
<br />
<pre><br />
call setFastQC<br />
<br />
set _JAVA_OPTIONS='-Dfastqc.output_dir=%OUT%'<br />
call fastqc thing.fq<br />
</pre><br />
<br />
=== MAFFT ===<br />
<br />
For version 7.212 (Win64), write <code>call setMAFFT.bat</code> at the top of your run script.<br />
<br />
=== Applied-Maths Open Source ===<br />
<br />
Write <code>call setAppliedMaths.bat</code> at the top of your run script, to add these to the path:<br />
<br />
{| border="1" cellspacing="0" cellpadding="5" align="center"<br />
|-<br />
! EXE File(s)<br />
! Description<br />
! Version<br />
|- <br />
| velvetg_mt_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_mt_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x86.exe<br />
| Graph Construction Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velvetg_st_x64.exe<br />
| Graph Construction Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_mt_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x86.exe<br />
| Hashing Multi-threaded, 32-bit<br />
| 1.01.04<br />
|-<br />
| velveth_st_x64.exe<br />
| Hashing Multi-threaded, 64-bit<br />
| 1.01.04<br />
|-<br />
| ray_x64.exe<br />
| Ray 64-bit (for MPI)<br />
| 1.6<br />
|-<br />
| ray_x86.exe<br />
| Ray 32-bit (for MPI)<br />
| 1.6<br />
|-<br />
| Mothur_x64.exe<br />
| Mothur 64-bit (for MPI)<br />
| 1.25.1<br />
|-<br />
| Mothur_x86.exe<br />
| Mothur 32-bit (for MPI)<br />
| 1.25.1<br />
|}<br />
<br />
=== Tensorflow ===<br />
<br />
Here is an example of getting Tensorflow (as a python package) available from an R script, using the cluster to install everything for you. It only works for Tensorflow 2.0 - older versions and newer versions seem to want combinations of DLLs that I can't satisfy. The following are I think minimal steps to get Tensorflow running from Python and R.<br />
<br />
* Create a folder for your python packages in your home directory (so the cluster can see it. I'm calling mine <code>Q:/python_37_repo</code> - and the cluster will call it \\fi--san03.dide.ic.ac.uk\homes\wrh1\python_37_repo - change wrh1 to your DIDE username of course.<br />
* Create a similar folder for R packages, which for me is <code>Q:/R4</code><br />
* And a third folder, Q:/Tensorflow to do this work in.<br />
* Download these DLLs and put them in the Q:/Tensorflow folder - at present they seem to be the ones Python/Tensorflow wants. Let's hope for the best... Download [https://mrcdata.dide.ic.ac.uk/resources/msvcp140.dll msvcp140.dll] and [https://mrcdata.dide.ic.ac.uk/resources/msvcp140_1.dll msvcp140_1.dll] into your <code>Q:/Tensorflow</code> folder.<br />
<br />
* Now, create a script <code>Q:/Tensorflow/prepare.bat</code>. We're going to launch this on the cluster, and get it to wire up the packages we want. Replace USER with your username...<br />
<br />
<pre><br />
net delete Q: /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set TMPDIR=%TEMP%<br />
set PYTHONPATH=\\qdrive.dide.ic.ac.uk\homes\USER\python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
pip install --no-cache-dir tensorflow==2.00 --target Q:\python_37_repo<br />
RScript Q:\Tensorflow\install_tensorflow.R<br />
</pre><br />
<br />
* Create also <code>Q:/Tensorflow/install_tensorflow.R</code> contains:<br />
<pre><br />
install.packages(c("reticulate", "tensorflow"),lib="Q:/R4",repos="https://cran.ma.imperial.ac.uk/")<br />
</pre><br />
<br />
* Now submit the prepare job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\prepare.bat</code> in the big window will do it. That will take a while, but you should notice lots of new files in your Q:/python_37_repo folder. You'll only need to do this once (unless you delete your Python repo folder, or it randomly breaks and you start again...)<br />
<br />
* Next, create a file <code>Q:/Tensorflow/run.bat</code> containing the following the two "call" lines near the end are the ones that do some "work" in python or R, so this script basically a template for running your future jobs. Just change those two calls to set off the python or R code you want to run that uses Tensorflow.<br />
<br />
<pre><br />
call set_python_37_64.bat<br />
call setr64_4_0_3.bat<br />
net use Q: /delete /y<br />
net use Q: \\qdrive.dide.ic.ac.uk\homes\USER<br />
set PYTHONPATH=Q:/python_37_repo<br />
set R_LIBS=Q:\R4<br />
set R_LIBS_USER=Q:\R4<br />
call python Q:\tensorflow\test_tensorflow.py<br />
call RScript Q:\tensorflow\test_tensorflow.R<br />
</pre><br />
<br />
* And finally, you'll need <code>Q:\tensorflow\test_tensorflow.py</code> and <code>Q:\tensorflow\test_tensorflow.R</code> to exist. Here are minimal tests that everything is wired up. <code.test_tensorflow.py</code> first:-<br />
<br />
<pre><br />
import tensorflow as tf;<br />
print(tf.reduce_sum(tf.random.normal([1000, 1000])));<br />
</pre><br />
<br />
and <code>Q:\Tensorflow\test_tensorflow.R</code> contains:<br />
<br />
<pre><br />
library(reticulate)<br />
reticulate::py_config()<br />
library(tensorflow)<br />
tf$constant('Hello, TensorFlow!')<br />
tf$Variable(tf$zeros(shape(1L)))<br />
</pre><br />
<br />
* Now submit the run job to the cluster with <code>job submit /scheduler:fi--dideclusthn /jobtemplate:GeneralNodes \\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> - or if you use the portal, just <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\run.bat</code> in the big window will do it - and perhaps paste <code>\\fi--san03.dide.ic.ac.uk\homes\USER\tensorflow\output.txt</code> in the stdout box to collect the input and see how it gets on. You should see sensible output from both tests.<br />
<br />
== Launching a job ==<br />
<br />
<br />
== Submitting many jobs ==<br />
<br />
Suppose you write an exe that you might run with… Mysim.exe init.txt 1 2 3, and you want to run it many times with a range of parameters. One way of many, is to write a launch.bat file that will run “job submit” separately, for example (thanks Tini/James!):-<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set initFile=\\networkshare\job\init.txt<br />
set exeFile=\\networkshare\job\mysim.exe<br />
<br />
%SubDetails% %exeFile% %initfile% 1 2 3<br />
%SubDetails% %exeFile% %initfile% 4 5 6<br />
%SubDetails% %exeFile% %initfile% 7 8 9<br />
</pre><br />
<br />
Suppose the job you want to run is an R script? To specify arguments to an R script, you have to add ′--args a=1 b=2′ - so… you might make launch.bat like this:-<br />
<br />
<pre><br />
@echo off<br />
set SubDetails=job submit /scheduler:fi--dideclusthn.dide.ic.ac.uk /jobtemplate:GeneralNodes /numcores:1<br />
set rbatFile=\\networkshare\R-scripts\run.bat<br />
<br />
%SubDetails% %rbatFile% 1 2<br />
%SubDetails% %rbatFile% 3 4<br />
</pre><br />
And make the significant line of your run.bat:-<br />
<code><br />
Rcmd BATCH ′--args a=%1 b=%2′ U:\R-scripts\test.R<br />
</code><br />
%1 and %2 will map to the first and second thing after the batch file. You can go all the way up to %9.<br />
<br />
IMPORTANT NOTE<br />
<br />
Make sure you get the apostrophe character right in the above example – NEVER copy and paste from a word document into a script. It will go hideously wrong. Type the apostrophes (and dashes for that matter) in a good text editor – you want just the standard old-fashioned characters.<br />
<br />
== Requesting resources ==<br />
<br />
The following modifiers next to the “/scheduler:” part of the job submit line (before your app.exe 1 2 3 part), will request things you might want…<br />
<br />
/numcores:8 - number of cores you want<br />
<br />
/numcores:8-12 - minimum and maximum cores appropriate for your job<br />
<br />
/memorypernode:1024 - amount of mem in MegaBytes needed.<br />
<br />
/workdir:\\networkshare\work - set working directory<br />
<br />
/stdout:\\networkshare\out.txt - divert stdout to a file<br />
<br />
/stderr: or /stdin: - similar for stderr and stdin<br />
<br />
== Troubleshooting / Miscellany / Q & A ==<br />
<br />
* My job doesn’t work.<br />
** Run the HPC Job Manager application. Find your job id, possible under “My Jobs, Failed”. Double click on it, then on “View All Tasks”. Perhaps something in the output section will help.<br />
<br />
* Check that the path to your job is visible everywhere.<br />
** Avoid spaces in your paths - “job submit” doesn’t like them very much. If you must have them, they’ll be ok in the “run.bat” batch file that the cluster will run – in which case, put the path in standard double-quotes ("). But avoid them in your “launch.bat” file – you may have to relocate your run.bat to a simple non-space-containing directory.<br />
** Rather than putting the full application and parameters on the job submit line, you might want to write a batch-file to do all that, and submit the batch file to the cluster. (See section 6 about R for example). But make sure the batch file is somewhere visible to the cluster.<br />
<br />
* My job seems to work, but reports as having failed.<br />
** The success/failure depends on the error code returned. If you’re running C code, end with “return 0;” for success.<br />
<br />
* job submit ..blah blah.. app.exe >out.txt doesn’t work!<br />
** The contents of out.txt will be the result of “job submit”, not the result of “app.exe”. You meant to say this job submit ..blah blah.. /stdout:out.txt app.exe correcting out.txt and app.exe to network paths of course.<br />
<br />
* My C/C++ Executable works perfectly on my desktop, but doesn't work on the cluster. What's wrong with the cluster?<br />
** Your EXE file probably relies on some DLL files that the cluster doesn't have. If you use Visual Studio, check that in Code Generation, you have "Multi-Threaded (/MT)" as your target - with no mention of DLLs. If your code uses OpenMP, check you've copied the right OpenMP DLL(s) (go back to the C section above). As a last resort, give your EXE to someone who doesn't have Visual Studio installed, and get them to run it, and see what DLL files it seems to ask for.<br />
<br />
== Contributing Authors ==<br />
<br />
Wes Hinsley, James Truscott, Tini Garske, Jeff Eaton, Hannah Clapham</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Remote_access_to_DIDE&diff=14253Remote access to DIDE2023-04-17T14:59:02Z<p>Admin: </p>
<hr />
<div>These instructions are now obsolete. All remote access to DIDE is now via [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/unified-access/ Zscaler]. ZScaler replaces both the Pulse Secure VPN and the Remote Desktop Gatway.<br />
<br />
== Remote Access to your Windows DIDE Desktop via ZScaler ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
# Ensure that you've uninstalled Pulse Secure VPN.<br />
# Ensure that you've installed ZScaler via above link.<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
== OBSOLETE SINCE ZSCALER - Remote Access to your Windows DIDE Desktop via Terminal Services Gateway ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
[[File:TS connection2.png|thumb|Steps 4 to 9]]<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Select the '''Advanced''' tab on the far right and click the '''Settings...''' button under Connect from anywhere.<br />
# Select the '''Use these RD Gateway server settings''' button.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the ''Server name'' field.<br />
# Select '''Allow me to select later''' in the ''Log-on method'' field. We'll add this later (see after these steps).<br />
# Ensure that '''Bypass RD Gateway server for local addresses''' is ''ticked''.<br />
# Ensure that '''Use my RD Gateway credentials for the remote computer''' at the bottom is ''unticked''. <br />
# Press the OK button<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
You will now have a new icon on your desktop and you never need to do the above again. To connect to your DIDE desktop, click the '''Connect''' button in your new Remote Desktop shortcut.<br />
<br />
You will now be asked for your Imperial credentials in order to access the Terminal Services Gateway. Enter your details as '''IC\username''' with your Imperial password.<br />
<br />
Once that allows you through you will be prompted for your DIDE credentials in order to access your PC. Enter your details as '''DIDE\username''' with your DIDE password. If this is the first time that you've connected to your DIDE PC for a while you'll get a certificate warning message. Tick the box and select '''Yes'''.<br />
<br />
If you don't get prompted for your DIDE account that probably means your PC is asleep. To wake it up [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/remotely-access-my-college-computer/wake-my-pc/ click here].<br />
<br />
'''N.B.''' If all the above seems too difficult, I've made a template file with all the settings. It is in the '''T:''' drive in folder '''IT\VPN''' and is called '''DIDE-Remote.RDP'''. Right-click and edit that to put your correct hostname and username in and all the other settings are done for you.<br />
<br />
== Remote Access to your Ubuntu DIDE Desktop via Terminal Services Gateway ==<br />
First of all, you will need to have enabled remote desktop in advance. [https://linuxconfig.org/ubuntu-remote-desktop-18-04-bionic-beaver-linux See here for example]. You should also check you have the Remmina app installed.<br />
<br />
# Open Remmina<br />
# Click the “+” button to add a new configuration, and give it a meaningful name.<br />
# Protocol: RDP - Remote Desktop Protocol<br />
# Open Remmina and on the Basic tab fill in the following details: <br />
#* Server: Your desktop PC FQDN (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''')<br />
#* User Name: '''Your Ubuntu username'''<br />
#* Password: '''Your Ubuntu password'''<br />
#* Domain: '''DIDE''' (only if your workstation is on the DIDE domain - very likely it's not).<br />
# Adjust colour depth to '''16 bit'''.<br />
# Go to to the Advanced Tab and set up the ICT RDP Gateway with the following details:<br />
#* RDG Server: '''ictsgw.cc.ic.ac.uk'''<br />
#* RDG username: '''Your IC username'''<br />
#* RDG password: '''Your IC password'''<br />
#* RDG domain: '''IC'''<br />
# Click '''Save''' and you will have a new configuration with the name you initially specified. <br />
# Click on this new entry to connect to your server.<br />
# Accept the certificate warnings. These should only appear the very first time you connect.<br />
# Your Ubuntu remote session should now be active.<br />
<br />
<br />
== Remote Access to your DIDE Desktop via Terminal Services Gateway on a Mac ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
<br />
# Start the '''Microsoft Remote Desktop 10''' app from the App Store and select '''Add desktop''' to add a new profile entry.<br />
# Enter a name for the remote desktop profile.<br />
# Enter the full name of your DIDE machine (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''').<br />
# Change the '''User Account''' drop down to Add User Account<br />
# Enter '''DIDE\username''' in the User name field.<br />
# Enter your DIDE password in the Password field.<br />
# Enter a friendly name if desired (optional)<br />
# Click '''Add''' and '''Save''' again.<br />
# Click the Cog in the top left of the screen then Preferences.<br />
# Click Gateways.<br />
# Click on the '''+''' symbol on the bottom left of the window.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the Server name field.<br />
# Select your Imperial '''username''' in the User name field.<br />
# Enter a name for the gateway in the Friendly Name field.<br />
# Click '''Save'''.<br />
# Close the screen.<br />
# Now choose the gateway you created from the Gateway dropdown list.<br />
# Click the pencil icon.<br />
# Under '''General''' select the gateway name in the Gateway field.<br />
# Click '''Save'''.<br />
# The main screen will display your new profile. Double click this to start a remote desktop session.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Remote_access_to_DIDE&diff=14252Remote access to DIDE2023-04-17T14:56:02Z<p>Admin: </p>
<hr />
<div>These instructions are now obsolete. All remote access to DIDE is now via [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/unified-access/ Zscaler]. ZScaler replaces both the Pulse Secure VPN and the Remote Desktop Gatway.<br />
<br />
== Remote Access to your Windows DIDE Desktop via ZScaler ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
[[File:TS connection2.png|thumb|Steps 4 to 9]]<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
== OBSOLETE SINCE ZSCALER - Remote Access to your Windows DIDE Desktop via Terminal Services Gateway ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
[[File:TS connection2.png|thumb|Steps 4 to 9]]<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Select the '''Advanced''' tab on the far right and click the '''Settings...''' button under Connect from anywhere.<br />
# Select the '''Use these RD Gateway server settings''' button.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the ''Server name'' field.<br />
# Select '''Allow me to select later''' in the ''Log-on method'' field. We'll add this later (see after these steps).<br />
# Ensure that '''Bypass RD Gateway server for local addresses''' is ''ticked''.<br />
# Ensure that '''Use my RD Gateway credentials for the remote computer''' at the bottom is ''unticked''. <br />
# Press the OK button<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
You will now have a new icon on your desktop and you never need to do the above again. To connect to your DIDE desktop, click the '''Connect''' button in your new Remote Desktop shortcut.<br />
<br />
You will now be asked for your Imperial credentials in order to access the Terminal Services Gateway. Enter your details as '''IC\username''' with your Imperial password.<br />
<br />
Once that allows you through you will be prompted for your DIDE credentials in order to access your PC. Enter your details as '''DIDE\username''' with your DIDE password. If this is the first time that you've connected to your DIDE PC for a while you'll get a certificate warning message. Tick the box and select '''Yes'''.<br />
<br />
If you don't get prompted for your DIDE account that probably means your PC is asleep. To wake it up [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/remotely-access-my-college-computer/wake-my-pc/ click here].<br />
<br />
'''N.B.''' If all the above seems too difficult, I've made a template file with all the settings. It is in the '''T:''' drive in folder '''IT\VPN''' and is called '''DIDE-Remote.RDP'''. Right-click and edit that to put your correct hostname and username in and all the other settings are done for you.<br />
<br />
== Remote Access to your Ubuntu DIDE Desktop via Terminal Services Gateway ==<br />
First of all, you will need to have enabled remote desktop in advance. [https://linuxconfig.org/ubuntu-remote-desktop-18-04-bionic-beaver-linux See here for example]. You should also check you have the Remmina app installed.<br />
<br />
# Open Remmina<br />
# Click the “+” button to add a new configuration, and give it a meaningful name.<br />
# Protocol: RDP - Remote Desktop Protocol<br />
# Open Remmina and on the Basic tab fill in the following details: <br />
#* Server: Your desktop PC FQDN (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''')<br />
#* User Name: '''Your Ubuntu username'''<br />
#* Password: '''Your Ubuntu password'''<br />
#* Domain: '''DIDE''' (only if your workstation is on the DIDE domain - very likely it's not).<br />
# Adjust colour depth to '''16 bit'''.<br />
# Go to to the Advanced Tab and set up the ICT RDP Gateway with the following details:<br />
#* RDG Server: '''ictsgw.cc.ic.ac.uk'''<br />
#* RDG username: '''Your IC username'''<br />
#* RDG password: '''Your IC password'''<br />
#* RDG domain: '''IC'''<br />
# Click '''Save''' and you will have a new configuration with the name you initially specified. <br />
# Click on this new entry to connect to your server.<br />
# Accept the certificate warnings. These should only appear the very first time you connect.<br />
# Your Ubuntu remote session should now be active.<br />
<br />
<br />
== Remote Access to your DIDE Desktop via Terminal Services Gateway on a Mac ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
<br />
# Start the '''Microsoft Remote Desktop 10''' app from the App Store and select '''Add desktop''' to add a new profile entry.<br />
# Enter a name for the remote desktop profile.<br />
# Enter the full name of your DIDE machine (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''').<br />
# Change the '''User Account''' drop down to Add User Account<br />
# Enter '''DIDE\username''' in the User name field.<br />
# Enter your DIDE password in the Password field.<br />
# Enter a friendly name if desired (optional)<br />
# Click '''Add''' and '''Save''' again.<br />
# Click the Cog in the top left of the screen then Preferences.<br />
# Click Gateways.<br />
# Click on the '''+''' symbol on the bottom left of the window.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the Server name field.<br />
# Select your Imperial '''username''' in the User name field.<br />
# Enter a name for the gateway in the Friendly Name field.<br />
# Click '''Save'''.<br />
# Close the screen.<br />
# Now choose the gateway you created from the Gateway dropdown list.<br />
# Click the pencil icon.<br />
# Under '''General''' select the gateway name in the Gateway field.<br />
# Click '''Save'''.<br />
# The main screen will display your new profile. Double click this to start a remote desktop session.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Remote_access_to_DIDE&diff=14251Remote access to DIDE2023-03-06T17:26:38Z<p>Admin: </p>
<hr />
<div>These instructions are now obsolete. All remote access to DIDE is now via [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/unified-access/ Zscaler].<br />
<br />
<br />
== Remote Access to your Windows DIDE Desktop via Terminal Services Gateway ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
[[File:TS connection2.png|thumb|Steps 4 to 9]]<br />
# Click on the Start button in your desktop toolbar, type '''Remote Desktop''' in the search field and press the Return key on your keyboard.<br />
# Press the '''Show Options''' arrow in the bottom left hand corner of the Remote Desktop window.<br />
# Select the '''Advanced''' tab on the far right and click the '''Settings...''' button under Connect from anywhere.<br />
# Select the '''Use these RD Gateway server settings''' button.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the ''Server name'' field.<br />
# Select '''Allow me to select later''' in the ''Log-on method'' field. We'll add this later (see after these steps).<br />
# Ensure that '''Bypass RD Gateway server for local addresses''' is ''ticked''.<br />
# Ensure that '''Use my RD Gateway credentials for the remote computer''' at the bottom is ''unticked''. <br />
# Press the OK button<br />
# Click the '''Display''' tab and ensure that the '''Colours''' menu says ''High Colour (16-bit)''.[[File:TS display.png|thumb|Step 10]]<br />
# Click on the '''Experience''' tab and select '''Low speed internet''', then re-tick the box for Font Smoothing.<br />
# Click the '''Local Resources''' tab and ensure that the '''Printers''' box is ''unticked''.[[File:TS printer.png|thumb|Step 11]]<br />
# Click the '''General''' tab.<br />
# Enter the full name of your DIDE PC (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''') and click ''Connect''.<br />
# Enter your DIDE username ('''DIDE\yourusername''') and click ''OK''.[[File:TS general.png|thumb|Steps 12 to 15]]<br />
# Finally, click the '''Save As...''' button and save these settings as something like '''Remote Desktop Connection''' to your desktop or wherever you like.<br />
<br />
You will now have a new icon on your desktop and you never need to do the above again. To connect to your DIDE desktop, click the '''Connect''' button in your new Remote Desktop shortcut.<br />
<br />
You will now be asked for your Imperial credentials in order to access the Terminal Services Gateway. Enter your details as '''IC\username''' with your Imperial password.<br />
<br />
Once that allows you through you will be prompted for your DIDE credentials in order to access your PC. Enter your details as '''DIDE\username''' with your DIDE password. If this is the first time that you've connected to your DIDE PC for a while you'll get a certificate warning message. Tick the box and select '''Yes'''.<br />
<br />
If you don't get prompted for your DIDE account that probably means your PC is asleep. To wake it up [https://www.imperial.ac.uk/admin-services/ict/self-service/connect-communicate/remote-access/remotely-access-my-college-computer/wake-my-pc/ click here].<br />
<br />
'''N.B.''' If all the above seems too difficult, I've made a template file with all the settings. It is in the '''T:''' drive in folder '''IT\VPN''' and is called '''DIDE-Remote.RDP'''. Right-click and edit that to put your correct hostname and username in and all the other settings are done for you.<br />
<br />
== Remote Access to your Ubuntu DIDE Desktop via Terminal Services Gateway ==<br />
First of all, you will need to have enabled remote desktop in advance. [https://linuxconfig.org/ubuntu-remote-desktop-18-04-bionic-beaver-linux See here for example]. You should also check you have the Remmina app installed.<br />
<br />
# Open Remmina<br />
# Click the “+” button to add a new configuration, and give it a meaningful name.<br />
# Protocol: RDP - Remote Desktop Protocol<br />
# Open Remmina and on the Basic tab fill in the following details: <br />
#* Server: Your desktop PC FQDN (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''')<br />
#* User Name: '''Your Ubuntu username'''<br />
#* Password: '''Your Ubuntu password'''<br />
#* Domain: '''DIDE''' (only if your workstation is on the DIDE domain - very likely it's not).<br />
# Adjust colour depth to '''16 bit'''.<br />
# Go to to the Advanced Tab and set up the ICT RDP Gateway with the following details:<br />
#* RDG Server: '''ictsgw.cc.ic.ac.uk'''<br />
#* RDG username: '''Your IC username'''<br />
#* RDG password: '''Your IC password'''<br />
#* RDG domain: '''IC'''<br />
# Click '''Save''' and you will have a new configuration with the name you initially specified. <br />
# Click on this new entry to connect to your server.<br />
# Accept the certificate warnings. These should only appear the very first time you connect.<br />
# Your Ubuntu remote session should now be active.<br />
<br />
<br />
== Remote Access to your DIDE Desktop via Terminal Services Gateway on a Mac ==<br />
The procedure below only needs to be followed once. Complete each step in turn.<br />
<br />
# Start the '''Microsoft Remote Desktop 10''' app from the App Store and select '''Add desktop''' to add a new profile entry.<br />
# Enter a name for the remote desktop profile.<br />
# Enter the full name of your DIDE machine (e.g. '''WPIA-DIDEx.dide.ic.ac.uk''').<br />
# Change the '''User Account''' drop down to Add User Account<br />
# Enter '''DIDE\username''' in the User name field.<br />
# Enter your DIDE password in the Password field.<br />
# Enter a friendly name if desired (optional)<br />
# Click '''Add''' and '''Save''' again.<br />
# Click the Cog in the top left of the screen then Preferences.<br />
# Click Gateways.<br />
# Click on the '''+''' symbol on the bottom left of the window.<br />
# Enter '''ictsgw.cc.ic.ac.uk''' in the Server name field.<br />
# Select your Imperial '''username''' in the User name field.<br />
# Enter a name for the gateway in the Friendly Name field.<br />
# Click '''Save'''.<br />
# Close the screen.<br />
# Now choose the gateway you created from the Gateway dropdown list.<br />
# Click the pencil icon.<br />
# Under '''General''' select the gateway name in the Gateway field.<br />
# Click '''Save'''.<br />
# The main screen will display your new profile. Double click this to start a remote desktop session.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Printing&diff=14250Printing2022-11-17T13:30:57Z<p>Admin: </p>
<hr />
<div>= Printing at Imperial College =<br />
The print system is handled by HP themselves on behalf of the college. You should follow [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Imperial's Mobile Print instructions]. This page is a guide to follow while you're working through the ICT Mobile Print page.<br />
<br />
== Advice for when following the Mobile Print instrucions ==<br />
Most of you are Windows users so I'll go over the key steps here. The main ICT instructions are quite good if you pay close attention, but they are from an old version of Windows 10. My page here is a supplement in case you get stuck.<br />
<br />
Log in to the [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Mobile Print site] with your Imperial credentials and then select '''Driver Print'''. Note that this page will only work on campus.<br />
<br />
[[file:mobileprint1.jpeg|border]]<br />
<br />
Now select your operating system with the large icons at the bottom (you are most likely Windows):<br />
<br />
[[file:mobileprint2os.jpeg|border]]<br />
<br />
After this you get a couple of pages with only one option each (there used to be more). Go through these formalities and select the only option you have:<br />
<br />
[[File:Mobileprint3option.jpeg|border]]<br />
<br />
We now get to a mildly tricky bit as we have to download the actual HP software for talking to the printers. There is a handy link on the Mobile Print page to the [https://ftp.hp.com/pub/softlib/software13/COL40842/ds-99374-24/upd-pcl6-x64-7.0.1.24923.exe HP FTP site] so click on that to download it.<br />
<br />
[[File:Mobileprint4drivers.jpeg|border]]<br />
<br />
Once downloaded, double click on it to run it and press the '''Unzip''' button. Go through the installer as normal but make sure you select '''USB Mode - Plug and Play''' as your printer type. Leave the two text boxes at the bottom ticked.<br />
<br />
[[File:Mobileprint5usb.png|border]]<br />
<br />
Now we get to add the actual print queue to your computer. From your start menu type '''printers''' and you'll see a menu item to add printers. Click on that and then select '''add a printer'''. It will have a think for a while and then show you some completely incorrect printers. As soon as you see the option '''my printer is not listed here''' click on that.<br />
<br />
[[File:Printernotlisted.png|border]]<br />
<br />
Now you can copy and paste the name of your print queue, which is listed in Step 4 of the Mobile Print page guide (below is a screenshot for your reference - you can't copy and paste this):<br />
<br />
[[File:Mobileprint6queue.jpeg|border]]<br />
<br />
You do need to copy and paste the whole address line, including the bit I've blurred out (I've blurred my exact address out or you would be printing as me with my print credit!). Remember that you're not copying and pasting this line, you're using the line from the Mobile Print web page!<br />
<br />
The rest of the ICT instructions should work ok, you merely have to select that you're adding an '''HP''' printer and you want to use '''HP Universal Printing PCL 6'''. There is no need to print a test page.<br />
<br />
== I don't have time for this and just want to print something! ==<br />
If you're in a rush and have to print right now, then email your document to [mailto:print@imperial.ac.uk print@imperial.ac.uk] from your Imperial email address. You may then go to any swipe printer and collect your document. Remember to delete the email from your Sent Items folder afterwards or you may be wasting mailbox space.<br />
<br />
== Gotchas ==<br />
If you are in the habit of using Remote Desktop and the VPN to connect to your DIDE PC, then there is a setting which you should '''untick''' in your Remote Desktop connection settings screen:<br />
<br />
[[File:Printer3.jpg]]<br />
<br />
Otherwise you'll end up with your office printers connected to your home computer and it has been known to mess things up.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Printing&diff=14249Printing2022-11-17T13:30:33Z<p>Admin: </p>
<hr />
<div>= Printing at Imperial College =<br />
The print system is handled by HP themselves on behalf of the college. You should follow [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Imperial's Mobile Print instructions]. This page is a guide to follow while you're working through the ICT Mobile Print page.<br />
<br />
== Advice for when following the Mobile Print instrucions ==<br />
Most of you are Windows users so I'll go over the key steps here. The main ICT instructions are quite good if you pay close attention, but they are from an old version of Windows 10. My page here is a supplement in case you get stuck.<br />
<br />
Log in to the [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Mobile Print site] with your Imperial credentials and then select '''Driver Print'''. Note that this page will only work on campus.<br />
<br />
[[file:mobileprint1.jpeg|border]]<br />
<br />
Now select your operating system with the large icons at the bottom (you are most likely Windows):<br />
<br />
[[file:mobileprint2os.jpeg|border]]<br />
<br />
After this you get a couple of pages with only one option each (there used to be more). Go through these formalities and select the only option you have:<br />
<br />
[[File:Mobileprint3option.jpeg|border]]<br />
<br />
We now get to a mildly tricky bit as we have to download the actual HP software for talking to the printers. There is a handy link on the Mobile Print page to the [https://ftp.hp.com/pub/softlib/software13/COL40842/ds-99374-24/upd-pcl6-x64-7.0.1.24923.exe HP FTP site] so click on that to download it.<br />
<br />
[[File:Mobileprint4drivers.jpeg|border]]<br />
<br />
Once downloaded, double click on it to run it and press the '''Unzip''' button. Go through the installer as normal but make sure you select '''USB Mode - Plug and Play''' as your printer type. Leave the two text boxes at the bottom ticked.<br />
<br />
[[File:Mobileprint5usb.png|border]]<br />
<br />
Now we get to add the actual print queue to your computer. From your start menu type '''printers''' and you'll see a menu item to add printers. Click on that and then select '''add a printer'''. It will have a think for a while and then show you some completely incorrect printers. As soon as you see the option '''my printer is not listed here''' click on that.<br />
<br />
[[File:Printernotlisted.png|border]]<br />
<br />
Now you can copy and paste the name of your print queue, which is listed in Step 4 of the Mobile Print page guide (below is a screenshot for your reference - you can't copy and paste this):<br />
<br />
[[File:Mobileprint6queue.jpeg|border]]<br />
<br />
You do need to copy and paste the whole address line, including the bit I've blurred out (I've blurred my exact address out or you would be printing as me with my print credit!). Remember that you're not copying and pasting this line, you're using the line from the Mobile Print web page!<br />
<br />
The rest of the ICT instructions should work ok, you merely have to select that you're adding an '''HP''' printer and you want to use '''HP Universal Printing PCL 6'''. There is no need to print a test page.<br />
<br />
== I don't have time for this and just want to print something! ==<br />
If you're in a rush and have to print right now, then email your document to [mailto:print@imperial.ac.uk print@imperial.ac.uk] from your Imperial email address. You may then go to any swipe printer and collect your document. Remember to delete the email from your Sent Items folder afterwards or you may be wasting mailbox space.<br />
<br />
=== Gotchas ===<br />
If you are in the habit of using Remote Desktop and the VPN to connect to your DIDE PC, then there is a setting which you should '''untick''' in your Remote Desktop connection settings screen:<br />
<br />
[[File:Printer3.jpg]]<br />
<br />
Otherwise you'll end up with your office printers connected to your home computer and it has been known to mess things up.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Printing&diff=14248Printing2022-11-17T13:29:40Z<p>Admin: </p>
<hr />
<div>= Printing at Imperial College =<br />
The print system is handled by HP themselves on behalf of the college. You should follow [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Imperial's Mobile Print instructions]. This page is a guide to follow while you're working through the ICT Mobile Print page.<br />
<br />
== I don't have time for this and just want to print something! ==<br />
If you're in a rush and have to print right now, then email your document to [mailto:print@imperial.ac.uk print@imperial.ac.uk] from your Imperial email address. You may then go to any swipe printer and collect your document. Remember to delete the email from your Sent Items folder afterwards or you may be wasting mailbox space.<br />
<br />
== Advice for when following the Mobile Print instrucions ==<br />
Most of you are Windows users so I'll go over the key steps here. The main ICT instructions are quite good if you pay close attention, but they are from an old version of Windows 10. My page here is a supplement in case you get stuck.<br />
<br />
Log in to the [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Mobile Print site] with your Imperial credentials and then select '''Driver Print'''. Note that this page will only work on campus.<br />
<br />
[[file:mobileprint1.jpeg|border]]<br />
<br />
Now select your operating system with the large icons at the bottom (you are most likely Windows):<br />
<br />
[[file:mobileprint2os.jpeg|border]]<br />
<br />
After this you get a couple of pages with only one option each (there used to be more). Go through these formalities and select the only option you have:<br />
<br />
[[File:Mobileprint3option.jpeg|border]]<br />
<br />
We now get to a mildly tricky bit as we have to download the actual HP software for talking to the printers. There is a handy link on the Mobile Print page to the [https://ftp.hp.com/pub/softlib/software13/COL40842/ds-99374-24/upd-pcl6-x64-7.0.1.24923.exe HP FTP site] so click on that to download it.<br />
<br />
[[File:Mobileprint4drivers.jpeg|border]]<br />
<br />
Once downloaded, double click on it to run it and press the '''Unzip''' button. Go through the installer as normal but make sure you select '''USB Mode - Plug and Play''' as your printer type. Leave the two text boxes at the bottom ticked.<br />
<br />
[[File:Mobileprint5usb.png|border]]<br />
<br />
Now we get to add the actual print queue to your computer. From your start menu type '''printers''' and you'll see a menu item to add printers. Click on that and then select '''add a printer'''. It will have a think for a while and then show you some completely incorrect printers. As soon as you see the option '''my printer is not listed here''' click on that.<br />
<br />
[[File:Printernotlisted.png|border]]<br />
<br />
Now you can copy and paste the name of your print queue, which is listed in Step 4 of the Mobile Print page guide (below is a screenshot for your reference - you can't copy and paste this):<br />
<br />
[[File:Mobileprint6queue.jpeg|border]]<br />
<br />
You do need to copy and paste the whole address line, including the bit I've blurred out (I've blurred my exact address out or you would be printing as me with my print credit!). Remember that you're not copying and pasting this line, you're using the line from the Mobile Print web page!<br />
<br />
The rest of the ICT instructions should work ok, you merely have to select that you're adding an '''HP''' printer and you want to use '''HP Universal Printing PCL 6'''. There is no need to print a test page.<br />
<br />
=== Gotchas ===<br />
If you are in the habit of using Remote Desktop and the VPN to connect to your DIDE PC, then there is a setting which you should '''untick''' in your Remote Desktop connection settings screen:<br />
<br />
[[File:Printer3.jpg]]<br />
<br />
Otherwise you'll end up with your office printers connected to your home computer and it has been known to mess things up.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Printing&diff=14247Printing2022-11-17T13:29:07Z<p>Admin: </p>
<hr />
<div>= Printing at Imperial College =<br />
The print system is handled by HP themselves on behalf of the college. You should follow [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Imperial's Mobile Print instructions]. This page is a guide to follow while you're working through the ICT Mobile Print page.<br />
<br />
== I don't have time for this and just want to print something! ==<br />
If you're in a rush and have to print right now, then email your document to [mailto:print@imperial.ac.uk print@imperial.ac.uk] from your Imperial email address. You may then go to any swipe printer and collect your document. Remember to delete the email from your Sent Items folder afterwards or you may be wasting mailbox space.<br />
<br />
== Advice for when following the Mobile Print instrucions ==<br />
Most of you are Windows users so I'll go over the key steps here. The main ICT instructions are quite good if you pay close attention, but they are from an old version of Windows 10. My page here is a supplement in case you get stuck.<br />
<br />
Log in to the [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Mobile Print site] with your Imperial credentials and then select '''Driver Print'''. Note that this page will only work on campus.<br />
<br />
[[file:mobileprint1.jpeg|border]]<br />
<br />
Now select your operating system with the large icons at the bottom (you are most likely Windows):<br />
<br />
[[file:mobileprint2os.jpeg|border]]<br />
<br />
After this you get a couple of pages with only one option each (there used to be more). Go through these formalities and select the only option you have:<br />
<br />
[[File:Mobileprint3option.jpeg|border]]<br />
<br />
We now get to a mildly tricky bit as we have to download the actual HP software for talking to the printers. There is a handy link on the Mobile Print page to the [https://ftp.hp.com/pub/softlib/software13/COL40842/ds-99374-24/upd-pcl6-x64-7.0.1.24923.exe HP FTP site] so click on that to download it.<br />
<br />
[[File:Mobileprint4drivers.jpeg|border]]<br />
<br />
Once downloaded, double click on it to run it and press the '''Unzip''' button. Go through the installer as normal but make sure you select '''USB Mode - Plug and Play''' as your printer type. Leave the two text boxes at the bottom ticked.<br />
<br />
[[File:Mobileprint5usb.png|border]]<br />
<br />
Now we get to add the actual print queue to your computer. From your start menu type '''printers''' and you'll see a menu item to add printers. Click on that and then select '''add a printer'''. It will have a think for a while and then show you some completely incorrect printers. As soon as you see the option '''my printer is not listed here''' click on that.<br />
<br />
[[File:Printernotlisted.png|border]]<br />
<br />
Now you can copy and paste the name of your print queue, which is listed in Step 4 of the Mobile Print page guide (below is a screenshot for your reference - you can't copy and paste this):<br />
<br />
[[File:Mobileprint6queue.jpeg|border]]<br />
<br />
You do need to copy and paste the whole address line, including the bit I've blurred out (I've blurred my exact address out or you would be printing as me with my print credit!). Remember that you're not copying and pasting this line, you're using the line from the Mobile Print web page!<br />
<br />
The rest of the ICT instructions should work ok, you merely have to select that you're adding an '''HP''' printer and you want to use '''HP Universal Printing PCL 6'''. There is no need to print a test page.<br />
<br />
=== Gotchas ===<br />
If you are in the habit of using Remote Desktop and the VPN to connect to your DIDE PC, then there is a setting which you should disable in your Remote Desktop connection settings screen:<br />
<br />
[[File:Printer3.jpg]]<br />
<br />
Otherwise you'll end up with your office printers connected to your home computer and it has been known to mess things up.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Printing&diff=14246Printing2022-11-17T13:27:58Z<p>Admin: </p>
<hr />
<div>= Printing at Imperial College =<br />
The print system is handled by HP themselves on behalf of the college. You should follow [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Imperial's Mobile Print instructions]. This page is a guide to follow while you're working through the ICT Mobile Print page.<br />
<br />
== I don't have time for this and just want to print something! ==<br />
If you're in a rush and have to print right now, then email your document to [mailto:print@imperial.ac.uk print@imperial.ac.uk] from your Imperial email address. You may then go to any swipe printer and collect your document. Remember to delete the email from your Sent Items folder afterwards or you may be wasting mailbox space.<br />
<br />
== Advice for when following the Mobile Print instrucions ==<br />
Most of you are Windows users so I'll go over the key steps here. The main ICT instructions are quite good if you pay close attention, but they are from an old version of Windows 10. My page here is a supplement in case you get stuck.<br />
<br />
Log in to the [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Mobile Print site] with your Imperial credentials and then select '''Driver Print'''. Note that this page will only work on campus.<br />
<br />
[[file:mobileprint1.jpeg|border]]<br />
<br />
Now select your operating system with the large icons at the bottom (you are most likely Windows):<br />
<br />
[[file:mobileprint2os.jpeg|border]]<br />
<br />
After this you get a couple of pages with only one option each (there used to be more). Go through these formalities and select the only option you have:<br />
<br />
[[File:Mobileprint3option.jpeg|border]]<br />
<br />
We now get to a mildly tricky bit as we have to download the actual HP software for talking to the printers. There is a handy link on the Mobile Print page to the [https://ftp.hp.com/pub/softlib/software13/COL40842/ds-99374-24/upd-pcl6-x64-7.0.1.24923.exe HP FTP site] so click on that to download it.<br />
<br />
[[File:Mobileprint4drivers.jpeg|border]]<br />
<br />
Once downloaded, double click on it to run it and press the '''Unzip''' button. Go through the installer as normal but make sure you select '''USB Mode - Plug and Play''' as your printer type. Leave the two text boxes at the bottom ticked.<br />
<br />
[[File:Mobileprint5usb.png|border]]<br />
<br />
Now we get to add the actual print queue to your computer. From your start menu type '''printers''' and you'll see a menu item to add printers. Click on that and then select '''add a printer'''. It will have a think for a while and then show you some completely incorrect printers. As soon as you see the option '''my printer is not listed here''' click on that.<br />
<br />
[[File:Printernotlisted.png|border]]<br />
<br />
Now you can copy and paste the name of your print queue, which is listed in Step 4 of the Mobile Print page guide (below is a screenshot for your reference - you can't copy and paste this):<br />
<br />
[[File:Mobileprint6queue.jpeg|border]]<br />
<br />
You do need to copy and paste the whole address line, including the bit I've blurred out (I've blurred my exact address out or you would be printing as me with my print credit!). Remember that you're not copying and pasting this line, you're using the line from the Mobile Print web page!<br />
<br />
The rest of the ICT instructions should work ok, you merely have to select that you're adding an '''HP''' printer and you want to use '''HP Universal Printing PCL 6'''. There is no need to print a test page.<br />
<br />
=== Gotchas ===<br />
If you are in the habit of using Remote Desktop and the VPN to connect to your DIDE PC, then there is a setting which you should disable in your Remote Desktop connection settings screen:<br />
[[File:Printer3.jpg|left]]<br />
Otherwise you'll end up with your office printers connected to your home computer and it has been known to mess things up.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Printing&diff=14245Printing2022-11-17T13:27:22Z<p>Admin: </p>
<hr />
<div>= Printing at Imperial College =<br />
The print system is handled by HP themselves on behalf of the college. You should follow [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Imperial's Mobile Print instructions]. This page is a guide to follow while you're working through the ICT Mobile Print page.<br />
<br />
== I don't have time for this and just want to print something! ==<br />
If you're in a rush and have to print right now, then email your document to [mailto:print@imperial.ac.uk print@imperial.ac.uk] from your Imperial email address. You may then go to any swipe printer and collect your document. Remember to delete the email from your Sent Items folder afterwards or you may be wasting mailbox space.<br />
<br />
== Advice for when following the Mobile Print instrucions ==<br />
Most of you are Windows users so I'll go over the key steps here. The main ICT instructions are quite good if you pay close attention, but they are from an old version of Windows 10. My page here is a supplement in case you get stuck.<br />
<br />
Log in to the [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Mobile Print site] with your Imperial credentials and then select '''Driver Print'''. Note that this page will only work on campus.<br />
<br />
[[file:mobileprint1.jpeg|border]]<br />
<br />
Now select your operating system with the large icons at the bottom (you are most likely Windows):<br />
<br />
[[file:mobileprint2os.jpeg]]<br />
<br />
After this you get a couple of pages with only one option each (there used to be more). Go through these formalities and select the only option you have:<br />
<br />
[[File:Mobileprint3option.jpeg]]<br />
<br />
We now get to a mildly tricky bit as we have to download the actual HP software for talking to the printers. There is a handy link on the Mobile Print page to the [https://ftp.hp.com/pub/softlib/software13/COL40842/ds-99374-24/upd-pcl6-x64-7.0.1.24923.exe HP FTP site] so click on that to download it.<br />
<br />
[[File:Mobileprint4drivers.jpeg]]<br />
<br />
Once downloaded, double click on it to run it and press the '''Unzip''' button. Go through the installer as normal but make sure you select '''USB Mode - Plug and Play''' as your printer type. Leave the two text boxes at the bottom ticked.<br />
<br />
[[File:Mobileprint5usb.png]]<br />
<br />
Now we get to add the actual print queue to your computer. From your start menu type '''printers''' and you'll see a menu item to add printers. Click on that and then select '''add a printer'''. It will have a think for a while and then show you some completely incorrect printers. As soon as you see the option '''my printer is not listed here''' click on that.<br />
<br />
[[File:Printernotlisted.png]]<br />
<br />
Now you can copy and paste the name of your print queue, which is listed in Step 4 of the Mobile Print page guide (below is a screenshot for your reference - you can't copy and paste this):<br />
<br />
[[File:Mobileprint6queue.jpeg]]<br />
<br />
You do need to copy and paste the whole address line, including the bit I've blurred out (I've blurred my exact address out or you would be printing as me with my print credit!). Remember that you're not copying and pasting this line, you're using the line from the Mobile Print web page!<br />
<br />
The rest of the ICT instructions should work ok, you merely have to select that you're adding an '''HP''' printer and you want to use '''HP Universal Printing PCL 6'''. There is no need to print a test page.<br />
<br />
=== Gotchas ===<br />
If you are in the habit of using Remote Desktop and the VPN to connect to your DIDE PC, then there is a setting which you should disable in your Remote Desktop connection settings screen:<br />
[[File:Printer3.jpg|left]]<br />
Otherwise you'll end up with your office printers connected to your home computer and it has been known to mess things up.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Printing&diff=14244Printing2022-11-17T13:25:19Z<p>Admin: </p>
<hr />
<div>= Printing at Imperial College =<br />
The print system is handled by HP themselves on behalf of the college. You should follow [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Imperial's Mobile Print instructions]. This page is a guide to follow while you're working through the ICT Mobile Print page.<br />
<br />
== I don't have time for this and just want to print something! ==<br />
If you're in a rush and have to print right now, then email your document to [mailto:print@imperial.ac.uk print@imperial.ac.uk] from your Imperial email address. You may then go to any swipe printer and collect your document. Remember to delete the email from your Sent Items folder afterwards or you may be wasting mailbox space.<br />
<br />
== Advice for when following the Mobile Print instrucions ==<br />
Most of you are Windows users so I'll go over the key steps here. The main ICT instructions are quite good if you pay close attention, but they are from an old version of Windows 10. My page here is a supplement in case you get stuck.<br />
<br />
Log in to the [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Mobile Print site] with your Imperial credentials and then select '''Driver Print'''. Note that this page will only work on campus.<br />
<br />
[[file:mobileprint1.jpeg]]<br />
<br />
Now select your operating system with the large icons at the bottom (you are most likely Windows):<br />
<br />
[[file:mobileprint2os.jpeg]]<br />
<br />
After this you get a couple of pages with only one option each (there used to be more). Go through these formalities and select the only option you have:<br />
<br />
[[File:Mobileprint3option.jpeg]]<br />
<br />
We now get to a mildly tricky bit as we have to download the actual HP software for talking to the printers. There is a handy link on the Mobile Print page to the [https://ftp.hp.com/pub/softlib/software13/COL40842/ds-99374-24/upd-pcl6-x64-7.0.1.24923.exe HP FTP site] so click on that to download it.<br />
<br />
[[File:Mobileprint4drivers.jpeg]]<br />
<br />
Once downloaded, double click on it to run it and press the '''Unzip''' button. Go through the installer as normal but make sure you select '''USB Mode - Plug and Play''' as your printer type. Leave the two text boxes at the bottom ticked.<br />
<br />
[[File:Mobileprint5usb.png]]<br />
<br />
Now we get to add the actual print queue to your computer. From your start menu type '''printers''' and you'll see a menu item to add printers. Click on that and then select '''add a printer'''. It will have a think for a while and then show you some completely incorrect printers. As soon as you see the option '''my printer is not listed here''' click on that.<br />
<br />
[[File:Printernotlisted.png]]<br />
<br />
Now you can copy and paste the name of your print queue, which is listed in Step 4 of the Mobile Print page guide (below is a screenshot for your reference - you can't copy and paste this):<br />
<br />
[[File:Mobileprint6queue.jpeg]]<br />
<br />
You do need to copy and paste the whole address line, including the bit I've blurred out (I've blurred my exact address out or you would be printing as me with my print credit!). Remember that you're not copying and pasting this line, you're using the line from the Mobile Print web page!<br />
<br />
The rest of the ICT instructions should work ok, you merely have to select that you're adding an '''HP''' printer and you want to use '''HP Universal Printing PCL 6'''. There is no need to print a test page.<br />
<br />
=== Gotchas ===<br />
If you are in the habit of using Remote Desktop and the VPN to connect to your DIDE PC, then there is a setting which you should disable in your Remote Desktop connection settings screen:<br />
[[File:Printer3.jpg|left]]<br />
Otherwise you'll end up with your office printers connected to your home computer and it has been known to mess things up.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Printing&diff=14243Printing2022-11-17T12:28:03Z<p>Admin: </p>
<hr />
<div>= Printing at Imperial College =<br />
The print system is handled by HP themselves on behalf of the college. You should follow [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Imperial's Mobile Print instructions].<br />
<br />
== I don't have time for this and just want to print something! ==<br />
If you're in a rush and have to print right now, then email your document to [mailto:print@imperial.ac.uk print@imperial.ac.uk] from your Imperial email address. You may then go to any swipe printer and collect your document. Remember to delete the email from your Sent Items folder afterwards or you may be wasting mailbox space.<br />
<br />
== Advice for when following the Mobile Print instrucions ==<br />
Most of you are Windows users so I'll go over the key steps here. The main ICT instructions are quite good if you pay close attention, but they are from an old version of Windows 10. My page here is a supplement in case you get stuck.<br />
<br />
Log in to the [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Mobile Print site] with your Imperial credentials and then select '''Driver Print'''. Note that this page will only work on campus.<br />
<br />
[[file:mobileprint1.jpeg]]<br />
<br />
Now select your operating system with the large icons at the bottom (you are most likely Windows):<br />
<br />
[[file:mobileprint2os.jpeg]]<br />
<br />
After this you get a couple of pages with only one option each (there used to be more). Go through these formalities and select the only option you have:<br />
<br />
[[File:Mobileprint3option.jpeg]]<br />
<br />
We now get to a mildly tricky bit as we have to download the actual HP software for talking to the printers. There is a handy link on the Mobile Print page to the [https://ftp.hp.com/pub/softlib/software13/COL40842/ds-99374-24/upd-pcl6-x64-7.0.1.24923.exe HP FTP site] so click on that to download it.<br />
<br />
[[File:Mobileprint4drivers.jpeg]]<br />
<br />
Once downloaded, double click on it to run it and press the '''Unzip''' button. Go through the installer as normal but make sure you select '''USB Mode - Plug and Play''' as your printer type. Leave the two text boxes at the bottom ticked.<br />
<br />
[[File:Mobileprint5usb.png]]<br />
<br />
Now we get to add the actual print queue to your computer. From your start menu type '''printers''' and you'll see a menu item to add printers. Click on that and then select '''add a printer'''. It will have a think for a while and then show you some completely incorrect printers. As soon as you see the option '''my printer is not listed here''' click on that.<br />
<br />
[[File:Printernotlisted.png]]<br />
<br />
Now you can copy and paste the name of your print queue, which is listed in Step 4 of the Mobile Print page guide (below is a screenshot for your reference - you can't copy and paste this):<br />
<br />
[[File:Mobileprint6queue.jpeg]]<br />
<br />
You do need to copy and paste the whole address line, including the bit I've blurred out (I've blurred my exact address out or you would be printing as me with my print credit!). Remember that you're not copying and pasting this line, you're using the line from the Mobile Print web page!<br />
<br />
The rest of the ICT instructions should work ok, you merely have to select that you're adding an '''HP''' printer and you want to use '''HP Universal Printing PCL 6'''. There is no need to print a test page.<br />
<br />
=== Gotchas ===<br />
If you are in the habit of using Remote Desktop and the VPN to connect to your DIDE PC, then there is a setting which you should disable in your Remote Desktop connection settings screen:<br />
[[File:Printer3.jpg|left]]<br />
Otherwise you'll end up with your office printers connected to your home computer and it has been known to mess things up.</div>Adminhttps://mrcdata.dide.ic.ac.uk/wiki/index.php?title=Printing&diff=14242Printing2022-11-17T12:24:20Z<p>Admin: </p>
<hr />
<div>= Printing at Imperial College =<br />
The print system is handled by HP themselves on behalf of the college. You should follow [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Imperial's Mobile Print instructions].<br />
<br />
== I don't have time for this and just want to print something! ==<br />
If you're in a rush and have to print right now, then email your document to [mailto:print@imperial.ac.uk print@imperial.ac.uk] from your Imperial email address. You may then go to any swipe printer and collect your document. Remember to delete the email from your Sent Items folder afterwards or you may be wasting mailbox space.<br />
<br />
== Advice for when following the Mobile Print instrucions ==<br />
Most of you are Windows users so I'll go over the key steps here. The main ICT instructions are quite good if you pay close attention, but they are from an old version of Windows 10. My page here is a supplement in case you get stuck.<br />
<br />
Log in to the [https://www.imperial.ac.uk/admin-services/ict/self-service/computers-printing/printing-photocopying-and-scanning/install-print-drivers/ Mobile Print site] with your Imperial credentials and then select '''Driver Print'''. Note that this page will only work on campus.<br />
<br />
[[file:mobileprint1.jpeg]]<br />
<br />
Now select your operating system with the large icons at the bottom (you are most likely Windows):<br />
<br />
[[file:mobileprint2os.jpeg]]<br />
<br />
After this you get a couple of pages with only one option each (there used to be more). Go through these formalities and select the only option you have:<br />
<br />
[[File:Mobileprint3option.jpeg]]<br />
<br />
We now get to a mildly tricky bit as we have to download the actual HP software for talking to the printers. There is a handy link on the Mobile Print page to the [https://ftp.hp.com/pub/softlib/software13/COL40842/ds-99374-24/upd-pcl6-x64-7.0.1.24923.exe HP FTP site] so click on that to download it.<br />
<br />
[[File:Mobileprint4drivers.jpeg]]<br />
<br />
Once downloaded, double click on it to run it and press the '''Unzip''' button. Go through the installer as normal but make sure you select '''USB Mode - Plug and Play''' as your printer type. Leave the two text boxes at the bottom ticked.<br />
<br />
[[File:Mobileprint5usb.png]]<br />
<br />
Now we get to add the actual print queue to your computer. From your start menu type '''printers''' and you'll see a menu item to add printers. Click on that and then select '''add a printer'''. It will have a think for a while and then show you some completely incorrect printers. As soon as you see the option '''my printer is not listed here''' click on that.<br />
<br />
[[File:Printernotlisted.png]]<br />
<br />
Now you can copy and paste the name of your print queue, which is listed in Step 4 of the Mobile Print page guide (below is a screenshot - you can't copy and paste this):<br />
<br />
[[File:Mobileprint6queue.jpeg]]<br />
<br />
You do need to copy and paste the whole address line, including the bit I've blurred out (I've blurred my exact address out or you would be printing as me with my print credit!). Remember that you're not copying and pasting this line, you're using the line from the Mobile Print web page!<br />
<br />
The rest of the ICT instructions should work ok, you merely have to select that you're adding an '''HP''' printer and you want to use '''HP Universal Printing PCL 6'''. There is no need to print a test page.<br />
<br />
=== Gotchas ===<br />
If you are in the habit of using Remote Desktop and the VPN to connect to your DIDE PC, then there is a setting which you should disable in your Remote Desktop connection settings screen:<br />
[[File:Printer3.jpg|left]]<br />
Otherwise you'll end up with your office printers connected to your home computer and it has been known to mess things up.</div>Admin