Microsoft Linux Cluster User Notes: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
|||
(One intermediate revision by the same user not shown) | |||
Line 22: | Line 22: | ||
* Obviously, replace <code><dide-username></code> with your username. So note here that <code>/homes/<dide-username></code> is what Linux refers to when you use <code>~</code>. | * Obviously, replace <code><dide-username></code> with your username. So note here that <code>/homes/<dide-username></code> is what Linux refers to when you use <code>~</code>. | ||
* The full path to your dide home directory, from a linux job, will therefore be: <code>/homes/<dide-username>/dide/home</code>. | * The full path to your dide home directory, from a linux job, will therefore be: <code>/homes/<dide-username>/dide/home</code>. | ||
== Running Something == | |||
=== Hello World in C === | |||
* We'll probably want the code to be compiled on the cluster, so here's a really simple example that compiles and runs something.... | |||
* Here's my complicated simulation code, which I'm going to save as <code>\\fi--san03\homes\wrh1\testlinux\hw.c</code>. | |||
<pre> | |||
#include <stdio.h> | |||
int main() { | |||
printf("Hello World!\n"); | |||
return 0; | |||
} | |||
</pre> | |||
* And now I'm going to write a script that compiles this code on Linux, and runs it collecting some results. | |||
* '''IMPORTANT:''' - this is going to be a <code>.sh</code> files, and must have Linux line-endings. | |||
<pre> | |||
cd ~/dide/home/testlinux | |||
gcc -o hw hw.c | |||
chmod 755 ./hw | |||
./hw > output.txt | |||
</pre> | |||
=== Launching it from Windows === | |||
* I'm going to write <code>\\fi--san03\homes\wrh1\testlinux\launch.bat</code> - which is a '''Windows''' batch file to launch the job. | |||
<pre> | |||
job submit /scheduler:fi--didelxhn /jobtemplate:LinuxNodes ~/dide/home/testlinux/run.sh | |||
</pre> | |||
* This launch.bat can be anywhere on your local machine, but I've kept everything in the same place for simplicity. | |||
* Run the launch.bat on your Windows machine, and when the job runs, you'll see the <code>hw</code> executable file, and the <code>output.txt</code> file appear. | |||
=== Launching it from the Web Portal === | |||
* If you're not on Windows (which may be one reason you want to run on a linux cluster...), then use the HPC Web Portal to launch the job. | |||
* Log into the portal - if you've got rights, you'll see <code>fi--didelxn</code> listed. (Otherwise, see the section at the top...!) | |||
* Submit Job on the <code>fi--didelxn</code>. | |||
* Paste <code>~/dide/home/testlinux/run.sh</code> in the main job window. Set other things if you like. And submit it. | |||
* Look for <code>hw</code> and <code>output.txt</code> to appear in the test folder. |
Latest revision as of 12:06, 10 October 2017
Using the Linux Cluster
Get in touch
This is still a bit experimental, so mail Wes to get started.
- On linux, the
~
symbol refers to something that is probably not what you want - it's your local linux home directory, on the particular node that your job happens to get run on. So different jobs might have a different local linux home directory, because they might be run on different nodes. - Instead, use
~/dide/home
- which will be mapped to your DIDE home directory. - If you want to have other shares mapped, then, edit or create a file in your DIDE home directory, called
.pam_mount.conf.xml
which contains something like this.
<?xml version="1.0" encoding="utf-8" ?> <pam_mount> <volume options="vers=2.1,nodev,nosuid" user="*" mountpoint="/homes/<dide-username>/dide/tmp" path="Tmp" server="fi--didef2.dide.ic.ac.uk" fstype="cifs" /> <volume options="vers=2.1,nodev,nosuid" user="*" mountpoint="/homes/<dide-username>/dide/f2gsim" path="GlobalSim" server="fi--didef2.dide.ic.ac.uk" fstype="cifs" /> <volume options="vers=2.1,nodev,nosuid" user="*" mountpoint="/homes/<dide-username>/dide/nas1gsim" path="Test" server="fi--didenas1.dide.ic.ac.uk" fstype="cifs" /> </pam_mount>
- Obviously, replace
<dide-username>
with your username. So note here that/homes/<dide-username>
is what Linux refers to when you use~
. - The full path to your dide home directory, from a linux job, will therefore be:
/homes/<dide-username>/dide/home
.
Running Something
Hello World in C
- We'll probably want the code to be compiled on the cluster, so here's a really simple example that compiles and runs something....
- Here's my complicated simulation code, which I'm going to save as
\\fi--san03\homes\wrh1\testlinux\hw.c
.
#include <stdio.h> int main() { printf("Hello World!\n"); return 0; }
- And now I'm going to write a script that compiles this code on Linux, and runs it collecting some results.
- IMPORTANT: - this is going to be a
.sh
files, and must have Linux line-endings.
cd ~/dide/home/testlinux gcc -o hw hw.c chmod 755 ./hw ./hw > output.txt
Launching it from Windows
- I'm going to write
\\fi--san03\homes\wrh1\testlinux\launch.bat
- which is a Windows batch file to launch the job.
job submit /scheduler:fi--didelxhn /jobtemplate:LinuxNodes ~/dide/home/testlinux/run.sh
- This launch.bat can be anywhere on your local machine, but I've kept everything in the same place for simplicity.
- Run the launch.bat on your Windows machine, and when the job runs, you'll see the
hw
executable file, and theoutput.txt
file appear.
Launching it from the Web Portal
- If you're not on Windows (which may be one reason you want to run on a linux cluster...), then use the HPC Web Portal to launch the job.
- Log into the portal - if you've got rights, you'll see
fi--didelxn
listed. (Otherwise, see the section at the top...!) - Submit Job on the
fi--didelxn
. - Paste
~/dide/home/testlinux/run.sh
in the main job window. Set other things if you like. And submit it. - Look for
hw
andoutput.txt
to appear in the test folder.