Microsoft Linux Cluster User Notes

Using the Linux Cluster

Get in touch

This is still a bit experimental, so mail Wes to get started.

Sort out your shares

• On linux, the ~ symbol refers to something that is probably not what you want - it's your local linux home directory, on the particular node that your job happens to get run on. So different jobs might have a different local linux home directory, because they might be run on different nodes.
• Instead, use ~/dide/home - which will be mapped to your DIDE home directory.
• If you want to have other shares mapped, then, edit or create a file in your DIDE home directory, called .pam_mount.conf.xml which contains something like this.

<?xml version="1.0" encoding="utf-8" ?>
<pam_mount>
  <volume options="vers=2.1,nodev,nosuid" user="*" mountpoint="/homes/<dide-username>/dide/tmp" path="Tmp" server="fi--didef2.dide.ic.ac.uk" fstype="cifs" />
  <volume options="vers=2.1,nodev,nosuid" user="*" mountpoint="/homes/<dide-username>/dide/f2gsim" path="GlobalSim" server="fi--didef2.dide.ic.ac.uk" fstype="cifs" />
  <volume options="vers=2.1,nodev,nosuid" user="*" mountpoint="/homes/<dide-username>/dide/nas1gsim" path="Test" server="fi--didenas1.dide.ic.ac.uk" fstype="cifs" />
</pam_mount>

• Obviously, replace <dide-username> with your username. So note here that /homes/<dide-username> is what Linux refers to when you use ~.
• The full path to your dide home directory, from a linux job, will therefore be: /homes/<dide-username>/dide/home.

Running Something

Hello World in C

• We'll probably want the code to be compiled on the cluster, so here's a really simple example that compiles and runs something....
• Here's my complicated simulation code, which I'm going to save as \\fi--san03\homes\wrh1\testlinux\hw.c.

#include <stdio.h>
int main() {
  printf("Hello World!\n");
  return 0;
}

• And now I'm going to write a script that compiles this code on Linux, and runs it collecting some results.
• IMPORTANT: - this is going to be a .sh files, and must have Linux line-endings.

cd ~/dide/home/testlinux
gcc -o hw hw.c
chmod 755 ./hw
./hw > output.txt

Launching it from Windows

• I'm going to write \\fi--san03\homes\wrh1\testlinux\launch.bat - which is a Windows batch file to launch the job.

job submit /scheduler:fi--didelxhn /jobtemplate:LinuxNodes ~/dide/home/testlinux/run.sh

• This launch.bat can be anywhere on your local machine, but I've kept everything in the same place for simplicity.
• Run the launch.bat on your Windows machine, and when the job runs, you'll see the hw executable file, and the output.txt file appear.

Launching it from the Web Portal

• If you're not on Windows (which may be one reason you want to run on a linux cluster...), then use the HPC Web Portal to launch the job.
• Log into the portal - if you've got rights, you'll see fi--didelxn listed. (Otherwise, see the section at the top...!)
• Submit Job on the fi--didelxn.
• Paste ~/dide/home/testlinux/run.sh in the main job window. Set other things if you like. And submit it.
• Look for hw and output.txt to appear in the test folder.