====== HPC wiki ======
Information about the HPC clusters of the University of Twente.
===== Getting Started =====
==== Account ====
For an new account on any of the clusters refer to the **[[start#cluster_table|Cluster table]]** below to see who is responsible for account creation on the cluster.
==== Basics ====
All home directory's are mounted on /home after a users in logged in. \\
Software is located under /software. \\
You can set the path and other variables to use the software by using the [[environment-modules|module]] for the software. \\
\\
There is a "scratch" space on all nodes in /local. \\
You can create a directory for yourself and put some temporary files there. \\
This can benefit the speed of your job because the local storage is much faster. Your home directory is on NFS which is slow. \\
This "scratch" space is **automatically cleaned every** day if the files are not changed for **90 days**! \\
If you just copy or move the files from this space to your home directory as the last command in your job you won't loose anything. \\
\\
==== Basic workflow ====
- [[#connecting_to_a_cluster|Connect]] to the [[#cluster_table|cluster]] using SSH \\
- Transfer files to the cluster \\
- Create a job script and submit/[[#scheduler_s|schedule]] your job \\
* Do you need a [[#environment_modules|module]]
* do you need to copy data to and from the node
- Wait until your job gets queued, it gets executed and it finishes \\
- Study the results generated by your job \\
* remove data from the cluster afterwards to free diskspace
==== Connecting to a Cluster ====
To access the cluster you will need to be working on a PC connected to the UT network or use [[https://www.utwente.nl/en/service-portal/hardware-software-network/network-eduroam-vpn-etc/vpn-eduvpn|EduVPN]].
Refer to the **[[start#cluster_table|Cluster table]]** below to see which headnode to use.
OpenSource SSH clients: \\
- [[https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html|PuTTy]]; SSH CLI terminal. \\
- [[https://www.microsoft.com/en-us/p/windows-terminal/9n0dx20hk701?activetab=pivot:overviewtab|Windows Terminal]]; Terminal for Windows systems only. \\
- [[https://mobaxterm.mobatek.net/|MobaXterm]]; Enchanced terminal for windows with X11 server, tabbed SSH client. \\
Refer to the **[[connecting|connecting]]** page for more information.
==== Environment Modules ====
See [[http://modules.sourceforge.net|Environment Modules on SourceForge]] for more in depth information.
=== Quick overview ===
The clusters have quite a few programs pre-installed. They are managed as optional modules that you have to load before you can use them. One reason for using them this way is that this makes it possible to have multiple versions of the same program on the same system. Below there is an basic explanation of module usage.
- Using modules \\
''module av'' to list all available modules on the cluster. \\
------------------------------------------------ /software/Modulefiles -------------------------------------------------
bison/2.7.1 centaur/14 clang/3.5.0 matlab/R2012b ofed/3.18 paraview/5.8.0
castep/20.11 centaur/14.1 clang/3.6.0 matlab/R2017a openfoam/20.06
centaur/11 centaur/14.5 cuda/7.0 matlab/R2020b paraview/5.4.1
centaur/12 centaur/15 firedrake/firedrake metis/5.1.0 paraview/5.6.1
centaur/13.5 centaur/15.1 mathematica/11.2 miniconda/3 paraview/5.6.1-GUI(default)
''module load '' to load a available module. \\
''module list'' to list all currently active modules. \\
Currently Loaded Modulefiles:
1) matlab/R2020b 2) paraview/5.6.1
''module unload '' to unload a loaded module. \\
''module purge'' to unload all loaded modules. \\
Refer to the **[[start#cluster_table|Cluster table]]** below to see the cluster software details.
==== Scheduler(s) ====
Refer to the corresponding scheduler page for more information :
* [[slurm:start|Slurm]] or [[https://slurm.schedmd.com/quickstart.html|the Slurm quickstart]]
* [[hadoop-yarn:start|Hadoop/Yarn]].
* t.b.d.
==== Changing your password ====
We use 2 different types of accounts: AD or local. \\
\\
If you login with you AD account you can change you password by typing: passwd \\
This can have a big impact because this account is also used for all of the other services on the University. Think twice before you use this. \\
\\
If you use a local (NIS) account you can change your password by typing: yppasswd \\
===== Cluster table =====
^Cluster Name ^Scheduler ^Monitor ^Contact ^
| [[AM Cluster|AM Cluster]] | [[slurm:start|Slurm]] | | [[https://people.utwente.nl/h.d.vancorbach|Herman van Corbach]] |
| [[CCP Cluster|CCP Cluster]] | [[slurm:start|Slurm]] | | [[https://people.utwente.nl/c.filippi|Claudia Filippi]] [[https://people.utwente.nl/m.bokdam|Menno Bokdam]] [[https://people.utwente.nl/l.leppert|Linn Leppert]] |
| [[eemcs-hadoop| EEMCS (DSI/CTIT) Hadoop Cluster]] | [[hadoop-yarn:start|Hadoop/Yarn]] [[slurm:start|Slurm]] | [[https://spark-nn.eemcs.utwente.nl:8088/cluster|Dashboard]] | [[https://people.utwente.nl/jan.flokstra|Jan Flokstra]] |
| [[eemcs-hpc| EEMCS (DSI/CTIT) HPC Cluster]] | [[slurm:start|Slurm]] | [[http://hpc-status.ewi.utwente.nl/slurm|Dashboard]] | [[https://people.utwente.nl/g.j.laanstra|Geert Jan Laanstra]] |
| [[MSM3 Cluster| MSM3 Cluster]] | [[slurm:start|Slurm]] | | [[https://people.utwente.nl/m.h.g.wilens|Martin Wilens]] |
| [[Serendipity Cluster|Serendipity Cluster ]] | [[slurm:start|Slurm]] | | [[https://people.utwente.nl/m.schlottbom|Matthias Schlottbom]] [[https://people.utwente.nl/b.j.geurts|Bernard Geurts]] |
| [[TFE2 Cluster|TFE2 Cluster]] | [[slurm:start|Slurm]] | | [[https://people.utwente.nl/m.h.g.wilens|Martin Wilens]] |