Running Ollama on the HPC Cluster
Starting an interactive session for the ollama user service
- Load slurm utils (sinteractive)
module load slurm/utils
- Start an interactive session using sinteractive, see : https://hpc.wiki.utwente.nl/slurm:sinteractive
sinteractive --gres=gpu:1
wait until you got resources assigned !!
Starting the ollama server and interacting with it
- Once you got an interactive session with gpu (a bit crowded right now), load the module ollama and start the user daemon
module load ollama/0.1.41
start-ollama
- Now you can use the download, list, run commands of ollama (here for example : run llama3)
ollama run llama3
- Once completed you can stop the user daemon and terminate the interactive session
stop-ollama
exit