====== Running Ollama on the HPC Cluster ======

===== Starting an interactive session for the ollama user service =====

  * Load slurm utils (sinteractive)

<code bash>
module load slurm/utils
</code>

  * Start an interactive session using sinteractive, see : https://hpc.wiki.utwente.nl/slurm:sinteractive

<code bash>
sinteractive --gres=gpu:1
</code>

wait until you got resources assigned !!

==== Starting the ollama server and interacting with it ====

  * Once you got an interactive session with gpu (a bit crowded right now), load the module ollama and start the user daemon

<code bash>
module load ollama/0.1.41
start-ollama
</code>

  * Now you can use the download, list, run commands of ollama (here for example : run llama3)

<code bash>
ollama run llama3
</code>

  * Once completed you can stop the user daemon and terminate the interactive session

<code bash>
stop-ollama
exit
</code>

===== Setting Up Environment Variables for Ollama =====

Because Ollama runs as a user service, it does not have access to the environment variables defined in your bash session. If you need to add environment variables for Ollama, you must include them in your local Ollama service file. To do this, start Ollama, and once it is running, enter:

<code bash>systemctl --user edit ollama</code>

This will open the local ''ollama.service'' configuration file, where you can modify various settings, including environment variables visible to the service.

After making any changes, save the file (Ctrl+O). Then reload the daemon and restart Ollama:

<code bash>
systemctl --user daemon-reload
systemctl --user restart ollama
</code>

See the examples below on how to set up a proxy or enable remote connections.

==== Proxy Setup ====

On the cluster, downloads are only possible through a proxy server. If you want to pull new models via Ollama, you can add the proxy settings by modifying the service file as shown below:

<code bash>### Anything between here and the comment below will become the new contents of the file

[Service]
Environment="HTTPS_PROXY=http://proxy.utwente.nl:3128"

### Lines below this comment will be discarded</code>

**Note**: Remember to reload the daemon and restart Ollama after saving your changes!

==== Remote Connection Setup ====

If you want to run the Ollama daemon on the cluster but connect to it from your local machine, modify your Ollama service file as follows:

<code bash>### Anything between here and the comment below will become the new contents of the file

[Service]
Environment="OLLAMA_HOST=0.0.0.0:8886"

### Lines below this comment will be discarded</code>

**Note 1**: Remember to reload the daemon and restart Ollama after saving your changes.\\
**Note 2**: The example above uses port **8886**, but you can change it to any open port in range **<port_A>–<port_B>** that is accessible on the nodes.

To connect from your local machine, forward a local port to the port on which Ollama is running on the cluster:

<code bash>
ssh -L 11434:node_name:8886 username@hpc-head1.ewi.utwente.nl
</code>

**Note 1**: In this example, **11434** is the local port on your machine. Ollama uses this port by default, so make sure you are not also running Ollama locally if you use **11434**.\\
**Note 2**: Replace ''node_name'' with the actual node where Ollama is running (for example, ''ctit082'').