Humus

Humus is a pair of computers located at Dalhousie University managed by ACENET on behalf of researchers who purchased the equipment. Humus is integrated with the Nefelibata cluster, but Humus and the balance of Nefelibata are owned by two different research groups.

Access

You may only access Humus with the permission of the contributing Principal Investigator. If you don't have access to Humus and believe you should, please write to support@ace-net.ca and say so. Copy the PI, and mention in the email your Digital Research Alliance of Canada username.

The two machines are named humus216 and humus217. (The numbers are in relation to the node numbering of Nefelibata.) Choose one to log into, like so:

 ssh -J username@nefelibata.ace-net.ca username@humus216

or

 ssh -J username@nefelibata.ace-net.ca username@humus217

Supply your Alliance password when prompted. The -J parameter above stands for "jump"; you are "jumping" through nefelibata.ace-net.ca to get to humus21[6,7].

We strongly recommend creating and installing an SSH key pair and using that for access after your first login.

• For more on key pairs, see https://docs.alliancecan.ca/wiki/SSH_Keys.
• For more on SSH in general, see https://docs.alliancecan.ca/wiki/SSH.
• Step-by-step instructions for setting up key-pair authentication from Windows using MobaXterm

Storage system

When you log in to Humus your default working directory will be /home/username on the shared Nefelibata filesystem. But there is very little space on it for Humus users.

For primary storage please use /scratch/username on each Humus node. Each node has approximately 58 TB of (unshared, node-local) storage space in /scratch.

Data transfer

sftp can be used for interactive command-line data transfer, following the pattern shown above for ssh:

sftp -J username@nefelibata.ace-net.ca username@humus216
> cd /scratch/username
> put somefile.txt
> get some-other-file
> exit

scp can be used similarly for wild-card or script-driven data transfers:

scp -J username@nefelibata.ace-net.ca somefiles.* username@humus216:/scratch/username/

Graphical (drag-and-drop) file transfer applications can be used if they implement the scp or sftp protocols. WinSCP has been tested successfully; use username@humus216.nefelibata.ace-net.ca etc.

Software

Humus uses the same modules system as Alliance clusters, providing access to the same list of available software.

For software which requires a graphical user interface (e.g. RStudio), you can connect to Humus using VNC. Here is a step-by-step guide to creating a VNC session.

Job scheduling

We recommend using batch jobs for your work whenever feasible, using the Slurm job scheduler. This allows you to queue up several tasks which will execute one after another without constant human supervision. It also allows group members to share the resources more effectively, 'e.g.' the scheduler will not try to run two different jobs that both need a GPU on the same node at the same time; it will either run them on different nodes or at different times.

The Humus nodes carry NVidia A30 GPUs, so the following command will run `jobscript` on one entire node:

sbatch --time=6:0:0 --gres=gpu:a30:1 --cpus-per-task=64 --mem=0 jobscript

If the data resides on (for example) humus217 but not on humus216, you should specify which node the job should execute on with `--nodelist`. Don't omit to specify the CPU, GPU and memory you need though! For example:

sbatch --nodelist=humus217 --time=6:0:0 --gres=gpu:a30:1 --cpus-per-task=64 --mem=0 jobscript

If you do not need the GPU for a certain calculation, or you need less than an entire node's CPUs and memory, please specify only what you need. The scheduler can run more than one such job on the node at a time. For example:

sbatch --time=24:0:0 --cpus-per-task=1 --mem=4G jobscript