1,453
edits
Changes
amend language to "billing hours"
* "Account" is not the same as "user" in this context. Your firm or organization has a contract with ACENET (or you probably wouldn't be reading this.) On Siku, that contract is represented by an "account" with a name like "pd-abc-123".
* "QoS" stands for "Quality of Service", but it might be better to think of a "QoS" as a software object which remembers how many CPU hours etc. an account is allowed to use, and how much has been used already. Each paid account is associated with its own QoS, and the QoS has the same name as the account, like "pd-abc-123".
* "Billing unitshours" measure the use of the system. One CPU-minute hour and associated RAM is worth one billing unithour. A GPU-minute hour is worth 35 billing unitshours. See below for a formula, and examples. ** A "billing minute" is simply one-sixtieth of a billing hour.** We have also used the term "CPU hours" in the past, with the same meaning.
== How much computing can I do? ==
You can see the number of billing units hours available to you through your QoS by running the utility <tt>acct-tool</tt>. To make them somewhat easier to interpret, they're translated from minutes into hours and referred to as "CPU hours". The output will look something like this:
Available QoSs: pd-abc-123
For QoS 'pd-abc-123' this quarter:
Billing (CPU -equiv) hours cap: 2190000.0 (131400000 billing unitsminutes) Billing (CPU -equiv) hours used: 57.7 (3461 billing unitsminutes)
When your team gets close to the limit, you may find that your jobs are not starting, but instead staying in PD (pending) state with "QOSGrpBillingMinutes" showing in the "Reason" field of squeue (or sq). Or, you may receive this pair of messages on trying to submit a job with with sbatch:
End time: 2019-11-14T20:27:50
Billing (CPU -equiv) hours per user:
198.4 alice
9817.9 bob
<pre>
</pre>
A job that reserves one CPU and 4G of RAM and runs for one minute hour consumes 1 billing unithour. One CPU and 10G of RAM for one minutehour? 2.15 billing unitshours.
One of our '''GPU-equipped nodes''' has 40 CPUs, 186G of RAM, and two GPUs. To use that node for 24 hours would cost
MAX( 40, 186*0.215, 2*35 ) * 24 * 60 = 70 * 24 * 60 = 100800 1680 billing unitshours.
The rate per GB of RAM is chosen so that using either all the CPUs or all the RAM on a basic node costs the same 40 billing units.
Requesting all the memory on a '''high-memory node''' costs 80 billing units per minute.
You can see get the billing ''rate'' for a live job like so:
[you@login1 ~]$ scontrol show job 7976 | grep billing
TRES=cpu=12,mem=108000M,node=1,billing=22
This job will be billed at a rate of 22 billing units per minute unit of elapsed time. You can derive determine the billing unit units consumed by a an individual job, once it has completed ,by examining the Slurm accounting record for the job withand multiplying the billing rate by the elapsed time:
[you@login1 ~]$ sacct -X --format=AllocTRES%40,Elapsed --noheader -j 7402
billing=40,cpu=40,mem=10G,node=1 00:02:09
== Why the funny word, QoS? ==