Many users come with Slurm job scripts, and a lot of AI tooling only knows sbatch and srun. If your cluster runs Open Cluster Scheduler, those scripts used to need rewriting first. slurm-shim removes that step. In an earlier post I explained how srun starts tasks across hosts. Reading about it is one thing. Seeing it run is better, so I recorded a video.

There is also news on MPI. slurm-shim now supports srun --mpi=pmix, so OpenMPI applications run with srun out of the box, just as on a Slurm cluster. mpirun inside a batch script works too. This came with slurm-shim v0.10.0, and the OpenMPI recipe shows both ways, also inside an Apptainer container. Note that --mpi is only in the latest slurm-shim release on GitHub for now. The shim in the Open Cluster Scheduler upstream packages, including 9.1.7, does not have it yet.

The title says what it is about: two front doors, one scheduler. Slurm users keep using sbatch, Grid Engine users keep using qsub, and both kinds of job end up in the same Open Cluster Scheduler.

Run sbatch Jobs on Open Cluster Scheduler: Two Front Doors, One Scheduler

The video is hosted on YouTube. Nothing is loaded from YouTube until you click the button. After that, the video is loaded from youtube-nocookie.com and Google processes your data according to its privacy policy.

Or watch it on YouTube

Now part of the 9.1.7 release

Until recently you had to get slurm-shim separately. With 9.1.7 that is no longer needed. The shim is delivered with Open Cluster Scheduler and Gridware Cluster Scheduler as a self-contained package next to the OCS and GCS packages. It contains the binaries for all supported architectures, a README, and its own license information and SBOM. The release also brings hot spare nodes for Slurm jobs: a job can hold spare nodes in reserve and survive losing up to k of them. All of this is in the 9.1.7 release post on the HPC Gridware blog, together with hosts that join the cluster on their own and configuration snapshots in Qontrol.

A new download page for Open Cluster Scheduler

Open Cluster Scheduler has a new download page: open.clusterscheduler.io. It has all releases since 9.0.0 and a nightly build, and you don't need an account. For 9.1.7 the shim sits under Extensions as ocs-9.1.7-slurm-shim.tar.gz, next to the binary packages for Linux (amd64, arm64, riscv64) and FreeBSD.

The page also works from scripts. The binary package alone is not enough: you also need the common package, which every binary package requires, and I highly recommend the documentation with the man pages. Since 9.1.7 the slurm-shim package is there as well. This fetches all four for Linux x86-64 from the latest release:

curl -fOJL https://open.clusterscheduler.io/latest/bin-lx-amd64.tar.gz
curl -fOJL https://open.clusterscheduler.io/latest/common.tar.gz
curl -fOJL https://open.clusterscheduler.io/latest/doc.tar.gz
curl -fOJL https://open.clusterscheduler.io/latest/slurm-shim.tar.gz

If you try the shim with your own Slurm scripts, I'd like to hear what works and what doesn't.