Cluster

Scale up on Cluster

Suggest edits

Documentation > Scale Up

Content:

1 - Using clusters with OpenMOLE

Using clusters with OpenMOLE 🔗

Batch systems 🔗

Many distributed computing environments offer batch processing capabilities. OpenMOLE supports most of the batch systems.
Batch systems generally work by exposing an entry point on which the user can log in and submit jobs. OpenMOLE accesses this entry point using SSH. Different environments can be assigned to delegate the workload resulting of different tasks or groups of tasks. However, not all clusters expose the same features, so options may vary from one environment to another.

Before being able to use a batch system, you should first provide your authentication information to OpenMOLE.

Singularity 🔗

To make sure that OpenMOLE tasks runs on your cluster you might want to ensure that Singularity or Apptainer is properly working on the cluster nodes.
To test it you should first use a Linux system on which are root. First, install Singularity on this Linux system and then build the lolcow.sif image:

sudo singularity build lolcow.sif library://sylabs-jms/testing/lolcow

Copy the lolcow.sif image on your cluster and log to a worker node (from the head node you can submit an interactive job, see the doc of your cluster). Then try running these 2 commands:

singularity overlay create -S -s 20000 test.img
singularity exec --overlay test.img lolcow.sif cowsay OpenMOLE

If you see a cow saying OpenMOLE, congrats the OpenMOLE tasks should work fine on the cluster. In case there is an error, you can report it to the cluster admin team and ask for help.

Sometimes to make Singularity or Apptainer available you might have to load the corresponding modules. To check the available modules:

module avail

And to load it:

module load apptainer

In the case you need to load a module, mind passing it to your OpenMOLE environment as well, for instance:

val env =
  SLURMEnvironment(
    "login",
    "machine.domain",
    modules = Seq("apptainer")
  )

Grouping 🔗

You should also note that the use of a batch environment is generally not suited for short tasks, i.e. less than 1 minute for a cluster. In case your tasks are short, you can group several executions with the keyword by in your workflow. For instance, the workflow below groups the execution of model by 100 in each job submitted to the environment:

// Define the variables that are transmitted between the tasks
val i = Val[Double]
val res = Val[Double]

// Define the model, here it is a simple task executing "res = i * 2", but it can be your model
val model =
  ScalaTask("val res = i * 2") set (
    inputs += i,
    outputs += (i, res)
  )

// Define a local environment
val env = SLURMEnvironment("login", "machine.domain")

// Make the model run on the the local environment
DirectSampling(
  evaluation = model hook display,
  sampling = i in (0.0 to 1000.0 by 1.0)
) on env by 100

PBS 🔗

PBS is a venerable batch system for clusters. It is also referred to as Torque. You may use a PBS computing environment as follows:

val env =
  PBSEnvironment(
    "login",
    "machine.domain"
  )