Photonics

Photonics

Concurrent computing on a Slurm-based cluster

    • pierre-luc.piveteau
      Subscriber

      Hello,

      I'm running Lumerical FDTD on my university on-premise cluster. On this cluster, jobs are submitted to Slurm via a batch script.

      How can I run multiple simulations at the same time (in the limit of our license capabilities, of course) ?

      I did some tests, but with the following line, it appears that only the last simulation (simulation_3.fsp) was submitted - regardless of the number of cores, nodes, or tasks involved.

    • Lito Yap
      Ansys Employee
      Please consult with your IT/cluster admin on a submission script for your job scheduler to submit a job to run all the simulation.fsp files on your specific "folder" in your cluster. Otherwise, you can run the simulation files on a specific folder in sequence on your cluster using the "fdtd-run-local.sh" script bundled with our installation. See this post for details.
    • Lito Yap
      Ansys Employee
      Try this sample script:
      #SBATCH --job-name=lumtest#Job name
      #SBATCH --nodes=3 #3 nodes
      #SBATCH --ntasks=48#48 total processes/tasks
      #SBATCH --cpus-per-task=1#1 cpu/core per task
      #SBATCH --mem=64gb#64gb memory
      #SBATCH --time=01:00:00#1 hr wall time (h:m:s)
      #change directory to the location of your simulation files
      cd /path_to/simulationfiles
      #load your MPI and Lumerical modules
      #module load intelmpi lumericalv221
      #run FDTD simulation with intel mpi
      srun -n 16 fdtd-engine-impi-lcl -t 1 simulationfile1.fsp&
      srun -n 16 fdtd-engine-impi-lcl -t 1 simulationfile2.fsp&
      srun -n 16 fdtd-engine-impi-lcl -t 1 simulationfile3.fsp&
      wait
    • pierre-luc.piveteau
      Subscriber
      Dear Thank you for your quick and detailed answer ! It's working well, using your sample script.
      I specified explicitly the RAM reservation for each srun, otherwise the whole node RAM was taken by the first job : e.g :
      srun -n 16 --mem =20G fdtd-engine-impi-lcl -t 1 simulationfile1.fsp &
      Thanks again !
    • Lito Yap
      Ansys Employee
      You are welcome. Yes, you can modify the sample script as needed. i.e. nodes, tasks, time, jobname, etc.
Viewing 4 reply threads
  • You must be logged in to reply to this topic.