Learning Forum

[pierre-luc.piveteau]

Hello,

I'm running Lumerical FDTD on my university on-premise cluster. On this cluster, jobs are submitted to Slurm via a batch script.

How can I run multiple simulations at the same time (in the limit of our license capabilities, of course) ?

I did some tests, but with the following line, it appears that only the last simulation (simulation_3.fsp) was submitted - regardless of the number of cores, nodes, or tasks involved.

[Lito Yap]

Please consult with your IT/cluster admin on a submission script for your job scheduler to submit a job to run all the simulation.fsp files on your specific "folder" in your cluster. Otherwise, you can run the simulation files on a specific folder in sequence on your cluster using the "fdtd-run-local.sh" script bundled with our installation. See this post for details.

[Lito Yap]

Try this sample script:
#SBATCH --job-name=lumtest#Job name
#SBATCH --nodes=3 #3 nodes
#SBATCH --ntasks=48#48 total processes/tasks
#SBATCH --cpus-per-task=1#1 cpu/core per task
#SBATCH --mem=64gb#64gb memory
#SBATCH --time=01:00:00#1 hr wall time (h:m:s)
#change directory to the location of your simulation files
cd /path_to/simulationfiles
#load your MPI and Lumerical modules
#module load intelmpi lumericalv221
#run FDTD simulation with intel mpi
srun -n 16 fdtd-engine-impi-lcl -t 1 simulationfile1.fsp&
srun -n 16 fdtd-engine-impi-lcl -t 1 simulationfile2.fsp&
srun -n 16 fdtd-engine-impi-lcl -t 1 simulationfile3.fsp&
wait

[pierre-luc.piveteau]

Dear Thank you for your quick and detailed answer ! It's working well, using your sample script.
I specified explicitly the RAM reservation for each srun, otherwise the whole node RAM was taken by the first job : e.g :
srun -n 16 --mem =20G fdtd-engine-impi-lcl -t 1 simulationfile1.fsp &
Thanks again !

[Lito Yap]

You are welcome. Yes, you can modify the sample script as needed. i.e. nodes, tasks, time, jobname, etc.