Learning Forum

[Mehdi Feizpour]

Hello community! 

I have a Python code that makes a group of 20 simulations with different geometries, one after the other. After each simulation is done, the monitor data is saved in an array and then these 20 arrays are analyzed. 

Recently, I am running some simulations on my university's cluster that uses Slurm scheduler. I would like to use the cluster for the aforementioned Python code. I am thinking to send the 20 simulations in parallel to the cluster and then when all are done, analyze their results. Needless to say that the simulations start based on the availability of resources. Thus, they will finish at different times. I would appreciate any input you might have for the asynchronous running of multiple Lumerical FDTD simulations.

[Guilin Sun]

If you have sufficent licenses, you can set resources if you run the python in GUI, and send all the files through Slurm scheduler. At this point it is up to the Slurm scheduler that assigns the simulation files for concurrent simulation. Please refer to

https://optics.ansys.com/hc/en-us/articles/360034620113-Lumerical-job-scheduler-integration-configuration-Slurm-Torque-LSF-SGE-

https://optics.ansys.com/hc/en-us/articles/360039028654-Job-scheduler-submission-scripts-SGE-Slurm-Torque-

https://optics.ansys.com/hc/en-us/articles/360043624373-Running-simulation-jobs-with-PBS-scheduler

https://optics.ansys.com/hc/en-us/articles/360026165714-Running-simulations-on-your-cluster

 

You can also fully utilize Lumerical's features:
1: if you can set the geometry properly in a structure group, or in "model", with an identifier that selects a specific configuration, or the geometry is simply changing its dimension/location with variables  (parameterized), you can use the sweep utility in "optimization and sweeps", and use Python to run the sweep.

script:

runsweep - Script command

 

2: You can also use Lumerical script

addjob

runjobs

addjob - Script command

runjobs - Script command

listjobs - Script command

then you can use Python to run them.

The key is to have the jobs in the queque for concurent simulation https://optics.ansys.com/hc/en-us/articles/360026162414-Concurrent-Parametric-Computing

with proper licenses.

 

The sweep utility has the capability to create all the simulation files, and send them to be simulated concurrently depedning on the resource configuration ("capacity"):   https://optics.ansys.com/hc/en-us/articles/360025161033-Compute-resource-configuration-use-cases 

[Mehdi Feizpour]

Dear Guilin, Thank you for the extensive explanation and the quick reply! It is very helpful. Going through the info you kindly provided, I think the most straightforward approach would be to use:

addjob - Script command

runjobs - Script command

As our cluster does not support a GUI. I will try it and will share my experience here.