Learning Forum

[PavanaKollipara]

Hi,

 

     Our group has purchased 5 licenses for FDTD modules. When we tried to run our cases on the Texas Advanced Computing Center (TACC), we found that we at most use 64 cores for the computation. However, it runs quite slow. For example, for one case requires ~30 GB RAM, after running 2 hours with 64 cores from TACC, the Autoshutoff min still at 1. Even compared with our local compature with 32 GB RAM, it looks abnormally slow, which makes us not able to perform our simulations on TACC. Furthermore, after we did some tests, we found one license at most requires 16 cores, which means that 4 licenses are avaliable on TACC rather than 5 we have bought. Our main concerns are: 1) Whether the one license correponding to 16 cores is correct and 2) How could we accelerate the simulation speed? P.S., the details of TACC source could be found in this link for your interest: https://portal.tacc.utexas.edu/user-guides/stampede2 .

 

Thanks for your help!

Pavana

[Lito Yap]

@Pavana,

1 FDTD solve license allows running a simulation up to a maximum of 32 cores as shown here. Ensure that you are submitting your simulation job with the "job scheduler submission script" to run the job on the cluster compute nodes and not on the login node. Depending on your simulation and requirements, you can accelerate the run time by increasing the number of processes used to run the simulation as shown here. 1 FDTD solve license will be checked out for each increment of 32 processes/cores running the simulation job.  

[Guilin Sun]

In addition to the above, I have the following supplementary comments:

The slow simulation may not be related to the total number of licenses. Depending on the memory requirement or simulation settings, a 30GB file may have small simulation volume but a large amount of frequency-domain calculations, for example, a 3D frequency-domain monitor with hundreds of frequency points. In that case it is not limited by memory. Rather it is by CPU speed. (to be continued)

[Guilin Sun]

One simulation file is run in distributed parallel computing which needs only one license. More licenses are needed for more than one independent simulation files, which is called concurrent parallel simulation: Concurrent Parametric Computing