Coupled Run partitioning on HPC
I recently was able to make a coupled system run on an HPC cluster, but I discovered that the simulation time is much slower than on my desktop PC. This is not the case when I run Fluent alone on the same cluster, so I think the trouble is in the partitioning.
I found a link ( https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v194/sysc_ug/sysc_commandline_analysis_setup_modifysettings.html?q=ParallelArguments ) that describes says ParalleArguments can be used in execCon to allocate the number of processes used by each of the coupling participants. I had to search for an example of how to use this command, and the only one I could find is in the section on Forte 2019R, Forte Beta Features Manual. This is the only documented use of this command, so I am not sure I am using it correctly, but this is what I put in the input file for the system coupling run:
execCon['Solution'].ExecutionControl.ParallelArguments = '-nprocesses 16'
So I put this in the input file for both Solution and Solution 1, since I am first using a single node with 32 cores. However, this results in an error:
Error: eval: unbound variable
Errro Object: -nporcesses
When that didn't work, I dug further into the documentation and found this command ( https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v194/sysc_ug/sysc_cl_PartitionParticipants.html ) and tried to do like example 46 where it is supposed to just divide the nodes equally between participants. But I got another error:
Traceback (most recent call last):
File "/apps/r/ansys/v193/SystemCoupling/PyLib/Controller.py", line 139, in <module>
File "/apps/r/ansys/v193/SystemCoupling/PyLib/Controller.py", line 135, in _run
File "/apps/r/ansys/v193/SystemCoupling/PyLib/Controller.py", line 107, in _executeScript
File "PyLib/kernel/commands/__init__.py", line 31, in readScriptFile
File "PyLib/kernel/commands/CommandManager.py", line 168, in readScriptFile
File "inputfile_p.in", line 3, in <module>
File "PyLib/kernel/commands/CommandDefinition.py", line 72, in func
File "PyLib/kernel/commands/__init__.py", line 28, in executeCommand
File "PyLib/kernel/commands/CommandManager.py", line 120, in executeCommand
File "PyLib/cosimulation/externalinterface/cosim_commands/partitioning.py", line 78, in execute
File "PyLib/cosimulation/partitioning/__init__.py", line 63, in partitionParticipants
File "PyLib/cosimulation/partitioning/__init__.py", line 119, in sharedAllocateMachinesByFraction
File "PyLib/cosimulation/partitioning/__init__.py", line 142, in __allocateCommon
File "PyLib/cosimulation/partitioning/machinelist.py", line 50, in loadMachines
RuntimeError: Host list not found.
From the error list it looks like ANSYS expects a hostlist even though the example I mentioned above issues the command exactly the way I did. The problem is I cannot specify the host names beforehand, as this is a cluster with hundreds of nodes and when I submitt a job I get whatever node is available according to the queuing system.
Question: How can I specify the number of nodes used in System Coupling through HPC? I need to run on both one node and several nodes, each with 32 cores. How can I accomplish this task?