October 20, 2022 at 1:16 pmdheryadiSubscriber
I manage Ansys installation and administration on our site. Our cluster is running RHEL8 and Univa Grid Engine (UGE) 8.6.18 is used for our job scheduler. Our end user reported that they're not able to run Fluent across multiple nodes. Basically, all Fluent processes are running on a single node, instead of distributed across nodes. I understand that Fluent comes with its own MPI implementation and was wondering whether this is a known issue with this software on RHEL 8 and Grid Engine 8.6. It would be greatly appreciated if you have any suggestions on how to resolve this issue (e.g. command line options to use, etc).
October 28, 2022 at 7:41 pmHunter WangAnsys Employee
If you use custom SGE scripts instead of relying on the standard Fluent option (either the -scheduler=sge option from the command line or the Use SGE option in Fluent Launcher, as described in the preceding sections), your environment variables related to the job scheduler will not be used unless you include the -scheduler_custom_script option with the Fluent options in your script.
Also try -scheduler_tight_coupling, which also internally invokes -scheduler_custom_script
October 31, 2022 at 5:51 pmdheryadiSubscriberThank you for the response.I'm using the standard fluent option, like so:fluent 3ddp -g -i journal_file -t$NSLOTSwhere NSLOTS is the core count reserved in the job script. This option used to work in the older version of SGE (8.6.2), but not the later one that we're running (SGE 8.6.18). I'm currently testing Grid Engine 8.7.2 and the same Fluent issue is unfortunately encountered in this version. According to this table (https://www.ansys.com/content/dam/it-solutions/platform-support/2022-r2-job-schedulers-queuing-systems-support.pdf), Fluent supports SGE 8.6. This support is probably for the earlier release of 8.6, not the very last one (8.6.18).Please note that I was able to run the following Ansys 2022R2's mpitest command across multiple hosts:mpitest222 -mpi openmpi -scheduler=sge -np $NSLOTSUsing our own OpenMPI installation, I was also able to run another Ansys 2022R2 product (called CFX) across multiple hosts with the following command:cfx5solve -parallel -part $NSLOTS -par-local -start-method "Open MPI Local Parallel" . . .I've tried many different Fluent options, but to no avail.
October 31, 2022 at 8:41 pmHunter WangAnsys Employee
I was able to replicate the issue you reported. Fixed by adding -scheduler_tight_coupling or -scheduler_custom_script.
fluent 3ddp -g -i journal_file -t$NSLOTS
fluent 3ddp -g -i journal_file -t$NSLOTS -scheduler_tight_coupling
fluent 3ddp -g -i journal_file -t$NSLOTS -scheduler_custom_script
It should not relate to OS version of UGE version, but Fluent version. The 2 flags work for 2021 R2 and new releases.
November 2, 2022 at 5:04 pmdheryadiSubscriber
Thank you so much for the suggestions. The '-scheduler_tight_coupling' flag (with the default Intel MPI) appears to work on both versions of Grid Engine: 8.6.8 and 8.7.2
- You must be logged in to reply to this topic.
Simulation World 2022
Check out more than 70 different sessions now available on demand. Get inspired as you hear from visionary companies, leading researchers and educators from around the globe on a variety of topics from life-saving improvements in healthcare, to bold new realities of space travel. Take a leap of certainty and check out a session today here.
Earth Rescue – An Ansys Online Series
The climate crisis is here. But so is the human ingenuity to fight it. Earth Rescue reveals what visionary companies are doing today to engineer radical new ideas in the fight against climate change. Click here to watch the first episode.
Subscribe to the Ansys Blog to get great new content about the power of simulation delivered right to your email on a weekly basis. With content from Ansys experts, partners and customers you will learn about product development advances, thought leadership and trends and tips to better use Ansys tools. Sign up here.
- Suppress Fluent to open with GUI while performing in journal file
- Heat transfer coefficient
- What are the differences between CFX and Fluent?
- Floating point exception in Fluent
- The solver failed with a non-zero exit code of : 2
- Getting graph and tabular data from result in workbench mechanical
- Difference between K-epsilon and K-omega Turbulence Model
- Time Step Size and Courant Number
- Mesh Interfaces in ANSYS FLUENT
- error in cfd post
© 2022 Copyright ANSYS, Inc. All rights reserved.