Ansys Products

Ansys Products

How to get FLUENT through workbench to use the GPU?

    • tyw7
      Subscriber
      As subject. I am using Fluent through workbench on a HPC. How to get FLUENT to use the GPU?n
    • tsiriaks
      Ansys Employee
      What is the job scheduler on the cluster ?nWhat is the OS version ?nWhat is the graphics card on this node ?.ormally, you may have use stand-alone Fluent to achieve that but it also depends on the info that i asked above.n
    • tyw7
      Subscriber
      Redhat linux. It's a direct submission using qsub and mobaxterm.
    • tsiriaks
      Ansys Employee
      I forgot to ask about ANSYS/Fluent version.nCan you provide a sample command (and script, if exist) that you are using to submit the job ?nPlease also provide the output of the following commandsncat /etc/*releasenlspci -vnn | grep VGA -A 12nglxinfo | grep OpenGLnn
    • tyw7
      Subscriber
      Version 2020R2.nnI have attached the GPU / Fluent / Workbench scripts.nnThe GPU one is pretty generic and doesn't show how to submit Fluent / Workbench to use the GPUnn
    • tyw7
      Subscriber
      I have attached the output of the strings you askedn
    • tsiriaks
      Ansys Employee
      Sorry, ANSYS employee are not allowed to download file attachment.nCould you provide that in screenshots inline with text or perhaps copy the content and post as text here ?n
    • tyw7
      Subscriber
      [[username]@crescent-login1 ~]$ cat /etc/*releasenNAME=Red Hat Enterprise Linux ServernVERSION=7.7 (Maipo)nID=rhelnID_LIKE=fedoranVARIANT=ServernVARIANT_ID=servernVERSION_ID=7.7nPRETTY_NAME=Red Hat Enterprise Linux Server 7.7 (Maipo)nANSI_COLOR=0;31nCPE_NAME=cpe:/o:redhat:enterprise_linux:7.7:GA:servernHOME_URL=https://www.redhat.com/nBUG_REPORT_URL=https://bugzilla.redhat.com/nnREDHAT_BUGZILLA_PRODUCT=Red Hat Enterprise Linux 7nREDHAT_BUGZILLA_PRODUCT_VERSION=7.7nREDHAT_SUPPORT_PRODUCT=Red Hat Enterprise LinuxnREDHAT_SUPPORT_PRODUCT_VERSION=7.7nRed Hat Enterprise Linux Server release 7.7 (Maipo)nRed Hat Enterprise Linux Server release 7.7 (Maipo)n[[username]@crescent-login1 ~]$ lspci -vnn | grep VGA -A 12n08:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106GL [Quadro P2000] [10de:1c30] (rev a1) (prog-if 00 [VGA controller])n Subsystem: NVIDIA Corporation Device [10de:11b3]n Physical Slot: 2-2n Flags: bus master, fast devsel, latency 0, IRQ 206, NUMA node 0n Memory at d1000000 (32-bit, non-prefetchable) [size=16M]n Memory at e0000000 (64-bit, prefetchable) [size=256M]n Memory at f0000000 (64-bit, prefetchable) [size=32M]n I/O ports at 1000 [size=128]n [virtual] Expansion ROM at d2000000 [disabled] [size=512K]n Capabilities: n Kernel driver in use: nvidian Kernel modules: nouveau, nvidiann--n0a:00.0 VGA compatible controller [0300]: Matrox Electronics Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1) [102b:0522] (rev 05) (prog-if 00 [VGA controller])n Subsystem: Intel Corporation Device [8086:0103]n Flags: bus master, fast devsel, latency 0, IRQ 19, NUMA node 0n Memory at f3000000 (32-bit, prefetchable) [size=16M]n Memory at d3010000 (32-bit, non-prefetchable) [size=16K]n Memory at d2800000 (32-bit, non-prefetchable) [size=8M]n Expansion ROM at d3000000 [disabled] [size=64K]n Capabilities: n Kernel driver in use: mgag200n Kernel modules: mgag200nn7f:08.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 QPI Link 0 [8086:3c80] (rev 07)n Subsystem: Intel Corporation Device [8086:3582]n[[username]@crescent-login1 ~]$ glxinfo | grep OpenGLnOpenGL vendor string: Mesa ProjectnOpenGL renderer string: Software RasterizernOpenGL version string: 1.4 (2.1 Mesa 10.5.4)nOpenGL extensions:nn
    • tyw7
      Subscriber
      #!/bin/bashn##n## ANSYS submission script for PBS on CRESCENTn## -------------------------------------------n##n## Follow the 6 steps below to configure your jobn## n## STEP 1:n##n## Enter a job name after the -N on the line below:n##n#PBS -N fluent_examplen##n## STEP 2:n##n## Select the number of cpus/cores required by modifying the #PBS -l select line belown##n## Normally you select cpus in chunks of 16 cpusn## The Maximum value for ncpus is 16 and mpiprocs MUST be the same value as ncpus.n##n## If more than 16 cpus are required then select multiple chunks of 16n## e.g.t16 CPUs: select=1:ncpus=16:mpiprocs=16n##t32 CPUs: select=2:ncpus=16:mpiprocs=16n##t48 CPUs: select=3:ncpus=16:mpiprocs=16n##t..etc..n##n#PBS -l select=1:ncpus=16:mpiprocs=16n##n## STEP 3:n##n## Select the correct queue by modifying the #PBS -q line belown##n## half_hourt- 30 minutesn## one_hourt-  1 hourn## three_hour  -  3 hoursn## six_hour   -  6 hoursn## half_dayt- 12 hoursn## one_dayt- 24 hoursn## two_dayt- 48 hoursn## five_dayt- 120 hoursn## ten_dayt- 240 hours (by special arrangement)n##n#PBS -q half_hourn##n## STEP 4:n##n## Replace the hpc@cranfield.ac.uk email addressn## with your Cranfield email address on the #PBS -M line below:n## Your email address is NOT your usernamen##n#PBS -m abe n#PBS -M hpc@cranfield.ac.ukn##n## ====================================n## DO NOT CHANGE THE LINES BETWEEN HEREn## ====================================n#PBS -l application=ansysn#PBS -j oen#PBS -W sandbox=PRIVATEn#PBS -kln -s $PWD $PBS_O_WORKDIR/$PBS_JOBIDn## Change to working directoryncd $PBS_O_WORKDIRn# Get node list from PBS and format for ANSYS:nANSYS_NODES= uniq -c ${PBS_NODEFILE} | awk -F. '{ print $1 }' | awk '{print $2 : $1}' | paste -s -d ":" n## load module required by ANSYS/16.2nmodule load libGLU/9.0.0-foss-2016bn## ========n## AND HEREn## ========n##n## STEP 5: n## n## Load the default application environmentn## For a specific version add the version number, e.g.n## module load ANSYS/17.1n##nmodule load ANSYS/16.2n##n## STEP 6: n## n## Put the correct parameters in the ansys execution linen## below to run the ANSYS launchern## Unless you are sure do not change the default parametersn##n## The main parameters to modify are -n## tapplication name - must match the version loadedn## tinput filename - change input.jou to your own filenamennansys162 -dis -machines $ANSYS_NODES -dir $PBS_O_WORKDIR -b -i input.inpnn## Tidy up the log directoryn## DO NOT CHANGE THE LINE BELOWn## ============================nrm $PBS_O_WORKDIR/$PBS_JOBIDn#nn
    • tyw7
      Subscriber
      #!/bin/bashn##n## GPU submission script for PBS on CRESCENTn## -----------------------------------------n##n## Follow the 6 steps below to configure your jobn## n## STEP 1:n##n## Enter a job name after the -N on the line below:n##n#PBS -N gpu_examplen##n## STEP 2:n##n## Select the number of cpus/cores and GPUs required by modifying the #PBS -l select line belown##n## The Maximum value for ncpus is 8 and mpiprocs MUST be the same value as ncpus.n## The Maximum value for ngpus is 1 n## e.g.t 1 GPU and 8 CPUs :select=1:ncpus=8:mpiprocs=8;ngpus=1n##n#PBS -l select=1:ncpus=8:mpiprocs=8:ngpus=1n##n## STEP 3:n##n## There is only one queue for GPU jobs as defined in the #PBS -q line belown##n#PBS -q gpun##n## The default walltime in the gpu queue is one day(24 hours)n## The maximum walltime in the gpu queue is five days(120 hours)n## In order to increase the walltime modify the #PBS -l walltime line belown## and remove one of the leading # characters n##n##PBS -l walltime=24:00:00n##n## STEP 4:n##n## Replace the hpc@cranfield.ac.uk email addressn## with your Cranfield email address on the #PBS -M line below:n## Your email address is NOT your usernamen##n#PBS -m abe n#PBS -M hpc@cranfield.ac.ukn##n## ====================================n## DO NOT CHANGE THE LINES BETWEEN HEREn## ====================================n#PBS -j oen#PBS -v CUDA_VISIBLE_DEVICES=n#PBS -W sandbox=PRIVATEn#PBS -kln -s $PWD $PBS_O_WORKDIR/$PBS_JOBIDn## Allocated gpu(s)necho CUDA_VISIBLE_DEVICES=$CUDA_VISIBLE_DEVICESn## Change to working directoryncd $PBS_O_WORKDIRn## Calculate number of CPUs and GPUsnexport cpus= cat $PBS_NODEFILE | wc -l nexport gpus= echo $CUDA_VISIBLE_DEVICES|awk -F, '{print NF}' n## ========n## AND HEREn## ========n##n## STEP 5: n## n## Load the default application environmentn## For a specific version add the version number, e.g.n## module load CUDA/8.0.44n##nmodule load CUDAn##n## STEP 6: n## n## Run gpu application n##n## Put correct parameters and cuda application in the line below:n##n/apps/software/CUDA/9.0.176/extras/demo_suite/deviceQuerynn## Tidy up the log directoryn## DO NOT CHANGE THE LINE BELOWn## ============================nrm $PBS_O_WORKDIR/$PBS_JOBIDn#nn
    • tyw7
      Subscriber
      #!/bin/bashn##n## Workbench submission script for PBS on CRESCENTn## -----------------------------------------------n##n## Follow the 6 steps below to configure your jobn## n## STEP 1:n##n## Enter a job name after the -N on the line below:n##n#PBS -N workbench-examplen##n## STEP 2:n##n## Select the number of cpus/cores required by modifying the #PBS -l select line belown##n## Normally you select cpus in chunks of 16 cpusn## The Maximum value for ncpus is 16 and mpiprocs MUST be the same value as ncpus.n##n## If more than 16 cpus are required then select multiple chunks of 16n## e.g.t16 CPUs: select=1:ncpus=16:mpiprocs=16n##t32 CPUs: select=2:ncpus=16:mpiprocs=16n##t48 CPUs: select=3:ncpus=16:mpiprocs=16n##t..etc..n##n#PBS -l select=1:ncpus=16:mpiprocs=16n##n## STEP 3:n##n## Select the correct queue by modifying the #PBS -q line belown##n## half_hourt- 30 minutesn## one_hourt-  1 hourn## half_dayt- 12 hoursn## three_hour  -  3 hoursn## six_hour   -  6 hoursn## one_dayt- 24 hoursn## two_dayt- 48 hoursn## five_dayt- 120 hoursn## ten_dayt- 240 hours (by special arrangement)n##n#PBS -q half_hourn##n## STEP 4:n##n## Replace the hpc@cranfield.ac.uk email addressn## with your Cranfield email address on the #PBS -M line below:n## Your email address is NOT your usernamen##n#PBS -m abe n#PBS -M hpc@cranfield.ac.ukn##n## ====================================n## DO NOT CHANGE THE LINES BETWEEN HEREn## ====================================n#PBS -l application=ansysn#PBS -j oen#PBS -W sandbox=PRIVATEn#PBS -kln -s $PWD $PBS_O_WORKDIR/$PBS_JOBIDn## Change to working directoryncd $PBS_O_WORKDIRn## Calculate number of CPUsnexport cpus= cat $PBS_NODEFILE | wc -l n# Get node list from PBS and format for ANSYSnANSYS_NODES= uniq -c ${PBS_NODEFILE} | awk -F. '{ print $1 }' | awk '{print $2 : $1}' | paste -s -d ":" n## load module required by ANSYSnmodule load libXt libxcbn## Set dummy Displaynexport DISPLAY=hpcxserv-1.central.cranfield.ac.uk:0.0n## ========n## AND HEREn## ========n##n## STEP 5: n## n## Load the default application environmentn## For a specific version add the version number, e.g.n## module load ANSYS/17.1n##nmodule load ANSYSn##n## STEP 6: n## n## Put correct parameters in workbench execution linen## below (i.e. replace wbsimulation.wbjn with your journal filename):n## see wbsimulation.wbjn in /apps/example/pbs for format of workbench journal filen##nnrunwb2 -b -r wbsimulation.wbjnnn## Tidy up the log directoryn## DO NOT CHANGE THE LINE BELOWn## ============================nrm $PBS_O_WORKDIR/$PBS_JOBIDn#nn
    • Mangesh Bhide
      Ansys Employee
      HellonIf all you need to do is run solvers then explore configuring up ANSYS RSM to PBS or running Fluent under a scheduler per the documentation.nThe graphics card model may not work for GPU acceleration and may not offer much benefit.nRunning Workbench in batch may not be necessary unless other things are also being done.nn
Viewing 11 reply threads
  • You must be logged in to reply to this topic.