Platform

Platform

Reading a simple .msh file last more than 30 min. when running in “Distributed Memory on a Cluster”

Tagged: , ,

    • iarauzo
      Subscriber
      Reading a simple .msh file last more than 30 minutes when running in "Distributed Memory on a Cluster" mode. If I choose the option "Shared Memory On Local Machine", it is read in a few minutes.
      
      I have copied the RHEL and fluent console during the instructions and I could upload them if necessary.
      
      The platform is an HPC Supermicro SYS-5028TK-HTR. It has 4 Intel Xeon Phi 7210 processors, each one with 64 cores, but we only use 2 Intel Xeon Phi 7210 processors with a maximum of 64 cores.
      
      Is such a long time normal? Can I do something to reduce this 30 min? The larger time-lapse is between these instructions in Fluent console:
      Multicore SMT processors detected. Processor affinity set!
      *****30 minutes waiting*****
      Reading "/home/user/Converging_Nozzle.msh"...
      Buffering for file scan...
      


    • George Karnos
      Ansys Employee
      Not sure if there is a solution to this as the Phi processor does not seem to be supported What happens when you run on another machine?
    • iarauzo
      Subscriber
      Thank you for your answer. I do not have another machine. Xeon Phi processor was supported when we bought ANSYS FLUENT, 4 years ago.
      Do you know where I can check supported processors, or until what version this processor is supported?
      Thank you in advance.
    • George Karnos
      Ansys Employee
      I am still looking into the Phi processor support.
      Yes, can you please post the RHEL and Fluent consoles. please post the text as we are not permitted to open files.
      Thank You.
    • iarauzo
      Subscriber
      RHEL CONSOLE
      /ansys_inc/v211/fluent/fluent21.1.0/cortex/lnamd64/cortex.21.1.0 -f fluent -license=enterprise -i /home/iarauzo/tutorial_fluent//_read_fluent_case_.jou (fluent "2d -pethernet-host -alnamd64 -r21.1.0 -t20 -mpi=intel -path/ansys_inc/v211/fluent -ssh")
      ^C
      [iarauzo@flutwin1 launcher]$ ^C
      [iarauzo@flutwin1 launcher]$ ^C
      [iarauzo@flutwin1 launcher]$ ./launcher.sh
      /ansys_inc/v211/fluent/fluent21.1.0/bin/fluent -r21.1.0 -cflush -x11
      Qt: Session management error: Could not open network socket
      /ansys_inc/v211/fluent/fluent21.1.0/bin/fluent -r21.1.0 2d -t20 -pethernet -mpi=intel -i /home/iarauzo/tutorial_fluent//_read_fluent_case_.jou -cnf=/home/iarauzo/.fluent.launcher.host -ssh -cflush -x11 -license=enterprise
      [iarauzo@flutwin1 launcher]$ Launching /ansys_inc/v211/fluent/fluent21.1.0/multiport/mpi_wrapper/bin/mpirun.fl --arch=lnamd64--ic=ethernet --ic_variant=default--mpi=intel--np=20--rsh=ssh--prefix=/ansys_inc/v211/fluent/fluent21.1.0/multiport --cflush --cnf=/home/iarauzo/.fluent.launcher.host/ansys_inc/v211/fluent/fluent21.1.0/multiport/mpi_wrapper/test/lnamd64/cflush node -mpiw intel -pic ethernet
      Starting /ansys_inc/v211/fluent/fluent21.1.0/multiport/mpi/lnamd64/intel/bin/mpirun -f /tmp/fluent-appfile.iarauzo.55078 --rsh=ssh -genv I_MPI_FABRICS shm:tcp -genv I_MPI_FALLBACK_DEVICE disable -genv FLUENT_ARCH lnamd64 -genv I_MPI_DEBUG 0 -genv I_MPI_ADJUST_REDUCE 2 -genv I_MPI_ADJUST_ALLREDUCE 2 -genv I_MPI_ADJUST_BCAST 1 -genv I_MPI_ADJUST_BARRIER 2 -genv I_MPI_ADJUST_ALLGATHER 2 -genv I_MPI_ADJUST_GATHER 2 -genv I_MPI_ADJUST_ALLTOALL 1 -genv I_MPI_ADJUST_SCATTER 2 -genv I_MPI_ADJUST_ALLGATHERV 2 -genv I_MPI_PLATFORM auto -genv PYTHONHOME /ansys_inc/v211/fluent/fluent21.1.0/../../commonfiles/CPython/3_7/linx64/Release/python -genv FLUENT_PROD_DIR /ansys_inc/v211/fluent/fluent21.1.0 -genv KMP_AFFINITY disabled -genv LD_PRELOAD /ansys_inc/v211/fluent/fluent21.1.0/multiport/mpi/lnamd64/intel/lib/libstrtok.so -genv TMI_CONFIG /ansys_inc/v211/fluent/fluent21.1.0/multiport/mpi/lnamd64/intel/etc/tmi.conf -machinefile /tmp/fluent-appfile.iarauzo.55078 -np 2 /ansys_inc/v211/fluent/fluent21.1.0/multiport/mpi_wrapper/test/lnamd64/cflush node -mpiw intel -pic ethernet

      Initiating flushing of file cache buffers...
      This process may take a few minutes depending on the total memory of the system.
      Current maximum cache ratio 0.885366% > 1%.
      1flutwin20.883873 /109.854GB 0.804587% 5045.32 0.00305176GB

      ----------------------------------------------------------------------------------
      idhostnamecached /total GBcached% fluent load/dev/shm GB
      ----------------------------------------------------------------------------------
      0flutwin10.972607 /109.854GB 0.885366% 7051.75 0.00366211GB
      Maximum cache ratio 0.885366% < 1%, no need to flush!
      /ansys_inc/v211/fluent/fluent21.1.0/cortex/lnamd64/cortex.21.1.0 -f fluent -license=enterprise -i /home/iarauzo/tutorial_fluent//_read_fluent_case_.jou (fluent "2d -pethernet-host -alnamd64 -r21.1.0 -t20 -mpi=intel -cnf=/home/iarauzo/.fluent.launcher.host -path/ansys_inc/v211/fluent -ssh")

    • iarauzo
      Subscriber
      FLUENT console
      Opening input/output transcript to file "/home/iarauzo/tutorial_fluent/fluent-20210610-233419-55307.trn".
      /ansys_inc/v211/fluent/fluent21.1.0/bin/fluent -r21.1.0 2d -pethernet -host -alnamd64 -t20 -mpi=intel -cnf=/home/iarauzo/.fluent.launcher.host -path/ansys_inc/v211/fluent -ssh -cx flutwin1:41787:42203
      Starting /ansys_inc/v211/fluent/fluent21.1.0/lnamd64/2d_host/fluent.21.1.0 host -cx flutwin1:41787:42203 "(list (rpsetvar (QUOTE parallel/function) "fluent 2d -flux -node -alnamd64 -r21.1.0 -t20 -pethernet -mpi=intel -cnf=/home/iarauzo/.fluent.launcher.host -ssh") (rpsetvar (QUOTE parallel/rhost) "") (rpsetvar (QUOTE parallel/ruser) "") (rpsetvar (QUOTE parallel/nprocs_string) "20") (rpsetvar (QUOTE parallel/auto-spawn?) #t) (rpsetvar (QUOTE parallel/trace-level) 0) (rpsetvar (QUOTE parallel/remote-shell) 1) (rpsetvar (QUOTE parallel/path) "/ansys_inc/v211/fluent") (rpsetvar (QUOTE parallel/hostsfile) "/home/iarauzo/.fluent.launcher.host") )"

      Welcome to ANSYS Fluent 2021 R1

      Copyright 1987-2021 ANSYS, Inc. All Rights Reserved.
      Unauthorized use, distribution or duplication is prohibited.
      This product is subject to U.S. laws governing export and re-export.
      For full Legal Notice, see documentation.

      Build Time: Nov 20 2020 18:46:04 ESTBuild Id: 10179


      --------------------------------------------------------------
      This is an academic version of ANSYS FLUENT. Usage of this product
      license is limited to the terms and conditions specified in your ANSYS
      license form, additional terms section.
      --------------------------------------------------------------
      Host spawning Node 0 on machine "flutwin1" (unix).
      /ansys_inc/v211/fluent/fluent21.1.0/bin/fluent -r21.1.0 2d -flux -node -alnamd64 -t20 -pethernet -mpi=intel -cnf=/home/iarauzo/.fluent.launcher.host -ssh -mport 155.210.150.19:155.210.150.19:40820:0
      Starting /ansys_inc/v211/fluent/fluent21.1.0/multiport/mpi/lnamd64/intel/bin/mpirun -f /tmp/fluent-appfile.iarauzo.57370 --rsh=ssh -genv I_MPI_FABRICS shm:tcp -genv I_MPI_FALLBACK_DEVICE disable -genv FLUENT_ARCH lnamd64 -genv I_MPI_DEBUG 0 -genv I_MPI_ADJUST_REDUCE 2 -genv I_MPI_ADJUST_ALLREDUCE 2 -genv I_MPI_ADJUST_BCAST 1 -genv I_MPI_ADJUST_BARRIER 2 -genv I_MPI_ADJUST_ALLGATHER 2 -genv I_MPI_ADJUST_GATHER 2 -genv I_MPI_ADJUST_ALLTOALL 1 -genv I_MPI_ADJUST_SCATTER 2 -genv I_MPI_ADJUST_ALLGATHERV 2 -genv I_MPI_PLATFORM auto -genv PYTHONHOME /ansys_inc/v211/fluent/fluent21.1.0/../../commonfiles/CPython/3_7/linx64/Release/python -genv FLUENT_PROD_DIR /ansys_inc/v211/fluent/fluent21.1.0 -genv KMP_AFFINITY disabled -genv LD_PRELOAD /ansys_inc/v211/fluent/fluent21.1.0/multiport/mpi/lnamd64/intel/lib/libstrtok.so -genv TMI_CONFIG /ansys_inc/v211/fluent/fluent21.1.0/multiport/mpi/lnamd64/intel/etc/tmi.conf -machinefile /tmp/fluent-appfile.iarauzo.57370 -np 20 /ansys_inc/v211/fluent/fluent21.1.0/lnamd64/2d_node/fluent_mpi.21.1.0 node -mpiw intel -pic ethernet -mport 155.210.150.19:155.210.150.19:40820:0

      -------------------------------------------------------------------------------
      IDHostnameCoreO.S.PIDVendor
      -------------------------------------------------------------------------------
      n10-19flutwin210/256Linux-64174857-174866Intel(R) Xeon Phi(TM) 7210
      n0-9flutwin110/256Linux-6457555-57564Intel(R) Xeon Phi(TM) 7210
      hostflutwin1Linux-6455686Intel(R) Xeon Phi(TM) 7210

      MPI Option Selected: intel
      Selected system interconnect: ethernet
      Multiple networks are configured on the system.
      Ensure optimal choice of network interface and FLUENT communicator!
      -------------------------------------------------------------------------------

      Cleanup script file is /home/iarauzo/tutorial_fluent/cleanup-fluent-flutwin1-55686.sh

      Reading journal file /home/iarauzo/tutorial_fluent//_read_fluent_case_.jou...

      > ;Journal file automatically created by Fluent!
      /file read-case /home/iarauzo/tutorial_fluent/Converging_Nozzle.msh

      Multicore SMT processors detected. Processor affinity set!
      **HERE IT IS WHEN THE COMPUTER IS 30 MINUTES WORKING. This comment is not in console, it is mine**
      Reading "/home/iarauzo/tutorial_fluent/Converging_Nozzle.msh"...

      Buffering for file scan...

      13805 nodes, binary.
      616 nodes, binary.
      27916 2D interior faces, zone1, binary.
      56 2D velocity-inlet faces, zone5, binary.
      252 2D wall faces, zone6, binary.
      56 2D pressure-outlet faces, zone7, binary.
      252 2D axis faces, zone8, binary.
      14112 quadrilateral cells, zone2, binary.

      Building...
      mesh
      auto partitioning mesh by Metis (fast) distributing mesh
      parts.................... faces.................... nodes.................... cells.................... inter-node communication reduction using architecture-aware remapping: 78%
      bandwidth reduction using Reverse Cuthill-McKee: 154/33 = 4.66667
      materials interface domains zones axis
      outlet
      wall
      inlet
      converging_nozzle
      interior-converging_nozzle
      parallel Done.

      Preparing mesh for display...
      Warning: The use of axis boundary conditions is not appropriate for
      a 2D/3D flow problem. Please consider changing the zone
      type to symmetry or wall, or the problem to axisymmetric.

      Warning: The use of axis boundary conditions is not appropriate for
      a 2D/3D flow problem. Please consider changing the zone
      type to symmetry or wall, or the problem to axisymmetric.

      Done.

      >
      adjoint/mesh/report/
      define/parallel/server/
      display/plot/solve/
      exitpreferences/surface/
      file/print-license-usageviews/

      >
      adjoint/mesh/report/
      define/parallel/server/
      display/plot/solve/
      exitpreferences/surface/
      file/print-license-usageviews/

      >
      >
    • iarauzo
      Subscriber
      Sorry, I thought I had posted consoles last week, but I did not realize that both consoles in a message were over the maximum length, and it was not recorded.
Viewing 6 reply threads
  • You must be logged in to reply to this topic.