Error in FSI - runs fine in PC but not on HPC

ansysuseransysuser USA Member Posts: 44

Hello,

 

I am trying to get a coupled system to run.  I found that the system runs to completion on my PC, but I get an error when I try to run it on HPC.

 *** ERROR ***                           CP =       1.126   TIME= 18:040
 The FSIN interface names of the mechanical model are different from the
 participant region names of the system coupling. 

 

Progress = 0; Message = "System coupling run completed with errors.  Fluid Flow (Fluent) (Solution) reported: The FSIN interface names of the mechanical model are different from the participant region names of the system coupling."

 

 

 

«1

Comments

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

    Anyone have any insight on this?

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

      +====================================================================+
     |                    Data Transfer Information (2)                   |
     +====================================================================+

     +--------------------------------------------------------------------+
     | Data Transfer: Data Transfer                                       |
     +--------------------------------------------------------------------+

     Source : Fluid Flow (Fluent)                                         
       Region                = fsi                                        
       Variable              = force                                      

     Target : Transient Structural                                        
       Region                = Fluid Solid Interface                      
       Variable              = Force                                      

     General Information :                                                
       Name                  = Data Transfer                              
       Execute Transfer At   = Start Of Iteration                         
       Convergence Option    = RMS Change In Data                         
         Target Value        = 0.01                                       
       Under Relax. Factor   = 1                                          
       Ramping               = None                                       

     +--------------------------------------------------------------------+
     | Data Transfer: Data Transfer 2                                     |
     +--------------------------------------------------------------------+

     Source : Transient Structural                                        
       Region                = Fluid Solid Interface                      
       Variable              = Incremental Displacement                   

     Target : Fluid Flow (Fluent)                                         
       Region                = fsi                                        
       Variable              = displacement                               

     General Information :                                                
       Name                  = Data Transfer 2                            
       Execute Transfer At   = Start Of Iteration                         
       Convergence Option    = RMS Change In Data                         
         Target Value        = 0.01                                       
       Under Relax. Factor   = 1                                          
       Ramping               = None     

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

      +====================================================================+
     |                    Data Transfer Information (2)                   |
     +====================================================================+

     +--------------------------------------------------------------------+
     | Data Transfer: Data Transfer                                       |
     +--------------------------------------------------------------------+

     Source : Fluid Flow (Fluent)                                         
       Region                = fsi                                        
       Variable              = force                                      

     Target : Transient Structural                                        
       Region                = Fluid Solid Interface                      
       Variable              = Force                                      

     General Information :                                                
       Name                  = Data Transfer                              
       Execute Transfer At   = Start Of Iteration                         
       Convergence Option    = RMS Change In Data                         
         Target Value        = 0.01                                       
       Under Relax. Factor   = 1                                          
       Ramping               = None                                       

     +--------------------------------------------------------------------+
     | Data Transfer: Data Transfer 2                                     |
     +--------------------------------------------------------------------+

     Source : Transient Structural                                        
       Region                = Fluid Solid Interface                      
       Variable              = Incremental Displacement                   

     Target : Fluid Flow (Fluent)                                         
       Region                = fsi                                        
       Variable              = displacement                               

     General Information :                                                
       Name                  = Data Transfer 2                            
       Execute Transfer At   = Start Of Iteration                         
       Convergence Option    = RMS Change In Data                         
         Target Value        = 0.01                                       
       Under Relax. Factor   = 1                                          
       Ramping               = None     

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

    Here is the system coupling window.  The solution ran just fine on my PC.

     

     

  • KeatonKeaton Posts: 148Forum Coordinator
    edited May 2019

    1) how are you transferring the project from PC to HPC, and 2) is your HPC linux-based?

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

    Hello, 

    Since posting several weeks ago I have made some progress.  I will updated the situation below:

     

    I have Mechanical export a .dat file. 

    I have Fluent export .cas.gz and .dat.gz

    I have the Coupler export .sci

    Then I put them all on the same Linux directory and call them with the slurm commands:

    module load ansys
    echo "Made to system coupling"
    /apps/r/ansys/v193/aisol/.workbench -cmd ansys.services.systemcoupling.exe -inputFile ${COUPLING} &
    echo "Made it past system coupling"
    fluent 3ddp -g -t${NPF} -driver null -ssh -scport=${HOSTPORT[0]} -schost=${HOSTPORT[1]} -scname="${JOB[4]}" < ${JOURNAL} > output.FLUENT &
    ansys193 -b -scport=${HOSTPORT[0]} -schost=${HOSTPORT[1]} -scname="${JOB[2]}" -i ${STRUCTURALDATA} -o output.ANSYS -np ${NPA}

    And in the slurm output file I see this:

    Made to system coupling
    Made it past system coupling
    [0] MPI startup(): Single-threaded optimized library
    ... lots of lines of MPI startup...
    Cleanup script file is /pfs/tsfs1/gscratch/....
    forrtl: No such file or directory
    forrtl: No such file or directory
    forrtl: No such file or directory
    forrtl: No such file or directory
    forrtl: No such file or directory
    forrtl: No such file or directory
    forrtl: severe (28): CLOSE error, unit 24, file "Unknown"

    In output.FLUENT I see it loads the case and data, builds mesh, distributed mesh, parts, faces, etc, then gets to Done. Then the next thing I see in there is:

    > Deleting Client...done
    Shutting down Server...done
    No error handler available
    System Coupling Service encountered fatal error.#f

    In the output.ANSYS file I see:

     *** ERROR ***
     Another ANSYS job with the same job name (file) is already running in this
     directory or the file.lock file has not been deleted from an abnormally
     terminated ANSYS run.  To disable this check, set the ANSYS_LOCK environment
     variable to OFF.

    Now I know the file.lock is deleted between each try of this, so that is not the problem.  Also, this is the only job running when I try to run it.  No other ANSYS jobs at all on the whole cluster.

     

    ANSYS wrote out a whole bunch of other files, so let me know if you need to know what is in those as well.

     

    Thanks

     

     

     

     

     

  • SteveSteve Posts: 153Forum Coordinator
    edited May 2019

    Hi there,

    It looks like you're using the old command line method to start this case. Please review the following tutorial that goes over how to run System Coupling from the command line in 2019R1.

    https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v193/sysc_tut/sysc_tut_oscplate_commandline.html

    You can put all the System Coupling commands into a text file, (run.py for example, but the name isn't important). Then in your slurm commands you would have ...systemcoupling -R run.py which would start all participants.

    Steve

     

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

     Thank you, I will give that a try if I can and let you know how it goes.

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

    Hello,

    I have followed the above as far as setting up the problem and getting the files onto HPC. Then I use this command in the batch file:

    /apps/r/ansys/v193/SystemCoupling/bin/systemcoupling -R run.py > outsyscoup.txt

    The only thing that is in the outsyscoup.txt is:

    Executing from: /apps/r/ansys/v193/SystemCoupling/bin/systemcoupling
    ... list of Point Releases and Patches....
    executing script 'run.py'

    +----------------------------------------------------------------------------+
    | The most recent execution of the LoadParticipants command reset the        |
    | datamodel.  All content has been cleared.                                  |
    +----------------------------------------------------------------------------+

    The contents of run.py are as follows (I was hoping to get a printout of the names as in the documentation so I could then do the next step, i.e., use GetRegionNamesForParticipant and then modify the timestep).  But these commands did not print out the ChildNames, as you can see from the output above.

    LoadParticipants(InputFiles = ['Mech_sys_couple_files.scp', 'Fluent_coupler.scp'])
    DatamodelRoot().CouplingParticipant.GetChildNames()

     

    What has gone wrong?

     

    Thanks

     

     

     

  • SteveSteve Posts: 153Forum Coordinator
    edited May 2019

    - Is the current directory of your linux terminal the same as the location of the .scp files?

    - Are the .scp files located in the same directory? You could also have them in separate directories with: LoadParticipants(InputFiles = ['./Structural/Mech_sys_couple_files.scp', './Fluid/Fluent_coupler.scp'])

    - If you open both .scp files in a text editor, is the <WorkingDirectory> set correctly?

    You could also use the workflow with setup in Workbench and execution in the command line. The workflow is a bit easier. I meant to send this link last time. https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v193/sysc_tut/sysc_tut_oscplate_sci-commandline.html.

    One other note, ANSYS doesn't officially support slurm.

    Steve

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

    Hello Steve,

     

    Thanks for your follow up.

    For more background: I am working remotely on HPC, so I don't exactly have a "terminal" that I can use real time.  I have to use a scheduler and the cluster uses SLURM through bash files.  All commands to run a job have to be in the bash file. I have been using ANSYS on SLURM for over 5 years on different clusters, but only with Fluent, and I have never had a problem until using the system coupling.  

    Now, all files are in the same directory, including the bash file that initiates the job, and the ANSYS files.  I made sure that the <WorkingDirectory> in the scp files is set to the local directory on the cluster, so this is not the problem.  Here is a screenshot of the directory, showing all files inside it.

     

    I can try the other method you suggest and let you know if that works.

     

    Thanks again.

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

    Hello,

     

    I tried to make it through that tutorial you linked to above, using the oscillating plate even, but there is a part where there seems to be something missing in the tutorial instructions.  On this page (which is the very first link of the page you gave)

     

    https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v193/sysc_tut/sysc_tut_oscplate_sci-commandline_steps.html

     

    Look under 1. where it says, "When setting up project systems, prepare input files for each system, (Structural, Fluid, and System Coupling) before closing the associated application"


    I went through the instructions under Structural and Fluid through those links, but when I click on the System Coupling link the instructions say, "On the Project Schematic, ensure that the System Coupling system's Setup cell (C2) is in an Up-to-Date (image&nbsp state."  My System Coupling has a question mark.  The tutorial has not covered how to set up System Coupling.  It skipped that step.

    Help!

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

    Hello,

    I was able to get to the system coupler setup by gong to the other track of the tutorial (setting up and solving in Workbench, as opposed to setting up in Workbench and solving at command line).  So when I ran the case this way, I get:

     

    Starting Command Console...
    Type ^Z<Enter> to exit
    >>> Exiting Command Console...
    /var/spool/slurmd/job2752553/slurm_script: line 12: syntax error near unexpected token `FilePath'

    I am not sure why the error about FilePath is there, because this is the exact command simply copied and pasted from the tutorial linked above.

    I appreciate any further input, as I am trying to evaluate the performance of HPC for coupled analysis for our group. We want to know if it is worthwhile to purchase several HPC licenses if the runs are significantly faster on HPC for these types of simulations than on our high end desktops.  I have been trying to get a simple coupled simulation to run on HPC for over a month now. Fluent runs just fine, and so I have been able to compare performance, but this coupled simulation seems to be much more trouble!

     

    Thanks

     

     

  • KeatonKeaton Posts: 148Forum Coordinator
    edited May 2019

    could be a result of copy-pasting  Are you operating with text files with the script and systems coupling commands on windows and then using them on linux, or are you creating those files directly on linux? I am asking because there is a difference between end of line character on DOS and Unix which may lead to an error when using Windows-created scripts on linux. The fix is to save text files in Unix format, which many text editors allow, or to use dos2unix command to convert the file on linux.

    If this doesn't help, we'll need to see your slurm and systems coupling scripts 

     

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

    Hello kkoubat,

    I have gotten further.  I am facing a new error, which is detailed below.  Recall that I am following Steve's advice for the setup under the tutorial "Setup with Workbench and run from Command line" for the oscillating plate model.

    Here is how I call ANSYS in the bash file:

    /apps/r/ansys/v193/SystemCoupling/bin/systemcoupling < inputfile.in

    And below is the contents of the file inputfile.in.  Note that this is taken directly from the tutorial.  I am aware of the eol difference across OS, so I made sure to paste into linux and then backspace and return over each line.  I don't think that is the issue.

    ImportSystemCouplingInputFile(FilePath = 'ScInput.sci')
    execCon = DatamodelRoot().CouplingParticipant
    execCon['Solution'].ExecutionControl.InitialInput = 'mapdl.dat'
    execCon['Solution'].ExecutionControl.WorkingDirectory = 'Structural_Run'
    execCon['Solution 1'].ExecutionControl.InitialInput = 'fluidFlow.cas'
    execCon['Solution 1'].ExecutionControl.WorkingDirectory = 'Fluid_Run'
    execCon['Solution'].ExecutionControl.PrintState()
    execCon['Solution 1'].ExecutionControl.PrintState()
    Solve()

    Now below is the content of the output from SLURM.  Note that it appears to be seeing the input files for both Fluent and Mechanical, and that system coupler seems to be starting.

    Starting Command Console...
    Type ^Z<Enter> to exit
               
    >>> >>> >>> >>> >>> >>> >>> CouplingParticipant:Solution
      ExecutionControl
        WorkingDirectory = Structural_Run
        ParallelArguments =
        InitialInput = mapdl.dat
        Option = ProgramControlled
        AdditionalArguments =

    >>> CouplingParticipant:Solution 1
      ExecutionControl
        WorkingDirectory = Fluid_Run
        ParallelArguments =
        InitialInput = fluidFlow.cas
        Option = ProgramControlled
        AdditionalArguments =

    >>> Host spawning Node 0 on machine "t416.cluster" (unix).
    /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/bin/fluent -r18.0.0 3ddp -mpi=ibmmpi -p -node -t1 -nodehomedir=False -flux -wait -nosge -mport 10.98.4.168:10.98.4.1687343:0
    ls: cannot access /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/cortex/lnamd64/cortex.*.*.*: No such file or directory
    Starting /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/multiport/mpi/lnamd64/ibmmpi/bin/mpirun -e MPI_IBV_NO_FORK_SAFE=1 -e MPI_USE_MALLOPT_MMAP_MAX=0 -np 1 /pfs/tsfs1/apps/el7-x86_64/r/ansys/v193/commonfiles/CPython/2_7_15/linx64/Release/python/bin/python2 /apps/r/ansys/v193/SystemCoupling/PyLib/kernel/Engine/ComputeNode.py /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/lnamd64/3ddp_node/fluent_mpi.18.0.0 node -mpiw ibmmpi -pic shmem -mport 10.98.4.168:10.98.4.1687343:0 -nodehomedir=False -wait -nosge

    -------------------------------------------------------------------------------
    ID          Hostname      Core  O.S.      PID   Vendor                     
    -------------------------------------------------------------------------------
    e0          t416.cluster  1/32  Linux-64  3320  Intel(R) Xeon(R) E5-2683 v4
    controller  t416.cluster        Linux-64  3002  Intel(R) Xeon(R) E5-2683 v4

    MPI Option Selected: ibmmpi
    -------------------------------------------------------------------------------


    ==============================================================================
    +============================================================================+
    |                                                                            |
    |                         Summary of Coupling Setup                          |
    |                                                                            |
    +============================================================================+
    ==============================================================================
               

    +============================================================================+
    |                              Coupling Control                              |
    +============================================================================+
    |                                                                            |
    | Analysis Type :                                                  Transient |
    | Duration Control                                                           |
    |    Option :                                                        EndTime |
    |    End Time :                                                     10.0 [s] |
    | Step Control                                                               |
    |    Maximum Number Of Iterations Per Step :                               5 |
    |    Minimum Number Of Iterations Per Step :                               1 |
    |    Time Step Size :                                                0.1 [s] |
    |                                                                            |
    +============================================================================+
    |                               Output Control                               |
    +============================================================================+
    |                                                                            |
    | Output Control Option :                                       StepInterval |
    | Output Frequency :                                                       2 |
    |                                                                            |
    +============================================================================+
    |                        Participants Information (2)                        |
    +============================================================================+
    |                                                                            |
    | Participant: Structural                                                    |
    |    Type :                                                            MAPDL |
    |    Update Controls:                                                        |
    |       Option :                                           ProgramControlled |
    |    Region: Fluid Solid Interface                                           |
    |       Topology :                                                   Surface |
    |       Input Variables :                                              Force |
    |       Output Variables :                          Incremental Displacement |
    |                                                                            |
    | Participant: Fluid                                                         |
    |    Type :                                                           FLUENT |
    |    Update Controls:                                                        |
    |       Option :                                           ProgramControlled |
    |    Region: wall_deforming                                                  |
    |       Topology :                                                   Surface |
    |       Input Variables :                                       displacement |
    |       Output Variables :                                             force |
    |                                                                            |
    +============================================================================+
    |                         Interfaces Information (1)                         |
    +============================================================================+
    |                                                                            |
    | Interface: interface-1                                                     |
    |    Sidene                                                                |
    |       Coupling Participant :                                    Structural |
    |       Region List :                                  Fluid Solid Interface |
    |    Side:Two                                                                |
    |       Coupling Participant :                                         Fluid |
    |       Region List :                                         wall_deforming |
    |    Transfer To Side One:Force                                              |
    |       Side One Variable :                                            Force |
    |       Side Two Variable :                                            force |
    |       Convergence Target :                                        1.00E-02 |
    |       Ramping Option :                                                None |
    |       Relaxation Factor :                                         1.00E+00 |
    |       Mapping Type :                                  Surface Conservative |
    |    Transfer To Side Two:Incremental Displacement                           |
    |       Side One Variable :                         Incremental Displacement |
    |       Side Two Variable :                                     displacement |
    |       Convergence Target :                                        1.00E-02 |
    |       Ramping Option :                                                None |
    |       Relaxation Factor :                                         1.00E+00 |
    |       Mapping Type :                            Surface Profile Preserving |
    |                                                                            |
    +============================================================================+
    +============================================================================+
    |                     Participants Execution Information                     |
    +============================================================================+
    |                                                                            |
    | Structural                                                                 |
    |   Execution Command:                                                       |
    |     "/apps/r/ansys/v193/ansys/bin/ansys193" -b nolist -s noread -o Solutio |
    |     n.out -scport 42633 -schost t416.cluster -scname "Solution" -scid="332 |
    |     0_t416.cluster" -sclic=2325@tlic1 -i mapdl.dat                         |
    |   Working Directory:                                                       |
    |     REDACTED                     |
    |                                                                            |
    | Fluid                                                                      |
    |   Execution Command:                                                       |
    |     "/apps/r/ansys/v193/fluent/bin/fluent" 3ddp -g -scport=42633 -schost=t |
    |     416.cluster -scname="Solution 1" -scid="3320_t416.cluster" -sclic="232 |
    |     5@tlic1" -i Solution1.jou                                              |
    |   Working Directory:                                                       |
    |     REDACTED                            |
    +============================================================================+
    Awaiting connections from coupling participants...Fatal error in PMPI_Init_thread: Other MPI error, error stack:
    MPIR_Init_thread(805)....: fail failed
    MPID_Init(1832)..........: channel initialization failed
    MPIDI_CH3_Init(126)......: fail failed
    MPID_nem_init_ckpt(1087).: fail failed
    MPIDI_PG_SetConnInfo(711): PMI_KVS_Put returned 14
    In: PMI_Abort(69253391, Fatal error in PMPI_Init_thread: Other MPI error, error stack:
    MPIR_Init_thread(805)....: fail failed
    MPID_Init(1832)..........: channel initialization failed
    MPIDI_CH3_Init(126)......: fail failed
    MPID_nem_init_ckpt(1087).: fail failed
    MPIDI_PG_SetConnInfo(711): PMI_KVS_Put returned 14)
    srun: Job step aborted: Waiting up to 32 seconds for job step to finish.

  • KeatonKeaton Posts: 148Forum Coordinator
    edited May 2019

    hmm, looks to be mechanical end throwing MPI errors. This is ibmmpi. Let me bring our installation folks into this discussion

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

    Thank you so much, kkourbat.

  • SteveSteve Posts: 153Forum Coordinator
    edited May 2019

    Hi,

    As a sanity check, can you try: systemcoupling -R inputfile.in, instead of: /apps/r/ansys/v193/SystemCoupling/bin/systemcoupling < inputfile.in. Please post the output for this. I have a few other things to check if this doesn't work.

    Steve

  • ansysuseransysuser USAPosts: 141Member
    edited May 2019

    Hello Steve,

    It looks the same to me.  I don't know if this helps or not, but I noticed when I look in my job queue that the node is still running this job even after no more output is printed to the output file.  It runs until the time requested is up.

    executing script 'inputfile.in'
    CouplingParticipant:Solution
      ExecutionControl
        WorkingDirectory = Structural_Run
        ParallelArguments =
        InitialInput = mapdl.dat
        Option = ProgramControlled
        AdditionalArguments =

    CouplingParticipant:Solution 1
      ExecutionControl
        WorkingDirectory = Fluid_Run
        ParallelArguments =
        InitialInput = fluidFlow.cas
        Option = ProgramControlled
        AdditionalArguments =

    Host spawning Node 0 on machine "t416.cluster" (unix).
    /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/bin/fluent -r18.0.0 3ddp -mpi=ibmmpi -p -node -t1 -nodehomedir=False -flux -wait -nosge -mport 10.98.4.168:10.98.4.168:41642:0
    ls: cannot access /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/cortex/lnamd64/cortex.*.*.*: No such file or directory
    Starting /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/multiport/mpi/lnamd64/ibmmpi/bin/mpirun -e MPI_IBV_NO_FORK_SAFE=1 -e MPI_USE_MALLOPT_MMAP_MAX=0 -np 1 /pfs/tsfs1/apps/el7-x86_64/r/ansys/v193/commonfiles/CPython/2_7_15/linx64/Release/python/bin/python2 /apps/r/ansys/v193/SystemCoupling/PyLib/kernel/Engine/ComputeNode.py /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/lnamd64/3ddp_node/fluent_mpi.18.0.0 node -mpiw ibmmpi -pic shmem -mport 10.98.4.168:10.98.4.168:41642:0 -nodehomedir=False -wait -nosge

    -------------------------------------------------------------------------------
    ID          Hostname      Core  O.S.      PID   Vendor                     
    -------------------------------------------------------------------------------
    e0          t416.cluster  1/32  Linux-64  5532  Intel(R) Xeon(R) E5-2683 v4
    controller  t416.cluster        Linux-64  5206  Intel(R) Xeon(R) E5-2683 v4

    MPI Option Selected: ibmmpi
    -------------------------------------------------------------------------------

    Cleanup script file is /pfs/tsfs1/gscratch/mfig/SysC_Run/cleanup-afd-t416.cluster-5206.sh


    ==============================================================================
    +============================================================================+
    |                                                                            |
    |                         Summary of Coupling Setup                          |
    |                                                                            |
    +============================================================================+
    ==============================================================================

    +============================================================================+
    |                              Coupling Control                              |
    +============================================================================+
    |                                                                            |
    | Analysis Type :                                                  Transient |
    | Duration Control                                                           |
    |    Option :                                                        EndTime |
    |    End Time :                                                     10.0 [s] |
    | Step Control                                                               |
    |    Maximum Number Of Iterations Per Step :                               5 |
    |    Minimum Number Of Iterations Per Step :                               1 |
    |    Time Step Size :                                                0.1 [s] |
    |                                                                            |
    +============================================================================+
    |                               Output Control                               |
    +============================================================================+
    |                                                                            |
    | Output Control Option :                                       StepInterval |
    | Output Frequency :                                                       2 |
    |                                                                            |
    +============================================================================+
    |                        Participants Information (2)                        |
    +============================================================================+
    |                                                                            |
    | Participant: Structural                                                    |
    |    Type :                                                            MAPDL |
    |    Update Controls:                                                        |
    |       Option :                                           ProgramControlled |
    |    Region: Fluid Solid Interface                                           |
    |       Topology :                                                   Surface |
    |       Input Variables :                                              Force |
    |       Output Variables :                          Incremental Displacement |
    |                                                                            |
    | Participant: Fluid                                                         |
    |    Type :                                                           FLUENT |
    |    Update Controls:                                                        |
    |       Option :                                           ProgramControlled |
    |    Region: wall_deforming                                                  |
    |       Topology :                                                   Surface |
    |       Input Variables :                                       displacement |
    |       Output Variables :                                             force |
    |                                                                            |
    +============================================================================+
    |                         Interfaces Information (1)                         |
    +============================================================================+
    |                                                                            |
    | Interface: interface-1                                                     |
    |    Sidene                                                                |
    |       Coupling Participant :                                    Structural |
    |       Region List :                                  Fluid Solid Interface |
    |    Side:Two                                                                |
    |       Coupling Participant :                                         Fluid |
    |       Region List :                                         wall_deforming |
    |    Transfer To Side One:Force                                              |
    |       Side One Variable :                                            Force |
    |       Side Two Variable :                                            force |
    |       Convergence Target :                                        1.00E-02 |
    |       Ramping Option :                                                None |
    |       Relaxation Factor :                                         1.00E+00 |
    |       Mapping Type :                                  Surface Conservative |
    |    Transfer To Side Two:Incremental Displacement                           |
    |       Side One Variable :                         Incremental Displacement |
    |       Side Two Variable :                                     displacement |
    |       Convergence Target :                                        1.00E-02 |
    |       Ramping Option :                                                None |
    |       Relaxation Factor :                                         1.00E+00 |
    |       Mapping Type :                            Surface Profile Preserving |
    |                                                                            |
    +============================================================================+
    +============================================================================+
    |                     Participants Execution Information                     |
    +============================================================================+
    |                                                                            |
    | Structural                                                                 |
    |   Execution Command:                                                       |
    |     "/apps/r/ansys/v193/ansys/bin/ansys193" -b nolist -s noread -o Solutio |
    |     n.out -scport 42400 -schost t416.cluster -scname "Solution" -scid="553 |
    |     2_t416.cluster" -sclic=2325@tlic1 -i mapdl.dat                         |
    |   Working Directory:                                                       |
    |                           |
    |                                                                            |
    | Fluid                                                                      |
    |   Execution Command:                                                       |
    |     "/apps/r/ansys/v193/fluent/bin/fluent" 3ddp -g -scport=42400 -schost=t |
    |     416.cluster -scname="Solution 1" -scid="5532_t416.cluster" -sclic="232 |
    |     5@tlic1" -i Solution1.jou                                              |
    |   Working Directory:                                                       |
    |                                 |
    +============================================================================+
    Awaiting connections from coupling participants...Fatal error in PMPI_Init_thread: Other MPI error, error stack:
    MPIR_Init_thread(805)....: fail failed
    MPID_Init(1832)..........: channel initialization failed
    MPIDI_CH3_Init(126)......: fail failed
    MPID_nem_init_ckpt(1087).: fail failed
    MPIDI_PG_SetConnInfo(711): PMI_KVS_Put returned 14
    In: PMI_Abort(69253391, Fatal error in PMPI_Init_thread: Other MPI error, error stack:
    MPIR_Init_thread(805)....: fail failed
    MPID_Init(1832)..........: channel initialization failed
    MPIDI_CH3_Init(126)......: fail failed
    MPID_nem_init_ckpt(1087).: fail failed
    MPIDI_PG_SetConnInfo(711): PMI_KVS_Put returned 14)
    srun: Job step aborted: Waiting up to 32 seconds for job step to finish.
    slurmstepd: error: *** STEP 2753601.0 ON t416 CANCELLED AT 2019-05-29T13:22:20 ***
    [mpiexec@t416.cluster] control_cb (../../pm/pmiserv/pmiserv_cb.c:798): connection to proxy 0 at host t416 failed
    [mpiexec@t416.cluster] HYDT_dmxu_poll_wait_for_event (../../tools/demux/demux_poll.c:76): callback returned error status
    [mpiexec@t416.cluster] HYD_pmci_wait_for_completion (../../pm/pmiserv/pmiserv_pmci.c:501): error waiting for event
    [mpiexec@t416.cluster] main (../../ui/mpich/mpiexec.c:1147): process manager error waiting for completion

  • SteveSteve Posts: 153Forum Coordinator
    edited May 2019

    Thanks. We'll take a look into this in more detail and get back to you with what we find.

    Steve

Sign In or Register to comment.