Fluids

Fluids

Fluent: Problem at partition boundary in parallel simulation using VOF and porous media

    • naibaf
      Subscriber

      Dear community,

      I have a problem in parallel simulation. I want to simulate fluid flow in a porous media representing a generic underground mine. Therefore, is set up the geometry and assign porous media. Other specifications of my model are:

      ·        VOF model used.

      ·        porous media with spatially varying porosity and permeability in the domain

      ·        two-phase (air, water) flow with compressible liquid for both phase

      ·        laminar flow conditions

      ·        UDF with mass source term for air-phase

      ·        14 Solver processes with activated hyperthreading, partition method X-Cartesian

      ·        Ryzen 7 3700X processor

      The code works in serial however I must investigate physics in more detail to check plausibility of the results. Serial simulation is very slow, so I wanted to start a parallel calculation. I parallelize my UDF’s, and the mesh is partitioned. But at some point, the calculation gives incorrect results. At a partition interface the volume fraction of water vanishes ahead of the fully water-filled area. Before there was a significant VOF of water. Additionally, a high negative static pressure evolves at the partition interface (see pictures). In serial mode there is no problem at this point.

      Does anybody faced such a problem in the past and may explain the reason for these 'artefact' and how I remove it? So far, I think there is a problem in communication between the node processes or with the water surface at the partition interface. The problem occurs even if I deactivate all my UDF's. When changing the partiton method the problem occurs at a different location due to changed location of the partition interface.

    • DrAmine
      Ansys Employee
      So running with single node or serial you do not see that issue?
    • naibaf
      Subscriber
      Dear DrAmine thank you for your reply. The problem does not occur when simulating with one node (select one solver process in Fluent launcher)

      Regards
    • Rob
      Ansys Employee
      What does the mass source do?
    • naibaf
      Subscriber
      Dear Rob Thank you for your reply. The mass source ÔÇ£absorbsÔÇØ the surplus air mass in the cells due to water rising/flow (a bid similar to the ÔÇ£degassing boundary udfÔÇØ in the manual). I cannot use an outlet-boundary condition in the model since the water should stay inside the domain and in theory has the possibility to flow into nearly every part of the model. At first, I use DEFINE_ADJUST to calculate mass and momentum sources for each cell in the model. Values are saved in UDM. The DEFINE_SOURCE access UDM and takes the value for the source in each cell. I also tried to use PRF_GSYNC(). However, the issue in parallel calculation occurs with or without my UDFs.
      The UDFs Looks like:
      DEFINE_ADJUST(calcSource,d)
      {
      #if!RP_HOST
      Thread*cthread,*ts;
      cell_tc;
      thread_loop_c(cthread,d)
      {
      begin_c_loop_int(c,cthread)
      {
      ts=THREAD_SUB_THREAD(cthread,1);
      C_UDMI(c,cthread,0)=-(C_R(c,ts)-1.225)*C_VOF(c,ts)/CURRENT_TIMESTEP*C_POR(c,cthread);
      C_UDMI(c,cthread,1)=-(C_R(c,ts)-1.225)*C_VOF(c,ts)/CURRENT_TIMESTEP*C_U(c,cthread)*C_POR(c,cthread);
      C_UDMI(c,cthread,2)=-(C_R(c,ts)-1.225)*C_VOF(c,ts)/CURRENT_TIMESTEP*C_V(c,cthread)*C_POR(c,cthread);
      C_UDMI(c,cthread,3)=-(C_R(c,ts)-1.225)*C_VOF(c,ts)/CURRENT_TIMESTEP*C_W(c,cthread)*C_POR(c,cthread);
      }
      end_c_loop_int(c,cthread)
      }
      #endif
      }

      DEFINE_SOURCE(DensitySource,cell,thread,dS,eqn)
      {
      #if!RP_HOST
      dS[eqn]=0.0;
      returnC_UDMI(cell,thread,0);
      #endif
      }

      DEFINE_SOURCE(x_recoil,cell,t,dS,eqn)
      {
      #if!RP_HOST
      dS[eqn]=0.0;
      returnC_UDMI(cell,t,1);
      #endif
      }

      DEFINE_SOURCE(y_recoil,cell,t,dS,eqn)
      {
      #if!RP_HOST
      dS[eqn]=0.0;
      returnC_UDMI(cell,t,2);
      #endif
      }

      DEFINE_SOURCE(z_recoil,cell,t,dS,eqn)
      {
      #if!RP_HOST
      dS[eqn]=0.0;
      returnC_UDMI(cell,t,3);
      #endif
      }

      Thank you in advance!
    • DrAmine
      Ansys Employee
      It is not clear for me where in your VOF picture the porous domain is starting. Can you highlight that? Have you tried without any porous domain?
    • naibaf
      Subscriber
      Dear DrAmine the whole domain is defined as porous media, since rock and backfill material is porous. So therefore I did not try without porous media since the porosity of rock has to be considered.
      In tests on a much smaller column model with constant porosity and permeability the issue doesn't occur within parallel calculation.
      Regards
    • DrAmine
      Ansys Employee
      We are trying to help you debugging the case: Will you see the issue if you disable the porous domain? Are you using physical or superficial velocity?
    • naibaf
      Subscriber
      Thank you very much for your help.
      I am using superficial velocity. And I try to perform the simulation without porous media model. However, convergence issues occur when disabling pororus media. I think this is due to absent permeability of the medium. Therefore my calculatiuon settings etc. are not sufficient anymore.
      Regards
    • naibaf
      Subscriber
      For a first look, the calculation is very unstable without porous media. But it seems that the problem remains.
      Fig 1 Vanishing volume fraction of water in the area of partition interface, ahead of the fully water-filled part.
      Fig 2 Residuals showing poor convergence of the simulation without porous media model.


    • Rob
      Ansys Employee
      You're trying to turn the flow in a very small (coarse mesh) space. What happens if you increase the resolution?
    • naibaf
      Subscriber
      Dear Rob increasing resolution might be a good point for further investigation, thank you. But (only for better understanding of the topic) what is the difference in calculating on a single node or calculating on several nodes, related to mesh resolution? I do not understand why the single node simulation doesn't show the issue.
      Regards
    • Rob
      Ansys Employee
      It shouldn't, but you also have a parallelised UDF. What do the UDM values show when compared to the single core model?
    • naibaf
      Subscriber
      At the beginning of the calculation the UDM values for the source term are nearly identical in both cases. Slightliy differences at the partition boundary are visible. With increasing simulation duration the error between the UDM values at the partiton interface increases (see Figures below). The calculated minimum mass source (outflow) is the same in both cases. But there is a significant maximum mass source (inflow) in the parallel case.
      But I am wondering since the issue remains even if I unhook and unload all the UDF's . As expected the UDMs without UDFs are zero everywhere. So no source should be applied, but the issue with the vanishing volume fraction of water remains.
      Fig 1 UDM value for mass source in parallel case.
      Fig 2 UDM value for mass source in single node case.

      Thank you in advance!

    • DrAmine
      Ansys Employee
      Please summarize now (still bit surprising for me how a non-porous case using where the porous one is using superficial velocity is having convergence issue).
      Can yo confirm these statements

      1/The issue occurs without UDF#s? If Yes go to 2/ If No: We cannot help
      2/The issue does not occur on Single CPU If Yes go to 3/
      3/Post for the same time step size the VOF fraction field on a plane without any node values and use the same range

      Depending on the answers above we will try to respond.
    • naibaf
      Subscriber
      The convergence issue may arise from my modelling approach. I initialize an "excess pressure" in a specified area in the mine. The excess pressure evolves due to a mechanical deformation, which I calculate within another software on the same grid. ANSYS Fluent should simulate the relaxation of the pressurized fluid to a nearly hydrostatic equilibrium. The initial pressure is therefore maybe too high for simulation without a viscous resistance. So, I think setting-changes are needed for such a case (time-step size etc.).
      I can confirm (to the best of my knowledge and conscience) that:
      1/ yes, it occurs even without UDFs
      2/ yes, the issue does not occur with one solver process
      3/I hope these are the figures you expected.
      The Figures show the VOF for water in the underground rooms and drifts as plane view. In Fig 1 for the parallel case the vof drops, right at the partition interface. If the simulation runs further the spatial distribution of the decreasein the vof becomes larger. In Fig 2 for single node the VOF do not drop. If the calculation on the single node is continued, the chambers are getting filled with water more and more. For the parallel case, the water does not flow further. Both figures are saved after fluid-time = 221146.63 s after 40 time steps are calculated. I use adaptive time advancement to gradually increase the time step. The time advancement settings are the same in both cases.

      Fig 1 Volume fraction of water for the parallel case at t = 221146.63 s after 40 time steps.

      Fig 2 Volume fraction of water for the single node case at t = 221146.63 s after 40 time steps.
      Fig. 3 Distribution of partitions.

      Thank you very much for your help and time!

    • Rob
      Ansys Employee
      Repartition onto a different number of nodes or with y-direction so the partition is in a different place. What happens? What is the flow field like?
    • naibaf
      Subscriber
      Dear Rob for my first test with the default METIS-partitioning-method. The issue was occuring on a different location but also at the partition. So I changed the partitioning method to X-Cartesian and the issue occured at a different time at the new partition locations.
    • naibaf
      Subscriber
      So from my point of view, the fluid flow is ok until it reaches the partition interfaces.
    • Rob
      Ansys Employee
      OK, so we have a gradient issue with a very coarse mesh at the parallel partition. Please refine the mesh and see what happens.
    • naibaf
      Subscriber
      Dear Rob ok thank you for the help. I will try a mesh refinement to check results in such a case. I will post my findings here asap. Maybe it helps other users.
Viewing 20 reply threads
  • You must be logged in to reply to this topic.