Fluids

Fluids

How to use DPM with multi processor cores?

    • reza121
      Subscriber

      Hello, I hope you are doing well.

      I am using DPM for my simulations. There are 11 different mono-size injections and about 700 parcels are released at the inlet in each of those injections (i.e. 7700 parcels in total). The airflow is steady but the DPM is in "unsteady particle tracking" mode. The given particle time step size is 1e-7 and for all of the injections, the injection start time is at 0s and it ends at 1e-4s. please refer to the attached photos below.

      The duct where I am studying the motion of the particles has a length of 10mm and the flow/particle velocity is 1 m/s. However, it takes more than a week to solve the particle paths from the inlet to the outlet no matter how many CPUs I use on the super-computer (I have tested 64 cores up to 640 cores). I have a feeling that only one core is used to track the particles. Is there any setting to allocate the number of cores for the particle trajectory calculation?

      Since the particle phase is dilute and there is a one-way interaction between the flow and the particles, I would like to assign each of the parcels to a core to speed up the calculations. Is there a way to do that?

      It is noteworthy that it takes only a few seconds to solve the particle trajectories when DPM is in steady mode. 

      Thanks in advance

       

    • Rob
      Ansys Employee

      With that time step you're adding 1000 parcels per inlet facet per injection, ie you add a parcel per facet per particle time step. That may result in several parcels in the domain: the solver will report this in the TUI. 

      You're also going to struggle with a max steps of 5,000,000 when the default is 500 (from memory). If you need that many steps to avoid incomplete trajectories you need a smaller time step. If you do need that many steps the cpu requirement is going to be excessive. 

    • reza121
      Subscriber

      Thanks for your reply Rob,

      What you said is exactly correct. Over the 1e-4s of injection time, 1000 parcels are injected from each facet. I have about 700 inlet facets, and my total number of injection groups is 11. All in all, there are about 8 million parcels to be tracked. I don't think that this is a significant number. Do you?

      You didn't answer my question specifically though. Is there any way that I can change my core allocation to DPM in the software settings? The solution time doesn't change no matter how many cores I allocate to the job. Again, it takes much less time to track DPM in steady mode. I understand that in the steady case, it will be a one-time injection from each facet for each of the 11 injection groups. Therefore, there are 1000 times more particles in the unsteady mode.

      However, the first one takes a few seconds while the latter takes more than a week to be solved. The difference in time is not reasonable. So, I am suspicious that something is not right in the core-allocation settings in the unsteady case.

    • reza121
      Subscriber

      By the way, I have a complementary question regarding how the time step is calculated in both the steady and unsteady DPM cases. This time step is required for the equation of motion of the particles. 

      I know that in the steady mode, the user can give either the "step length factor" or the "length scale". However, in the unsteady DPM case, this panel remains active even though the user will assign a particle time step directly. In the unsteady DPM case, what is the use of the "step length factor" or the "length scale"? 

      Will they be ignored and only the DPM time step will be considered maybe?

       

    • Rob
      Ansys Employee

      8M particles is a large number, so will cause some slow down. You're also calculating particles on some nodes: if they're all near the inlet you'll find they're all on one node. So, it's not particles being brought back to the host over particles swamping one node. How many cells have you got? 

      If you want one pulse/band reduce the injection stop time to be 1-2 time steps. 

      DPM time step is how long a flight time is modelled for, so 1e-7s in your case. The steps or length factor is then how often the position and vector are checked (minimum is some per cell) per time step. In your case I suspect some parcels are bouncing between very few cells and therefore using many steps over the track. 

    • reza121
      Subscriber

      Thanks again for your reply Rob,

      I am not quite sure what you mean by "You're also calculating particles on some nodes". I chose surface injection and chose my inlet surface for that. I have about 2 Million cells in my entire domain and the inlet facets are about 700. 

      Yes, reducing the number of time steps by reducing the stop time is a good idea. The main purpose is studying the efficiency of a filter, so I just need a few parcels in each injection group to judge the number of trapped particles at the filter. 

      Correct me if I'm wrong: Based on what I understood from what you said, the "DPM time step" defines how often a parcel is injected from the designated injection location. In my case, it is creating a parcel every 1e-7 s. On the other side, the required time step for discretizing the equation of motion of particles (du_p / dt = Sigma F) is still calculated based on the "step length factor" or "length scale". right? If this is true, yes I will have an accumulation of multiple particles in my cells because the injection rate is higher than the speed of particles moving forward in comparison to my grid resolution. 

      If that is the case, I do not need to set my "DPM time step" to such a small number not so ever! My particle relaxation time is about 6e-7 so I chose the DPM time step as 1e-7 while I should have calculated the "step length factor" or "length scale" in a way to satisfy my relaxation time. right?

       

    • Rob
      Ansys Employee

      Fluent splits the mesh over some nodes. By default the particles are not considered in the partitioning checks so if the whole inlet is on one node all of the particles will be solved on that node: that can slow the whole calculation down. 

      I'd not use much more than 4 cores for 2M cells, if I had plenty of cpu I'd not expect to see much speed up beyond 20 cores. But with a non-uniform parcel distribution may mean you don't see much speed up beyond 4-5. 

      The particle time step governs the release rate and how long the particle is tracked for. So, every 1e-7s a new parcel is injected. Every parcel in the system has it's position moved by (velocity x 1e-7s) : the number of steps or length factor govern how often the parcel trajectory is checked, minimum is as a parcel enters a cell & then leaves so if a particle crosses many cells or has a complex flight path you need more steps or the parcel is aborted before it's travelled for the time step. 

    • reza121
      Subscriber

      Thank you so much for the help, Rob.

      The last question is to please guide me on how to create multiple nodes at the inlet so the DPM track can be solved over multiple cores parallelly as you said. How can I do that? Should I split my inlet rectangle surface into smaller sub-divided rectangles in my CAD software and then name them all as inlet? will it work? Or is there any better way? That would be great if you could refer me to a user-guide section if there is any in this regard. Thanks

       

    • Rob
      Ansys Employee

      Have a look at Load Balancing in the Parallel tab. I think there's a DPM option in there, if not look at physics options. 

Viewing 8 reply threads
  • You must be logged in to reply to this topic.