damian.marek
Subscriber

Hello,

HFSS does not explicitly say out of memory, but the job scheduler reports that the processes have been killed by the out-of-memory handler. In the design I have imposed a very fine mesh in order to stress test HFSS.

Here is the output of the simulation, are my settings correctly using both nodes?:

First-time configuration completed successfully.
[progress:  0%] (1) Reading...: Creating Component Defintions... (3:37:18 PM  Aug 30, 2022)
[info] Running "SLURM" job with ID "8087710".  Command line: "ansysedt.exe -monitor -useElectronicsPPE -ng -Distributed includetypes=Mesh Assembly, Domain Solver, Iterative Solver, Direct Solver, Mesher, Solver, Solvers -machinelist list=nia1335:40:40:90,nia1338:40:40:90 -batchoptions HPCLicenseType=pool HFSS/EnableGPU=0 -batchsolve benchmark1_80p.aedt".
[info] Project:benchmark1_80p, Design:homo (Terminal Network), Setup1 : LastAdaptive distributing Direct Solver Memory (3:37:22 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Design:homo (Terminal Network), Setup1 : Sweep distributing Direct Solver Memory (3:37:22 PM  Aug 30, 2022)
[progress:100%] (2) benchmark1_80p - homo - Starting simulation engine for setup: Setup1 on nia1335 - RUNNING (3:37:24 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Solution Process : Start Time: 08/30/2022 15:37:32, Host: nia1335.scinet.local, Processor: 80, OS: Linux 3.10.0-1160.76.1.el7.x86_64, Product: HFSS Version 2022.2.0 (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        Executing from /software/commercial/ansys/2022r2/v222/Linux64/HFSSCOMENGINE.exe (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        HPC: Enabled (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        Allow off core: True (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        HPC settings: Manual (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        Distribution Types: Domain Solver, Direct Solver, Iterative Solver, Mesh Assembly (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        Two level: Disabled (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        Machines: (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        nia1335 [189 GB]: RAM Limit: 90.000000%, 40 tasks, 40 cores, Free Disk Space: 138 GB (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        nia1338 [189 GB]: RAM Limit: 90.000000%, 40 tasks, 40 cores, Free Disk Space: 138 GB (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        Solution Basis Order: 1 (3:37:32 PM  Aug 30, 2022)
[progress:  0%] (2) benchmark1_80p - homo - Setup1: Part validation on nia1335:   (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Design Validation : Elapsed time : 00:00:00 , HFSS ComEngine Memory : 118 M (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        Perform full validations with standard port validations (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Initial Meshing : Time: 08/30/2022 15:37:32 (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Mesh Phi : Real Time 00:00:00 : CPU Time 00:00:00 : Memory 118 M : Tetrahedra: 1267 (3:37:32 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Mesh Post : Real Time 00:00:00 : CPU Time 00:00:00 : Memory 118 M : Tetrahedra: 1692 (3:37:33 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Mesh Refinement : Manual Seed Based (3:37:33 PM  Aug 30, 2022)
[progress: 59%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 8 ; Elements needing refinement= 50; Target= top 50; Ntets= 5640 (3:37:34 PM  Aug 30, 2022)
[progress: 15%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 21 ; Elements needing refinement= 223; Target= top 223; Ntets= 18601 (3:37:37 PM  Aug 30, 2022)
[progress: 46%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 22 ; Elements needing refinement= 417; Target= top 417; Ntets= 22011 (3:37:39 PM  Aug 30, 2022)
[progress: 83%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 25 ; Elements needing refinement= 227; Target= top 227; Ntets= 29837 (3:37:42 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 28 ; Elements needing refinement= 10; Target= top 10; Ntets= 40464 (3:37:46 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 39 ; Elements needing refinement= 27; Target= top 27; Ntets= 65772 (3:37:51 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 47 ; Elements needing refinement= 6; Target= top 6; Ntets= 80416 (3:37:59 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 54 ; Elements needing refinement= 228; Target= top 228; Ntets= 114708 (3:38:10 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 63 ; Elements needing refinement= 9; Target= top 9; Ntets= 153044 (3:38:14 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 69 ; Elements needing refinement= 8; Target= top 8; Ntets= 181751 (3:38:25 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 71 ; Elements needing refinement= 9; Target= top 9; Ntets= 197708 (3:38:29 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 74 ; Elements needing refinement= 28; Target= top 28; Ntets= 224855 (3:38:33 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 78 ; Elements needing refinement= 4; Target= top 4; Ntets= 249631 (3:38:49 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 81 ; Elements needing refinement= 33; Target= top 33; Ntets= 290991 (3:38:52 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 85 ; Elements needing refinement= 18; Target= top 18; Ntets= 310706 (3:38:56 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 88 ; Elements needing refinement= 35; Target= top 35; Ntets= 332634 (3:39:01 PM  Aug 30, 2022)
[progress: 83%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 94 ; Elements needing refinement= 4; Target= top 4; Ntets= 375421 (3:39:28 PM  Aug 30, 2022)
[progress: 15%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 98 ; Elements needing refinement= 376; Target= top 376; Ntets= 458410 (3:39:30 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 101 ; Elements needing refinement= 5407; Target= top 5407; Ntets= 463565 (3:39:41 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 102 ; Elements needing refinement= 32; Target= top 32; Ntets= 536551 (3:39:46 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 106 ; Elements needing refinement= 38; Target= top 38; Ntets= 570723 (3:40:25 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 110 ; Elements needing refinement= 56; Target= top 56; Ntets= 686699 (3:40:36 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 113 ; Elements needing refinement= 653; Target= top 653; Ntets= 787599 (3:40:39 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 114 ; Elements needing refinement= 17; Target= top 17; Ntets= 796081 (3:41:16 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 116 ; Elements needing refinement= 67; Target= top 67; Ntets= 922975 (3:41:18 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 118 ; Elements needing refinement= 12; Target= top 12; Ntets= 933671 (3:41:35 PM  Aug 30, 2022)
[progress: 15%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 122 ; Elements needing refinement= 188; Target= top 188; Ntets= 1052073 (3:41:37 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 125 ; Elements needing refinement= 9899; Target= top 9899; Ntets= 1054815 (3:41:54 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 129 ; Elements needing refinement= 14217; Target= top 14217; Ntets= 1176586 (3:42:38 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 132 ; Elements needing refinement= 1048; Target= top 1048; Ntets= 1333399 (3:42:42 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 135 ; Elements needing refinement= 2; Target= top 2; Ntets= 1350740 (3:42:49 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 140 ; Elements needing refinement= 24; Target= top 24; Ntets= 1389222 (3:43:33 PM  Aug 30, 2022)
[progress: 79%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 142 ; Elements needing refinement= 93; Target= top 93; Ntets= 1667236 (3:43:38 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 144 ; Elements needing refinement= 475; Target= top 475; Ntets= 1722463 (3:43:43 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 146 ; Elements needing refinement= 12; Target= top 12; Ntets= 1740660 (3:43:46 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 149 ; Elements needing refinement= 12; Target= top 12; Ntets= 1753264 (3:44:56 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 152 ; Elements needing refinement= 108; Target= top 108; Ntets= 1997878 (3:44:59 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 154 ; Elements needing refinement= 29; Target= top 29; Ntets= 2012234 (3:45:07 PM  Aug 30, 2022)
[progress: 83%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: Doing seed refinement (3:45:38 PM  Aug 30, 2022)
[progress: 84%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 170 ; Elements needing refinement= 76; Target= top 76; Ntets= 2074134 (3:45:41 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 175 ; Elements needing refinement= 3; Target= top 3; Ntets= 2089579 (3:45:44 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 178 ; Elements needing refinement= 6; Target= top 6; Ntets= 2104780 (3:45:46 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 184 ; Elements needing refinement= 4; Target= top 4; Ntets= 2133138 (3:45:51 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 186 ; Elements needing refinement= 11; Target= top 11; Ntets= 2158692 (3:45:54 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 190 ; Elements needing refinement= 1; Target= top 1; Ntets= 2192401 (3:46:02 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 194 ; Elements needing refinement= 5; Target= top 5; Ntets= 2232341 (3:46:08 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 200 ; Elements needing refinement= 2; Target= top 2; Ntets= 2290466 (3:46:13 PM  Aug 30, 2022)
[progress: 85%] (2) benchmark1_80p - homo - Setup1: Seed Initial Mesh on nia1335: PassNo 202 ; Elements needing refinement= 7; Target= top 7; Ntets= 2313244 (3:46:16 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Mesh (volume, seed) : Real Time 00:09:48 : CPU Time 00:09:46 : Memory 2.58 G : Tetrahedra: 2336616 (3:47:21 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Mesh Refinement : Lambda Based (3:47:23 PM  Aug 30, 2022)
[progress:  0%] (2) benchmark1_80p - homo - Setup1: Lambda-Refine Mesh on nia1335:   (3:47:23 PM  Aug 30, 2022)
[progress: 15%] (2) benchmark1_80p - homo - Setup1: Lambda-Refine Mesh on nia1335: Reading the mesh stats. (3:48:07 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Mesh (lambda based) : Real Time 00:01:35 : CPU Time 00:01:35 : Memory 1.64 G : Tetrahedra: 2336452 (3:48:58 PM  Aug 30, 2022)
[progress:  1%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Setting up simulation (3:48:59 PM  Aug 30, 2022)
[progress: 10%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Setting up simulation (3:49:42 PM  Aug 30, 2022)
[progress: 50%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Setting up simulation (3:49:51 PM  Aug 30, 2022)
[progress: 90%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Setting up simulation (3:49:56 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Simulation Setup : Real Time 00:00:57 : CPU Time 00:00:57 : Memory 3.53 G : Disk: 0 Bytes,  (3:49:57 PM  Aug 30, 2022)
[progress:  1%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Building matrix (3:50:00 PM  Aug 30, 2022)
[progress:  2%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Building matrix (3:50:05 PM  Aug 30, 2022)
[progress:  4%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Building matrix (3:50:07 PM  Aug 30, 2022)
[progress:  8%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Building matrix (3:50:12 PM  Aug 30, 2022)
[progress: 10%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Building matrix (3:50:15 PM  Aug 30, 2022)
[progress: 28%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Building matrix (3:50:24 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Port Adaptation : Real Time 00:00:28 : CPU Time 00:00:28 : Memory 3.78 G : Disk: 13 KB, Tetrahedra: 1897433 (3:50:25 PM  Aug 30, 2022)
[progress:  0%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335:   (3:50:27 PM  Aug 30, 2022)
[progress: 15%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Reading the mesh stats. (3:50:58 PM  Aug 30, 2022)
[progress: 50%] (2) benchmark1_80p - homo - Setup1: Applying port refinement on nia1335: Reading the mesh stats. (3:51:12 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Mesh (port based) : Real Time 00:01:38 : CPU Time 00:01:38 : Memory 1.64 G : Tetrahedra: 2336918 (3:52:05 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        Length1, Length2 (3:52:06 PM  Aug 30, 2022)
[progress:  0%] (2) benchmark1_80p - homo - Construction of the starting mesh completed on nia1335 (3:52:06 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Stop : Elapsed Time: 00:14:34 (3:52:06 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Adaptive Meshing : Time: 08/30/2022 15:52:06 (3:52:06 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Adaptive Pass 1 : Frequency:  10GHz, Time: 08/30/2022 15:52:06 (3:52:06 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        Solving with MPI (Intel MPI) (3:52:06 PM  Aug 30, 2022)
[progress:  0%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1: Simulating ... on nia1335:   (3:52:15 PM  Aug 30, 2022)
[progress:  1%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335:   (3:52:24 PM  Aug 30, 2022)
[progress: 10%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335:   (3:53:21 PM  Aug 30, 2022)
[progress: 30%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335:   (3:53:23 PM  Aug 30, 2022)
[progress: 50%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335:   (3:53:33 PM  Aug 30, 2022)
[progress: 90%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335:   (3:53:39 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Simulation Setup : Real Time 00:01:32 : CPU Time 00:01:32 : Memory 3.75 G : Disk: 291 KB, Tetrahedra: 1897811 (3:53:57 PM  Aug 30, 2022)
[progress:  0%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335: Building matrix (3:53:57 PM  Aug 30, 2022)
[progress:  1%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335: Building matrix (3:54:02 PM  Aug 30, 2022)
[progress:  2%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335: Building matrix (3:54:08 PM  Aug 30, 2022)
[progress:  4%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335: Building matrix (3:54:11 PM  Aug 30, 2022)
[progress:  8%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335: Building matrix (3:54:16 PM  Aug 30, 2022)
[progress: 10%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335: Building matrix (3:54:20 PM  Aug 30, 2022)
[progress: 15%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Solving single frequency for adaptive meshing ... on nia1335: Building matrix (3:54:48 PM  Aug 30, 2022)
[error] Project:benchmark1_80p, Design:homo (Terminal Network), Solving adaptive frequency ... (Distributed direct matrix solver), process hf3d exited with code 149. -- Simulating on machine: nia1335 (3:55:00 PM  Aug 30, 2022)
[progress:  0%] (2) benchmark1_80p - homo - Setup1: Adaptive Pass #1 - Completed solving frequencies ... on nia1335 (3:55:20 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Stop : Elapsed Time: 00:03:28 (3:55:34 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Stop : Elapsed Time: 00:03:28 (3:55:34 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Simulation Summary (3:55:34 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Design Validation : Elapsed Time: 00:00:00, Total Memory: 118 MB (3:55:34 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Initial Meshing : Elapsed Time: 00:14:34, Total Memory: 3.78 GB (3:55:34 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Adaptive Meshing : Elapsed Time: 00:03:28, Total Memory: 0 Bytes (3:55:34 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Stop :   (3:55:34 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE]        Stop Time: 08/30/2022 15:55:34, Status: Engine Detected Error (3:55:34 PM  Aug 30, 2022)
[info] Project:benchmark1_80p, Setup1 : [PROFILE] Stop : Elapsed Time: 00:18:02, ComEngine Memory: 121 M (3:55:34 PM  Aug 30, 2022)
[progress:100%] (3) Updating all derived data centrally...: Create Plots (3:55:37 PM  Aug 30, 2022)
[error] Project:benchmark1_80p, Design:homo (Terminal Network), Simulation completed with execution error on server: nia1335. (3:55:37 PM  Aug 30, 2022)
slurmstepd: error: Detected 1104 oom-kill event(s) in StepId=8087710.batch. Some of your processes may have been killed by the cgroup out-of-memory handler.