Learning Forum

[randyk]

Hi Ehsan,

Consider basing your script on this example - I highlighted items to watch/change to match your system.

Create "job.sh" with the following contents:

 

#!/bin/bash

#SBATCH -N 3        # allocate 3 nodes

#SBATCH -n 12       # 12 tasks total

#SBATCH -J AnsysEMTest   # sensible name for the job

 

#Set job folder, scratch folder, project, and design (Design is optional)

JobFolder=$(pwd)

ProjName=OptimTee-DiscreteSweep-FineMesh.aedt

DsnName="TeeModel:Nominal:Setup1"

 

# Executable path

AppFolder=//AnsysEM/v222/Linux64

 

# Copy AEDT Example to $JobFolder for this initial test

cp ${AppFolder}/schedulers/diagnostics/Projects/HFSS/${ProjName} ${JobFolder}/${ProjName}

 

# setup environments and srun

export ANSYSEM_GENERIC_MPI_WRAPPER=${AppFolder}/schedulers/scripts/utils/slurm_srun_wrapper.sh

export ANSYSEM_COMMON_PREFIX=${AppFolder}/common

export ANSYSEM_TASKS_PER_NODE=${SLURM_TASKS_PER_NODE}  

 

# If enabling Intel MPI 2021 on AEDT 2022R2 to support RHEL8.x (beta flag)

export ANSYSEM_FEATURE_F539685_MPI_INTEL21_ENABLE=1

 

# setup srun

srun_cmd="srun --overcommit --export=ALL -n 1 --cpu-bind=none --mem-per-cpu=0 --overlap "

# note: srun '--overlap' option was introduced in SLURM VERSION 20.11 - if customer using older SLURM version, remove the "--overlap" 

 

# MPI timeout set to 30min default for cloud suggest lower to 120 or 240 seconds for onprem

export MPI_TIMEOUT_SECONDS=120

 

# System networking environment variables - HPC system dependent should not be user edits!

# export ANSOFT_MPI_INTERCONNECT=ib

# export ANSOFT_MPI_INTERCONNECT_VARIANT=ofed

 

# Skip dependency check

export ANS_NODEPCHECK=1

 

# Setup Batchoptions

echo "\$begin 'Config'" > ${JobFolder}/${JobName}.options

# echo "'Desktop/Settings/ProjectOptions/HPCLicenseType'='Pack'" >> ${JobFolder}/${JobName}.options

echo "'HFSS/RAMLimitPercent'=90" >> ${JobFolder}/${JobName}.options

echo "'HFSS/RemoteSpawnCommand'='scheduler'" >> ${JobFolder}/${JobName}.options

# echo "'Desktop/Settings/ProjectOptions/AnsysEMPreferredSubnetAddress'='192.168.16.0/24'" >> ${JobFolder}/${JobName}.options

echo "\$end 'Config'" >> ${JobFolder}/${JobName}.options

 

# Run Job

${srun_cmd} ${AppFolder}/ansysedt -ng -monitor -waitforlicense -useelectronicsppe=1 -distributed -auto -machinelist numcores=12 -batchoptions ${JobFolder}/${JobName}.options -batchsolve ${DsnName} ${JobFolder}/${ProjName}

#note the "numcores=xx"  must match the assigned resource core count.

 

Then run it:

$ dos2unix ./job.sh

$ chmod +x  ./job.sh

$ sbatch ./job.sh