Learning Forum

[michail.stockfelt]

Hello

I do ANSYS Mechanical on a minor company that does not have a budget for Xeon/Epyc clusters, and I am very interested in the upcoming AMD Threadripper Pro.

AVX-512 and 8-channel RAM, moderate price sounds nice. I may even manage to convince my boss to cash up for a HPC pack so we can use more cores - although Mechanical does not scale as well as CFD with more cores, it is the 16-core CPUs I am interested in.

The one thing I wonder about is this: on my current Ryzen I had to enable a system setting - "MKL_DEBUG_CPU_TYPE=5" to make ANSYS utilize the AVX2 functionality, which right about doubled performance.

As I understand, this is because ANSYS is compiled with an Intel compiler that only sees AVX on Intel CPUs, and ignores it on AMD CPUs unless that setting is enabled. Intel's excuse is that they cannot guarrantee that the functionality of other CPUs comply with the expected behavior, which is a weak argument since there is supposedly established standards and the onus is on AMD to follow them. Considering the extensive reliance on other-party libraries in the software world, nothing would work if programmers followed Intels logic here, so I can only assume it is deliberate to make AMD CPUs perform worse.

So my question is really for ANSYS to answer: will we be able to use the Threadripper Pro 7000 series with AVX-512 extensions without having to rely on compiler workaround hacks this time?

Thanks for any input.

[Hunter Wang]

AMD Threadripper Pro 7000 series has not been released yet. Rumor says desktop and mobile Zen 4 based processor will be released next month. Assuming workstation (Threadripper Pro) and server (EYPC) grade could be later.

Refer Ansys 2021 R2 Release Notes (https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v212/en/ai_rn/rnsolvers.html%23d0e3050):

2.3.1. Improved Performance on AMD Processors

Support for the AMD BLIS library has been added, providing faster performance when running on systems with AMD processors. Mechanical APDL uses the BLIS library when running on AMD processors (and continues to use the MKL library when running on Intel processors). For most simulations, sparse direct solver performance with BLIS on AMD processors is 5-20 percent faster (depending on your AMD hardware and the number of CPU cores used). For harmonic analyses, the sparse direct solver runs up to 2x faster.

Also refer https://www.amd.com/system/files/documents/amd-epyc-7003-pb-hpc-ansys-mechanical-aocl-optimized-performance.pdf

Assuming MKL_DEBUG_CPU_TYPE=5 is no longer required for 2021 R2 and newer releases. But you could try to compare if you would like. 

As far as I know, Ansys developers are working closely with AMD (and also all partners). I believe there is no reason for future version Ansys not to support AVX-512 for better performance if AMD's upcoming processors could support complete and reliable AVX-512 instruction set (or similar) and offer correspondig math libraries. 

[michail.stockfelt]

Oh. BLIS? Neat!

I must investigate that immediately. :) Haven't touched the MKL thing since last Windows installation, long ago.

Thanks!