Ansys mechanical runs only after 2 attempts

- July 18, 2023 at 7:30 pm
  
  Tomas Llano-Rios
  Subscriber
  
  When running Ansys (2023R1) Mechanical on any node from a cluster, it only acquires the license successfully after the first run has failed. This happened before with fluent, but I fixed it by setting ANSYSLI_TIMEOUT_FLEXLM=20, ANSYSLI_FLEXLM_TIMEOUT_ENV=2000000, ANSYSCL_TIMEOUT_CONNECT=60, and ANSYSCL_TIMEOUT_RESPONSE=300 in ansyslmd.ini (https://forum.ansys.com/forums/topic/fluentansys-2023r1-fails-to-acquire-license-on-1st-run-but-succeeds-on-2nd/). Why do these timeout values not work for Mechanical?
  Any help you can give me is highly appreciated.
  Example command I use to invoke Ansys Mechanical:
  ansys231 -b nolist -apip on -p ansys -i remote.dat -o solve.out -dis -machines node01:4 -dir "/path/to/data" –usessh
  Here are some relevant logs:
  /home/user/.ansys/licdebug.node01.FEAT_ANSYS.231.out
  2023/07/18 18:09:02    INFO                Starting Licensing Client Proxy server.
  2023/07/18 18:09:02    INFO                /apps/ansys/2023r1/v231/licensingclient/linx64/ansyscl -acl 91922.75297 -nodaemon -log /home/user/.ansys/ansyscl.node01.91922.75297.log
  2023/07/18 18:09:02    INFO                Started ANSYSLI server.
  2023/07/18 18:11:02    CONNECT_ERROR       ANSYS                                                                                0/0/0/0   91922:FEAT_ANSYS:user@node01.maas:linx64          0:127.0.1.1
                  ANSYSLI exited or could not read server port ansyscl.node01.91922.75297.
                  Please refer /home/user/.ansys/ansyscl.node01.91922.75297.log for more information.
  2023/07/18 18:11:02    CONNECT_ERROR       ANSYS                                                                                0/0/0/0   91922:FEAT_ANSYS:user@node01.maas:linx64          0:127.0.1.1
                  ANSYSLI exited or could not read server port ansyscl.node01.91922.75297.
                  Please refer /home/user/.ansys/ansyscl.node01.91922.75297.log for more information.
  /home/user/.ansys/ansyscl.node01.log
  2023/07/18 18:11:03    INFO                ANSYSLI_CMD=/apps/ansys/2023r1/v231/licensingclient/linx64/ansyscl -acl 91922.75297 -nodaemon -log /home/user/.ansys/ansyscl.node01.91922.75297.log
  2023/07/18 18:11:03    INFO                ANSYSLI_INITIALIZATION_FILE='/apps/ansys/2023r1/shared_files/licensing/ansyslmd.ini'
  2023/07/18 18:11:03    INFO                ANSYSLI_PRODORD_FILE='/apps/ansys/2023r1/shared_files/licensing/prodord/ansysli.prodord.xml'
  2023/07/18 18:11:03    INFO                ANSYSLI_TIMEOUT_FLEXLM=20
  2023/07/18 18:11:03    INFO                Configuring ACL Core
  2023/07/18 18:11:03    INFO                ACL Core Initialized
  2023/07/18 18:11:03    INFO                ANSYSCL_PORT=39841
  2023/07/18 18:11:03    INFO                Listen Socket Created
  2023/07/18 18:11:03    INFO                ANSYSLI_CLIENT_IDLE_TIMEOUT=0
  2023/07/18 18:11:03    INFO                ANSYSCL_FNP_PATH=1055@licensing.server
  2023/07/18 18:11:03    INFO                Ready to accept connections.
  2023/07/18 18:11:03    INFO                ANSYSLI_IP_OVERRIDE option is off.
  2023/07/18 18:11:18    INFO                ACL Shutdown
  2023/07/18 18:11:19    INFO                Shutting Down ACL Core
  2023/07/18 18:11:19    INFO                ACL Core Shutdown
  2023/07/18 18:11:19    INFO                License server shutdown is complete. Thank you.
  Additionally, If I run ansys231 interactively it shows these errors:
  ANSYS LICENSE MANAGER ERROR:
  ANSYSLI exited or could not read server port ansyscl.node01.126749.30360.
  Please refer /home/user/.ansys/ansyscl.node01.126749.30360.log for more information.
  
  ANSYS LICENSE MANAGER ERROR:
  Request name mech_2 does not exist in the licensing pool.
  No such feature exists.
  Feature:       mech_2
  License path: 1055@licensing.server:
  FlexNet Licensing error:-5,147
  
  ANSYS LICENSE MANAGER ERROR:
  Request name mech_1 does not exist in the licensing pool.
  No such feature exists.
  Feature:       mech_1
  License path: 1055@licensing.server:
  FlexNet Licensing error:-5,147
  
  ANSYS LICENSE MANAGER ERROR:
  Request name meba does not exist in the licensing pool.
  No such feature exists.
  Feature:       meba
  License path: 1055@licensing.server:
  FlexNet Licensing error:-5,147
  
  Ansys Mechanical Enterprise Academic Research
- July 19, 2023 at 7:16 pm
  
  Mangesh Bhide
  Ansys Employee
  
  Hello
  Does setting
  FLEXLM_TIMEOUT=20000000
  help ?
  - July 19, 2023 at 7:35 pm
    
    Tomas Llano-Rios
    Subscriber
    
    Hi,
    no, it didn't help. I added FLEXLM_TIMEOUT=20000000 to ansyslmd.ini but the issue persists.
- July 19, 2023 at 7:45 pm
  
  George Karnos
  Ansys Employee
  
  Hello,
  Can you try the following:
  Set the following Environment Variable:
  FLEXLM_TIMEOUT=60000000
  
  Other possibilities are a slow disk or a security software may not be letting ansyscl startup fast enough.
  
  After 2nd try of the binary, any libraries etc might be getting into system cache and getting reused - but this is only my hypothesis based on the behavior reported
  - July 19, 2023 at 8:23 pm
    Tomas Llano-Rios
    Subscriber
    Hi,
    I changed FLEXLM_TIMEOUT to 60000000, but saw no improvement. For the other possibilities you describe:
    Slow disk: Since the data was within a NFS export, I did the following:
    Copied the data (size is 5.2G) to a local SSD: cp -r /home/user/data /tmp/data
    Changed directory to /tmp/data: cd /tmp/data
    Launched mapdl from /tmp/data as follows: mapdl -b nolist -apip on -p ansys -i remote.dat -o solve.out -dis -machines node02:4 -dir "/tmp/data" -usessh
    It still failed in the same way. I also tried launching mapdl from a local and remote ansys install (the remote one residing on an NFS export), but both of them have the exact same error.
    Security software: The node runs Ubuntu 20.04, but I double checked and UFW is turned off. The coordinador node is the gateway and routes traffic via nftables. I have also tried disabling non-forwarding related rules on nftables, thus allowing all traffic from the licensing server to come in, but it still does not work.
- July 19, 2023 at 9:16 pm
  
  Mangesh Bhide
  Ansys Employee
  
  Are the compute machines and license server on the same network (same LAN) ?
  - July 20, 2023 at 2:03 am
    
    Tomas Llano-Rios
    Subscriber
    
    No. The compute nodes are on a separate LAN. The coordinator (or master) node has two NICS; one in the same network as the license server and another in the same network as the compute nodes. It acts as a router for the compute nodes by forwarding traffic from one LAN to the other via nftables rules.
- July 20, 2023 at 3:37 am
  
  Mangesh Bhide
  Ansys Employee
  
  if simply 2 different subets but same LAN then it should be fast. what might be reasons for delays?
  Network Latency: any reason the latency might be higher between networks? perhaps heavy network load? routes? multiple DNS ? (try seting environment variable FNP_IP_ENV=1 on compute nodes ?
  License Server Resources: Is the license server under heavy load? perhaps also serving other licenses or running other programs?
  - July 20, 2023 at 1:27 pm
    Tomas Llano-Rios
    Subscriber
    The license server IP and hostname are both within /etc/hosts on every node, so lookups are fast. Moreover, I use the license server IP instead of its hostname when setting up the SERVER variable inside ansyslmd.ini. This is how it currently looks:
    SERVER=1055@x.x.x.x
    ANSYSLI_SERVERS=2325@x.x.x.x
    ANSYSLI_FNP_IP_ENV=1
    ANSYSLI_TIMEOUT_FLEXLM=20
    ANSYSLI_FLEXLM_TIMEOUT_ENV=2000000
    ANSYSCL_TIMEOUT_CONNECT=120
    ANSYSCL_TIMEOUT_RESPONSE=300
    ANSYSLI_TIMEOUT_TCP=300
    FLEXLM_TIMEOUT=60000000
    The nodes can reach the license server fairly quick:
    Coordinator node (TCP test):
    traceroute -T x.x.x.x
    
    traceroute to x.x.x.x (x.x.x.x), 30 hops max, 60 byte packets
    1 license.server (x.x.x.x) 0.274 ms 0.210 ms 0.239 ms
    node02 (TCP test):
    traceroute -T x.x.x.x
    
    traceroute to x.x.x.x (x.x.x.x), 30 hops max, 60 byte packets
    1 node00.maas (y.y.y.y) 0.275 ms 0.229 ms 0.213 ms
    2 license.server (x.x.x.x) 0.393 ms 0.418 ms 0.404 ms
    Coordinator node (ICMP test):
    
    PING x.x.x.x (x.x.x.x) 56(84) bytes of data.
    64 bytes from x.x.x.x: icmp_seq=1 ttl=127 time=0.564 ms
    64 bytes from x.x.x.x: icmp_seq=2 ttl=127 time=0.562 ms
    64 bytes from x.x.x.x: icmp_seq=3 ttl=127 time=0.534 ms
    --- x.x.x.x ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2043ms
    rtt min/avg/max/mdev = 0.534/0.553/0.564/0.013 ms
    node02 (ICMP test):
    
    PING x.x.x.x (x.x.x.x) 56(84) bytes of data.
    64 bytes from x.x.x.x: icmp_seq=1 ttl=128 time=0.314 ms
    64 bytes from x.x.x.x: icmp_seq=2 ttl=128 time=0.298 ms
    64 bytes from x.x.x.x: icmp_seq=3 ttl=128 time=0.320 ms
    
    --- x.x.x.x ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2052ms
    rtt min/avg/max/mdev = 0.298/0.310/0.320/0.009 ms
    
    I tried setting FNP_IP_ENV=1 when I executed mapdl, but the issue persists. From the command line:
    FNP_IP_ENV=1 mapdl -b nolist -apip on -p ansys -i remote.dat -o solve.out -dis -machines node02:4 -dir “/tmp/zdusa2sc.s1v” -usessh
    As for the license server, It serves licenses to multiple machines, but it is not under heavy load. When I run mapdl from the coordinator node there are not issues. It acquires the license on the first try and runs fine every time (when this happened with fluent in the past I noticed the same behaviour; the coordinator would acquire the license without issues, but the compute nodes would take more time to the point I had to set longer timeouts for FLEXLM and ANSYSCL in ansyslmd.ini).

Viewing 4 reply threads

You must be logged in to reply to this topic.

Ansys Innovation Space

Boost Ansys Fluent Simulations with AWS

Computational Fluid Dynamics (CFD) helps engineers design products in which the flow of fluid components is a significant challenge. These different use cases often require large complex models to solve on a traditional workstation. Click here to join this event to learn how to leverage Ansys Fluids on the cloud, thanks to Ansys Gateway powered by AWS.

Earth Rescue – An Ansys Online Series

The climate crisis is here. But so is the human ingenuity to fight it. Earth Rescue reveals what visionary companies are doing today to engineer radical new ideas in the fight against climate change. Click here to watch the first episode.

Ansys Blog

Subscribe to the Ansys Blog to get great new content about the power of simulation delivered right to your email on a weekly basis. With content from Ansys experts, partners and customers you will learn about product development advances, thought leadership and trends and tips to better use Ansys tools. Sign up here.

Trending discussions