ANSYS on HPC Systems
What is the ANSYS Software Package?
The ANSYS Software Package is an engineering and scientific software suite applicable for the simulation of computational fluid dynamics (CFD), acoustics, computational structural mechanics (CSM), electro-magnetics (EMAG), thermodynamics and many more applications. The different ANSYS simulation tools and applications can be accessed either via the ANSYS Workbench GUI, the GUI's of standalone applications like ANSYS CFX or ANSYS Fluent and through command line interfaces, e.g. in so-called batch mode on the Linux Cluster and SuperMUC-NG. Furthermore the ANSYS Software Package includes the Postprocessing Tools CFD-Post and ANSYS Ensight.
With a separate installation package ANSYS distributes the ANSYS EM (EM=Electro-Magnetics) applications with their own integrative platform ANSYS Electronics Desktop (ansysedt). Currently at LRZ both Maxwell-2D/-3D (simulation of static electro-magnetic fields) as well as HFSS (propagation of low and high frequency electro-magnetic waves) are provided for parallel simulations on LRZ Linux Clusters. This parallelization approach based on the SLURM scheduler is still in a testing phase.
Software Producer
ANSYS, Inc.
Southpointe
275 Technology Drive
Canonsburg, PA 15317
Telephone: (724) 746-3304
Software Vendor and Channel Partner
CADFEM GmbH
Marktplatz 2 85567 Grafing
Tel.: 08092/7005-0
Fax: 08092/7005-77
Email:
General information: info@cadfem.de
Information on seminars: seminar@cadfem.de
The ANSYS Applications
ANSYS CFX (Fluid Dynamics)
ANSYS Fluent (Fluid Dynamics)
ANSYS HFSS
ANSYS LS-Dyna (FEA & Multiphysics)
ANSYS Lumerical (Optics / Photonics)
ANSYS Maxwell-2D/-3D
ANSYS Mechanical (CSM)
ANSYS Meshing & ICEM/CFD
ANSYS Postprocessing
ANSYS System Coupling
Troubleshooting the Access to ANSYS Academic Research/Teaching Licenses
Licensing
The Leibniz Supercomputing Center (LRZ) provides access to the ANSYS Software, which is limited by the number of concurrently available software licenses as well as by the terms and conditions for the usage of the ANSYS Academic Research & Teaching Licenses. These licenses may only be used for research and teaching by the institutes of the Bavarian Academy of Sciences and the Munich Universities, i.e. TUM, LMU, HM. The licenses allow the usage of ANSYS products for non-commercial purposes only.
ANSYS Academic Teaching licenses are provided in the Munich Scientific Network (MWN) free of charge. The main difference of the ANSYS Academic Teaching licenses in comparison to the Free ANSYS Student Software is the full availability of all ANSYS products in the software installation, which are included in the full ANSYS Academic Research license as well. Examples are the mesh generation tool ICEM/CFD Hexa/Tetra and the ANSYS OptiSlang and Rocky DEM products which are available with the ANSYS Academic Teaching license, but not with the Free ANSYS Students Software. Furthermore the ANSYS Academic Teaching licenses do not have resource limitations on mesh cell count, number of geometry entities and the like.
In case of difficulties with the ANSYS Software installation from the provided LRZ installation resourses and in using the ANSYS Academic Teaching licenses within the MWN network, please read the following troubleshooting document first, before you are submitting an LRZ Service Request.
The LRZ is providing full capability ANSYS Academic Research Multiphysics licenses and a larger number of High-Performance Computing (ANSYS HPC) licenses free of charge for the usage on LRZ Linux Clusters, our supercomputer SuperMUC-NG and Remote Visualization Systems (RVS). For academic research purposes on local compute resources the institutions of the Munich Universities (TUM, LMU, HM) can purchase ANSYS Academic Research and ANSYS HPC licenses from the LRZ pool of an ANSYS Campus license for an annual license fee. Further information on ANSYS software licensing at LRZ is available in the corresponding documentation at https://doku.lrz.de/display/PUBLIC/ANSYS-Lizenzen. Additional licenses purchased and owned by other institutions or university departments and which are aimed for use at the LRZ Linux Clusters and supercomputers can be hosted at LRZ (license server hosting).
Important Note:
Prior to use the ANSYS Software on any LRZ computer system you need to register with LRZ user support in order to obtain an entry of your LRZ user-ID in the ANSYS license server operated and managed by LRZ. Only afterwards a user will be aible to checkout a license for any of the ANSYS software applications from the LRZ license server. Granted access to LRZ Linux Cluster and/or SuperMUC-NG does not automatically imply a granted access to any licensed software. Access to licensed software is granted on individual request!
For local installation of the ANSYS software please consult our software download page. To access this site you need a password that you will receive by contacting the LRZ user support. Furthermore, the most contemporary version of the ANSYS software (Windows, Linux) can be downloaded directly from the ANSYS Customer Portal after registering with ANSYS, Inc. as a software user.
License Preference Settings
ANSYS allows users to make there license server / license preference settings prior to start any of the ANSYS applications by using the following ANSYS license administration GUI, e.g. on a Windows installation:
ANSYS Licensing Settings <Release>
in order to adjust their license preference settings. In the GUI under the menue option "License Preference Settings" is an overview provided over the different ANSYS software versions and available licenses. You can set your prefered usage profile there. Please note, that the license preferences are ANSYS version specific, i.e. they have to be specified and adjusted for each individual ANSYS version.
ANSYS Free Students Edition vs. ANSYS Academic Teaching & Research Campus Version
Essentially one need to distinguish between the following three license types (mainly focussing here on ANSYS CFD and ANSYS Mechanical), which are inter-linked with the corresponding software installation and applying specific software limitations. All ANSYS software licenses are subject to the governing ANSYS licensing terms & conditions. All information in the following table is given with absolutely no warranty and is subject to change by ANSYS Inc. in any future version of the software, so that the following information is provided here for ANSYS 2021.R1 to 2024.R1 releases to our best knowledge.
For ANSYS Software Releases >=2021.R1:
| Topic | ANSYS Free Students Edition | ANSYS Academic Teaching Mechanical & CFD & EM | ANSYS Academic Research Mechanical & CFD & EM LS-Dyna, OptiSlang, Zemax, Rocky-DEM, etc. |
|---|---|---|---|
| Included parallel licenses | run on <=2 CPU cores, no HPC | run on <=4 CPU cores, no HPC | run on 4 CPU cores + HPC licenses |
| Limitations in mesh / element cell count | Limitations to number of geometric entities and mesh cell counts apply | No limitations | No limitations |
| LRZ License Server | not applicable | licansysteach.lrz.de | licansys.lrz.de |
| License Feature Name | (aa_t_i ??) | advanced_meshing (tp:at) | advanced_meshing (tp:a) |
| License Availability | free & local, | 5000, | 500 + 16.384 HPC; |
Actual Version
Since January 2024 the actual version of the ANSYS Software Package is named "ANSYS 2024.R1", while in accordance to the default installation path the software is named v241. This new version ANSYS 2024.R1 has been made the default version / software module of the ANSYS software on the LRZ Linux Clusters CoolMUC-2/4 and on SuperMUC-NG, while at the same time older versions of the ANSYS software are still supported and provided as separate modules in the module system. At the same time the outdated ANSYS software versions 2020.R1 and 2020.R2 have been deprecated as a LRZ software module. Furthermore after some updates to the SuperMUC-NG SLES15 operating system some older versions of ANSYS CFX and Fluent stopped working properly. The modules of corresponding versions show a corresponding comment and point to newer and contemporary versions of the ANSYS solvers which are known to work properly on this system.
SSH User Environment Settings
As described in the LRZ documentation "Using Secure Shell on LRZ HPC Systems → LRZ internal configuration", some applications require special SSH keys for the cluster internal communication. ANSYS Fluent, and potentially other packages from the overall ANSYS software portfolio as well, belongs to this class of applications.
Therefore ANSYS software users should follow the SSH setup steps as outlined here. This section is only relevant for setting up ssh to work without passwords (and without ssh-agent) inside the cluster. This is needed for e.g. batch process startup in some MPI implementations. None of the keys (public or private) generated here should be transferred outside the cluster, since this may incur security problems. The following commands must therefore be executed on the LRZ target system, i.e. one of the Linux Cluster or SuperMUC-NG login nodes.
| Commands | Remarks |
|---|---|
cd $HOME mkdir ~/.ssh | Make in your $HOME directory a hidden subfolder for the SSH-Keys and configuration files (if not existing yet). |
chmod 700 ~/.ssh | Make this .ssh subfolder accessible only for yourself, not for the project group and not for others. This is a mandatory setting. With different permission settings SSH connections might not work correctly on LRZ Linux Clusters. |
cd ~/.ssh ssh-keygen -t ed25519 -a 100 | Generate an ed25519 key. The command will respond with Generating public/private ed25519 key pair. to which you respond by typing the ENTER key. Next, you are prompted for a passphrase Enter passphrase (empty for no passphrase): to which you should respond by typing ENTER twice (no passphrase). |
cd ~/.ssh cat id_ed25519.pub >> authorized_keys | Add internal public SSH key to the list of authorized keys. |
| Testing of the SSH user environment is no longer possible, since 2FA is required between Linux Cluster login nodes as well. The passphrase-free SSH connection is only available between Linux Cluster compute nodes. |
Getting Started
LRZ users are adviced NOT to use Linux Cluster or SuperMUC-NG login nodes for any kind of ANSYS simulations which have the potential to put some heavy processor load on these login node systems or which are potentially consuming large amounts of memory in order not to disturb other cluster users. For such purposes large memory nodes are provided e.g. in interactive cluster queues. It is even more recommended to use for e.g. ANSYS meshing, pre- and postprocessing purposes the LRZ Remote Visualization Systems.
Once you are logged into one of these systems, you can check the availability (i.e. installation) of ANSYS software by:
> module avail ansys
Load the prefered ANSYS version environment module, e.g.:
> module load ansys/2024.R1
Due to the rather special requirements of ANSYS Workbench and its components for the underlying operating system configuration and graphical environment, this part of the ANSYS software can only be supported on the LRZ Remote Visualization System. Once the user has obtained access to one of the RVS machines, the software can e.g. be invoked by:
> module load wb/2024.R1 > vglrun runwb2
For parallel execution of the ANSYS individual software applications (ANSYS CFX, ANSYS Fluent, ANSYS Mechanical, ANSYS Maxwell, HFSS, LS Dyna, etc.) in the LRZ Linux Cluster and SuperMUC-NG batch queuing systems (SLURM) please refer to the provided application-specific subpages of this documentation. If the software of your choice is not yet listed as an available software module, please contact the LRZ application support team.
Since ANSYS Workbench unfortunately does not support SLURM as a scheduler or batch queuing system, it is currently not possible at the compute resources of LRZ to execute ANSYS Workbench projects in distributed parallel mode. This would require an ANSYS Remote Solver Manager (RSM) instance being able to access the SLURM queues, which is currently not a functionality supported by the ANSYS software. The same applies for the currently not possible parametric studies in distributed parallel mode, which makes use of the parameter manager and RSM as well. Coupled simulations of different ANSYS software components, like e.g. fluid-structure interaction simulations involving ANSYS CFD and ANSYS Structural Mechanics, can be executed in distributed parallel mode by applying ANSYS System Coupling in scripted mode and by using appropriately formulated SLURM scripts. Users who are interested in running such coupled simulations are adviced to get in contact with the LRZ Application Support Team through the LRZ Service Desk.
Documentation
The ANSYS documentation can be accessed from within ANSYS Workbench or from within each individual ANSYS application through the provided help menue. Furthermore ANSYS documentation is provided for registered users through the ANSYS Customer Portal.
On LRZ systems with available ANSYS installation the online documentation can further be accessed by:
> module load ansys/2024.R1 > anshelp
Mailing List for ANSYS Users
Users of the LRZ ANSYS licenses may join the mailinglist ansys@lists.lrz.de. This list will provide the user with important information about ANSYS software installation at LRZ (e.g. announcement of new versions, system configuration changes or important updates).
More information about ANSYS mailinglist at LRZ may be found here.
User Support
In case of any observed issues in the usage of the ANSYS software on LRZ managed compute resources or any arising questions, please feel free to contact the LRZ support. Please submit your LRZ Service Request with a clear specification of the encountered issue with the ANSYS software, e.g. by indicating in the request subject "ANSYS Fluent problem: ....". This will help the LRZ operators to assign your support request to the most appropriate LRZ staff to assist you.