VisIt
At the moment the VisIt infrastructure does not work correctly for compatibility issues with our MFA system. We are aware of it and discussing possible solutions internally.
We are sorry for the inconvenience.
Overview
From (VisIt HomePage) :
VisIt is an Open Source, interactive, scalable, visualization, animation and analysis tool. From Unix, Windows or Mac workstations, users can interactively visualize and analyze data ranging in scale from small (101 cores) desktop-sized projects to large (105 cores) leadership-class computing facility simulation campaigns. Users can quickly generate visualizations, animate them through time, manipulate them with a variety of operators and mathematical expressions, and save the resulting images and animations for presentations. VisIt contains a rich set of visualization features to enable users to view a wide variety of data including scalar and vector fields defined on two- and three-dimensional (2D and 3D) structured, adaptive and unstructured meshes. Owing to its customizable plugin design, VisIt is capable of visualizing data from over 120 different scientific data formats (see this partial list). See a table of key features and a complete table of the tool's features.
This is the user's documentation of the VisIt module present on the Linux Cluster and SuperMUC-NG supercomputer at LRZ. The module is usable either in batch mode through VisIt python scripting interface, or through interactive GUI accessible from the users' local machines, via remote host configuration.
Remote GUI usage
VisIt is configured to launch jobs on either machine's regular queues. That sets up a remote engine that connects automatically to your local open GUI, allowing access to the remote data with your local visit configuration. No dedicated server is used.
Publications acknowledgment
Visualizations displayed in this work have been produced via the VisIt remote GUI provided by the Leibniz Supercomputing Centre (LRZ) on the [SuperMUC-NG]/[Linux Cluster] system. We thank LRZ and its VisIt maintainer Dr. Salvatore Cielo for keeping this service optimized.
To set it up, please follow these configuration steps.
Check to have a working SSH connection
You will only need a machine capable of connecting to your server (e.g. check this page for SuperMUC-NG) running a local instance of VisIt FOR ONE OF THE VERSION present on the machine (e.g. 3.2). VisIt essentially uses the SSH protocol under the hood, so all the configurations needed for a Secure Shell connection apply. Setting up an SSH key would speed up connection, but beware of the restrictions for SuperMUC-NG.
The first step is testing to have a working SSH connection over command line interface before proceeding with the remote GUI setup.
After that, you won't need a CLI connection anymore; host configuration and remote connection will happen through your local VisIt GUI.
Download the provided host configuration files
Once you have a working SSH connection, in order to set the remote engine you just need a configuration file: please save the following files under your local ~./visit/hosts directory (C:\Users\<my user name>\Documents\VisIt\hosts on Windows): host_linux_cluster.xml and host_supermuc_ng.xml.
Alternatively, since version 3.1.1 you should be able to access those from the list of known remote hosts: simply open your local VisIt, select Options → Host Profiles ... → Remote Profiles → Update. You may have to check the dev branch. Then select the file for either SuperMUC-NG or the Linux cluster and click on Import. The files presented on this page are updated more frequently, though.
FInally, if you need a custom connection AND you know what you are doing, you may also fill or edit all Options → Host Profiles ... → Host Settings tabs manually by yourself.
Customize the host configuration with your LRZ account
In either case you need to update the host configuration with your account data. Enter you username in the Options → Host Profiles ... → Host Settings → Username field. You will have the option to change username via a pop-up window at connection time.
For SuperMUC-NG you also have to enter your SLURM account number (i.e. your LRZ project number), just like for any regular job submission via SLURM. Please add that under Options → Host Profiles ... → Launch Profiles → Advanced → Launcher Arguments.
You may have to select Options → Save Settings on the menu to make such edits permanent. All the other parameters should be set correctly.
Launching a remote engine
At this point you can launch a remote host simply by opening a file on the remote server in the Open dialog box, a very intuitive procedure. Select the desired host from the Host menu; you will then be connected to your remote filesystem. Select the file you want to open; the following dialog box will ask to select a job profile.
The provided hostfiles contain an exhaustive collection of launch profiles, corresponding exactly to the different partitions of the linux cluster and queues of SuperMUC-NG, with both salloc and sbatch/mpiexec; just select the desired job class, based on needs and machine availability (you can check that with the sinfo command). You will be able to select the number of processes (i.e. MPI tasks), nodes and the timeout of your job in the next popup.
Using 2 or 4 MPI tasks per node is a reasonable choiche when using OSPRay volume rendering (which is STRONGLY advised, since LRZ machines do not include accelerators at the time of writing). For other rendering methods, use more tasks (e.g. all or half the cores per node of the machine you are using) but expect inferior perfomance.
The default values of these parameters can be customized in the Options → Host Profiles ... → Launch Profiles → Parallel tab. Select Options → Save Settings on the menu to make such edits permanent.
Then proceed using VisIt normally. At this point or at the first Draw operation the job will be queued and your remote engine will be made available for you.
Warnings when using the remote GUI
The status of the launched job (e.g. start time, residual time, all other parameters) can be verified at anytime by logging into the server, via squeue or sview.
Closing a remote engine
On either machine, it is good practice to double-check that the remote engine job is actually concluded after use.
You can check the status of your engine from the File→Compute Engine menu. When no computation is running, just click on the Close Engine button to cancel the remote job.
You can also log onto the remote machine via command line SSH and end it manually with the scancel command. Only at thi point you can close the VisIt GUI.
Batch jobs and Python Scripting
For the largest jobs that cannot fit into interactive queues, users must submit scripts with VisIt python Command Line Interface via batch SLURM jobs, instead of using the GUI. Python scripting in VisIt is as complete and powerful as the GUI. The manuals contain an exhaustive guide. Assistance, including examples and tips for python can also be found on this page of the VisIt users' forum. Here we showcase the basic workflow on LRZ machines, provide some working examples, and address some common problems.
To get started, either on the login nodes or in SLURM scripts, you can just type module load visit while module avail visit will list all available versions.
Launching scripted jobs
We provide sample job scripts for SLURM files specific for LRZ machines.
Tips and tricks
Shortcuts to VisIt's python API.
The following two options suffice in most cases, and may save you from getting your hands too dirty:
- When generating a movie with the GUI, the user can select
Later, tell me the command to runwhen deciding how to allocate the resources for that rendering. This command can be launched in batch, although it relies on the user's filesystem (session, preferences, ...).
Also, the session should contain a parallel engine identical to the one that needs to be used; so you cannot use this feature to move your job to a larger allocation. But you can of course reserve more time, which could be impractical for the interactive job (long queue times). - The Control → Command Menu in the GUI VisIt is capable of recording and translating to python all the actions performed by the user between the push of record and stop buttons. Besides reverse-engineering all the syntax, one can record automatic actions, such as the camera positions obtained automatically in Spin Mode in the window toolbar.
One can thus produce movies or entire scripts in this way, without using taxing Keyframing.
Limitations of Control→Command
- Unlike the entries the Option menu, those of Preferences have no python equivalent. So you can't set these via scripting, you have to save them in the session used.
When performing a Volume Rendering, each additional color and opacity control point (besides the one in the default colormap) has to be initialised manually (this is a bug!):
VolumeAtts.colorControlPoints.AddControlPoints(ColorControlPoint()) # Always [...] VolumeAtts.opacityControlPoints.AddControlPoints(GaussianControlPoint()) # Only when using gaussian transfer functions
Some useful commands to have at hand
You are told how to execute queries with the Control→Command approach; yet in batch the result is neither collected nor printed on screen by default. After each query, you then need to add:
my_variable = GetQueryOutputValueimport time; time.clock()gives you an effective python timer, measuring wall clock time. But you can use any python timer. Visit has its own timers as well (check withvisit --fullhelp).DrawPlots()will draw all active plots,SaveWindow()will print the image according to the setSaveOptions
Important difference between windowed and `-nowin` mode
When in windowed mode, DrawPlots() will BOTH initialize the scene AND render it. A scalable rendering is then REPEATED during SaveWindow(), unless File → Set Save Option → Screen Capture is ticked.
That may double your workload when scripting in windowed mode. In (-cli) -nowin mode, as it is always the case in batch, only SaveWindow() will trigger the scalable rendering (no screen capture is possible).
Sample python scripts
The following scripts may be used for testing and as a basis to produce your own. Many of the following have been generated by Control→Command and some edits. Input test files are provided in the VisIt webpage.
Never forget to invoke quit() at the end of each script, or the job may not terminate (wasteful for your own compute budget!).