Page Content

Introduction

The purpose of the MATLAB Compiler Runtime is to run MATLAB standalone applications on systems without a MATLAB installation. The standalone application is compiled by the MATLAB Compiler which requires a licensed version of MATLAB. In contrast, the standalone application should be executable on any system providing a MCR installation, and without the need of a MATLAB license. On LRZ systems we provide all MCR versions which correspond to the installed MATLAB versions.

MCR-based standalones may run embarrassingly parallel. So-called MPMD (Multiple-Program Multiple-Data) or job-farming techniques are used to run many processes of a program simultaneously and independently, that is, without the need of communication. Typical applications are:

parameter studies,
independent processing of large datasets which are split into subsets.

Example: Basic Parallel MPMD Application with MATLAB MCR

Prerequisites

The job must be parallelizable. That is, the work can be divided into independent subtasks (e. g. embarrassing parallelization).
The compilation of the application's MATLAB script(s) must be feasible. That is, some MATLAB components may not be compiled or my not be available in the MCR.

Now what? A brief summary

Your MATLAB script needs to be serial with respect to PCT parallelization. If necessary, convert your MATLAB script, e. g. by removing parfor loops. The work of the parfor loops will be done by the MPMD concept. MATLAB's intrinsic multithreading is still usable.
Create a standalone application by compiling your script(s) within MATLAB.
The execution of the standalone application will require the MATLAB Compiler Runtime (MCR).
In your Slurm job script, run the standalone application via MPI. MPI will distribute the application to all resources you have allocated in the job. That is, multiple instances of your application will run simultaneously and independently.
How does each instance of my application know, what task to do?
MPI assigns an ID to each instance (in case of Intel MPI stored in environment variable PMI_RANK), which can be evaluated within the job script or even the MATLAB script.

General Workflow

The following table depicts the procedure from creating the standalone application to the submission of a Slurm job.

Step	Comment
1. Create folder structure of example
mcr_mpmd_example ├── bin ├── data ├── mcr_build.m ├── mcrjob.cmuc2.slurm ├── output └── src ├── dep1 │ └── matmul_serial.m ├── dep2 │ └── plot_save_matrix.m └── mcr_run.m	-> base path -> path for executable file -> path to data files created by application (log files, plots) -> build script running MATLAB Compiler -> Slurm script to run MCR job -> directory for Slurm job output files -> application sources: m-files -> dependencies -> serial matrix-matrix multiplication -> dependencies -> plot result -> run script of application ("main" function)
2. Prepare application sources
function mcr_run(size_A_str, size_B_str) %=============================================================================== % MATLAB MPMD EXAMPLE: RUNTIME CODE % % This function will be compiled by Matlab Compiler. This function % - summarizes some configuration steps, e. g. defining dependencies % - calls the Matlab function implemented by the user % % This function calls matmul_serial.m (matrix-matrix-multiplication with % threading support) as dependency 1 and a plotting function as dependency 2. % % Each MPI process will run this code by evaluating its own rank. %=============================================================================== %% ============================================================================= % Each process will use the MPI rank as task identifier. % Get my MPI rank from the MPI environment. %% ============================================================================= myrank = str2num(getenv('PMI_RANK')); %% ============================================================================= % Create a log file for this task to avoid cluttered output of all MPI tasks in % the Slurm job output file. %% ============================================================================= fp = fopen(sprintf('data/JOB_%s_TaskID_%d.log', getenv('SLURM_JOB_ID'), myrank), 'w'); %=============================================================================== % All processes say hello. % Show some information, also useful for troubleshooting %=============================================================================== fprintf(fp, 'MPI process rank=%d: I run on compute node %s and will handle task %d.\n', ... myrank, getenv('HOSTNAME'), myrank); fprintf(fp, 'MATLAB temporary directory = %s\n', tempdir); fprintf(fp, 'MCR_CACHE_ROOT = %s\n', getenv('MCR_CACHE_ROOT')); fprintf(fp, 'MCR root dir = %s\n', ctfroot); %=============================================================================== % Run user code. % Scale result by taskID to show MPMD effect. %=============================================================================== % get sizes of input matrices for computation of C = AB from commandline % arguments size_A = str2num(size_A_str); size_B = str2num(size_B_str); [C, comptime] = matmul_serial(size_A, size_B); C = C myrank; plot_save_matrix(C, ['data/matmul_result_taskID_' num2str(myrank)]); fprintf(fp, '(rank=%d) computation time = %.4f s.\n', myrank, comptime); fprintf(fp, '\tcheck (mean of elements scaled by MPI rank): %f\n', mean(C(:))); fprintf('myrank=%d: check (mean of elements of C = A*B scaled by MPI rank): %f\n', myrank, mean(C(:))); %% ============================================================================= % close log %% ============================================================================= fclose(fp);	This is the "main" function. It will be compiled by MATLAB Compiler. In this example, mcr_run.m will act as purely serial standalone application. It can be adjusted to any usecase. Main purposes are: organizing the parallel environment, calling application code, displaying debug information. Within the scope of the job-farming approach, MPI will be used to spawn multiple instances of compiled mcr_run.exe. Therefore, each instance needs to identify its individual sub-task. Consequently, we need one important modification. The easiest way: Each instance obtains its MPI rank from the MPI environment. The rank will be used to identify sub-task(s) assigned to each instance. The MPI rank is unique. In general, the range is 0, 1, ..., ("number of Slurm tasks" - 1).
function [C, comptime] = matmul_serial(size_A, size_B) %=============================================================================== % MATLAB EXAMPLE: SERIAL HELLO WORLD % -> matrix-matrix multiplication C = AB % % INPUT % size_A, size_B ... 2-element row vectors defining sizes of A and B % OUTPUT % C ................ result % comptime ......... computation time (matrix product only) %=============================================================================== %=============================================================================== % Check input %=============================================================================== if nargin~=2 error('Invalid number of input arguments!'); end if size_A(2)~=size_B(1) error(sprintf('Dimension mismatch of A (%d columns) and B (%d rows)!',... size_A(2), size_B(1))); end %=============================================================================== % Work %=============================================================================== % Hello message from compute node fprintf('Hello from MATLAB process PID=%d running on node %s!\n',... feature('getpid'),... strtrim(evalc('system(''hostname'');'))); % generate well-defined matrices NA = prod(size_A); NB = prod(size_B); A = reshape( linspace( 1,NA, NA), size_A ); B = reshape( linspace(NB, 1, NB), size_B ); % compute tic; C = AB; comptime = toc; fprintf('serial computation of matrix-matrix product:\n'); fprintf('\ttime = %.2f s\n', comptime); `function plot_save_matrix(A, fname) %=============================================================================== % Plot matrix A and save figure to file fname. %=============================================================================== fig = figure('visible','off'); imagesc(A); colorbar; saveas(gcf, [fname '.png']);`	dependencies of application mcr_run.m: matmul_serial.m: similar to example shown in section Common Batch Jobs plot_save_matrix.m: auxiliary plotting function
3. Check build script
function mcr_build(srcpath, mainfunc, threadingmode) %=============================================================================== % MATLAB MCR EXAMPLE: BUILD CODE % % This function calls Matlab Compiler to build an executable from user code. % % INPUT % srcpath ............ base path to sources (m-files) % mainfunc ........... name of main M-function to be compiled % -> file name of function only (no path) % -> without extension ".m" % -> according m-file must be located in srcpath % threadingmode ...... set multithreading mode (optional) % -> 'disableThreading', 'enableThreading' (default) %=============================================================================== %=============================================================================== % Construct basic compiler command. %=============================================================================== if nargin < 3 \|\| (nargin==3) && strcmp(lower(threadingmode), 'enablethreading') cmd = ['mcc' ... ' -m' ... ' -R -nodisplay' ... ' -o ' mainfunc ... ' ' srcpath '/' mainfunc '.m']; elseif (nargin==3) && strcmp(lower(threadingmode), 'disablethreading') cmd = ['mcc' ... ' -m' ... ' -R -nodisplay' ... ' -R -singleCompThread' ... ' -o ' mainfunc ... ' ' srcpath '/' mainfunc '.m']; end %=============================================================================== % Find dependencies and add them to the command. % -> all subdirectories in srcpath are assumed to be dependencies %=============================================================================== fcont = dir(srcpath); for d = fcont' if d.isdir && ~strcmp(d.name,'.') && ~strcmp(d.name,'..') cmd = [cmd ' -a ' fullfile(srcpath, d.name)]; end end cmd = strrep(cmd, '//', '/'); %=============================================================================== % Compile! %=============================================================================== fprintf('Compiling %s via following mcc-command:\n\t%s\n\n', mainfunc, cmd); eval(cmd); movefile(mainfunc, ['./bin/' mainfunc '.exe']);	This is a generic build script which can be used for any use case. It should not be necessary to modify it.
4. Compile application and set options to disable/enable multithreading in compiled application
1. Load Matlab module > module switch spack/23.1.0 > module load <MATLAB MODULE> # EDIT HERE (see supported releases) 2. Compile (multithreading in application disabled) > matlab -nodisplay -r "mcr_build('./src/','mcr_run','disableThreading');quit" 2. Compile (multithreading in application enabled) > matlab -nodisplay -r "mcr_build('./src/','mcr_run','enableThreading');quit" 2. Compile (multithreading in application enabled) > matlab -nodisplay -r "mcr_build('./src/','mcr_run');quit"	The "standalone" application will be created. In this example we will get "mcr_run.exe". This step can be done on any computer providing a Matlab installation and all necessary toolboxes. The MATLAB release version used for compilation has to match the MCR version at runtime! The update level may be different. If you work on the Linux Cluster, please load the desired MATLAB module and then call the command to compile. It is allowed to run this compilation procedure on the login nodes.
5. (A) Prepare Slurm batch script
> module switch spack/23.1.0 > module avail matlab-mcr #!/bin/bash #SBATCH -o ./output/%x.%j.%N.out #SBATCH -e ./output/%x.%j.%N.err #SBATCH -D ./ #SBATCH -J mcr_mpmd #SBATCH --get-user-env #SBATCH --export=NONE #SBATCH --clusters=cm2_tiny #SBATCH --partition=cm2_tiny #SBATCH --nodes=1 #SBATCH --tasks-per-node=10 #SBATCH --cpus-per-task=1 #SBATCH --time=0:10:00 #=============================================================================== # USER-DEFINED INPUT #=============================================================================== WORKDIR=MY_WORKING_DIR # Edit here -> set path name to working directory APPNAME=./bin/mcr_run.exe # Edit here -> name MCR compiled executable MCRMODULE=MATLAB-MCR-MODULE # Edit here -> set MCR module SIZE_MATRIX_A="[2000 1000]" SIZE_MATRIX_B="[1000 8000]" #=============================================================================== # SET MODULES AND ENVIRONMENT VARIABLES #=============================================================================== module load slurm_setup # Load Matlab Compiler Runtime. module rm matlab module load $MCRMODULE export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/runtime/glnxa64 export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/bin/glnxa64 export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/sys/os/glnxa64 export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/sys/opengl/lib/glnxa64 # Set MCR cache path to SCRATCH. Don't use HOME directory! export MCR_CACHE_ROOT=${SCRATCH}/MCR_MPMD_JOBID${SLURM_JOB_ID} # Set general temp. path (some MATLAB releases want to have TMP). export TMP=$SCRATCH #=============================================================================== # RUN APPLICATION #=============================================================================== cd $WORKDIR mpiexec $APPNAME "${SIZE_MATRIX_A}" "${SIZE_MATRIX_B}"	First panel: show available MCR modules on LRZ systems Second panel: Job script to run application on CoolMUC-2 The MATLAB release version used for compilation has to match the MCR version at runtime! The update level may be different. In the script, you need to load a Matlab MCR module, not Matlab!
5. (B) Adjust Slurm batch script on SuperMUC-NG
#!/bin/bash #SBATCH -o ./output/%x.%j.%N.out #SBATCH -e ./output/%x.%j.%N.err #SBATCH -D ./ #SBATCH -J mcr_mpmd #SBATCH --get-user-env #SBATCH --export=NONE #SBATCH --partition=test #SBATCH --nodes=1 #SBATCH --tasks-per-node=10 #SBATCH --cpus-per-task=1 #SBATCH --time=0:10:00 #SBATCH --account=MY_PROJECT_ID ##SBATCH --ear=off # switch on/off as needed #=============================================================================== # USER-DEFINED INPUT #=============================================================================== WORKDIR=MY_WORKING_DIR # Edit here -> set path name to working directory APPNAME=./bin/mcr_run.exe # Edit here -> name MCR compiled executable MCRMODULE=MATLAB-MCR-MODULE # Edit here -> set MCR module SIZE_MATRIX_A="[2000 1000]" SIZE_MATRIX_B="[1000 8000]" #=============================================================================== # SET MODULES AND ENVIRONMENT VARIABLES #=============================================================================== module load slurm_setup # Load Matlab Compiler Runtime. module rm matlab module switch spack/23.1.0 module load $MCRMODULE export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/runtime/glnxa64 export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/bin/glnxa64 export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/sys/os/glnxa64 export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/sys/opengl/lib/glnxa64 # Workaround on SuperMUC-NG to link to missing libraries export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lrz/sys/spack/release/23.1.0/opt/skylake_avx512/libxt/1.1.5-gcc-dzla4qt/lib # Set MCR cache path to SCRATCH. Don't use HOME directory! export MCR_CACHE_ROOT=${SCRATCH}/MCR_MPMD_JOBID${SLURM_JOB_ID} # Set general temp. path (some MATLAB releases want to have TMP). export TMP=$SCRATCH #=============================================================================== # RUN APPLICATION #=============================================================================== cd $WORKDIR mpiexec $APPNAME "${SIZE_MATRIX_A}" "${SIZE_MATRIX_B}"	Workaround for "missing library" error Jobs (mainly on SuperMUC-NG) may show the following error: libXt.so.6: cannot open shared object file: No such file or directory As a workaround you may modify the LD_LIBRARY_PATH environment variable in your job script.
6. Run job
For example: > sbatch mcrjob.supermuc-ng.slurm

Example

Construction site