Navigation and service

Usage of PETSc on JURECA

PETSc version 3.11.1 has been installed on JURECA in several versions. All versions for Intel compilers use mkl and include several other libraries.
PETSc/3.11.1 is configured with standard double precision and integer values
PETSc/3.11.1_debug is configured with --with-debugging=1
PETSc/3.11.1_int8 is configured with --with-64-bit-indices=1
PETSc/3.11.1_int8_debug is configured with --with-64-bit-indices=1 and --with-debugging=1
PETSc/3.11.1_complex is configured with --with-scalar-type=complex
PETSc/3.11.1_complex_debug is configured with --with-scalar-type=complex and --with-debugging=1

The debug versions are only available in Stages/Devel-2019a
PETSc version 3.13 is available in Stages/Devel-2019a. To access those version say

ml use $OTHERSTAGES
ml load Stages/Devel-2019a

Preparations necessary to use PETSc on JURECA

First you have to load the PETSc version you want, for example:

module load intel-para
module load PETSc/3.11.1
With that command the variables PETSC_DIR and PETSC_ARCH are set.

If you don't want to use the PETSc makefiles make sure that your makefile contains the statements

include ${PETSC_DIR}/ ${PETSC_ARCH}/lib/petsc/conf/petscvariables
include ${PETSC_DIR}/ ${PETSC_ARCH}/lib/petsc/conf/petscrules

Examples

Examples are available in $PETSC_DIR/src.

To run for instance ex1 from $PETSC_DIR/src/vec/vec/examples/tutorials you have to do the following:

# load the PETSc you want

module load intel-para
module load PETSc/3.11.1

# copy ex1.c and the makefile to the current directory

cp $PETSC_DIR/share/petsc/examples/src/vec/vec/examples/tutorials/ex1.c .
cp $PETSC_DIR/share/petsc/examples/src/vec/vec/examples/tutorials/makefile .

# compile and link the example code
make ex1

To execute the example on 2 processors you have to write an batchfile runex1 with the following commands:

#!/bin/bash -x
#SBATCH --nodes=1
#SBATCH --ntasks=2
#SBATCH --partition=batch
srun -np 2 ex1

To submit the batchfile to the slurm batchsystem say
sbatch batchfile

In the sources of the examples you find a section //TEST in the end where you can see how the examples is to be called.