Compiling OpenFOAM 3.0.1 on ARCHER

This page outlines the procedure we used for building OpenFOAM 3.0.1 on ARCHER. We downloaded the OpenFOAM-3.0.1 and ThirdParty-3.0.1 sources from the website and unpacked them on the /work filesystem on ARCHER.

Modifications

To build OpenFOAM we needed to make some small modifications to the configuration files. Firstly, we change the following lines in etc/bashrc:

Set this to the location of your OpenFOAM directory
foamInstall= 
....
Set the WM_MPLIB to CRAY_MPICH
export WM_MPLIB=CRAY_MPICH

Secondly, etc/config/settings.sh was updated as follows:

    x86_64)
        case "$WM_ARCH_OPTION" in
        32)
            export WM_COMPILER_ARCH=64
            export WM_CC='cc'
            export WM_CXX='CC'
            export WM_CFLAGS='-m32 -fPIC'
            export WM_CXXFLAGS='-m32 -fPIC'
            export WM_LDFLAGS='-m32'
            ;;
        64)
            WM_ARCH=linux64
            export WM_COMPILER_LIB_ARCH=64
            export WM_CC='cc'
            export WM_CXX='CC'
            export WM_CFLAGS='-m64 -fPIC'
            export WM_CXXFLAGS='-m64 -fPIC'
            export WM_LDFLAGS='-m64'
            ;;

.....

and add a new MPI library configuration before MPICH) as follows:

CRAY_MPICH)
    export WM_MPLIB=CRAY_MPICH
    export FOAM_MPI=cray_mpich
    export MPI_ARCH_PATH=$MPICH_DIR
    export LD_LIBRARY_PATH=$MPI_ARCH_PATH/lib:$LD_LIBRARY_PATH
    export WM_OPTIONS=$WM_ARCH$WM_COMPILER$WM_PRECISION_OPTION$WM_COMPILE_OPTION
    ;;

We also needed to change this line in etc/config/scotch.sh:

export SCOTCH_ARCH_PATH=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER/$SCOTCH_VERSION

Next we changed the c and c++ files in wmake/rules/linuxGcc replacing gcc with cc and g++ with CC. We also added two files, called mplibCRAY_MPICH and mplibCRAY_MPICH2 in the same directory with this content:

PFLAGS     = -DMPICH_SKIP_MPICXX
PINC       = -I$(MPI_ARCH_PATH)/include
PLIBS      = -L$(MPI_ARCH_PATH)/lib -lmpich

The same changes were made in the wmake/rules/linux64Gcc directory.

Some changes were also necessary in ThirdParty-3.0.1 directory. We updated ThirdParty-3.0.1/etc/wmakeFiles/scotch/Makefile.inc.i686_pc_linux2.shlib-OpenFOAM-64Int32 replacing gcc with cc and g++ with CC.

Build OpenFOAM

We used the following modules and environment variables when building:

module load PrgEnv-gnu
module load zlib
module load boost
module load cmake
export CRAYPE_LINK_TYPE=dynamic

To build OpenFOAM you then need to source to etc/bashrc in the OpenFOAM directory:

source etc/bashrc

And then run Allwmake from the OpenFOAM directory:

./Allwmake