Quick Start

Set up a COMPASS repository: for beginners

To begin, obtain the master branch of the compass repository with:

git clone git@github.com:MPAS-Dev/compass.git
cd compass
git checkout legacy
git submodule update --init --recursive

The MPAS repository is a submodule of COMPASS repository. For example, to compile MPAS-Ocean:

cd E3SM-Project/components/mpas-ocean
# load modules (see machine-specific instructions below)
make gfortran

For MALI:

cd MALI-Dev/components/mpas-albany-landice
# load modules (see machine-specific instructions below)
make gfortran

compass conda environment

The compass conda environment includes all the python libraries needed to run compass scripts. It is maintained on our standard machines (Grizzly, Badger, Anvil, Compy and Cori). Here are the commands for each machine:

• grizzly and badger:

source /usr/projects/climate/SHARED_CLIMATE/anaconda_envs/load_latest_compass.sh

• anvil (blues):

source /lcrc/soft/climate/e3sm-unified/load_latest_compass.sh

• compy:

source /share/apps/E3SM/conda_envs/load_latest_compass.sh

• cori:

source /global/cfs/cdirs/e3sm/software/anaconda_envs/load_latest_compass.sh

To install your own compass conda environment on other machines, first, install Miniconda3 (if miniconda is not already installed), then create a new conda environment as follows:

conda create -n compass -c conda-forge -c e3sm python=3.8 compass

Each time you want to work with COMPASS, you will need to run:

conda activate compass

Setting up a test case

To see all available test cases you can set up in compass, starting in the base of your local compass repo or branch, run:

./list_testcases.py

and you get output like this:

69: -o ocean -c global_ocean -r QU240 -t init
70: -o ocean -c global_ocean -r QU240 -t performance_test

To set up a particular test case, you can either use the full sequence of flags:

./setup_testcase.py \
  --config_file config.ocean \
  --work_dir $WORKDIR \
  --model_runtime runtime_definitions/mpirun.xml \
  -o ocean -c global_ocean -r QU240 -t init

or you can replace the last line with the simple shortcut: -n 69.

Here $WORKDIR is a path, usually to your scratch space. For example,

--work_dir /lustre/scratch4/turquoise/$USER/runs/191210_test_new_branch

and config.ocean is a config file that specifies directory and file paths. You can make a copy of the template config file for your core (e.g. cp general.config.ocean config.ocean) and modify it with the appropriate paths to the appropriate MPAS-Model build and local caches for meshes and initial-condition data files. The documentation for Setting config options for ocean test cases includes some examples you can use as a starting point for specific machines. (Similar documentation for the landice core will is coming soon.)

The --model_runtime is either srun or mpirun, depending whether your machine uses the SLURM queuing system or not.

Running a test case

After compiling the code and setting up a test case, you can log into an interactive node (see machine instructions below) and then

cd $WORKDIR
cd ocean/global_ocean/QU240/init
./run.py

Note the sequence of subdirectories is the same as the flags used to set up the case.

In order to run a bit-for-bit test with a previous case, use -b $PREVIOUS_WORKDIR.

Regression suites

We have assembles suites of test cases for code regressions and bit-for-bit testing. For the ocean core, they are here:

ls ocean/regression_suites/
   land_ice_fluxes.xml  light.xml  nightly.xml  rpe_tests.xml

You can set up a regression as follows:

./manage_regression_suite.py -s \
   --config_file config.ocean \
   -t ocean/regression_suites/nightly.xml \
   --model_runtime runtime_definitions/mpirun.xml \
   --work_dir $WORKDIR

where the details are mostly the same as for setting up a case. You can use the same config.ocean file and use -b $PREVIOUS_WORKDIR for bit-for-bit comparison of the output with a previous nightly regression suite.

To run the regression suite, log into an interactive node, load your modules, and

cd $WORKDIR
./nightly_ocean_test_suite.py

Set up a COMPASS repository with worktrees: for advanced users

This section uses git worktree, which provides more flexibility but is more complicated. See the beginner section above for the simpler version. In the worktree version, you will have many unix directories, and each corresponds to a git branch. It is easier to keep track of, and easier to work with many branches at once. Begin where you keep your repositories:

mkdir compass
cd compass
git clone git@github.com:MPAS-Dev/compass.git legacy
cd master
git checkout legacy

The MPAS-Dev/compass repo is now origin. You can add more remotes. For example

git remote add mark-petersen git@github.com:mark-petersen/compass.git
git fetch mark-petersen

To view all your remotes:

git remote -v

To view all available branches, both local and remote:

git branch -a

We will use the git worktree command to create a new local branch in its own unix directory.

cd compass/master
git worktree add -b newBranchName ../newBranchName origin/master
cd ../newBranchName

In this example, we branched off origin/master, but you could start from any branch, which is specified by the last git worktree argument.

In each new branch directory that you make, you will need to make a copy of general.config.ocean or general.config.landice and alter the copy to point to the MPAS executable and files. There are two ways to point to the MPAS executable:

1. Compass submodule (easier): This guarantees that the MPAS commit matches compass.

   git submodule update --init --recursive
   cd E3SM-Project/components/mpas-ocean
   # load modules (see machine-specific instructions below)
   make gfortran
   

2. Other E3SM directory (advanced): Create your own E3SM-Project/E3SM or MALI-Dev/E3SM repository elsewhere on disk, make a copy of general.config.ocean or general.config.landice, and point the copy to the MPAS standalone code path in your repo. The user must ensure that flag names and test cases match appropriately. The simplest way to set up a new repo in a new directory is:

   git clone git@github.com:E3SM-Project/E3SM.git your_new_branch
   cd your_new_branch
   git checkout -b your_new_branch origin/master
   cd components/mpas-ocean
   

Note

For ocean development, you will clone E3SM-Project/E3SM and build in components/mpas-ocean; for MALI development, the repo is MALI-Dev/E3SM and the build directory is components/mpas-albany-landice.