Home

Jump to Instructions

Overview

The purpose of the Offline Land Model Testbed (OLMT) is to simplify the workflows for single site, regional and ensemble offline ELM simulations, which are otherwise cumbersome using only CIME. The new version in this repository replaces the command line interface. The user now can run a simulation using a single python run script that will perform the entire workflow. This usually involves setting up three cases for biogeochemistry-enabled runs: the ad spinup, final spinup and transient simulations. It is also possible to add additional cases beginning in later years where we apply treatment effects or otherwise modify forcings. For example, in the SPRUCE study we have 10 experimental treatments (different levels of temperature and CO2 modifications) that begin in 2015. For those point simulations, we have a single run script that launches and manages 13 cases.

For each case, the runscript will perform the create_newcase, case setup, and submission. The case.build will be performed on the first case only, and then the same executable will be used for following cases. When submitting cases, the correct dependencies will be applied, such that the second case will start running after the first has finished, etc.

Users can customize the simulations to run single points, a list of lat/lon coordinates, rectangular regions or global simulations. OLMT assumes surface, land use and domain data already exist (using the defaults for specified compsets and resolution) and will extract points or regions from these data. The user can also set custom files, for example ultra high-resolution files created from kilocraft. For single point runs, the PFT and soil information can be set to match observations, for example at AmeriFlux sites.

Finally, OLMT also has the capability to perform ensemble simulations. Users can specify a list of parameters, with allowable ranges for each. Random samples can then be created. Alternatively, the user can provide their own files with different parameter combinations. When this ensemble option is enabled, the cases will be set up in the same way as above, but then multiple copies of run directories will be created. We then use another MPI-enabled python script to manage the multiple simulations in parallel. Users can also specify a list of output variables and time frequency for which to postprocess. A matrix of output values for all ensemble members is then created at the end of the simulation, which can then be used in Uncertainty quantification applications discussed in the other epics.

Instructions

These instructions currently work on 3 machines only: Perlmutter (pm-cpu), Chrysalis, and CADES baseline. A more general set of instructions for installing on a new machine will be provided soon. Installing OLMT and conda environment:

Clone this repository:
git clone [email protected]:dmricciuto/elm-olmt.git
cd elm-olmt

Load the module for anaconda, if you have not already:
module load <anaconda3>
Where <anaconda3> represents machine-specific modules:

• Chrysalis: anaconda3
• Cades-baseline: python/3.11-anaconda3
• Perlmutter: conda/Miniforge3-24.7.1-0 AND python/3.11

yml files are available for three machines: Perlmultter (pm), Chrysalis and CADES-baseline (ORNL only). the following commands will set up and activate the conda environment using these files (replace "machine" with the name of the machine you are on):

cd conda_envs
conda env create -f OLMT_<machine>.yml
conda activate OLMT_<machine>

To use OLMT, you will need to activate this conda environment each time you log in. Now, you are ready to view and edit the run script.

cd ../runscripts
Using your favorite editor, view the file run_BGC.py or click here to view on Github. The default setup should work out of the box. If you want to get started right away, you can launch this runscript simply by typing python run_BGC.py
This script will launch a full site simulation at the UMBS flux site (US-UMB) with short ad_spinup and final_spinups (for demonstration purposes), and a full transient simulation from 1850-2014.

The first ~90 lines of the file are editable by the user to customize paths, run types and namelist options. the machine defaults are retrieved from the get_machine_info function, You may customize these paths in the script: caseroot - where new cases will go runroot - where the run directories will go modelroot - path to E3SM code directory. You must clone the E3SM repository separately.

The following variables can be use to customize the type of run:
runtype = 'site' site,latlon_list (list of lat/lon coordinates),or latlon_bbox (rectangular regions defined by min/max lat and lon values).
mettype = 'gswp3' Type of met forcing to use (site, gswp3, crujra currently supported).
case_suffix = '' Unique identifier for the case. Case name will be (date)(site/region)(compset)_(suffix).

If you are running a site simulation, add the site name or list of site names under the "if runtype == 'site" block. The site information is obtained from:
/lnd/clm2/PTCLM/(sitegroup)_sitedata.txt (site coordinates).
/lnd/clm2/PTCLM/(sitegroup)_pftdata.txt (site plant functional types).
/lnd/clm2/PTCLM/(sitegroup)_soildata.txt (site soil texture).
Add the site group information in the same block. For single site simulations, "numproc" should always be 1. "numproc" is the number of processors assigned to each individual simulation. New sites and sitegroups can be added in the PTCLM directory.

For a list of point simulations to run ("latlon_list" option), the lat/lon coordinates are specified as a .csv file set as the "point_list_file". For running a rectangular region ("latlon_bbox" option), the latitude and longitude bounds are specified below.
lat_bounds = [-90,90]
lon_bounds = [-180,180]
For both the latlon_list, latlon_bbox options, the number of processors may set using the numproc variable. Note that the number of processors must be less than or equal to the number of active land grid cells in the simulation.

The resolution is set by the "res" variable below.
res = 'hcru_hcru' #Resolution of global files to extract from.

Additional variables can be set to customize the simulation.

Calibration example

An example script for running model calibration is run_FATES_calibrationexample.py. Using your favorite editor, you can view and edit this file. As with the previous example, this setup should work out of the box. If you want to get started right away, you can launch this runscript simply by typing python run_FATES_calibrationexample.py