-
Notifications
You must be signed in to change notification settings - Fork 18
Adding new observation types for the RDASApp ctests
RDASApp uses a method for merging individual observation-space yamls into one "super yaml" for its ctests. If you have validated a new observation type and wish to add it to the RDASApp ctests, you will need to make a few changes in addition to adding the yaml for the new observation type. Please see the detailed list below for additional developments that need to be made during your PR.
This step can be done by contacting someone on the fix file management team. You can also include the path to this data in your PR so that we can update the staged data on each machine.
Please just include the lines of the yaml corresponding to the new observation you are adding (i.e., start the yaml with the line - obs space:
). Also note that additional sections need to be included in the yaml such that the yaml can be used for both Var-based DA or EnKF. These include the distribution
and obs localizations
headers. Please see the example yamls already included in the rrfs-test/validated_yamls/templates/obtype_config
directory.
Additionally, please do not include the input obsfile
name or distribution
name directly in the yaml. Instead, you will need to pass in strings that can be replaced when merging the yamls. See below for an example for mesonet yamls:
- obs space:
name: msonet
distribution:
name: "@DISTRIBUTION@"
halo size: 100e3
obsdatain:
engine:
type: H5File
obsfile: "@OBSFILE@"
3. Add the new observation type to the merging script in rrfs-test/validated_yamls/gen_yaml_ctest.sh
and run the script
This script controls the observation-space yamls that are merged together for the ctests. First, you need to add any new yamls to the lists with the names *_obtype_configs
. Separate lists are created for each set of yamls that will use a different observation file. For example, if you are adding a new yaml that uses the same observation file as msonet_uv_288.yaml
, then you can add the new yaml under msonet_obtype_configs
. But if you are adding a yaml that uses a new observation type, you will need to create a new list along the lines of ${OBNAME}_obtype_configs
.
If you are creating a new list, you will also need to add a new line that calls the process_obtypes
function for that observation. This function replaces the @OBSFILE@
string described in the above step with the observation file name. Please see the existing calls to process_obtypes
for examples of what to add.
After adding the new lines, please run this script to generate the new "super yamls" inside rrfs-test/testinput
. This mega yaml should also be included in your PR.
Once your "super yaml" has been generated, you are ready to rerun the ctests. We expect the ctest to fail initially due to the output not matching the previous version of the ctest reference output in RDASApp. The ctest will generate new test output that will become the updated test reference data.
Navigate to the individual ctest run directories (e.g., build/rrfs-test/rundir-rrfs_mpasjedi_2024052700_Ens3Dvar
) and locate the rrfs-mpasjedi-*.out
file. You will then need to rename this file to rrfs-mpasjedi-*.ref
and copy it into build/rrfs-test/testoutput
.