Add abi_g16 & abi_g18 YAML files into templates #329
Conversation
Sfcshp 282 yaml passed phase 1 validation
… templates (NOAA-EMC#298) This PR addresses NOAA-EMC#273 to generate the ctest yamls when RDASApp is built. This will allow us to keep better sync between the ctests and updates to `rrfs-test/validated_yamls/templates`. It will also make reviewing PRs easier if we don't also need to make changes to the super yamls.
…cate action key. (NOAA-EMC#314) In rrfs-workflow, GSL colleagues (Chunhua and others) found that aircar_airTemperature_133 assimilation experiments had unrealistic increments. It was found that this is due to the jediyaml not correctly handling missing QualityMarker values. Essentially, those observations were getting into the analysis causing the very large and unreasonable analysis increments.
Adding phase 1 adpupa (132/232) yamls. There were no obs in the ctest case to fully validate. This isn't a surprise since these ob types were marked to be pretty sparse in the observation spreadsheet. These yamls were developed following the (120/220) yamls exactly with the exception of the ObsErrors and gross error checks for winds. The point of this PR is to establish these yamls in RDASApp. This won't affect cycling DA results because we will only use configurations that have passed phase 3 validation.
Added the abi_g16 and abi_g18 ctest into gen_yaml_ctest.sh.
xyzemc
requested review from
hu5970,
ShunLiu-NOAA,
delippi,
HaidaoLin-NOAA and
SamuelDegelia-NOAA
|
There was a problem hiding this comment.
@xyzemc good work! I just had a few minor and very picky suggestions. It is nice to have a comment for each filter. It helps non radiance people understand what is happening and it also helps to visually break the yaml up when you're viewing it.
Added the 'obs localizations' and 'halo size' into abi yaml file.
|
All mpasjedi ctests have been passed on Hera. |
|
FAILED on hercules started build_and_test on hercules at UTC time: Wed Mar 26 03:05:54 UTC 2025 workdir: /work/noaa/wrfruc/rrfsbot/PRs_RDASApp/329 |
|
@xyzemc Would you mind also committing the new test reference files for the fv3-jedi ctests? Regarding the failed mpas-jedi ctests on Hercules, this could just be related to the tolerance needing to be increased. But I cannot check that right now since Orion/Hercules are in PM right now. |
@SamuelDegelia-NOAA So if I use the gen_yaml_ctest.sh to generate the fv3_jedi yaml file, the above two lines will be included in the fv3_jedi yaml file. Then the fv3_jedi run will crash. |
|
@xyzemc We can use the Can you share one of your super yamls that are generated for the fv3-jedi ctests (which cause the failure)? I'll see if I can edit |
@SamuelDegelia-NOAA Sure. |
|
@xyzemc Do those lines need to be commented out for the atms yaml too, or just ABI? |
@SamuelDegelia-NOAA atms and amsua need to be commented out too for the fv3_jedi yaml file. |
|
All fv3_jedi ctests have passed on Hera with the updated fv3_jedi yaml file as described above. /scratch1/NCEPDEV/stmp2/Xiaoyan.Zhang/RDASApp_abi_csr/build/rrfs-test
100% tests passed, 0 tests failed out of 8 Label Time Summary: Total Test time (real) = 1024.25 sec |
* Updated the gen_yaml_ctest and commentQC.py to comments out two unecessary lines in fv3_jedi ctest yaml file
|
FAILED on hercules started build_and_test on hercules at UTC time: Fri Mar 28 01:31:19 UTC 2025 workdir: /work/noaa/wrfruc/rrfsbot/PRs_RDASApp/329 |
|
@SamuelDegelia-NOAA It seems the hercules fv3_jedi ctests still failed even with the updated reference files and the gen_yaml*sh. Could you help to check it one more time? Or we don't need to care about the Hercules ctests at this moment. |
|
@xyzemc The failure for The This is not a failure due to any reason with JEDI. That might have just been a temporary issue on Hercules? I am not really sure what would suddenly cause an error like that. But either way, I think these failures on Hercules are not a concern and that this PR can move forward. |
Thanks for the check and confirmation. |
Description
Issue(s) addressed
Resolves/Results are documented in Issue #
#57
#292
#306
#249
Checklist
I have performed a self-review of my own code.
I have run rrfs tests before creating the PR.
Ctest with abi_g16Ctest with abi_g18 on top of abi_g16