Add logic for running offline IODA tools. #600
Conversation
delippi
requested review from
MatthewPyle-NOAA,
ShunLiu-NOAA,
JacobCarley-NOAA and
BenjaminBlake-NOAA
as code owners
scripts/exrrfs_ioda_bufr.sh
Outdated
| ${cpreq} ${HOMErrfs}/sorc/RDASApp/rrfs-test/IODA/offline_add_var_to_ioda.py . | ||
| ioda_files=$(ls ioda*nc) | ||
| for ioda_file in ${ioda_files[@]}; do | ||
| python offline_add_var_to_ioda.py -o ${ioda_file} |
There was a problem hiding this comment.
I think NCO wants the script (including *.py) to run from the command line directly without the leading "python".
There was a problem hiding this comment.
Also, has the Python environment been correctly loaded for this tool?
There was a problem hiding this comment.
@guoqing-noaa, thanks for the comments. The tool already had #!/usr/bin/env python at the top. Its just habit to run it with python anyway. I just removed it in my most recent commit and it still works. As for the python environment it seems everything that is needed is already loaded so I didn't have to load anything else.
|
@delippi Did you test within the leading python to make sure it all still works? If so, think we can merge it. |
|
@MatthewPyle-NOAA hold off on merging though. There are some dependencies. Edit: The dependencies are documented in the initial comment. Please don't merge until those are satisfied, otherwise the python tool won't be available. I need some people to review that NOAA-EMC/RDASApp#259. |
|
I also just noticed that this |
scripts/exrrfs_ioda_bufr.sh
Outdated
| ${cpreq} ${HOMErrfs}/sorc/RDASApp/rrfs-test/IODA/offline_add_var_to_ioda.py . | ||
| ioda_files=$(ls ioda*nc) | ||
| for ioda_file in ${ioda_files[@]}; do | ||
| offline_add_var_to_ioda.py -o ${ioda_file} |
There was a problem hiding this comment.
Sorry for another comment, could you add ./ before offline_add_var_to_ioda.py?
Not all environments put the current directory in the $PATH variable.
(for example, our Jet role account cron jobs set $PATH to empty)
@delippi Thanks for noticing this! Yes, we will need to update the bufr2ioda yaml files. For all PARM files, rrfs-workflow will be self-contained. That means, RDASApp manages its parm/test files while rrfs-workflow adapts a given version from RDASApp. This adheres to the NCO standard generating PARM files on the fly from the |
|
I would suggest we hold this PR until the discussion in NOAA-EMC/RDASApp#259 is resolved. |
|
RDASApp was updated yesterday with NOAA-EMC/RDASApp#259, but in order for this PR to work, the version of RDASApp needs updated to the latest. Should I add that to this PR or should that be done separately? |
|
@delippi I would say yes to updating the hash of RDASApp to pick up what is needed within this PR. Unless others object. |
|
Yes, the same here. Vote for updating the RDASApp hash.
|
|
This PR can be merged after #618 is merged, |
|
All good on your end, @delippi ? |
|
Wow, thanks for updating the hash to RDASApp @guoqing-noaa. Yeah, I think we are good to go then. |
DESCRIPTION OF CHANGES:
exrrfs_ioda_bufr.shto run an offline tool to process IODA files. The offline tool addsMetaData/longitude_latitude_pressureto the IODA file which is used as the group category for duplicate checking based on RDASApp#259TESTS CONDUCTED:
Tested on Hera conus 12km retro runs.
ISSUE:
Dependencies: