Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Brings RRFS ensemble post into authoritative repository (#5)
* Adds fork modifications back to remade fork.
* Update README.md
* Update README.md
* updates .xml files for NAM on Dell, and new OCONUS FV3SAR location.
* Finalized the move to "oconus" for this workflow.
* Fixes some FV3 member QPF issues (no more negative precip), turns on AK
* Set of changes for 48 h capability in 48h_new branch
* Tweaks .xml files
* Fixes typos in 48h version of JHREF_EAS
* A number of changes to support 48 h in preproc, fixes link of variable table.
* Some diag prints for trouble 48h_new pmatch_mean_loc.f file
* Switches to Cray fv3regional, fixes FV3 QPF for 48 h runs, fixes UPHL related issue with FV3.
* Splits up JHREF_PREPROC into three distinct jobs for the different input models being processed.
* Some task count cleanup for PR/HI, some 48 h fixes. Uncertain href_get_prcip.f changes.
* Change to non-hardwired WGRIB2 version in EAS. CFP for EAS. Skin temp mean output
* Adds prototype for point probs filling the blank boundaries around EAS probs.
* Slight tweak to updated QPF eas script
* Updates ahead of machine switch. Some fixes for climo/calibration stuff.
* Commits what might be a proper make_easfracqpf_combo.py for boundary region. Still need to modify snow script.
* Turns off longer accumulations of PM and LPM means.
* Fixes bug in build_qpf_climo.py that prevented HIresW data from being found.
* Fixes for EAS calib stuff. Error checking in preproc jobs.
* Finally commits some code cleanup items. slr_derv gone.
* Various changes for 06/18Z conus, nssl-->arw2 for EAS stuff, meanwind code change for reduced FV3SAR output.
* Changes for changes for FV3 output name (hiresw.tXXz.fv3_), and
turning off some calibration stuff in eas_config
* Revamping of how FV3 buckets are done (now done in post) - no need for it in
preproc jobs now. Also fills boundaries in LPM mean product with regular PM mean. Cleans out some calibration stuff from qpf python code.
* Fix for FV3 snow bitmap. Adds "alt" AK to process off time HRRR files. Cleans up
the MAXUVV filtering (was keying on 400-1000 hPa for some models).
* uphl and mxuphl product changes that also impact SBN output.
* Two important fixes. One is to honor the bitmap for HRRR cloud base. Other is
to use the 1 h APCP from HRRR for Alaska (fix was in for hrrr but not hrrrak).
Other various cleanup items.
* Fixes the getflight.f fix from yesterday for the small domains that lack gfld%bmap
* Set of changes before prod switch
* Set of changes to extend WMO output to 48 h (3 hourly beyond f30)
* Set of AWIP product related changes. Gets LPM stuff in the PMMN files. Adds
low end snow probabilities to be EAS based.
* Important bug fixes for EAS snow products (NAM and FV3 were being ignored). Various
other small tweaks.
* Fixes HRRR 3h pcp preproc script for precip code change yesterday.
* Some error checking changes and stripping cycle from FV3 path.
* Fixes issue with FFG generation after DCOM location moved to Dell.
* Changes related to pointing at the new "hiresw" FV3 path and at 48 h of the HRRRv3 parallel.
* Off cycle CONUS GEMPAK to 48 h. Separates build and install for HREF build.
* Some changes for HRRR cloud products, adds "lavg" product to test LOCPM average QPF. Cleans out
some no longer needed calib/climo stuff in scripts directory.
* Various cleanups plus fix for HRRR ceilings
* href_get_prcip.f fix to define nbits based on computed values and an assigned scaling.
* Changes binary scaling from +5 to -5 in href_get_prcip.f.
* Some preprocessing err checking changes.
* GEMPAK resources, error checking in preprocess_fv3, luna/surge comments in rocoto run job.
* Commits GSL HRRR pointing .xml files for AK and CONUS.
* Set of changes that encodes the FFG exceedance probabilities more properly,
and also adds them to the AWIPS output
* Updates README.md
* Changes namx to nam for the filter file for NAM.
* Cleans out some old CLIMO/CALIB J-jobs, ffair stuff. Adds templates
for non-CONUS (noffri) prob files.
* Cleans up modulefiles/HREF directory, renames v2.0.0 to v3.0.0
* Cleans up USH scripts a bit, improves error checking messages
and adds crude doc block to new preprocess scripts.
* Renames USH preprocess scripts (strips .ecf termination), adjusts J-jobs
accordingly.
* Scrubs launch_info, updates README, renames link script.
* Tweaks CFP aspects of JHREF jobs.
* Reworks build to have individual code build scripts that write log files.
* Moves some hrefv3_ parm files to be href_. Changes ush script accordingly. Change
to name of FFG files (adds cycle).
* Adds prototype CFP version of JHREF_PREPROC_* jobs. Give them a test.
* PREPROC jobs shift to using cfp parallelism. .xml file node counts change accordingly.
Fixes MXUHPL parm entries so only expects the 3 thresholds.
Removes href_fv3bucket.fd and changes build and install scripts accordingly.
* Scrubs out generation of low end point probability QPF (lowest thresholds EAS only)
* Trying to eliminate href_fv3bucket.fd directory
* Increases EAS node count for Alaska...KEEPDATA change for CONUS .xml files.
* Work on establishing GESROOT and making the preprocessing system more EE2 compliant.
* Cleans up the HREF rocoto xml files a bit (removes some environmental variables no longer used in the scripts)
* Cleans up some of the python scripts (removes commented out or unused material)
* Adds the clean_href.sh script to clean out the build remnants.
* Updates GEMPAK/nawips script name to be in line with other href scripts.
* A few cleanup items, including cleaning out the lavg file type.
* Points at hiresw.v8.0.0 FV3 output.
* Cleans up FFG script a bit, changes link_hrf_fix.sh script to copy.
* More ERR_EXIT stuff if input files are missing.
* Cleans up JHREF_FFG_GEN, removes JHREF_HPSS_ARCHIVE job. Moves
rocoto launch.ksh script from jobs to rocoto directory. Changes .xml files accordingly.
* Cleans out scripts/exhref_archive.sh.ecf
* Fixes PRE definition in rocoto .xml files
* Adds a .txt version of google docs release notes.
* Updates 10 m wind prob thresholds in parm/wmo for 3 hourly times.
* Pushes updated wmo templates with corrected 10 m wind probabilities.
* Big change to encode the 10 m wind probabilities correctly as 194 (NBMAX).
Also changes the wmo parm files accordingly.
* Updates for rerelease and for new tag name.
* Changes location of HRRR inputs in rocoto drive files.
* Commits changes made to .xml for ops HRRRv4.
* Set of changes for issues exposed by running with traps.
Lack of bmap in PR/HI the biggest item, and other more subtle issues
with array bounds or undefined variables.
* Changes related to bulk wind shear - might be mostly prints
* Replaces release_notes_href_v3.0.0.txt with updated release_notes_href_v3.0.0.pdf
* Adds HPBL to parm/href_hrrr_filter.txt for air quality people.
* Adds HPBL line to parm/href_hrrr_filter.txt for air quality people.
* Some changes based on NCOs mods...could swear had pushed these before.
* Changes related to speeding up HREFv3 delivery times, at least for the
first 36 h. As of yet, not tested. Also has changes for PREPROC_FV3 and PREPROC_NAM
to only process as much as is needed for current cycle, and the needed but unprocessed
files from the previous cycle of that input.
* mainly rocoto changes that seem to get the split ENSPROD_1/ENSPROD_2 and EAS_1/EAS_2 jobs working. More
testing needed. Also fixes the GEMPAK table so the mean SLP field (now EMSL) is generated in GEMPAK output.
* More changes for the split HREF system (first 36 h, then the final hours later).
Mostly rocoto, but also eliminates unneeded JHREF_PREPROC_FV3, which required changes
now that the fv3 data is not in nwges.
* Eliminates jobs/JHREF_PREPROC_FV3_old
* Scrubs original JHREF_EAS and JHREF_ENSPROD jobs
* Also eliminates scripts/exhref_eas.sh.ecf
* Scrubs JHREF_PREPROC_NAM_old
* Removes COMINfv3 in ush script since really points at COMINhiresw now.
* Changes to hopefully speed up awips and gempak product generation.
* Changes triggers for AWIPS and GEMPAK jobs, and allows awips ush script
to start before all times are available (adds wait loops for needed
inputs to become available).
* Initial changes for HREF on wcoss2
* Brings in the operational HREF J-jobs (mostly cosmetic differences from
mine).
* Brings in some ops href.v3.0.3 changes that were never
in my HREF fork.
* Big set of changes that look to have things working pretty well
at least through ensprod and eas steps.
* Stuff to launch and run the GEMPAK and AWIPS jobs - they work on WCOSS2.
* Fixes some things to allow running on other domains.
* Changes to how the poe jobs for the PREPROC pieces are submitted.
launch script changes to make it easier to tweak resources for the
PREPROC jobs.
* Adds prelim versions of build.ver and run.ver
* Folds in the versions/build.ver and versions/run.ver concept.
* Large set of changes attempting to bring things closer to WCOSS2
standards for how COM is defined.
* Adds err_chk for JHREF_PREPROC_HRRR job, adds debug and all builds to all
makefiles.
* Big set of changes to follow the NCO request to use compath to define
COMOUT and COMIN types of variables within the J-job (backup defs).
* Big set of changes that seem to work with href_ver=3.1.0 set. Still
not entirely clear about what COMROOT should be. compath is a headache.
* Cleans out requirement for COMINfv3 to be defined in EAS python scripts.
* Renames things to make clear is v3.1.0
* Updates README.md
* Changes adopting to v3.1.0 module file.
* Tweaks HREF/v3.1.0 module file cosmetically.
* Changes for launch run jobs to reflect the renamed directory.
* Changes submission scripts for v3.1.0 directory name.
* Adds a release_notes_href_v3.1.0.txt copy of transition release notes.
* Changes certain J-jobs from being #!/bin/sh to #!/bin/bash
* Adds OMP_NUM_THREADS=1 for EAS and PREPROC jobs. PREPROC now runs on a
single node quickly. Brings EAS timings in line with WCOSS Cray.
* Updates release notes to reflect the re-release.
* Removes NWROOT and jlogfile references. Adds job definition to launch
scripts.
* Bringing some cloud RRFS material in on top of the WCOSS2 branch.
Shifting toward pure FV3 system with no time lagging.
* Scrubs some NAM/HRRR preprocessing things
* Cleans up some launch scripts (adds FV3 preproc...eliminates NAM/HRRR
preproc).
* Some initial EAS python changes - still need to tidy up naming
convention (COMINfv3 or COMINrrfs), and is the python code looking
at full files or preprocessed (with 1 h and 3 h QPF/WEASD added) files?
* Set of changes that allowed the FV3 PREPROC job to run properly. Other
cleanup items.
* More changes to get things running. ENSPROD now works. EAS stuff
starting to work (at least QPF version).
* Fixes the EAS snow script.
* Changes model name in output files from "href.t*" to "rrfs.t*"
* Changes RUN to rrfs. Makes more use of the ${RUN} variable for defining
names of output files.
* Points at ${RUN} nwges stuff. Also tweaks to "cloud top" VSBY field.
* Switches NET to rrfs
* More updates for new Cactus environment. Bug fix in the non-HREFv3 code
builds, and also in the fv3_3hqpf code. New aws_pull parm file for
transfers from S3.
* Enables "set -x" line in href_getmbr.sh script to get more log output.
* Adds "set -x" to some preprocess ush scripts.
* Some cleanup of FFG stuff, but also removed from the variable tables for
now.
* Tries moving README.md under main code.
* Cleans out the hard href.v3.1.0 layer.
* Moves .gitignore into main directory
* More use of $USER, finishes revamping to rrfs.v3.1.0 structure.
* Cleans up some build items
* Makes some changes to generate products out to 60 h.
* Pulling in huge set of changes from Jun Du (lots of renaming, and
setting up capability for single and time-lagged ensembles).
* Makes it more emc.lam focused in launch directory.
* Restores executable permissions.
* Pushes changes to run RT enspost and transfers to AWS (rocoto stuff).
* Some various J-job and ush changes that were needed mainly for the
time-lagged system.
* Adds fix to JRRFS_ENSPOST_PREPROC_FV3 to avoid use of "backdate" file -
use variable instead. Deletes JHREF jobs.
* Some transfer extension to push enspost output to dev wcoss, and other
rocoto tweaks.
* Probably unnecessary changes on TCDC field processing.
* Various roocoto related changes, particularly with transfer jobs. Also
removes downweighting of older runs.
* Big update for OCONUS domains, and also cleans out some HREF relics that
Jun renamed/modified as "enspost" items.
* Fixes location of enspost fix files.
* Pushes ush changes somehow missed in big update.
* updates to v0.6.5 RRFS code
* Adds 500' ceiliing and 3 and 5 mile visibility probs at request of AWC.
* Pushes the updated .xml and run_aws_trans jobs being used into github.
* Streamlines JRRFS_ENSPOST_PREPROC_FV3 job to only process ontime members
of timelagged ensemble. Shifts to more efficient way of running routine
cycles.
* Updates rocoto .xml to look for v0.6.7 files.
* pushes changes that splits the 1 h, 3 h, and 6 h QPF EAS processing into
low/med/high categories similar to what is done for 12 h and 24 h QPF
processing. Still a bit of a work in progress. Doing from 2 nodes to 4
nodes, and with more tasks, it sped up the AK and CONUS EAS jobs by
roughly 6-7 minutes (say from 33 minutes to 27 minutes).
* Bug fix for 1 h qpf in EAS scripts (was only capturing the high
thresholds due to error).
* Adds some initial HRRR preprocessing pieces from HREF, and cleans out
HREF ex-scripts.
* Adds PREPROC_HRRR J-job that somehow didn't make first commit.
* Pushes changes that seems to allow the HRRR preprocessing to work at
least for CONUS.
* Pushes HRRR capable version of enspost_getmbr.sh
* Pushes HRRR capable version of enspost_getmbr.sh
* Pushes updated .xml file before switching to dev machine to test.
* Cleans out HREF sorc/build items.
* Cleans HREF items from rocoto directory
* Cleans out HREF items from rocoto directory
* Pushes a number of items that mostly has things working.
* Adds a v1.1 Hawaii .xml file to see if things work without adding the
HRRR as a member.
* Initial commit to 1.2.0_winter branch just getting things going.
* Bug fix so FRZR field gets preprocessed for FV3 member.
* Changes that have 3 h FRZR accumulation making it into files.
* Set of changes that have FRZR and initial ASNOW stuff mostly working
* Changes that seemed to get it working with HRRR over CONUS.
* Some final initial changes for FRZR/ASNOW
* Some changes to support HRRRAK, and to generate new winter products for
CONUS.
* Set of changes that gets more of the HRRR/winter weather stuff working.
Adds some asnow/frzr prepreprocessing for HRRR, and special treatment
for it within get_precip code.
* Fixes final get_prcip issue (?) for AK using HRRR and new winter wx
products.
* Some tweaks to v1.1.0 system prior to making it the real-time version.
* Changes to remove timelag from the COMOUT kind of path, also some
tweaks that became needed after recent WCOSS upgrades (to avoid
environmental variables being lost within launch.ksh script).
* Pushes EAS changes to hopefully incorporate HRRR and HRRRAK members.
Utilized HREF scripts for guidance.
* Fixes hrrrak in QPF EAS script
* Commiting test fix to avoid skipping new_fog for 1 h forecast.
* Pushes .xml changes to make system more robust by making sure that the
previous cycle preproc* jobs also completed.
* Brings in local changes made under emc.lam account within rrfs.v1.1.0
* Changes to add use of AK MASK for EAS, and define zero EAS probability
if exceed3d sum is zero.
* Pulls in latest local changes from dogwood.
* Changes asnow probs to point probs for now.
* Fixes ASNOW probs and AK precip graphics issues. Also updates things to
get away from single_physics, and to v0.7.7
---------
Co-authored-by: Shun Liu <[email protected]>- Loading branch information
1 parent
3260ebe
commit 18ed09b