Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update operational workflow to work with GSI develop branch #3068

Merged

Conversation

ADCollard
Copy link
Contributor

Update operational workflow to work with GSI develop branch. This mostly involves changing the links to elements that have been moved out of the GSI repository since GFSv16.3. The ultimate aim is to be able to cycle with v16 workflow but using a version of GSI close to develop. This will not reproduce operations.

Resolves #3045

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? YES
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? YES
    • GSI

How has this been tested?

Testing is limited to WCOSS2. Initialised full resolution cycling experiment. Will compare to operational performance.

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

KateFriedman-NOAA and others added 18 commits December 20, 2024 09:06
The order of the upgrade to obsproc/v1.3 and the new AFWA global file
was moved back and thus the release branch needed to be remade. This PR
includes all updates for the GFS upgrade to obsproc/v1.3 and the new
global AFWA data file:

- new UFS_UTILS tag (`ops-gfsv16.3.20`) to support new global AFWA file
- new GSI tag (`gfsda.v16.3.22`) to support obsproc/v1.3 and saildrone
- workflow changes to support obsproc/v1.3 (and prepobs/v1.2 internal to
obsproc) and AFWA global file
- also workflow changes to allow developers to run the emcsfc_sfc_prep
job within the rocoto mesh via `DO_SFCPREP=YES` in config.base

This PR is a repeat of prior ones to get this package in place.

Refs NOAA-EMC#2913
NCO provided increase in ncpus to avoid ncpu burst warnings.

Refs NOAA-EMC#2913
…3.21) (NOAA-EMC#3200)

Update the `sorc/syndat_getjtbul.fd/getjtbul.f` code to resolve a problem parsing 3-character storm names.
This update is being included in the GFSv16.3.21 package (with approval by NCO).

From the developer @Qingfu-Liu:

> The `gfs_atmos_tropcy_qc_reloc` job did not properly parse the vitals
info from a raw bulletin with a tab in the storm name field for storm
"SIX". The code sorc/syndat_getjtbul.fd/getjtbul.f converting a tab to a
space worked well for storms that have at least 4 characters in their
names, but does not work as well for storms with 3-character names. This
code change will fix the parsing problem for 3-character storms.

Refs NOAA-EMC#2591
…release/gfs.v16.3.22

* origin/release/gfs.v16.3.21:
  Fix for syndat_getjtbul code parsing 3-character storm names (GFSv16.3.21) (NOAA-EMC#3200)
The WAFS is now a separate package in operations and is no longer a part of the GFS.

Also included in this package are the following unrelated updates:

1. improvements to the enkfgdas_update job resource and environment settings
2. a fix for the syndat_getjtbul code when parsing 3-character storm names

Refs NOAA-EMC#2591
The `ww3_outp` is optimized to read per time step binary output and create per point spectral and bull/cbull output.

Refs NOAA-EMC#2913
…6.3.22

* origin/dev/gfs.v16:
  GFSv16.3.21 - WAFS removal from GFS (NOAA-EMC#3281)
Update the name of the "Gulf of Mexico" to the "Gulf of America".
This includes fix/parm file updates and a new UWM tag for WW3 rename updates.

Refs NOAA-EMC#3326
Make additional "Gulf of Mexico" to "Gulf of America" renames.
The bufr station list is updated and the EMC_verif-global tag
is updated to a version with the same name change.

Refs NOAA-EMC#3326
Introduces bufr station updates into the GFSv16.3.23 operational
package. There are three updated stations and 121 new stations added to
the bufr station parm files. Changes provided by @BoCui-NOAA .

Refs NOAA-EMC#2913
…MC#3387)

Adjust the package to the new v16.3.23 and incorporate updates
from the new v16.3.22 package (now ahead of this update). 
Additionally, the obsproc (and prepobs) updates are backed 
out for a later upgrade. Now use `obsproc/v1.2.3`.

Refs NOAA-EMC#2913
…se/gfs.v16.3.23

* origin/release/gfs.v16.3.22:
  Update release notes
  Additional Gulf of Mexico name changes in GFS (NOAA-EMC#3353)
  Rename Gulf of Mexico to Gulf of America (NOAA-EMC#3352)
- The radmon_diag_ck.sh and rstprod.sh scripts were missing
from the link script step and were causing errors in ops.
- Add the two missing scripts to the link script and
the gitignore file.

Refs NOAA-EMC#2913
@ADCollard
Copy link
Contributor Author

Code has been updated to release/gfs.v16.3.23.

The workflow before these changes successfully ran for seven cycles. I still need to do a longer test with the latest version, but following conversations with @aerorahul, I think this is ready to take out of draft.

Note that CRTMv3 is not longer part of this upgrade.

@ADCollard
Copy link
Contributor Author

Actually I have one question. Externals.cfg has the following lines:

[GSI]
tag = gfsda.v16.3.22
local_path = sorc/gsi.fd
repo_url = https://github.com/NOAA-EMC/GSI.git
protocol = git
required = True

What does this do? I am checking out the develop version of GSI right now.

@KateFriedman-NOAA
Copy link
Member

What does this do? I am checking out the develop version of GSI right now.

@ADCollard We aren't using checkout_externals anymore but I'm keeping Externals.cfg synced with the tags in sorc/checkout.sh for consistency. Once you have a GSI tag for sorc/checkout.sh, please also set it in Externals.cfg. Thanks!

@KateFriedman-NOAA
Copy link
Member

Code has been updated to release/gfs.v16.3.23.

The workflow before these changes successfully ran for seven cycles. I still need to do a longer test with the latest version, but following conversations with @aerorahul, I think this is ready to take out of draft.

Awesome, feel free to move this out of draft. Note, I need to make a GFSv16.3.24 package that will have the obsproc/prepobs version updates. I'll let you know when I've done that so you can sync that into the v16.4 release branch or I can do the sync if it happens after this goes into the release branch. Just a heads up.

Note that CRTMv3 is not longer part of this upgrade.

Noted, thanks!

@ADCollard ADCollard marked this pull request as ready for review March 5, 2025 16:34
@ADCollard
Copy link
Contributor Author

@KateFriedman-NOAA Just to confirm that we will re-tag FV3GFS to 16.4, even though there will not be changes?

@KateFriedman-NOAA
Copy link
Member

@KateFriedman-NOAA Just to confirm that we will re-tag FV3GFS to 16.4, even though there will not be changes?

The ufs-weather-model was tagged for v16.4 because some WW3 updates were made and requested to be included in v16.4. So the g-w release branch will include the DA updates and a UWM tag update.

@ADCollard
Copy link
Contributor Author

@aerorahul Is this enough to kick off phases 1 and 2 that we discussed yesterday.

As I said then, I am on leave from tonight until 3/14 without WCOSS access, so cannot kick off a parallel before then. Hopefully we could still do a code handoff around May 1st.

Thanks.

Copy link
Member

@KateFriedman-NOAA KateFriedman-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ADCollard ! I am going to merge, sync it with v16.3.24 changes, and then review further.

@KateFriedman-NOAA KateFriedman-NOAA merged commit 49aed3f into NOAA-EMC:release/gfs.v16.4 Mar 5, 2025
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
production update Processing update in production
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants