Skip to content

Commit 66d132b

Browse files
kuanchihwangkuanchihwang
authored
Merge pull request #1297 from kuanchihwang/staging/mpas-dycore-mov-mtn-gw
cam6_4_091: Implement support for moving mountain gravity wave scheme in MPAS dycore
2 parents 4a8bf71 + c3fbc63 commit 66d132b

File tree

7 files changed

+357
-13
lines changed

7 files changed

+357
-13
lines changed

‎bld/build-namelist

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3852,10 +3852,10 @@ if (!$simple_phys) {
38523852

38533853
if ($phys =~ /cam7/) {
38543854
#
3855-
# moving mountains only supported by SE dycore
3855+
# moving mountains only supported by MPAS and SE dycore
38563856
# (since vorticity needs to be passed to physics in dp_coupling)
38573857
#
3858-
if ( $dyn =~ /se/ ) {
3858+
if ( $dyn =~ /mpas|se/ ) {
38593859
add_default($nl, 'use_gw_movmtn_pbl', 'val'=>'.true.');
38603860
} else {
38613861
add_default($nl, 'use_gw_movmtn_pbl', 'val'=>'.false.');
@@ -3864,10 +3864,9 @@ if (!$simple_phys) {
38643864

38653865
my $use_gw_movmtn_pbl = $nl->get_value('use_gw_movmtn_pbl');
38663866
if ($use_gw_movmtn_pbl =~ /$TRUE/io) {
3867-
if ( ! ($dyn =~ /se/) ) {
3868-
die "$ProgName - ERROR: use_gw_movmtn_pbl is only available with the SE dycore \n";
3869-
3870-
}
3867+
if ( ! ($dyn =~ /mpas|se/) ) {
3868+
die "$ProgName - ERROR: use_gw_movmtn_pbl is only available with MPAS and SE dycore\n";
3869+
}
38713870
}
38723871

38733872
add_default($nl, 'use_gw_rdg_gamma' , 'val'=>'.false.');

‎cime_config/testdefs/testlist_cam.xml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1677,6 +1677,16 @@
16771677
<option name="comment">CAM7 low top ~40 km with archiving</option>
16781678
</options>
16791679
</test>
1680+
<test compset="FHISTC_LTso" grid="mpasa480_mpasa480" name="ERS_D_Ln9" testmods="cam/outfrq9s_mpasa480">
1681+
<machines>
1682+
<machine name="derecho" compiler="intel" category="aux_cam"/>
1683+
<machine name="izumi" compiler="gnu" category="aux_cam"/>
1684+
</machines>
1685+
<options>
1686+
<option name="wallclock">00:30:00</option>
1687+
<option name="comment">CAM7 low top ~40 km with MPAS dynamical core</option>
1688+
</options>
1689+
</test>
16801690
<test compset="FHISTC_MTso" grid="ne30pg3_ne30pg3_mt232" name="SMS_D_Ln9" testmods="cam/outfrq9s">
16811691
<machines>
16821692
<machine name="derecho" compiler="intel" category="aux_cam"/>

‎cime_config/testdefs/testmods_dirs/cam/outfrq9s_mpasa480/user_nl_clm

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
flanduse_timeseries = ''
12
hist_nhtfrq = 9
23
hist_mfilt = 1
34
hist_ndens = 1

‎doc/ChangeLog

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,179 @@
11
===============================================================
22

3+
Tag name: cam6_4_091
4+
Originator(s): kuanchihwang
5+
Date: May 8, 2025
6+
One-line Summary: Implement support for moving mountain gravity wave scheme in MPAS dycore
7+
Github PR URL: https://github.com/ESCOMP/CAM/pull/1297
8+
9+
Purpose of changes (include the issue number and title text for each relevant GitHub issue):
10+
11+
This PR implements support for moving mountain gravity wave scheme in MPAS dycore.
12+
13+
The `use_gw_movmtn_pbl` namelist option will now default to `.true.` when MPAS dycore and CAM7
14+
physics are both selected.
15+
16+
The moving mountain gravity wave scheme needs relative vorticities at cell points as input.
17+
However, because MPAS uses staggered C-grid for spatial discretization, where wind vectors are
18+
located at edge points, it calculates relative vorticities at vertex points instead. As a result,
19+
this PR introduces a new functionality in MPAS subdriver to regrid vertex values to cell values.
20+
The regridding functionality is also generalized so that it will work with all variables at vertex
21+
points.
22+
23+
Subsequently, relative vorticities are passed to physics buffer during dynamics-physics coupling,
24+
after which the moving mountain gravity wave scheme can query and use them as input.
25+
26+
Closes #1253 (Support for new moving mountains gravity wave trigger with MPAS dycore)
27+
Closes #1277 (MPAS dycore support for moving mountains parameterization)
28+
29+
Describe any changes made to build system:
30+
31+
None
32+
33+
Describe any changes made to the namelist:
34+
35+
The `use_gw_movmtn_pbl` namelist option will now default to `.true.` when MPAS dycore and CAM7
36+
physics are both selected.
37+
38+
List any changes to the defaults for the boundary datasets:
39+
40+
None
41+
42+
Describe any substantial timing or memory changes:
43+
44+
None
45+
46+
Code reviewed by:
47+
48+
PeterHjortLauritzen, mgduda, nusbaume
49+
50+
List all files eliminated:
51+
52+
None
53+
54+
List all files added and what they do:
55+
56+
None
57+
58+
List all existing files that have been modified, and describe the changes:
59+
60+
M bld/build-namelist
61+
* Allow the moving mountain gravity wave scheme to be selected with MPAS dycore
62+
M cime_config/testdefs/testlist_cam.xml
63+
* Add testing for CAM7 low top with MPAS dynamical core
64+
M src/dynamics/mpas/dp_coupling.F90
65+
* Pass relative vorticities to physics buffer during dynamics-physics coupling
66+
M src/dynamics/mpas/driver/cam_mpas_subdriver.F90
67+
* Implement the computation of relative vorticities at cell points
68+
M src/dynamics/mpas/dyn_comp.F90
69+
* Pass relative vorticities to physics buffer during dynamics-physics coupling
70+
71+
If there were any failures reported from running test_driver.sh on any test
72+
platform, and checkin with these failures has been OK'd by the gatekeeper,
73+
then copy the lines from the td.*.status files for the failed tests to the
74+
appropriate machine below. All failed tests must be justified.
75+
76+
NOTE:
77+
78+
Since cam6_4_089, `model_version`, which is a variable that stores the current model version,
79+
has been introduced to `nuopc.runconfig`. However, the testing framework will detect it as a
80+
significant difference and fail at "NLCOMP" incorrectly.
81+
82+
All tests currently exhibit "NLFAIL" in their overall results due to those "NLCOMP" failures.
83+
84+
They are harmless and can be ignored.
85+
86+
derecho/intel/aux_cam:
87+
88+
ERS_D_Ln9.mpasa480_mpasa480.FHISTC_LTso.derecho_intel.cam-outfrq9s_mpasa480 (Overall: DIFF) details:
89+
FAIL ERS_D_Ln9.mpasa480_mpasa480.FHISTC_LTso.derecho_intel.cam-outfrq9s_mpasa480 BASELINE (Baseline directory does not exist)
90+
* New test
91+
SMS_D_Ln9_P1280x1.ne0CONUSne30x8_ne0CONUSne30x8_mt12.FCHIST.derecho_intel.cam-outfrq9s (Overall: FAIL) details:
92+
FAIL SMS_D_Ln9_P1280x1.ne0CONUSne30x8_ne0CONUSne30x8_mt12.FCHIST.derecho_intel.cam-outfrq9s SETUP
93+
* Pre-existing failures due to build-namelist error requiring CLM/CTSM external update
94+
95+
NOTE:
96+
97+
SMS_D_Ln9_P1280x1.ne0ARCTICne30x4_ne0ARCTICne30x4_mt12.FHIST.derecho_intel.cam-outfrq9s (Overall: NLFAIL) details:
98+
PASS SMS_D_Ln9_P1280x1.ne0ARCTICne30x4_ne0ARCTICne30x4_mt12.FHIST.derecho_intel.cam-outfrq9s RUN
99+
* This test is unstable. Sometimes (like 2 out of 3) it fails at "RUN", but with enough retries it eventually passes.
100+
When it does fail, a stack trace similar to the following can be reliably observed:
101+
102+
forrtl: error (65): floating invalid
103+
Image PC Routine Line Source
104+
libpthread-2.31.s 000015356741A8C0 Unknown Unknown Unknown
105+
libesmf.so 000015356DD8E72A _ZN5ESMCI3XXE10ps Unknown Unknown
106+
libesmf.so 000015356DD82D90 _ZN5ESMCI3XXE10ps Unknown Unknown
107+
libesmf.so 000015356DD777B0 _ZN5ESMCI3XXE10ps Unknown Unknown
108+
libesmf.so 000015356DD6DBB4 _ZN5ESMCI3XXE4exe Unknown Unknown
109+
libesmf.so 000015356DD6D36D _ZN5ESMCI3XXE4exe Unknown Unknown
110+
libesmf.so 000015356DCC66CD _ZN5ESMCI26sparse Unknown Unknown
111+
libesmf.so 000015356DCBACAF _ZN5ESMCI5Array18 Unknown Unknown
112+
libesmf.so 000015356DCB970E _ZN5ESMCI5Array17 Unknown Unknown
113+
libesmf.so 000015356DD1E340 c_esmc_arraysmmst Unknown Unknown
114+
libesmf.so 000015356E0A419F _Z19ESMCI_regrid_ Unknown Unknown
115+
libesmf.so 000015356E030568 _ZN5ESMCI7MeshCap Unknown Unknown
116+
libesmf.so 000015356E0E0E42 c_esmc_regrid_cre Unknown Unknown
117+
libesmf.so 000015356E7787B1 esmf_regridmod_mp Unknown Unknown
118+
libesmf.so 000015356E4E198C esmf_fieldregridm Unknown Unknown
119+
cesm.exe 00000000080D8794 lnd_set_decomp_an 497 lnd_set_decomp_and_domain.F90
120+
cesm.exe 00000000080D0F4E lnd_set_decomp_an 128 lnd_set_decomp_and_domain.F90
121+
cesm.exe 00000000080A2F3F lnd_comp_nuopc_mp 644 lnd_comp_nuopc.F90
122+
libesmf.so 000015356DE68020 _ZN5ESMCI6FTable1 Unknown Unknown
123+
libesmf.so 000015356DE67A6C ESMCI_FTableCallE Unknown Unknown
124+
libesmf.so 000015356E153DE4 _ZN5ESMCI2VM5ente Unknown Unknown
125+
libesmf.so 000015356DE68704 c_esmc_ftablecall Unknown Unknown
126+
libesmf.so 000015356E359B9D esmf_compmod_mp_e Unknown Unknown
127+
libesmf.so 000015356E5DD298 esmf_gridcompmod_ Unknown Unknown
128+
libesmf.so 000015356E999B87 nuopc_driver_mp_l Unknown Unknown
129+
libesmf.so 000015356E98F82B nuopc_driver_mp_i Unknown Unknown
130+
libesmf.so 000015356DE68020 _ZN5ESMCI6FTable1 Unknown Unknown
131+
libesmf.so 000015356DE67A6C ESMCI_FTableCallE Unknown Unknown
132+
libesmf.so 000015356E153DE4 _ZN5ESMCI2VM5ente Unknown Unknown
133+
libesmf.so 000015356DE68704 c_esmc_ftablecall Unknown Unknown
134+
libesmf.so 000015356E359B9D esmf_compmod_mp_e Unknown Unknown
135+
libesmf.so 000015356E5DD298 esmf_gridcompmod_ Unknown Unknown
136+
libesmf.so 000015356E999B87 nuopc_driver_mp_l Unknown Unknown
137+
libesmf.so 000015356E98F92C nuopc_driver_mp_i Unknown Unknown
138+
libesmf.so 000015356E97F883 nuopc_driver_mp_i Unknown Unknown
139+
libesmf.so 000015356DE68020 _ZN5ESMCI6FTable1 Unknown Unknown
140+
libesmf.so 000015356DE67A6C ESMCI_FTableCallE Unknown Unknown
141+
libesmf.so 000015356E153DE4 _ZN5ESMCI2VM5ente Unknown Unknown
142+
libesmf.so 000015356DE68704 c_esmc_ftablecall Unknown Unknown
143+
libesmf.so 000015356E359B9D esmf_compmod_mp_e Unknown Unknown
144+
libesmf.so 000015356E5DD298 esmf_gridcompmod_ Unknown Unknown
145+
cesm.exe 00000000004502F8 MAIN__ 128 esmApp.F90
146+
cesm.exe 000000000042711D Unknown Unknown Unknown
147+
libc-2.31.so 0000153562F2429D __libc_start_main Unknown Unknown
148+
cesm.exe 000000000042704A Unknown Unknown Unknown
149+
150+
Line 497 of lnd_set_decomp_and_domain.F90 is a call to ESMF_FieldRegridStore, which is outside of our (CAM) control.
151+
152+
derecho/nvhpc/aux_cam:
153+
154+
All pass
155+
156+
izumi/nag/aux_cam:
157+
158+
All pass
159+
160+
izumi/gnu/aux_cam:
161+
162+
ERS_D_Ln9.mpasa480_mpasa480.FHISTC_LTso.izumi_gnu.cam-outfrq9s_mpasa480 (Overall: DIFF) details:
163+
FAIL ERS_D_Ln9.mpasa480_mpasa480.FHISTC_LTso.izumi_gnu.cam-outfrq9s_mpasa480 BASELINE (Baseline directory does not exist)
164+
* New test
165+
166+
CAM tag used for the baseline comparison tests if different than previous
167+
tag:
168+
169+
cam6_4_090
170+
171+
Summarize any changes to answers:
172+
173+
None
174+
175+
===============================================================
176+
3177
Tag name: cam6_4_090
4178
Originator(s): lizziel
5179
Date: April 30, 2025

‎src/dynamics/mpas/dp_coupling.F90

Lines changed: 47 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -38,15 +38,15 @@ module dp_coupling
3838
!=========================================================================================
3939

4040
subroutine d_p_coupling(phys_state, phys_tend, pbuf2d, dyn_out)
41-
use cam_mpas_subdriver, only: cam_mpas_update_halo
41+
use cam_mpas_subdriver, only: cam_mpas_update_halo, cam_mpas_vertex_to_cell_relative_vorticities
4242

4343
! Convert the dynamics output state into the physics input state.
4444
! Note that all pressures and tracer mixing ratios coming from the dycore are based on
4545
! dry air mass.
4646
use cam_history, only: hist_fld_active
47-
use dyn_comp, only: frontgf_idx, frontga_idx
47+
use dyn_comp, only: frontgf_idx, frontga_idx, vort4gw_idx
4848
use mpas_constants, only: Rv_over_Rd => rvord
49-
use phys_control, only: use_gw_front, use_gw_front_igw
49+
use phys_control, only: use_gw_front, use_gw_front_igw, use_gw_movmtn_pbl
5050
use cam_budget, only : thermo_budget_history
5151

5252
! arguments
@@ -101,6 +101,12 @@ subroutine d_p_coupling(phys_state, phys_tend, pbuf2d, dyn_out)
101101
real(r8), allocatable :: frontgf_phys(:,:,:)
102102
real(r8), allocatable :: frontga_phys(:,:,:)
103103

104+
! Temporary arrays to hold vorticity for the "moving mountain" gravity wave scheme.
105+
real(r8), allocatable :: vort4gw(:, :) ! Data are unchunked.
106+
real(r8), allocatable :: vort4gw_phys(:, :, :) ! Data are chunked.
107+
! Pointer to vorticity in physics buffer for the "moving mountain" gravity wave scheme.
108+
real(r8), pointer :: pbuf_vort4gw(:, :)
109+
104110
type(physics_buffer_desc), pointer :: pbuf_chnk(:)
105111

106112
integer :: lchnk, icol, icol_p, k, kk ! indices over chunks, columns, physics columns and layers
@@ -116,6 +122,11 @@ subroutine d_p_coupling(phys_state, phys_tend, pbuf2d, dyn_out)
116122
character(len=*), parameter :: subname = 'd_p_coupling'
117123
!----------------------------------------------------------------------------
118124

125+
nullify(pbuf_chnk)
126+
nullify(pbuf_frontgf)
127+
nullify(pbuf_frontga)
128+
nullify(pbuf_vort4gw)
129+
119130
compute_energy_diags=thermo_budget_history
120131

121132
nCellsSolve = dyn_out % nCellsSolve
@@ -155,9 +166,6 @@ subroutine d_p_coupling(phys_state, phys_tend, pbuf2d, dyn_out)
155166

156167
if (use_gw_front .or. use_gw_front_igw) then
157168
call cam_mpas_update_halo('scalars', endrun) ! scalars is the name of tracers in the MPAS state pool
158-
nullify(pbuf_chnk)
159-
nullify(pbuf_frontgf)
160-
nullify(pbuf_frontga)
161169
!
162170
! compute frontogenesis function and angle for gravity wave scheme
163171
!
@@ -195,6 +203,16 @@ subroutine d_p_coupling(phys_state, phys_tend, pbuf2d, dyn_out)
195203

196204
end if
197205

206+
if (use_gw_movmtn_pbl) then
207+
call cam_mpas_vertex_to_cell_relative_vorticities(vort4gw)
208+
209+
allocate(vort4gw_phys(pcols, pver, begchunk:endchunk), stat=ierr)
210+
211+
if (ierr /= 0) then
212+
call endrun(subname // ': Failed to allocate vort4gw_phys')
213+
end if
214+
end if
215+
198216
call t_startf('dpcopy')
199217

200218
ncols = columns_on_task
@@ -225,6 +243,10 @@ subroutine d_p_coupling(phys_state, phys_tend, pbuf2d, dyn_out)
225243
frontgf_phys(icol_p, k, lchnk) = frontogenesisFunction(kk, i)
226244
frontga_phys(icol_p, k, lchnk) = frontogenesisAngle(kk, i)
227245
end if
246+
247+
if (use_gw_movmtn_pbl) then
248+
vort4gw_phys(icol_p, k, lchnk) = vort4gw(kk, i)
249+
end if
228250
end do
229251

230252
do k = 1, pverp
@@ -261,6 +283,25 @@ subroutine d_p_coupling(phys_state, phys_tend, pbuf2d, dyn_out)
261283
deallocate(frontogenesisAngle)
262284
end if
263285

286+
if (use_gw_movmtn_pbl) then
287+
!$omp parallel do private (lchnk, ncols, icol, k, pbuf_chnk, pbuf_vort4gw)
288+
do lchnk = begchunk, endchunk
289+
ncols = get_ncols_p(lchnk)
290+
pbuf_chnk => pbuf_get_chunk(pbuf2d, lchnk)
291+
292+
call pbuf_get_field(pbuf_chnk, vort4gw_idx, pbuf_vort4gw)
293+
294+
do k = 1, pver
295+
do icol = 1, ncols
296+
pbuf_vort4gw(icol, k) = vort4gw_phys(icol, k, lchnk)
297+
end do
298+
end do
299+
end do
300+
301+
deallocate(vort4gw)
302+
deallocate(vort4gw_phys)
303+
end if
304+
264305
call t_stopf('dpcopy')
265306

266307
call t_startf('derived_phys')

0 commit comments

Comments
 (0)