NoahMP as a submodule in WRF (wrf-model#1653)

TYPE: enhancement

KEYWORDS: NoahMP, submodule

SOURCE: Cenlin He (RAL/NCAR), internal

DESCRIPTION OF CHANGES:
This is an initial effort to include an external stand-alone repository into the
WRF system using git "submodules".

Here is the recipe, should we be interested in doing this again. 

The code does not build after this initial step, as we are removing mandatory source files that are
required for the build of the physics directory.

1. Remove the existing WRF-versions of the NoahMP source code
```
cd WRF/phys
git rm module_sf_noahmp_glacier.F
git rm module_sf_noahmp_groundwater.F
git rm module_sf_noahmpdrv.F
git rm module_sf_noahmplsm.F
```
... and the single table over which NoahMP has dominion
```
cd WRF/run
git rm MPTABLE.TBL
```
2. Add the NoahMP repository as a submodule
```
cd WRF/phys
git submodule add https://github.com/NCAR/noahmp noahmp
```
This generates the new file WRF/.gitmodules

3. Along with the rest of the staged files, include the SHA1 of the NoahMP repository
as part of the what the WRF model keeps permanently.
```
cd WRF/phys/noahmp
git checkout release-v4.4-WRF
cd ..
git add noahmp
```
Right now, this is SHA1 aed82e17e95b5 from NoahMP
NCAR/noahmp@aed82e1

LIST OF MODIFIED FILES:
A .gitmodules
D phys/module_sf_noahmp_glacier.F
D phys/module_sf_noahmp_groundwater.F
D phys/module_sf_noahmpdrv.F
D phys/module_sf_noahmplsm.F
A phys/noahmp
D run/MPTABLE.TBL

The code builds after this step, which includes the mods to the build system (the variously impacted Makefiles).
LIST OF MODIFIED FILES:
M Makefile
M clean
M phys/Makefile

In the top-level WRF Makefile, the `physics` target now includes a pre-step
to first build the NoahMP submodule files. If the NoahMP source files exist 
(the *.F files). 
   * If so, then proceed as normally (they must already be linked in). 
   * If the files do not exist, then 
     `git submodule update --init --recursive` to bring in
     the NoahMP repository, and then manually link the files into the
     original WRF locations. Also, if the source files did not exist, 
     assume that the NoahMP table file (MPTABLE.TBL) is also missing, 
     so put that file in the run directory.

Since these files are not really part of the WRF repo anymore, we remove
them when asked for a super-duper cleaning: `clean -aa`. Please note that
usage of the `clean -aa` script removes the backup configure.wrf file.

An "if" test in the phys/Makefile detects if the NoahMP source files
exist. If not, then the git submodule command is used. The important
item is that the top-level Makefile has a submodules target that is processed
for each build.

The NoahMP files are linked into the physics directory. This emulates the existing paradigm
of putting all of the physics source into a single location. This minimizes the infrastructure
modifications, and becomes almost transparent to a casual user. Care will need to be taken
so that no files associated with NoahMP are accidentally re-incorporated back into WRF.

Considerations:
The introduction of an external repository is a complication. We choose to avoid modifying the WRF build 
system, meaning that the entire physics directory must be WHOLE for a successful build. This is handled 
by copying files from the NoahMP repository directory location to the WRF repository directory location.
The infrastructure has been put in place in the WRF build to copy the files to the expected WRF locations.
Therefore, starting from a clean clone and without a preceding build, a user will not find the NoahMP source 
files in the physics directory. The NoahMP repository could be manually included as a submodule with the 
following command:
```
cd WRF
git submodule update --init --recursive
```
However, again, until a build occurs, the files are not moved from the NoahMP repository directory structure into
the proper and expected locations in the WRF directory structure.

TESTS CONDUCTED:
1. The code builds locally. When there are no NoahMP files, they are available with a git submodule
command, and subsequent linking.
2. With print statements in the Makefiles, when existing NoahMP files are detected, the submodule
step is bypassed.
3. If the required files are not available, an error condition is raised, and the compilation stops:
```
------------------------------------------------------------------------------
Error Error Error NoahMP submodule files not populating WRF directories
------------------------------------------------------------------------------
```
4. When the NoahMP files are available, a message is printed to the build log:
```
------------------------------------------------------------------------------
NoahMP submodule files populating WRF directories
------------------------------------------------------------------------------
```
5. Jenkins  - all tests are pass.

RELEASE NOTE: The NoahMP external repository is now a submodule for WRF.

Author: davegill

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "phys/noahmp"]
+	path = phys/noahmp
+	url = https://github.com/NCAR/noahmp

Makefile

@@ -108,6 +108,22 @@ framework_only : configcheck
 
 wrf : framework_only
 	$(MAKE) MODULE_DIRS="$(ALL_MODULES)" physics
+	if [ \( ! -f run/MPTABLE.TBL \) -o \
+	     \( ! -f phys/module_sf_noahmpdrv.F \) -o \
+	     \( ! -f phys/module_sf_noahmp_glacier.F \) -o \
+	     \( ! -f phys/module_sf_noahmp_groundwater.F \) -o \
+	     \( ! -f phys/module_sf_noahmplsm.F \) ] ; then \
+	   echo " " ; \
+	   echo "------------------------------------------------------------------------------" ; \
+	   echo "Error Error Error NoahMP submodule files not populating WRF directories" ; \
+	   echo "------------------------------------------------------------------------------" ; \
+	   echo " " ; \
+	   exit 31 ; \
+	else \
+	   echo "------------------------------------------------------------------------------" ; \
+	   echo "NoahMP submodule files populating WRF directories" ; \
+	   echo "------------------------------------------------------------------------------" ; \
+	fi
 	if [ $(WRF_CHEM) -eq 1 ]    ; then $(MAKE) MODULE_DIRS="$(ALL_MODULES)" chemics ; fi
 	if [ $(WRF_EM_CORE) -eq 1 ]    ; then $(MAKE) MODULE_DIRS="$(ALL_MODULES)" em_core ; fi
 	if [ $(WRF_HYDRO) -eq 1 ]   ; then $(MAKE) MODULE_DIRS="$(ALL_MODULES)" wrf_hydro ; fi
@@ -1009,20 +1025,20 @@ chemics :
 physics :
 	@ echo '--------------------------------------'
 	if [ $(WRF_CHEM) -eq 0 ] ; then \
-		( cd phys ; $(MAKE) CF2=" " ) ; \
+		( cd phys ; $(MAKE) submodules ; $(MAKE) CF2=" " ) ; \
 		if [ $(WRF_CMAQ) -eq 1 ] ; then \
 			@ echo '----------- make cmaq ----------------' ; \
 			( rm -f main/libcmaqlib.a; cd cmaq ; $(MAKE) -f Makefile.twoway ) ; \
 		fi \
 	else \
-		( cd phys ; $(MAKE) CF2="$(CHEM_FILES2)" ) ; \
+		( cd phys ; $(MAKE) submodules ; $(MAKE) CF2="$(CHEM_FILES2)" ) ; \
 	fi
 
 physics_plus :
 	if [ $(WRF_PLUS_CORE) -eq 0 ] ; then \
-	   ( cd phys ; $(MAKE) PHYS_PLUS=" " PHYS_MP=" " PHYS_BL=" " PHYS_CU=" " ) ; \
+	   ( cd phys ; $(MAKE) submodules ; $(MAKE) PHYS_PLUS=" " PHYS_MP=" " PHYS_BL=" " PHYS_CU=" " ) ; \
 	else \
-	   ( cd phys ; $(MAKE) PHYS_PLUS="$(MODS4)" PHYS_MP="$(MODMP)" PHYS_BL="$(MODBL)" PHYS_CU="$(MODCU)" ) ; \
+	   ( cd phys ; $(MAKE) submodules ; $(MAKE) PHYS_PLUS="$(MODS4)" PHYS_MP="$(MODMP)" PHYS_BL="$(MODBL)" PHYS_CU="$(MODCU)" ) ; \
 	fi
 
 wrftlmadj :

clean

@@ -80,6 +80,9 @@ if ( "$arg" == '-a' || "$arg" == '-aa' ) then
     /bin/rm -f configure.wrf.backup
     /bin/rm -f Registry/Registry.backup
     /bin/rm -f run/namelist.input.backup.*
+    /bin/rm -f phys/module_sf_noahmpdrv.F phys/module_sf_noahmp_glacier.F \
+               phys/module_sf_noahmp_groundwater.F phys/module_sf_noahmplsm.F \
+               run/MPTABLE.TBL
   endif
 endif
 

phys/Makefile

@@ -240,6 +240,20 @@ include ../configure.wrf
 
 non_nmm : $(MODULES) $(FIRE_MODULES) $(WIND_MODULES) $(OBJS) $(DIAGNOSTIC_MODULES_EM)
 
+submodules :
+	@if [ \( ! -f module_sf_noahmpdrv.F \) -o \( ! -f module_sf_noahmp_glacier.F \) -o \
+	      \( ! -f module_sf_noahmp_groundwater.F \) -o \( ! -f module_sf_noahmplsm.F \) ] ; then \
+		echo Pulling in NoahMP submodule ; \
+		( cd .. ; git submodule update --init --recursive ) ; \
+		ln -sf noahmp/drivers/wrf/module_sf_noahmpdrv.F . ; \
+		ln -sf noahmp/src/module_sf_noahmp_glacier.F . ; \
+		ln -sf noahmp/src/module_sf_noahmp_groundwater.F . ; \
+		ln -sf noahmp/src/module_sf_noahmplsm.F . ; \
+		( cd ../run ; ln -sf ../phys/noahmp/parameters/MPTABLE.TBL . ) ; \
+	else \
+		echo No action required for NoahMP submodule ; \
+	fi
+
 clean:
 	@ echo 'use the clean script'