Incorporate Sondes DA into JEDI for HAFS #16
Comments
Start with the
|
|
I've assigned the error as what HAFS GSI does with |
|
Adding observation inflation using |
|
Test on specificHumidity |
|
Test with uv. Here for the wind observations, introducing hofx scaling into the observation operator. if not use GSI geovals, then there are slight difference in JEDI and GSI hofx. The hofx scaling also did some correction here. HOFX after scaling - HOFX no scaling |
|
Rerun the GSI with
|
|
Though the IODA NC file is created from GSI diag file, but it tends out several points are out of JEDI domain boundary. Using the RRFS domain filter tool to further filter out points outside or near the boundary. |
|
After applying the domain check, the hofx difference between jedi and gsi are smaller. But the large difference near top of the model remains. |
|
Select the point where has max hofx difference, the location is lat=4.75 lon=285.80 FROM GSI DIAG: FROM ncdiag file converted IODA nc: |
|
Check the gsi and jedi geoval files to see the "air_pressure" distribution difference. GSI geofiles contains lat and lon information, so it's easier to locate the obs location that located near "4.75 285.80".
From GSI For JEDI, |
|
The index match method may not correct. So check the 1st point in both GSI and JEDI GeoVals. |
|
Found a bug in finding the matched point index in JEDI GEOVaLs. From gsi geoval file nc: At least we know the difference in hofx shouldn't be coming from air_pressure. |
|
@JingCheng-NOAA, Thanks for testing this. It looks like you are using GSI-IODA but not using GSI-geovals and just letting JEDI compute its own geovals, is that correct? Did you say that you get identical results when using the GSI-geovals? Can you add the the same pressure vs hofx difference plot for when using GSI-geovals? |
Yes the results of using same GSI GeoVaLs are showed in https://github.com/NOAA-EMC/HDASApp/issues/16#issuecomment-2372193384 And if plot the difference in vertical distribution: |
|
@JingCheng-NOAA, interesting. I would expect that if you get identical results with GSI-geovals then you would have identical results at levels 1-15 and still have differences elsewhere. But you might be using gph as vertical coordinate or something else instead of pressure. Check what is used in GSI. |
I checked the setupw.f90 in GSI and for this type of observations (otype=220), the GSI is using pressure levels. |
|
An update was made in comparison GSI test, block the otype 232 and only keep the otype 220. |
|
|
|
|
|
|
|
|
|
I've noticed this discussion in GDASApp, which compared the difference in Cube and Gaussian grids. This may explain our issue found in the difference of geovals in GSI and JEDI. |
|
Move to phase 2 for sounding data. |
|
Modified the bufr2ioda python script to add in MetaData/timeOffset. This variable will be used to QC the observation that fall out of time window |
|
Regenerated IODA nc file with updated bufr2ioda yaml file. Here are some statistics:
*JEDI with obs filter applied:
|
|
The observation distribution plot illustrated where the missing observations in JEDI are. |
|
Updated IODA yaml file with correct time and recreated observation file shows correct observation numbers that similar to GSI. |
|
Check for airTemperature and specificHumidity |
|
Thanks to @delippi 's plotting scripts, I've checked the stats after QC for each variable. Some of the observation error still has difference. |
|
An update was made in Wind yaml file, including adjusting the gross error and a bug fix in
|
|
For specific Humidity, adjusted the gross error min/max in JEDI yaml; And fixed a bug in the initial error assignment in JEDI yaml (the value grabbed from GSI errtable should be divided by 10 for q) |
|
The observation error pattern mismatch in airTemperature. |
|
In GSI read_prepbufr.f90, the error is inflated by 1.2, which leads to I am wondering which function in JEDI can do similar inflation. |
|
@JingCheng-NOAA, yep this should be fairly easy and probably a few ways to do it. One of the easiest ways would probably be to use the |
|
Thank you Donnie @delippi , this is very helpful. I will test it too once Hera comes back. |
|
@JingCheng-NOAA, if you hadn't already figured out what was wrong with my previous try, this works. I was close. |
|
Thanks, @delippi . I've tried this one and it works for my single observation test. The observation error in JEDI now looks the same as in GSI single ob exp. |
|
After apply the inflation fix to the phase 2 test, the results look improved. |
|
@JingCheng-NOAA, for the 1.2 inflation, it looks like this is done for all conventional obs and for when QualityMarker==3 or 7. Is that correct? I think I should add this to my configurations. |
|
@delippi, according to read_prepbufr.f90, this inflation only happen to the Temperature observation on pressure level 100hPa or above; and wind observation on pressure level 50hPa or above. I think the QM is irrelevant here. So based on what you taught me in previous comment, I added following part right after the initial error assignment, |
|
Actually, I think we're both only half right. I see the following in read_prepbufr for temperature. The obs can get inflated twice by 1.2. inflate error comes from |
|
Yes I think you are right. My case doesn't have qm 3 or qm 7 that's why I thought the inflate_error is disabled. But this part should be added in too. |
|
Updated the yaml file to added in the qm 3 or qm 7 inflation.
|
|
The remain missing observation error inflation is related to the duplication check that designed in setupt.f90. The corresponding obsFunction in JEDI is So far, besides stationPressure (requires further changes in IODA converter), the phase 2 validation of sonde yamls is considered completed. |
As the single ob DA is done within JEDI for HAFS, this issue is opened for more sounding obs DA in JEDI and comparison with GSI.
This issue will record the process for IODA file generation, JEDI yaml file for upper air DA, validation of Hx.
The text was updated successfully, but these errors were encountered: