[AWIPS #CWJ-540985]: grb2 decode question.

> Greetings.  A follow up question.   The attached small grib2 file has two 
> fields containing locally generated diagnostics stored with standard names
> 
> 1:0:d=2022050200:GUST Wind Speed (Gust) [m/s]:surface:anl:
> 2:3491:d=2022050200:MAXGUST Maximum Wind Speed [m/s]:surface:anl:
> 
> Recall that GUST and MAXGUST are non-standard fields - layer average and 
> layer max values of wind speed in the layer between the ground and the first 
> inversion but had to be named with standard WMO names due to the limitations 
> of the cfgrib python module.
> 
> Anyway -- after imparting the EDEX changes below we were expecting both to be 
> in the /awips2/edex/data/hdf5/grid directory -- but we only see the following 
> directory created  GFS_FE-GSTX  (were expecting also to see a GFS_FE-GSTF dir 
> too).
> 
> We note that in step 4 below there are identical entries for these fields in 
> the model definition file except for the name tag -- we suspect that EDEX 
> cant distinguish between them because of this reason and uses the last entry 
> in the model definition file and therefore only processes the last of the two 
> grib fields in the grib2 file.
> 
> Question is - what do you recommend changing to allow both these non-standard 
> fields to be processed with one submission -- it seems we have three options 
> and were wondering in your experience which would be the better path...
> Option 1 - create a grid proj file specific for each non-standard field (gets 
> messy if you have a lot of non-standard fields)
> Option 2 - assign a new process id to the grib fields in the grib2 file (this 
> is possible with wgrib2 -set analysis_or_forecast_process_id xxx  but since 
> xxx can only range from 0 to 254 this will limit you to 255 different 
> non-standard fields for a given model -- which may be enough)
> Option 3 - something else we hadn’t thought of that you can recommend
> 
> Here are the details.
> 
> 1. For  both fields the Center=7, Subcenter = 0, Discipline=0, Category=2.   
> The ID for GUST is 21 and the ID for MAXGUST is 22.   And the processid for 
> both is 128  (we think this may be the problem)
> 
> 2. Added the two lines at the end of 
> /awips2/edex/data/utility/common_static/base/grib/tables/7/0/4.2.0.2.table
> 21:21:Momentum Xfer Lyr Max:m/s:GSTX
> 22:22:Momentum Xfer Lyr Avg:m/s:GSTF
> 
> 3.  Created a grid projection file   
> /awips2/edex/data/utility/common_static/base/grib/grids/FE_48_by_23_grid_at_0.250000_degrees_horizontal_res.xml
>    containing the following lines
> <latLonGridCoverage>
> <name>4823</name>
> <description>48 by 23 grid at 0.250000 degrees horizontal res</description>
> <la1>42.500000</la1>
> <lo1>274.750000</lo1>
> <firstGridPointCorner>UpperLeft</firstGridPointCorner>
> <nx>48</nx>
> <ny>23</ny>
> <dx>0.250000</dx>
> <dy>0.250000</dy>
> <spacingUnit>degree</spacingUnit>
> </latLonGridCoverage>
> 
> 4. Added the following lines to 
> /awips2/edex/data/utility/common_static/base/grib/models/gribModels_NCEP-7.xml
> <model>
> <name>GFS_FE-GSTF</name>
> <center>7</center>
> <subcenter>0</subcenter>
> <grid>4823</grid>
> <process>
> <id>128</id>
> </process>
> </model>
> <model>
> <name>GFS_FE-GSTX</name>
> <center>7</center>
> <subcenter>0</subcenter>
> <grid>4823</grid>
> <process>
> <id>128</id>
> </process>
> </model>
> 
> 5. Restart ingestgrib  as awips user with the command  sudo service 
> edex_camel restart ingestGrib
> 
> 6. Cleared out the hdf5 grid dir with /bin/rm -rf 
> /awips2/edex/data/hdf5/grid/*
> 
> 7. Resubmitted the test.grib2 file to edex ingest with the command  cp 
> test.grib2 /awips2/data_store/ingest/
> 
> 8. We get in the edex ingest log confirmation it was accepted
> INFO  2022-05-04 15:54:57,310 4079 [GribPersist-4] Ingest: EDEX: Ingest - 
> grib2:: /awips2/data_store/manual/grib/20220504/15/test.grib2 processed in: 
> 2.1820 (sec) Latency: 4.6170 (sec)
> INFO  2022-05-04 15:54:57,310 4080 [GribPersist-8] Ingest: EDEX: Ingest - 
> grib2:: /awips2/data_store/manual/grib/20220504/15/test.grib2 processed in: 
> 2.1860 (sec) Latency: 4.6170 (sec)
> 
> 9. But only see one subdir in /awips2/edex/data/hdf5/grid   GFS_FE_GSTX (and 
> only one hd5f file    
> /awips2/edex/data/hdf5/grid/GFS_FE-GSTX/SFC/GFS_FE-GSTX-2022-05-03-00-FH-000.h5
> 

Hi Pete,

You're really close! 

So the grid projection file (step 3) and the entry in gribModels_NCEP-7.xml 
(step 4) is just information about the "model" (ex GFS, RAP, GFS_FE, etc.). 
GUST and MAXGUST are two different variables from the same "model" (Just like 
the GFS model has Temp, Dewpoint, Wind, etc. variables) and are only referenced 
via the tables. So you should only have one entry in gribModels_NCEP-7.xml 
because you only have one model. You could probably rename it to be called 
"GFS_FE". 

As you were seeing in the logs, both files were getting ingested still. The 
directory in /awips2/edex/data/hdf5/grid corresponds only to the model, not the 
variable. Most likely the second variable was being stored with the first 
"model name". You can check this by doing an "h5ls 
GFS_FE-GSTX-2022-05-03-00-FH-000.h5 | more" and you should see both variables 
listed there. Additionally, if you look in the product browser you should see 
both of them listed as well (probably under GFS_FE-GSTX --> SFC).

In short, remove the second entry in step #4 rename the model name to be a 
little more generic. Then restart ingestGrib.

For the volume browser, you will add the model name to the volume.xml file and 
then both variables to the misc.xml file.



Thanks,

Tiffany Meyer
AWIPS Lead Software Engineer
UCAR-Unidata

If you're interested, please feel free to fill out a survey about the support 
you receive:  
https://docs.google.com/forms/d/e/1FAIpQLSeDIkdk8qUMgq8ZdM4jhP-ubJPUOr-mJMQgxInwoAWoV5QcOw/viewform

Ticket Details
===================
Ticket ID: CWJ-540985
Department: Support AWIPS
Priority: Critical
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.