Re: Problems with GRIB feature collections

Greetings Roy,

For the first question about the clean vs dirty lat/lon's, this turns out to be a floating point math problem. Specifically, here:

https://github.com/Unidata/thredds/blob/master/grib/src/main/java/ucar/nc2/grib/grib1/Grib1Gds.java#L394

In the case of the FNMOC NAVGEM grib records, we get the following:

code: la1 = getOctet3(11) * scale3;

math: (int) -90000 * (float) 1.0e-3 -> (float) -90.00001

I think a float *should* be ok for storing la1 and computing the lat array, but not for the initial computation of of la1. I'm pretty sure we can get this addressed (I'll open a github issue).

For the second question regarding "full" vs "best", this has to do with how the overall collection is aggregated. For the half-degree NAVGEM, you have monthly collections of grids, but none of the forecasts overlap. Therefore, the overall collection aggregation exposes all of the grids. For the one degree, it looks like the TDS thinks there was an overlap in the grids (i.e. two grids with the same valid time), and so one of the overlapped grids was "better" (i.e. closer to a runtime) and the other was dropped to make a "best" collection. However, I don't see that happening in any of the monthly collections listed, so it is probably a bug. In the xml for these two collections, can you add "TwoD" to the datasetTypes attribute of the gribConfig element? That will give us a bit more information.

Cheers,

Sean

On Tue, Dec 5, 2017 at 4:52 PM, Roy Mendelssohn - NOAA Federal <address@hidden> wrote:

Hi Sean:

I am having even more problems I don't understand.  I have attached two catalogs I am using,  one for NAVGEM 1-degree data,  one for NAVGEM half-degree data,  all GRIB-1.  Now as best as I can see these tow catalogs are the same except for the obvious places where the paths and links etc etc must be different.  Yet the one-degree produces a "Best" link:

http://thredds1.pfeg.noaa.gov/thredds/catalog/FNMOC/NAVGEM/onedegree/pressure/6hr/catalog.html


while the half-degree produces a "Full Collection" link:

http://thredds1.pfeg.noaa.gov/thredds/catalog/FNMOC/NAVGEM/halfdegree/pressure/6hr/catalog.html

I have no idea why they would come up different ,  as I believe the catalogs are basically the same.  Moreover for the one-degree,  if you go to:

http://oceanwatch.pfeg.noaa.gov/thredds/dodsC/FNMOC/NAVGEM/onedegree/pressure/6hr/Best.ascii?time[6680:1:6687]

You will see that the last time. with units "Hour since 2013-02-21T00:00:00Z" is something like Dec 8.  I do not have that problem with half-degree.  I have checked the recent files and they are all good,  none show that time.

I am completely stumped by this.

Thanks for any help,

-Roy



> On Dec 5, 2017, at 12:05 PM, Sean Arms <address@hidden> wrote:
>
> Greetings Roy,
>
> I'm seeing the same thing in our 0.5 degree NAVGEM collection as well:
>
> http://thredds.ucar.edu/thredds/dodsC/grib/FNMOC/NAVGEM/Global_0p5deg/FNMOC_NAVGEM_Global_0p5deg_20171205_1200.grib1.ascii?lat[0:1:360],lon[0:1:719]
>
> I do not see the same behavior in our GFS 0.5 degree collection, though:
>
> http://thredds.ucar.edu/thredds/dodsC/grib/NCEP/GFS/Global_0p5deg/GFS_Global_0p5deg_20171205_1200.grib2.ascii?lat[0:1:360],lon[0:1:719]
>
>
> Of course, this is grib1 vs grib2, so different parts of the code. Let me take a look and see what I can find.
>
> Cheers,
>
> Sean
>
>
> On Tue, Dec 5, 2017 at 12:47 PM, Roy Mendelssohn - NOAA Federal <address@hidden> wrote:
> Hi Sean:
>
> We are running TDS 4.6.10. I am using catalog.xml similar to what we discussed in our back and forth.  Please look at say:
>
> http://oceanwatch.pfeg.noaa.gov/thredds/dodsC/FNMOC/NAVGEM/halfdegree/pressure/6hr/TP.ascii?lat[0:1:360],lon[0:1:719]
>
> Note that the "lats" aren't "clean".  In some of theaters, it is the "lons", in some it is both.  Can send you sample files.
>
> -Roy
>
>
>
>
>
>
> **********************
> "The contents of this message do not reflect any position of the U.S. Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> ***Note new street address***
> 110 McAllister Way
> Santa Cruz, CA 95060
> Phone: (831)-420-3666
> Fax: (831) 420-3980
> e-mail: address@hidden www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
>
>

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: address@hidden www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.