[netCDFJava #PYP-475421]: Bug in NetCDF-Java 4.3 exposed by this GRIB2 file

Hi Rich:

It is an error in the CFSR files (they have lots). We have a workaround in NcML 
and if served from the TDS. Can you live with that?

John

> Lansing,
> 
> I was expecting you to say that this issue was a bug in the GRIB
> reader, and it would get a ticket to be fixed.   But I think you are
> saying that the data was written wrong, and therefore the issue will
> not be fixed?
> 
> This is one of those "big deal" datasets that it would be great to
> read correctly.
> http://rda.ucar.edu/pub/cfsr.html
> 
> It's good to know that NcML can fix the issue for a file, but there
> are hundreds of these, so I would be better to fix the reader if you
> can.
> 
> Thanks,
> -Rich
> 
> On Tue, Jun 12, 2012 at 12:53 PM, Unidata netCDF Java Support
> <address@hidden> wrote:
> >> Lansing,
> >>
> >> It looks like you are using ToolsUI 4.2, not ToolsUI 4.3.   In toolsUI
> >> 4.2, you see one variable, but the times are wrong.   In 4.3, you see
> >> two variables with correct times, but we want one variable with
> >> correct time!
> >>
> >> Try this:
> >> http://www.unidata.ucar.edu/software/netcdf-java/v4.3/webstart/netCDFtools.jnlp
> >>
> >> On Mon, Jun 11, 2012 at 1:10 PM, Unidata netCDF Java Support
> >> <address@hidden> wrote:
> >> >> NetCDF-Java folks:
> >> >>
> >> >> There appears to be a bug in the GRIB2 decoding problem using
> >> >> NetCDF-Java 4.3 on some of the surface pressure data we got from
> >> >> http://rda.ucar.edu/pub/cfsr.html
> >> >>
> >> >> This data is supposed to have a single variable (pressure) with
> >> >> uniform hourly sampling, but when we bring it up in NetCDF 4.3 tools,
> >> >> we see two variables instead of one, with the 1st having a uniform
> >> >> time base of 6 hours (0,6,12), and the 2nd having a non-uniform time
> >> >> base that fills in the rest of the hourly data  (1,2,3,4,5
> >> >> 7,8,9,10,11  13,14...)
> >> >>
> >> >> Screenshot: http://screencast.com/t/DWEgKMfy
> >> >>
> >> >> I put a sample file here:
> >> >> http://geoport.whoi.edu/thredds/catalog/usgs/data1/rsignell/test/catalog.html?dataset=usgs/data1/rsignell/test/pressfc.gdas.199801.grb2
> >> >>
> >> >> Thanks,
> >> >> Rich
> >> >> --
> >> >> Dr. Richard P. Signell   (508) 457-2229
> >> >> USGS, 384 Woods Hole Rd.
> >> >> Woods Hole, MA 02543-1598
> >> >>
> >> >>
> >> > Hello Rich,
> >> >
> >> > I'm not sure what's going on, but I tried your file and saw only one 
> >> > variable reported.  I've attached a screenshot from NetCDF-Java ToolsUI. 
> >> >  Maybe I'm missing something obvious here?
> >> >
> >> > Regards,
> >> >  Lansing Madry
> >> >  Unidata
> >> >  Boulder, Colorado
> >> >
> >> > Ticket Details
> >> > ===================
> >> > Ticket ID: PYP-475421
> >> > Department: Support netCDF Java
> >> > Priority: Normal
> >> > Status: Open
> >>
> >>
> >>
> >> --
> >> Dr. Richard P. Signell   (508) 457-2229
> >> USGS, 384 Woods Hole Rd.
> >> Woods Hole, MA 02543-1598
> >>
> >>
> > Rich,
> >
> > Thanks for educating me on 4.2 vs. 4.3 -- details, details!
> >
> > I dug a little deeper to see what is going on with the data in the file you 
> > sent.  It looks like, at the byte level, there are some differences that 
> > 4.2 conveniently ignored and 4.3 notices.  Specifically, if you look at the 
> > file in the iosp/GRIB2/GRIB2Collection tab, you'll notice that two 
> > variables seem to show up, with separate GDS hash codes.  In that view, if 
> > you look down to the Grib2GridDefinitionSection, highlight a row, 
> > right-click on the row, and select "Show GDS", you'll see the information 
> > for Lo2 differs for the two "different" variables:
> >
> > Lo2 - longitude of last grid point == 359687000
> > Lo2 - longitude of last grid point == 359688000
> >
> > This is really a defect in how the data was written out - perhaps a 
> > rounding error somewhere, or some other loss of precision.
> >
> > In the meantime, you can "correct" the issue using a bit of NcML to change 
> > the GDS hashcode.  Details are here:
> >
> > http://www.unidata.ucar.edu/software/netcdf-java/formats/GribFiles.html
> >
> > and I've tried this bit of NcML and found it to work in this case:
> >
> > <?xml version="1.0" encoding="UTF-8"?>
> > <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; 
> > location="C:/Users/madry/rsignell/pressfc.gdas.199801.grb2">
> >  <iospParam>
> >   <gdsHash from="1450192070" to="1450218978"/>
> >  </iospParam>
> > </netcdf>
> >
> > However, you will need to delete the associated .ncx file before opening 
> > this NcML file in the toolsUI-4.3 viewer.
> >
> > Let me know if this doesn't work for you, and I'll take another crack at it.
> >
> > Cheers,
> >  Lansing
> >
> > Ticket Details
> > ===================
> > Ticket ID: PYP-475421
> > Department: Support netCDF Java
> > Priority: Normal
> > Status: Open
> >
> 
> 
> 
> --
> Dr. Richard P. Signell   (508) 457-2229
> USGS, 384 Woods Hole Rd.
> Woods Hole, MA 02543-1598
> 
> 


Ticket Details
===================
Ticket ID: PYP-475421
Department: Support netCDF Java
Priority: Normal
Status: Open